Measuring Performance issues of Dynamics 365 with Dynatrace and StresStimulus

Here’s the design and the operationalization of the architecture related to Dynamics 365 / Dynatrace / StresStimulus. I intend to describe below how we measure the performance issue s of the CRM Servers with Dynatrace and StresStimulus.

Context and problem

 

Here’s the characteristics of the Architecture related to Dynamics 365 Servers :

  • It’s a point-to-point integration architecture between the CRM Architecture (CRM System with multiple nodes) and external applications : point-to-point communication;

  • No Messaging infrastructure connects the components and services : highly coupled;

  • No local orchestration in CRM Architecture and all communication is synchronous;

  • CRM Architecture does not include Caching Server; but, use of AppFabric Server as Caching Service (for external app);

  • CRM Reporting Servers not in Frontend or Backend Servers;

  • CRM Architecture has been scaled :

    • Vertical scaling has been done (CPU, RAM, …) : servers have been boosted

    • Horizontal scaling has been done (more node been added) : 6 nodes (4 nodes for the Frontend Services, 2 nodes for the backend Services, 2 nodes for SQL Servers including an CRM Always On Cluster)

  • CRM Architecture uses Load Balancing (F5) - Round-Robin pattern is implemented.

High-Level Architecture

Issues and problems : what do we know ?

  • Number of applicative errors (timeoutexception, etc.) is increasing because the communication between the systems is growing : errors are related to CRM frontend and backend servers;

  • CRM applicative errors impact the communication with the other servers (integration & other), getting for example timeoutexception…;

  • Errors are related to performance issues;

  • We know that CRM (frontend services) generate constantly dynamic variables and we absolutely need to catch them during our performance tests.

Question : what do we need ?

We need to dig into the performance issues and so, we need to measure the performance of CRM servers.

High Level and Applicative Errors

Solutions

Architecture Components - Our performance framework.

Architecture components

 

How does work our Architecture ?

STEPS

DESCRIPTION

STEPS

DESCRIPTION

1

Tidal executes the Bat file.

2

The bat file execute StresStimulus which executes transaction in the CRM Platform : CRUD Operations.

3

Tidal executes the PowerShell Code (Custom Performance Framework - CPF).

4

The PowerShell Code executes :

  • SQL Queries to extract Wait Stats.

  • Executes PerfMon to extract stats (perf. counters and IIS events).

5

The PowerShell Code executes a monitoring : each transaction and code execution will be logged. The log will be accessible by Tidal.

6

  • StresStimulus stores the results in its SQL Database : time response.

  • The CPF stores the results related to the Wait Stats in a SQL Database.

  • PerfMon will generate the results related to Performance Counters and IIS Events in multiple files.

7

  • The configuration of the http-headers variables in StresStimulus will allow Dynatrace to catch the CRM transactions triggered by StresStimulus.

  • Dynatrace will generate a reporting accessible through its Web Interface.

  • StresStimulus will generate a reporting accessible through its application.

8

All the data needed to analyze the performance will be accessible and categorized.

 

Benefits and results

  • We managed to prove that the performance issue was not related to the database servers (after scaling in and out).

  • We managed to prove that the performance issue was related to the front-end servers : web requests.

  • We managed to prove that the performance issue was related to the back-end servers.

  • We managed to get consistent data and to show that the results from extracted by StresStimulus were correlated by Dynatrace.

Issues and considerations

  • Needed more time to dig into the details, to cross data in order to find the true problem.

  • Consider the use of a lot of components.

  • Consider a team with different expertise to implement this architecture.