Page Comparison

Summary

The data migration process requires a methodological approach to address any errors generated by our validation tests once the data is transferred. The data migration process must be accompanied by a “Troubleshooting” process. The latter should provide appropriate, accurate and relevant solutions to help make the data migration a success.

Two types of errors are possible: a generic type error and a specific type error. Because of the content of errors generated by the CRM platform, it is natural to categorize them according to its two types.

To provide a better understanding of the content of these errors (generic and specific), tools are available (platform log, CRM trace log, server log, SQL query tracking, decompilation of .NET Framework CRM services). Thus, using these tools will generate a process (containing inclusive steps), creating inclusive levels of Troubleshooting. Once the errors are entered in a transparent manner, the solutions can be applied and thus put an end to the error generated by the platform.

Finally, the white paper shows how much CRM service contract extensions (“EntityServiceInternal<T>”) could play a key role in a problem-solving process.

Keywords

Dynamics 365, Data Migration, Dynamics CRM, Troubleshooting, CRM Trace, CRM Exceptions, Event Viewer, Decompiler, Dynamics Link Library (DLL), MS SQL Profiler, Stored Procedures, OrganizationServiceFault, Database, FaultException, C#, CRM SDK365, Microsoft.Crm.ObjectModel, Asynchronous CRM services, Synchronous CRM services.

Confidentiality

The white paper does not provide any confidential client data related to data migration. All the code in this white paper is only related to the CRM SDK365 code.

INTRODUCTION: How to resolve a post-data migration error in Dynamics 365?

Troubleshooting is a systematic approach to problem solving that could used to find and resolve problems related to asynchronous and synchronous CRM services. The first step is to collect information related to the problem, such as undesirable behavior, a lack of expected functionality, or a crash in functionality related to the triggering of services (synchronous and asynchronous) that rely on CRM service contracts (Create Request, Update Request, SetState Request, etc.), etc.

Context : data migration phases

The migration from the CRM 2011 database to the D365 database takes place in an "unrestricted" environment from which we had access to all services and components of the Dynamics 365 platform. Our migration process has been executed in 4 steps:

...

Finalization related to deploying CRM components using SDK

1. PROBLEM: Description of CRM Errors Post-Migration

Visualizing CRM Error or CRM Exception

When did a CRM error or exception occur during migration process?

Questions asked to troubleshoot CRM errors

• How? How do I understand CRM error or exception?

• How? How do I understand CRM error or exception and find the right solution?

What type of CRM error or exception occurs when migrating data?

Contents of CRM errors or exceptions

Depth of CRM errors or exceptions

Some examples

Error regarding entities « Business Unit » and « Team »

Unhandled Exception: System.ServiceModel.FaultException[[Microsoft.Xrm.Sdk.OrganizationServiceFault, Microsoft.Xrm.Sdk, Version=8.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35]]: 
Can't add members to the default business unit team. Detail: 
<OrganizationServiceFault xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/xrm/2011/Contracts">
  <ActivityId>8efcd1b6-5bf4-49f0-87cf-e42452c9fdc2</ActivityId>
  <ErrorCode>-2147187957</ErrorCode>
  <ErrorDetails xmlns: d2p1="http://schemas.datacontract.org/2004/07/System.Collections.Generic" />
  <Message>Can't add members to the default business unit team</Message>
  <Timestamp>2018-09-24T18:31:28.3317847Z</Timestamp>
  <ExceptionRetriable>false</ExceptionRetriable>
  <ExceptionSource i:nil="true" />
  <InnerFault i:nil="true" />
  <OriginalException i:nil="true" />
  <TraceText i:nil="true" />
</OrganizationServiceFault>[1]

Error regarding entities « Role » et « Privilege »

Unhandled Exception: System.ServiceModel.FaultException[[Microsoft.Xrm.Sdk.OrganizationServiceFault, Microsoft.Xrm.Sdk, Version=8.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35]]: System.Web.HttpUnhandledException: Microsoft Dynamics CRM has experienced an error. Reference number for administrators or support: #9DA12221Detail:
<OrganizationServiceFault xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/xrm/2011/Contracts">
  <ActivityId>bd4af73c-a11c-474d-8cff-a076416208aa</ActivityId>
  <ErrorCode>-2147220970</ErrorCode>
  <ErrorDetails xmlns:d2p1="http://schemas.datacontract.org/2004/07/System.Collections.Generic" />
  <Message>System.Web.HttpUnhandledException: Microsoft Dynamics CRM has experienced an error. Reference number for administrators or support: #9DA12221</Message>
  <Timestamp>2019-05-11T23:50:33.5128137Z</Timestamp>
  <ExceptionRetriable>false</ExceptionRetriable>
  <ExceptionSource i:nil="true" />
  <InnerFault>
    <ActivityId>bd4af73c-a11c-474d-8cff-a076416208aa</ActivityId>
    <ErrorCode>-2147220960</ErrorCode>
    <ErrorDetails xmlns:d3p1="http://schemas.datacontract.org/2004/07/System.Collections.Generic" />
    <Message>RoleService::VerifyCallerPrivileges failed. User: ac8026fc-16e8-e311-9247-005056b80603, PrivilegeName:   prv..., PrivilegeId: 2caa63a2-00d6-42dd-9262-058c6b61ee01, Depth: Global, BusinessUnitId: db002ef6-16e8-e311-9247-005056b80603</Message>
    <Timestamp>2019-05-11T23:50:33.5138137Z</Timestamp>
…
  </InnerFault>
  <OriginalException i:nil="true" />
  <TraceText i:nil="true" />
</OrganizationServiceFault>[2]

Error as generic error

Unhandled Exception: System.ServiceModel.FaultException[[Microsoft.Xrm.Sdk.OrganizationServiceFault, Microsoft.Xrm.Sdk, Version=8.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35]]: An unexpected error occurred. Detail:
<OrganizationServiceFault xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/xrm/2011/Contracts">
  <ActivityId>425234c8-811f-4484-a2ff-2f7a3bf98487</ActivityId>
  <ErrorCode>-2147220970</ErrorCode>
  <ErrorDetails xmlns:d2p1="http://schemas.datacontract.org/2004/07/System.Collections.Generic" />
  <Message>An unexpected error occurred</Message>
  <Timestamp>2018-09-24T18:15:17.2530019Z</Timestamp>
...
  <TraceText i:nil="true" />
</OrganizationServiceFault>[3]

2. Means or tools to address CRM errors or exceptions : Logs contents

Log generated by CRM platform

When the platform executes the user's request for an action through the interface using the service of the CRM (OrganizationService), that action gets an error or a CRM exception that appears in a "Pop-up" window. From this window, the user can download the error in question (see points on "Concrete examples").

...

Log generated by the server through "Tracing CRM" feature

In general, the "Tracing CRM"[4] function is not enabled. However, during a data migration, its activation becomes essential to understand and understand a CRM error or exception generated through the CRM interface. Once the error has occurred, it will have to be repeated once the "Tracing CRM" has been activated. In doing so, the log will be placed in the directory defined in the server registry keys.

...

Log generated in the server via "Event Viewer" feature

The function of Event Viewer in a server (installation of the platform in full server mode) could play a key role in the "Troubleshooting" in general. Therefore, it was clear that all applications and behaviors of the latter would be logged in the server and categorized by the system (error, warning, information).

In the D365 platform, all "warning" logs must be considered in connection with an exception or an error generated during a query via the CRM interface. In fact, this type of error ("warning") in the server could hide a CRM error.

...

"Tracking" tool of SQL queries generated through CRM platform

Because data migration is a database migration to a database in an "on-premise" context (on-site), it became undeniable that the SQL Server Profiler tool could play a crucial role in the "Troubleshooting" of our CRM errors because we knew that queries executed via the platform (CRM web services contracts) turned into SQL queries and used procedures stored or native views D365.

...

Tool to decompile synchronous and asynchronous CRM services (.NET libraries)

While there are recognized Dynamics CRM errors caused by asynchronous and synchronous service behaviors during a data migration, there have been no errors in this migration directly related to asynchronous CRM services. Errors appear at first glance as errors from the OrganizationService (CRM) synchronous services.

The free decompiling tool referred to is provided by Jet Brains dotPeek (.NET Decompiler). This allows to decompile the .NET libraries used by the D365 platform on both the client (synchronous services) and server (asynchronous service) sides.

...

3. Figuring out and resolving CRM errors

3 inclusive levels of "Troubleshooting"

CRM error handling and resolution process could be characterized by 3 levels of "Troubleshooting": extracting logs, tracking SQL Server queries, and finally, decompiling the D365 platform's native code source. These 3 levels of analysis should not be separated from each other but should instead be included in a comprehensive approach and considered complementary to each other.

...

« Troubleshooting » steps

Preliminary Step:

→ Code execution (SSIS package).

...

We know that level 1 of "troubleshooting" is not enough to make sense of the error and even more so to link to SQL queries executed or queries executed by .NET code (via SDK) during the transaction. That said, level 1 will at least be our starting point for the "troubleshooting" process. 2 scenarios are possible:

Scenario 1 - from Level 1 to Level 3: We can go from level 1 to level 3 directly.

• An unexpected error has occurred: It may be better to move immediately to level 3 by searching the services mentioned in the .NET library (Microsoft.Crm.ObjectModel) because the transaction executed through the CRM interface is related to the entity involved in the transaction at first sight. In addition, it would be better to "tracker" all SQL queries (views, stored procedures..." via the "SQL Profiler" tool. However, the error does not always allow us to focus our search on the CRM native stored procedures.

• Case of a specific error "Can't add members to the default business unit team": It is also preferable to take the error message (descriptive content) and search the .NET library (Microsoft.Crm.ObjectModel) and the native SQL queries of the CRM. We can access source code: search by keywords does not give anything and therefore, the focus should be on methods in services related to the following entities: Team ("TeamServiceInternal", Business Unit ("BusinessUnitServiceInternal") and System User ("SystemUserServiceInternal").

• Case of a specific error "RoleService::VerifyCallerPrivileges failed": It is also preferable to search the SDK's .NET library (Microsoft.Crm.ObjectModel) by keywords. Also, since the contents of the error refer to concepts such as "Role" and "Privilege", the latter can use the search for native SQL queries in the MSCRM database.

Scenario 2 - from level 1 to level 2, then level 2 to level 3: Level 2 is an intermediate passage

• Case of a generic error "An unexpected error has occurred" and Case of a specific error "Can't add members to the default business unit team" and "RoleService::VerifyCallerPrivileges failed": It may be better to move to level 2 in other words:

...

As a result, the further you go in the level, the more generic the error can become. It is from level 2 that the generic error becomes a specific error.

...

Error resolutions and CRM exceptions

Here, the resolution process answers the question: how can the errors or CRM exceptions mentioned above be resolved: "Can't add members to the default business unit team"; "RoleService::VerifyCallerPrivileges failed" and finally, "An unexpected error has occurred"?

How to fix the generic error: an unexpected error has occurred?

Step 1: The server log ("Event Viewer") looks at the request executed considering the exact timing of execution: the warning and error types are considered. In the CRM, the "warning" type is to be considered an error-type error.

...

The CRM's SDK contains an error list that the platform categorizes with an enum-type object (see next list). The generic error can be found in the "ErrorCode" class in the .NET bookstore "Microsoft.Crm.Constants":

ErrorCodes.ErrorMessages.Add((object) -2147220970, (object) -"An unexpected error occurred.");

ErrorCodes.ErrorTypes.Add((object)-2147220970,(object) ErrorCodes.ErrorType.SystemFailure);

In the case of migration, the CRM error will be conducive to being system-type more often even if we migrate data related to the CRM's basic security model.

Step 5: We try to cross the request launched with the decompiled code and always considering the object model that we test (entity and therefore, for example, "RoleServiceInternal" if our transaction concerns the CRM security model). It is enough to reach the method executed by the SDK of the CRM and thus analyze its content.

How to resolve generic error: an unexpected error has occurred? - Additional point

By default, all CRM transactions from the client (interface) use the .NET library "Microsoft.Crm.Core.WebServices",  more precisely, uses a base class of type "sealed" and therefore cannot be instantiated.

...

public static ExecuteTransactionFault ConvertToExecuteTransactionFault(BaseServiceFault serviceFault)
{
      ExecuteTransactionFault transactionFault = new ExecuteTransactionFault();
      transactionFault.ErrorCode = serviceFault.ErrorCode;
      transactionFault.ErrorDetails = serviceFault.ErrorDetails;
      transactionFault.ExtensionData = serviceFault.ExtensionData;
      transactionFault.InnerServiceFault = serviceFault.InnerServiceFault;
      transactionFault.Message = serviceFault.Message;
      transactionFault.Timestamp = serviceFault.Timestamp;
      transactionFault.ActivityId = serviceFault.ActivityId;
      OrganizationServiceFault organizationServiceFault = serviceFault as OrganizationServiceFault;
      if (organizationServiceFault != null)
      {
        transactionFault.InnerFault = organizationServiceFault.InnerFault;
        transactionFault.TraceText = organizationServiceFault.TraceText;
      }
      return transactionFault;
}

How to resolve the specific error: RoleService::VerifyCallerPrivileges failed?

The error is of type CrmException not defined in the "ErrorCode" class of the .NET library "Microsoft.Crm.Constants", but defined in the .NET library "Microsoft.Crm.ObjectModel":

...

private static void ThrowPrivilegeNotFoundException(Guid userId, RolePrivilege rolePrivilege, ExecutionContext context)
{
      Microsoft.Crm.Metadata.Privilege privilege = MetadataCache.GetInstance((IOrganizationContext)    context).Privileges[(object) rolePrivilege.PrivilegeId];
      throw new CrmSecurityException(string.Format((IFormatProvider) CultureInfo.InvariantCulture, "RoleService::VerifyCallerPrivileges failed. User: {0}, PrivilegeName: {1}, PrivilegeId: {2}, Depth: {3}, BusinessUnitId: {4}", (object) userId, privilege != null ? (object) privilege.Name : (object) "Privilege not found in cache", (object) rolePrivilege.PrivilegeId, (object) rolePrivilege.Depth, (object) rolePrivilege.BusinessUnitId), -2147220960);
}

How to resolve the specific error: Can't add members to the default business unit team?

The error is of type CrmException and type "SystemFailure":

ErrorCodes.ErrorMessages.Add((object) - 2147187957, (object) "Can't add members to the default business unit team");
ErrorCodes.ErrorTypes.Add((object) - 2147187957, (object) ErrorCodes.ErrorType.SystemFailure);

Step 1: Extract the method associated with the error on transaction from the "CRM Trace": "CheckPrivilegesAndAccess". Retrieving should allow us to determine the service generates the error, in this case "TeamServiceInternal".

...

private bool CheckPrivilegesAndAccess(Guid teamId, int errorCode, bool allowDefaultBuTeam, ExecutionContext context)
{
      bool flag = this.AllowDefaultBuTeamMembership(allowDefaultBuTeam, context);
      if (!this.Provisioning && !context.Caller.Equals(SecurityLibrary.GetOrganizationSystemUser(context.OrganizationId, (IOrganizationContext) context)))
      {
        if (!flag && this.IsDefaultTeam(teamId, context))
          throw new CrmException(errorCode, new object[0]);
        this.CheckPrivilegeAndAccess(teamId, PrivilegeType.Write, AccessRights.WriteAccess, context);
        this.ValidateCallerPrivileges(teamId, context);
        this.ValidateCallerAttributePrivileges(teamId, context);
      }
      return flag;
}
private void CheckPrivilegeAndAccess(Guid teamId, PrivilegeType privType, AccessRights accessrights, ExecutionContext context)
{
      EntityMetadata entity = MetadataCache.GetInstance((IOrganizationContext) context).GetEntity(this.PlatformName);
      SecurityLibrary.CheckPrivilege(context.Caller, entity.GetPrivilege(privType).Id, context);
      BusinessManagementUtility.CheckAccess(new BusinessEntityMoniker(teamId, this.PlatformName, (IOrganizationContext) context), accessrights, context);
}

CONCLUSION: GENERIC AND SPECIFIC CRM ERROR

"Troubleshooter" a CRM error during a data migration is mainly related to the D365 security model.

...

Finally, successful data migration requires a correction of data related to the CRM security model: System User, Team, Business Unit, Role, and Privilege

The extension of service contracts related to CRM entities: "EntityServiceInternal<T>".

Regardless of whether an error is generic or specific and regardless of which entity the exception generated by the CRM platform is concerned, the dynamic library (DLL) "Microsoft.Crm.ObjectModel" plays a key role in the "troubleshooting" process.

...

The extension of the service contracts of the CRM "EntityServiceInternal<T>" generates various types of exceptions: "CRM Exception", "DataServiceRequestException", "Exception" to generate specific exceptions ("CrmArgumentException", "CrmArgumentNullException"...).
Typically, the error or exception generated by the CRM platform contains an error code that can be traced into the "ErrorCodes" class that can be retrieved from the .NET (DLL) "Microsoft.Crm" 

...