Posts Tagged With MS Dynamics - Musing, Rants & Jumbled Thoughts

Header Photo Credit: Lorenzo Cafaro (Creative Commons Zero License)

I put this together last year while looking into ways to improve the amount of time it took to download an approximation 45MB payload SOAP response from a Microsoft Dynamics CRM service (ie: a giant XML document).

In the process, I found surprising results at how much better the user experience was when you combine GZIP compression with SSL encryption.

While the below write-up is specific to CRM in IIS, it should apply much more generally. I hope you find this helpful.

Summary / Real-World Proof

I implemented the below described ssl + gzip IIS configuration to speed up the metadata downloads on an internal server. While I saw no measurable difference in download times from my dev machine on the local network (which downloaded the metadata at approx. 25 seconds), my coworker, who is connecting over a VPN from two timezones away, saw download times go from approx. 5 mins before the change to approx. 30 secs after.

This can be contributed exclusively to the ssl + gzip config change, as we were already running just gzip on the one of our servers and just ssl on another, which were both taking the full amount of time to download metadata. It was only once I enabled gzip and ssl that the times dropped so significantly. Ultimately, this is due to the drastically reduced payload size (data going across the wire) when you combine those two technologies.

Overview

After some investigation on how to improve the download times for the CRM metadata, I think on of our best options is to suggest users enable dynamic compression for SOAP data, and utilize SSL. This will significantly reduce the payload size going across the network by ~96%, which represents the overwhelming majority of the user's wait time.

Findings

Out-of-the-box, Dynamics CRM will enable the dynamic (GZIP) compression setting for the web interfaces (including WCF services), but IIS7’s default configuration does not consider SOAP to be compressible. You must manually add SOAP to the list of dynamicTypes, which is a host-wide config change. Further, enabling SSL with compression significantly reduces the payload size.

Estimated download payloads and timings:^

  • Default install (IIS7, no dynamic compression, no SSL): 44.5 MB = 8min
  • With GZIP compression for SOAP: 33 MB = 6 mins
  • With SSL only: 33 MB = 6 min
  • With GZIP and SSL: 1.5 MB = 17sec

^Times are best-case, assuming you’re using a network connection with 768Kbps (.09MBps) download speed, the average DSL speed in America. Actual times will likely be slower.

That’s not a typo – enabling both SSL and GZIP took the time down to 17 seconds, or ~3.5% of the original time.

How To:

Step 1: Enable dynamic compress for soap data in the IIS applicationHost.conf

Enable compression by manually updating the ApplicationHost.Config

  • On the CRM Server Navigate to: C:\\Windows\\System32\\Inetsrv\\Config\\applicationHost.config and open it with notepad.
  • Search for the Section: <dynamicTypes> and in that section you should fine an entry that looks like this: <add mimeType="application/x-javascript" enabled="true" />
  • Below that, add the following line: <add mimeType="application/soap+xml; charset=utf-8" enabled="true" />
  • Save the file and reset IIS for the setting to take effect.

Step 2: Ensure dynamic compression is enabled for the Dynamics service:

Note: This should already be enabled in the default configs, but may have been changed by sysadmin

In IIS Manager, open the compression settings for the host:

Ensure dynamic compression is checked.

Open the Dynamics site compression settings:

Ensure dynamic compression is enabled:

Step 3: Enable SSL using a self-signed cert

Follow these instructions to enabled SSL with a self-signed cert.

Step 4: Export the cert and install on desktop

The CRM SDK won't connect to a site with certificate errors, so if using an untrusted (self-signed) cert, you'll need to add it to the desktop's trusted certs.

In IIS Manager, from the Server Certifications page, click Export, select a location to save the file and enter a password.

Copy that file to your desktop machine and double-click the file, which should open the certificate import wizard.

Select Current User (or, to make the cert apply to all users on the machine, select Local Machine) and complete the wizard, using the same password when prompted as you entered on the server during export.

When prompted for which certificate store to use, select "Place all certificates in the following store" and browse to the "Trusted Root Certificate Authorities". Finish the wizard and agree to all of the security warnings (there may be several).

You may need to restart your desktop machine for the certificate settings to take affect.

References:



In this post, I describe how to create class instances and access/set members from other assemblies when their access levels are set to internal or private.

When unit testing, you often need to generate instances of your domain objects within your tests.  Hopefully, these object implements an interface you can mock with Rhino.Mocks (or your preferred mocking tool), or if not, provide virtual members that can be overridden in a derived class by a mocking tool.  But sometimes, you run into a nasty class that makes your life difficult.  I ran into this recently with the MS Dynamics 2011 SDK, where the Entity class, which drives most of the domain objects in the SDK, does not provide an interface, has internal-only setters on it's properties and has an internal-only constructor. And since Rhino.Mocks can't/won't mock non-virtual members, setting values to what you need for your tests is impossible without some work.

Luckily, you can get around internal and private access restrictions using reflection.

Note: I'll post the full set of classes, including some unit tests, at the end of this post, so in my examples here, I'm not going to include the using statements, etc.

Update 5/2/2012: Added Non-Public Fields section; Added base class recursive lookup for private fields/properties.

Non-Public Constructors:

So, let's start with the constructor, since you'll need an object to work with. The issues to deal with here is finding the right constructor based on the parameter list, so I'll use the Type.GetConstructor method.

public static T CreateInstance<T>(params object[] args)
{
    Type typeToCreate = typeof(T);

    Type[] parameterTypes = args.Select(arg => arg.GetType()).ToArray();

    // Use reflection to get the ConstructorInfo object that matches our parameters
    // even if it's not public.
    ConstructorInfo constructorInfoObj = typeToCreate.GetConstructor(
            BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic, 
            null, parameterTypes, null);

    return (T) constructorInfoObj.Invoke(args);
}

To use it, you need only make a call like this:

var entity = MockingHelper.CreateInstance<ClassWithPrivateConstructors>(param1, param2);

Non-Public Properties:

Ok, now you want to set the value of a property where the setter is non-public.  For this, I have a helper method that takes in the name of the property you want to set as a string, however I highly suggest you use the technique I suggested in my previous post for using reflection to get that string, thus limiting your risk of renaming the actual property and breaking your unit tests. This won't work, however, if the member you're trying to reach isn't visible to you, so you may have to resort to string-based names. (see unit tests at end of this post for full examples)

Note that if a member is private and is defined in a base class (ie: not the Type you're actually working with, but one of it's parent types), then you have to recursively walk the type hierarchy until you find what you're looking for.

public static void SetPropertyValue(object target, string memberName, object newValue)
{
    PropertyInfo prop = GetPropertyReference(target.GetType(), memberName);
    prop.SetValue(target, newValue, null);
}

private static PropertyInfo GetPropertyReference(Type targetType, string memberName)
{
    PropertyInfo propInfo = targetType.GetProperty(memberName,
                                          BindingFlags.Public |
                                          BindingFlags.NonPublic |
                                          BindingFlags.Instance);

    if (propInfo == null && targetType.BaseType != null)
    {
        //if the member isn't actually on the type we're working on, rather it's
        //defined in a base class as private, it won't be returned in the above call,
        //so we have to walk the type hierarchy until we find it.
        // See: http://agsmith.wordpress.com/2007/12/13/where-are-my-fields/

        return GetPropertyReference(targetType.BaseType, memberName);

    }
    return propInfo;
}

public static string GetPropertyName<T>(Expression<Func<T>> property)
{
    LambdaExpression lambdaExpression = (LambdaExpression)property;
    var memberExpression = lambdaExpression.Body as MemberExpression ?? 
        ((UnaryExpression)lambdaExpression.Body).Operand as MemberExpression;         
    return memberExpression.Member.Name;       
}

To use it, you need only make a call like this:

var entity = new ClassWithPrivatePropertySetter();
var memberName = MockingHelper.GetPropertyName(() => entity.MyString);
MockingHelper.SetPropertyValue(entity, memberName, "New Value");

Non-Public Fields:

Similar to properties, but a different call off Type:

public static void SetFieldValue(object target, string fieldName, object newValue)
{           
    FieldInfo field = GetFieldReference(target.GetType(), fieldName);
    field.SetValue(target, newValue);
}

private static FieldInfo GetFieldReference(Type targetType, string fieldName)
{
    FieldInfo field = targetType.GetField(fieldName,
                                          BindingFlags.Public |
                                          BindingFlags.NonPublic |
                                          BindingFlags.Instance);

    if (field == null && targetType.BaseType != null)
    {
        //if the field isn't actually on the type we're working on, rather it's
        //defined in a base class as private, it won't be returned in the above call,
        //so we have to walk the type hierarchy until we find it.
        // See: http://agsmith.wordpress.com/2007/12/13/where-are-my-fields/

        return GetFieldReference(targetType.BaseType, fieldName);

    }
    return field;
}

To use it, you need only make a call like this:

var entity = new ClassWithPrivatePropertySetter();
MockingHelper.SetFieldValue(entity, "_myStringField", "New Value");

Putting It All Together:

This is all good and whatnot, but to really make this clean, I would suggest creating some extension classes, or at least a helper class that will make these easier to call for your domain object, if you're going to do any really extensive testing.  For example, I created these extension methods to allow me to set the internal-only Attributes property on the Dynamics EntityMetadata object. These are pretty simple, but I have others that create the entity (which has an internal-only constructor) and set several properties all in one swoop.

public static void SetAttributesViaReflection( this EntityMetadata entityMetadata,
                                                       AttributeMetadata [] attributes)
{
    SetPropertyValue(entityMetadata, 
                     GetPropertyName(() => entityMetadata.Attributes ),
                     attributes);
}

public static void SetOneToManyRelationshipsViaReflection( 
                               this EntityMetadata entityMetadata, 
                               OneToManyRelationshipMetadata [] relationships)
{
    SetPropertyValue(entityMetadata, 
                     GetPropertyName(() => entityMetadata.OneToManyRelationships ),
                     relationships);
}

Good Luck and happy reflecting!

The Full Source Code:

using System;
using System.Linq;
using System.Linq.Expressions;
using System.Reflection;

namespace jwright.blog
{
    internal static class MockingHelper
    {

        public static T CreateInstance<T>(params object[] args)
        {
            Type typeToCreate = typeof(T);

            Type[] parameterTypes = args.Select(arg => arg.GetType()).ToArray();

            // Use reflection to get the ConstructorInfo object that matches our parameters
            // even if it's not public.
            ConstructorInfo constructorInfoObj = typeToCreate.GetConstructor(
                BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic, 
                null, parameterTypes, null);

            return (T) constructorInfoObj.Invoke(args);
        }
  
  

        public static void SetPropertyValue(object target, string memberName, object newValue)
        {
            PropertyInfo prop = GetPropertyReference(target.GetType(), memberName); 
            prop.SetValue(target, newValue, null);
        }

        private static PropertyInfo GetPropertyReference(Type targetType, string memberName)
        {
            PropertyInfo propInfo = targetType.GetProperty(memberName,
                                                  BindingFlags.Public |
                                                  BindingFlags.NonPublic |
                                                  BindingFlags.Instance);

            if (propInfo == null && targetType.BaseType != null)
            {
                //if the member isn't actually on the type we're working on, rather it's
                //defined in a base class as private, it won't be returned in the above call,
                //so we have to walk the type hierarchy until we find it.
                // See: http://agsmith.wordpress.com/2007/12/13/where-are-my-fields/

                return GetPropertyReference(targetType.BaseType, memberName);
            }
            return propInfo;
        }

        public static void SetFieldValue(object target, string fieldName, object newValue)
        {
            FieldInfo field = GetFieldReference(target.GetType(), fieldName);
            field.SetValue(target, newValue);
        }

        private static FieldInfo GetFieldReference(Type targetType, string fieldName)
        {
            FieldInfo field = targetType.GetField(fieldName,
                                                  BindingFlags.Public |
                                                  BindingFlags.NonPublic |
                                                  BindingFlags.Instance);

            if (field == null && targetType.BaseType != null)
            {
                //if the field isn't actually on the type we're working on, rather it's
                //defined in a base class as private, it won't be returned in the above call,
                //so we have to walk the type hierarchy until we find it.
                // See: http://agsmith.wordpress.com/2007/12/13/where-are-my-fields/

                return GetFieldReference(targetType.BaseType, fieldName);
            }
            return field;
        }


        public static string GetPropertyName<T>(Expression<Func<T>> property)
        {
            LambdaExpression lambdaExpression = (LambdaExpression)property;
            var memberExpression = lambdaExpression.Body as MemberExpression ?? 
                ((UnaryExpression)lambdaExpression.Body).Operand as MemberExpression;         
            return memberExpression.Member.Name;       
        }

    }
}

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace jwright.blog
{

    public class ClassWithPrivateConstructors : BaseClassWithPrivateMembers
    {
        private ClassWithPrivateConstructors()
        {
            MyString = "parameterless";
        }

        private ClassWithPrivateConstructors(string stringToUse)
        {
            MyString = stringToUse;
        }

        public string MyString { get; set; }
    }

    public class ClassWithPrivatePropertySetter : BaseClassWithPrivateMembers
    {
        public ClassWithPrivatePropertySetter()
        {
            MyStringProperty = "parameterless";
        }

        public string MyStringProperty { get; private set; }
           
        private string _myStringField;
        public string MyStringFieldWrapper { get { return _myStringField; } }
    }

    public abstract class BaseClassWithPrivateMembers
    {
        private string _privateStringField;
        public string PrivateStringFieldWrapper { get { return _privateStringField; } }
           
        private String PrivateStringProperty { get; set; }
        public string PrivateStringPropertydWrapper { get { return PrivateStringProperty; } } 
    }

}
  
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using NUnit.Framework;

namespace jwright.blog
{
    [TestFixture]
    public class UnitTests
    {
        [Test]
        public void VerifyPropertyNamesCorrectlyDetermined()
        {
            var entity = new ClassWithPrivatePropertySetter();
            var memberName = MockingHelper.GetPropertyName(() => entity.MyString);
            Assert.That(memberName, Is.EqualTo("MyString"));
        }

        [Test]
        public void VerifyPropertyValueSet()
        {
            var entity = new ClassWithPrivatePropertySetter();
            var memberName = MockingHelper.GetPropertyName(() => entity.MyString);

            var newValue = "New Value";

            MockingHelper.SetPropertyValue(entity, memberName, newValue);

            Assert.That(entity.MyString, Is.EqualTo(newValue));
        }


        [Test]
        public void VerifyPrivateBaseClassPropertyValueSet()
        {
            var entity = new ClassWithPrivatePropertySetter();
            var memberName = "PrivateStringProperty";

            var newValue = "New Value";

            MockingHelper.SetPropertyValue(entity, memberName, newValue);

            Assert.That(entity.PrivateStringPropertydWrapper, Is.EqualTo(newValue));
        }
  
        [Test]
        public void VerifyFieldValueSet()
        {
            var entity = new ClassWithPrivatePropertySetter();
            var memberName = "_myStringField";

            var newValue = "New Value";

            MockingHelper.SetFieldValue(entity, memberName, newValue);

            Assert.That(entity.MyStringFieldWrapper, Is.EqualTo(newValue));
        }

        [Test]
        public void VerifyPrivateBaseClassFieldValueSet()
        {
            var entity = new ClassWithPrivatePropertySetter();
            var memberName = "_privateStringField";

            var newValue = "New Value";

            MockingHelper.SetFieldValue(entity, memberName, newValue);

            Assert.That(entity.PrivateStringFieldWrapper, Is.EqualTo(newValue));
        }

        [Test]
        public void CreateInstanceTest_NoParams()
        {
            var entity = MockingHelper.CreateInstance<ClassWithPrivateConstructors>();

            Assert.That(entity.MyString, Is.EqualTo("parameterless"));
        }

        [Test]
        public void CreateInstanceTest_WithParam()
        {
            var testString = "a different value";
            var entity = MockingHelper
                             .CreateInstance<ClassWithPrivateConstructors>(testString);

            Assert.That(entity.MyString, Is.EqualTo(testString));
        }
    }
}


Lately, I've been doing work with Microsoft's Dynamics CRM 2011. Specifically, I've been integrating InRule's flagship product into Dynamics, which includes utilizing the Plugin feature of Dynamics.  Dynamic provides the option of registering a plugin to run in "sandbox" mode, which is a Partial Trust process.  While the on-premise Dynamics software will allow you to run plugins in or out of sandbox mode, the Dynamics Online (Microsoft's hosted solution) will only run in sandbox mode.

Unfortunately, there's not really any good documentation saying what is or is not allowed in the sandbox mode.  Even when I spoke to a group of Microsoft Dynamics team members and MVPs at the Dynamics Acceleration Lab earlier this month on Microsoft's Redmond campus, there wasn't a known set.

So, I basically had to take the approach of just trying to run the software in isolated mode and see what fails.  As I uncovered a new security failure, I'd create a whitebox plugin that would test the specific scenario to ensure it was indeed a sandbox-induced issue and to test any potential workarounds.  The good news is that the sandbox mode for on-premise installations is the same as the Dynamics OnLine environment, so I could test things locally.

Below is a list of items I discovered during my testing.

This is not an exhaustive list by any means, as I focused only on functionality I needed for the InRule integration. I will continue to update it as I come across additional items.

Exception classes: With .Net 4, there was a change to how you must construct your Exception classes if you want to include any custom data when serializing your object.  Previously, you would override the GetObjectData() method, but with .Net 4, this has been slapped with the [SecurityCritical] attribute, which stops you from using it in partial trust environments.  Instead, you would need to change the implementation per this link: http://msdn.microsoft.com/en-us/library/system.runtime.serialization.isafeserializationdata.aspx

Using any of the following cause a security exception (not an exhaustive list):

  • Attempting to use the AppDomain.CurrentDomain.AssemblyResolve event
  • System.IO.Path.GetTempPath() [System.Security.Permissions.EnvironmentPermissionException]
  • Any filesystem access code [System.Security.Permissions.FileIOPermissionException]
  • Attempting to use the EventLog [System.Diagnostics.EventLogPermissionException]
  • Attempting to use IsolatedStorage [System.Security.Permissions.IsolatedStoragePermissionException]
  • Any references to Thread.CurrentThread caused a security failure. 

Usage of the XmlReader class using a StringReader to provide the XML, caused a security failure.  I haven't yet been able to narrow down the specifics as to where or why this was disallowed, but here is the code that fails:

using(XmlReader reader = XmlReader.Create( new StringReader (_xmlString ))
{
    reader.MoveToContent(); //Security exception occurs here.

As I determine more, I'll update this post.

Note: My testing was with Dynamics CRM 2011 Rollup 7 running on a Windows 2008 server virtual machine inside VMWare Player.

Update: I came across this MSDN article Plug-in Isolation, Trusts, and Statistics which provides some info on restrictions related to opening networking connections, including a registry edit you can make to ease the restrictions. Noteworthy excerpt from the article:

Sandboxed plug-ins and custom workflow activities can access the network through the HTTP and HTTPS protocols. This capability provides 
support for accessing popular web resources like social sites, news feeds, web services, and more. The following web access restrictions
apply to this sandbox capability.

* Only the HTTP and HTTPS protocols are allowed.
* Access to localhost (loopback) is not permitted.
* IP addresses cannot be used. You must use a named web address that requires DNS name resolution.
* Anonymous authentication is supported and recommended. There is no provision for prompting the 
  on user for credentials or saving those credentials.