Custom fields support with SqlMetadataStore in the sync framework

I am putting together a sync framework provider and have come across some issues that do not appear to be documented. I’m adding custom fields to the item metadata so that I can store additional information for items that are synchronised with another provider. To do this, you define the custom fields for items as part of initializing the metadata store. The tricky thing is that it isn’t clear what data types are supported and how to correctly use the data types that are supported.

I have the following initialization code.

List<FieldSchema> customFields = new List<FieldSchema>
                                     {
                                         new FieldSchema("IsContainer", typeof(Boolean)),
                                         new FieldSchema("Name", typeof(String)),
                                         new FieldSchema("Path", typeof(String))
                                     };
String[] indexFieldNames = new[]
                               {
                                   "IsContainer", "Name", "Path"
                               };
List<IndexSchema> indexFields = new List<IndexSchema>
                                    {
                                        new IndexSchema(indexFieldNames, true)
                                    };

Metadata = MetaStore.InitializeReplicaMetadata(IdFormats, ReplicaId, customFields, indexFields);

This code throws the following exception:

Microsoft.Synchronization.MetadataStorage.MetadataStoreInvalidOperationException: Incorrect constructor used

It turns out that defining a FieldSchema as a string requires a defined length, hence the incorrect constructor has been used as there is another constructor that takes the size in addition to the name and type used here. This is not a good exception message as it does not adequately describe why there is a problem or how to fix it. The remarks documentation for the name and type constructor overload is also incorrect as it says the following.

This form of the constructor sets MaxLength to the size of dataType, in bytes.

There must be something wrong with the implementation of this constructor as this does not appear to be the case. If the length is provided to the constructor overload then no exception is thrown when constructing FieldSchema instances.

The code now looks like the following:

List<FieldSchema> customFields = new List<FieldSchema>
                                     {
                                         new FieldSchema("IsContainer", typeof(Boolean)),
                                         new FieldSchema("Name", typeof(String), 256),
                                         new FieldSchema("Path", typeof(String), 256)
                                     };
String[] indexFieldNames = new[]
                               {
                                   "IsContainer", "Name", "Path"
                               };
List<IndexSchema> indexFields = new List<IndexSchema>
                                    {
                                        new IndexSchema(indexFieldNames, true)
                                    };

Metadata = MetaStore.InitializeReplicaMetadata(IdFormats, ReplicaId, customFields, indexFields);

This now throws an exception saying:

System.NotSupportedException: The data type is not supported.

With a stack trace that looks like the following:

Microsoft.Synchronization.MetadataStorage.Utility.ConvertDataTypeToSyncMetadataFieldType(Type dataType)
Microsoft.Synchronization.MetadataStorage.SqlMetadataStore.CreateCustomFieldDefinition(FieldSchema fieldSchema, String parameterName)
Microsoft.Synchronization.MetadataStorage.SqlMetadataStore.InitializeReplicaMetadata(SyncIdFormatGroup idFormats, SyncId replicaId, IEnumerable`1 customItemFieldSchemas, IEnumerable`1 customIndexedFieldSchemas)
Neovolve.Jabiru.Client.Synchronization.Remote.StoreSyncProvider.LoadMetadataStore() in R:\Codeplex\Jabiru\Jabiru\Neovolve.Jabiru.Client\Neovolve.Jabiru.Client.Synchronization.Remote\StoreSyncProvider.cs: line 668
Neovolve.Jabiru.Client.Synchronization.Remote.StoreSyncProvider..ctor(IStoreProcessor processor) in R:\Codeplex\Jabiru\Jabiru\Neovolve.Jabiru.Client\Neovolve.Jabiru.Client.Synchronization.Remote\StoreSyncProvider.cs: line 64
Neovolve.Jabiru.Client.Synchronization.Remote.UnitTests.ServiceSyncProviderTests.RunSync(String localPath, IStoreProcessor processor, Guid storeGuid) in R:\Codeplex\Jabiru\Jabiru\Neovolve.Jabiru.Client\Neovolve.Jabiru.Client.Synchronization.Remote.UnitTests\ServiceSyncProviderTests.cs: line 160
Neovolve.Jabiru.Client.Synchronization.Remote.UnitTests.ServiceSyncProviderTests.SyncDeletesItemFromProcessorWhenFileIsDeletedTest() in R:\Codeplex\Jabiru\Jabiru\Neovolve.Jabiru.Client\Neovolve.Jabiru.Client.Synchronization.Remote.UnitTests\ServiceSyncProviderTests.cs: line 116

If you follow down the rabbit hole using Reflector, Microsoft.Synchronization.MetadataStorage.Utility.ConvertDataTypeToSyncMetadataFieldType has the following code:

internal static SYNC_METADATA_FIELD_TYPE ConvertDataTypeToSyncMetadataFieldType(Type dataType)
{
    if (dataType == typeof(byte[]))
    {
        return SYNC_METADATA_FIELD_TYPE.SYNC_METADATA_FIELD_TYPE_BYTEARRAY;
    }
    if (dataType == typeof(string))
    {
        return SYNC_METADATA_FIELD_TYPE.SYNC_METADATA_FIELD_TYPE_STRING;
    }
    if (dataType == typeof(byte))
    {
        return SYNC_METADATA_FIELD_TYPE.SYNC_METADATA_FIELD_TYPE_UINT8;
    }
    if (dataType == typeof(ushort))
    {
        return SYNC_METADATA_FIELD_TYPE.SYNC_METADATA_FIELD_TYPE_UINT16;
    }
    if (dataType == typeof(uint))
    {
        return SYNC_METADATA_FIELD_TYPE.SYNC_METADATA_FIELD_TYPE_UINT32;
    }
    if (dataType == typeof(ulong))
    {
        return SYNC_METADATA_FIELD_TYPE.SYNC_METADATA_FIELD_TYPE_UINT64;
    }
    if (dataType != typeof(Guid))
    {
        throw new NotSupportedException(StringResources.NotSupportedDataType);
    }
    return SYNC_METADATA_FIELD_TYPE.SYNC_METADATA_FIELD_TYPE_GUID;
}

The answer to this issue is that the Boolean data type is not supported. Unfortunately the documentation for InitializeReplicaMetadata does not to mention this. The way around this is to use a byte value to represent the Boolean state.

Code that successfully executes now looks like this:

List<FieldSchema> customFields = new List<FieldSchema>
                                     {
                                         new FieldSchema("IsContainer", typeof(Byte)),
                                         new FieldSchema("Name", typeof(String), 256),
                                         new FieldSchema("Path", typeof(String), 256)
                                     };
String[] indexFieldNames = new[]
                               {
                                   "IsContainer", "Name", "Path"
                               };
List<IndexSchema> indexFields = new List<IndexSchema>
                                    {
                                        new IndexSchema(indexFieldNames, true)
                                    };

Metadata = MetaStore.InitializeReplicaMetadata(IdFormats, ReplicaId, customFields, indexFields);

Transport connection closed exception with nDumbster

I have been using nDumbster on a project to unit test sending emails. It has been an interesting experience working with this little tool. It is a great product but has been out of development for a while and has some issues.

The more I started using nDumbster in test runs, the more I was finding that it wasn’t working so well. I was consistently getting the following exception:

Test method [TestName] threw exception:  System.Net.Mail.SmtpException: Failure sending mail. --->  System.IO.IOException: Unable to read data from the transport connection: net_io_connectionclosed..

System.Net.Mail.SmtpReplyReaderFactory.ProcessRead(Byte[] buffer, Int32 offset, Int32 read, Boolean readLine)

System.Net.Mail.SmtpReplyReaderFactory.ReadLines(SmtpReplyReader caller, Boolean oneLine)

System.Net.Mail.SmtpReplyReaderFactory.ReadLine(SmtpReplyReader caller)

System.Net.Mail.CheckCommand.Send(SmtpConnection conn, String& response)

System.Net.Mail.MailCommand.Send(SmtpConnection conn, Byte[] command, String from)

System.Net.Mail.SmtpTransport.SendMail(MailAddress sender, MailAddressCollection recipients, String deliveryNotify, SmtpFailedRecipientException& exception)

System.Net.Mail.SmtpClient.Send(MailMessage message)

System.Net.Mail.SmtpClient.Send(MailMessage message)

I worked on this one for a while. Without wanting to get my fingers into the nDumbster code, I found a solution that works from within the unit testing code itself. The solution is to create a new instance of the nDumbster SMTP server using a unique port number for each test. This ensures that there is a fresh connection for each test in the test run.

The unit test code looks like the following.

/// <summary>
/// Stores the random generator.
/// </summary>
private static readonly Random RandomPort = new Random(Environment.TickCount);

/// <summary>
/// Stores the SMTP server.
/// </summary>
private static SimpleSmtpServer _server;

#region Setup/Teardown

/// <summary>
/// Cleans up after running a unit test.
/// </summary>
[TestCleanup]
public void TestCleanup()
{
    _server.Stop();
    _server = null;
}

/// <summary>
/// Initializes the test.
/// </summary>
[TestInitialize]
public void TestInitialize()
{
    // HACK: The test must recreate the nDumbster server for each test using a different port number
    // This gets around issues where some tests consistently failed when the whole test class was run because the server
    // was not handling the connection properly. This appears to be the only solution that works.
    Int32 nextPortNumber = RandomPort.Next(1024, 6666);

    _server = SimpleSmtpServer.Start(nextPortNumber);
}

#endregion

Mocking IUnityContainer and avoiding BadImageFormatException

There is an issue with mocking IUnityContainer from RhinoMocks. A BadImageFormatException is thrown when you attempt to create a mock or a stub of IUnityContainer. For example:

using Microsoft.Practices.Unity;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Rhino.Mocks;

namespace TestProject1
{
    public interface ITestInterface
    {
        void DoSomething();
    }
 
    [TestClass]
    public class UnitTest1
    {
        [TestMethod]
        public void MockingIUnityContainerFailsTest()
        {
            // This throws a BadImageFormatException
            IUnityContainer container = MockRepository.GenerateStub<IUnityContainer>();
            ITestInterface test = MockRepository.GenerateStub<ITestInterface>();

            container.Stub(x => x.Resolve<ITestInterface>()).Return(test);

            ITestInterface resolvedTest = container.Resolve<ITestInterface>();

            resolvedTest.DoSomething();
        }
    }
}

This test results in the following failure:

Test method TestProject1.UnitTest1.MockingIUnityContainerFailsTest threw exception: System.BadImageFormatException: An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B).

System.Reflection.Emit.TypeBuilder._TermCreateClass(Int32 handle, Module module)
System.Reflection.Emit.TypeBuilder.CreateTypeNoLock()
System.Reflection.Emit.TypeBuilder.CreateType()
Castle.DynamicProxy.Generators.Emitters.AbstractTypeEmitter.BuildType()
Castle.DynamicProxy.Generators.Emitters.AbstractTypeEmitter.BuildType()
Castle.DynamicProxy.Generators.InterfaceProxyWithTargetGenerator.GenerateCode(Type proxyTargetType, Type[] interfaces, ProxyGenerationOptions options)
Castle.DynamicProxy.DefaultProxyBuilder.CreateInterfaceProxyTypeWithoutTarget(Type theInterface, Type[] interfaces, ProxyGenerationOptions options)
Castle.DynamicProxy.ProxyGenerator.CreateInterfaceProxyTypeWithoutTarget(Type theInterface, Type[] interfaces, ProxyGenerationOptions options)
Castle.DynamicProxy.ProxyGenerator.CreateInterfaceProxyWithoutTarget(Type theInterface, Type[] interfaces, ProxyGenerationOptions options, IInterceptor[] interceptors)
Castle.DynamicProxy.ProxyGenerator.CreateInterfaceProxyWithoutTarget(Type theInterface, Type[] interfaces, IInterceptor[] interceptors)
Rhino.Mocks.MockRepository.MockInterface(CreateMockState mockStateFactory, Type type, Type[] extras)
Rhino.Mocks.MockRepository.CreateMockObject(Type type, CreateMockState factory, Type[] extras, Object[] argumentsForConstructor)
Rhino.Mocks.MockRepository.Stub(Type type, Object[] argumentsForConstructor)
Rhino.Mocks.MockRepository.GenerateStub(Type type, Object[] argumentsForConstructor)
Rhino.Mocks.MockRepository.GenerateStub[T](Object[] argumentsForConstructor)
TestProject1.UnitTest1.MockingIUnityContainerFailsTest() in C:\Users\rprimrose\Documents\Visual Studio 2008\Projects\TestProject1\TestProject1\UnitTest1.cs: line 19

You can see in the stack trace that this is not actually an issue with RhinoMocks but an issue with Castle. As Moq also uses Castle internally it suffers from the same problem. Many people have raised this as an issue and asked for a solution. There does not seem to be an adequate answer out there other than a solution from Roy Osherove which is a bit heavy handed for what I want.

The answer is actually really simple. Reflector indicates the inheritance hierarchy of IUnityContainer as the following.

You can use either of these types to get the expected outcome and avoiding the BadImageFormatException. This exception seems to be an issue with mocking/stubbing the interface rather than its implemented types.

For example, both of the following tests pass.

[TestMethod]
public void MockingUnityContainerBaseSucceedsTest()
{
    IUnityContainer container = MockRepository.GenerateStub<UnityContainerBase>();
    ITestInterface test = MockRepository.GenerateStub<ITestInterface>();

    container.Stub(x => x.Resolve<ITestInterface>()).Return(test);

    ITestInterface resolvedTest = container.Resolve<ITestInterface>();

    resolvedTest.DoSomething();
}

[TestMethod]
public void MockingUnityContainerSucceedsTest()
{
    IUnityContainer container = MockRepository.GenerateStub<UnityContainer>();
    ITestInterface test = MockRepository.GenerateStub<ITestInterface>();

    container.Stub(x => x.Resolve<ITestInterface>()).Return(test);

    ITestInterface resolvedTest = container.Resolve<ITestInterface>();

    resolvedTest.DoSomething();
}

Easy fix.