Anonymous type to dictionary using DynamicMethod

May 3, 2008 at 12:24 PMAndre Loker

C# 3.0 offers a variety of new language features like anonymous types, object and collection initializers and extension methods. With some creativity these features can be used to reduce the amount of code being written and to make the code more readable. Imaging you have a method that expects a dictionary of configuration values, like:

   1: public class MyClass {
   2:     public void DoStuff(IDictionary<string, object> dictionary) {
   3:         // .. do something
   4:     }
   5: }

A typical .NET 2 client would use the method like this:

   1: public static void DotNet2(MyClass mc) {
   2:     Dictionary<string, object> dictionary = new Dictionary<string, object>();
   3:     dictionary["date"] = new DateTime(1970, 6, 12);
   4:     dictionary["foo"] = 123;
   5:     dictionary["name"] = "Some text";
   6:     mc.DoStuff(dictionary);
   7: }

C# 3.0 adds the collection initializer syntax, which reduces the repetitive code parts a lot:

   1: public static void CollectionInitializer(MyClass mc) {
   2:     var dictionary = new Dictionary<string, object> {
   3:         {"date", new DateTime(1970, 6, 12)},
   4:         {"foo", 123},
   5:         {"name", "Some text"}
   6:     };
   7:     mc.DoStuff(dictionary);
   8: }

Currently Microsoft is working on ASP.NET MVC, an MVC implementation for ASP.NET comparable to Ruby On Rails or Castle MonoRail. To make the code more compact - especially in the HTML views - they use anonymous objects instead of dictionaries. In our example a first approach could look like this:

   1: public static void AnonymousWithReflection(MyClass mc) {
   2:     var data = new {
   3:         date = new DateTime(1970, 6, 12),
   4:         foo = 123,
   5:         name = "Some text"
   6:     };
   7:     var dictionary = ObjectHelper.TurnObjectIntoDictionary(data);
   8:     mc.DoStuff(dictionary);
   9: }

This looks a lot cleaner to me. Certainly the first question is: how do we convert the object into a dictionary? A simple implementation could look like this:

   1: public static class ObjectHelper {
   2:     public static IDictionary<string, object> TurnObjectIntoDictionary(object data) {
   3:         var attr = BindingFlags.Public | BindingFlags.Instance;
   4:         var dict = new Dictionary<string, object>();
   5:         foreach (var property in data.GetType().GetProperties(attr)) {
   6:             if (property.CanRead) {
   7:                 dict.Add(property.Name, property.GetValue(data, null));
   8:             }
   9:         }
  10:         return dict;
  11:     }
  12: }

This method uses reflection to grab the values of all readable properties and puts them into a dictionary. Not very difficult to understand, but because of the reflection certainly not the fastest thing in the world as we will see later.

Intermezzo: using extension methods to improve readability

Before we continue our journey lets see whether we can improve the API. The call to ObjectHelper.TurnObjectIntoDictionary looks quite verbose to me, so lets add an extension method to ObjectHelper:

   1: public static IDictionary<string, object> ToDictionaryR(this object obj) {
   2:     return TurnObjectIntoDictionary(obj);
   3: }

I named the method ToDictionaryR to remind me that it uses reflection. This is not meant for production code, just for our little prove of concept here. While the code is not spectacular it improves usability:

   1: public static void AnonymousWithReflection2(MyClass mc) {
   2:     var data = new {
   3:        date = new DateTime(1970, 6, 12),
   4:        foo = 123,
   5:        name = "Some text"
   6:    };
   7:     mc.DoStuff(data.ToDictionaryR());
   8: }

We could tweak the whole thing into another direction by providing an overload of DoStuff that accepts an object which is converted to a dictionary as needed. If we deal with legacy code, we might as well use extension methods again:

   1: public static class MyClassExtensions {
   2:     public static void DoStuff(this MyClass client, object data) {
   3:         if (data is IDictionary<string, object>){
   4:             client.DoStuff((IDictionary<string, object>)data);
   5:         } else {
   6:             client.DoStuff(data.ToDictionaryR());
   7:         }
   8:     }
   9: }

And here the client that uses the extension:

   1: public static void AnonymousWithReflection3(MyClass mc) {
   2:     var data = new {
   3:        date = new DateTime(1970, 6, 12),
   4:        foo = 123,
   5:        name = "Some text"
   6:    };
   7:     mc.DoStuff(data);
   8: }

Looks neat! I do like the compactness of the new API. Let us see how this performs at runtime.

Give me numbers!

As mentioned earlier this approach uses a lot of reflection. This is bad, or isn't it? Benchmarks to the rescue! Here comes the almighty benchmarking program:

   1: private static void Main(string[] args) {
   2:     var mc = new MyClass();
   3:     for (int i = 0; i < 5; ++i) {
   4:         Benchmark("DotNet2", DotNet2, mc);
   5:         Benchmark("CollectionInitializer", CollectionInitializer, mc);
   6:         Benchmark("AnonymousWithReflection3", AnonymousWithReflection3, mc);
   7:     }
   8: }
   9:  
  10: public static void Benchmark(string name, Action<MyClass> exec, MyClass mc) {
  11:     Console.Out.Write("Benchmarking {0,-30}:", name);
  12:     const int count = 100000;
  13:     var sw = new Stopwatch();
  14:     sw.Start();
  15:     for (var i = 0; i < count; ++i) {
  16:         exec(mc);
  17:     }
  18:     sw.Stop();
  19:     Console.Out.WriteLine("{0} ms", sw.ElapsedMilliseconds);
  20: }

We simply invoke the approaches presented before one million times (and that 5 times in a row to let the values settle down) and see how fast they are. Certainly this is not very representative for code in practice. Still I want to get a rough estimate how expensive all that reflection mumbo jumbo really is. So, here are the results on my machine (Intel E6750, 4GB RAM, Vista Ultimate 64):

   1: Benchmarking DotNet2                       :32 ms
   2: Benchmarking CollectionInitializer         :29 ms
   3: Benchmarking AnonymousWithReflection3      :1822 ms
   4: Benchmarking DotNet2                       :28 ms
   5: Benchmarking CollectionInitializer         :29 ms
   6: Benchmarking AnonymousWithReflection3      :1783 ms
   7: Benchmarking DotNet2                       :28 ms
   8: Benchmarking CollectionInitializer         :28 ms
   9: Benchmarking AnonymousWithReflection3      :1789 ms
  10: Benchmarking DotNet2                       :28 ms
  11: Benchmarking CollectionInitializer         :28 ms
  12: Benchmarking AnonymousWithReflection3      :1852 ms
  13: Benchmarking DotNet2                       :29 ms
  14: Benchmarking CollectionInitializer         :29 ms
  15: Benchmarking AnonymousWithReflection3      :1809 ms

The DotNet2 and the CollectionInitializer version perform equally. This was expected, as collection initializers are syntactic sugar only. But the reflection version does bad, I mean REALLY bad: it is roughly 60 times slower than the non-reflective approach.

DynamicMethod to the rescue

Luckily we can minimize the overhead. Instead of using reflection all the time we convert the object we will use reflection one time to create code on the fly that can be reused afterwards. With the DynamicMethod class, this is rather easy. You only have to understand IL (intermediate language). If you have no experience with IL, grab Reflector and poke around in existing code. You'll get the idea.

Here is the code that creates a delegate that will convert an object to a dictionary:

   1: public static Func<object, IDictionary<string, object>> CreateObjectToDictionaryConverter(Type itemType) {
   2:     var dictType = typeof (Dictionary<string, object>);
   3:  
   4:     // setup dynamic method
   5:     // Important: make itemType owner of the method to allow access to internal types
   6:     var dm = new DynamicMethod(string.Empty, typeof (IDictionary<string, object>), new[] {typeof (object)}, itemType);
   7:     var il = dm.GetILGenerator();
   8:  
   9:     // Dictionary.Add(object key, object value)
  10:     var addMethod = dictType.GetMethod("Add");
  11:  
  12:     // create the Dictionary and store it in a local variable
  13:     il.DeclareLocal(dictType);
  14:     il.Emit(OpCodes.Newobj, dictType.GetConstructor(Type.EmptyTypes));
  15:     il.Emit(OpCodes.Stloc_0);
  16:  
  17:     var attributes = BindingFlags.Instance | BindingFlags.Public | BindingFlags.FlattenHierarchy;
  18:     foreach (var property in itemType.GetProperties(attributes).Where(info => info.CanRead)) {
  19:         // load Dictionary (prepare for call later)
  20:         il.Emit(OpCodes.Ldloc_0);
  21:         // load key, i.e. name of the property
  22:         il.Emit(OpCodes.Ldstr, property.Name);
  23:  
  24:         // load value of property to stack
  25:         il.Emit(OpCodes.Ldarg_0);
  26:         il.EmitCall(OpCodes.Callvirt, property.GetGetMethod(), null);
  27:         // perform boxing if necessary
  28:         if (property.PropertyType.IsValueType) {
  29:             il.Emit(OpCodes.Box, property.PropertyType);
  30:         }
  31:  
  32:         // stack at this point
  33:         // 1. string or null (value)
  34:         // 2. string (key)
  35:         // 3. dictionary
  36:  
  37:         // ready to call dict.Add(key, value)
  38:         il.EmitCall(OpCodes.Callvirt, addMethod, null);
  39:     }
  40:     // finally load Dictionary and return
  41:     il.Emit(OpCodes.Ldloc_0);
  42:     il.Emit(OpCodes.Ret);
  43:  
  44:     return (Func<object, IDictionary<string, object>>) dm.CreateDelegate(typeof (Func<object, IDictionary<string, object>>));
  45: }

The result of this method is a delegate that expects an object and returns a dictionary. Invoking CreateObjectToDictionaryConverter is rather expensive, so lets cache the result by providing a factory/registry:

   1: /// <summary>
   2: /// Loads the values of an object's properties into a <see cref="IDictionary{String,Object}"/>
   3: /// </summary>
   4: public class ObjectToDictionaryRegistry {
   5:     private static readonly Dictionary<Type, Func<object, IDictionary<string, object>>> cache = new Dictionary<Type, Func<object, IDictionary<string, object>>>();
   6:     private static readonly ReaderWriterLockSlim rwLock = new ReaderWriterLockSlim();
   7:  
   8:     /// <summary>
   9:     /// Loads the values of an object's properties into a <see cref="IDictionary{String,Object}"/>.
  10:     /// </summary>
  11:     /// <param name="dataObject">The data object.</param>
  12:     /// <returns>If <paramref name="dataObject"/> implements <see cref="IDictionary{String,Object}"/>, 
  13:     /// the object is cast to <see cref="IDictionary{String,Object}"/> and returned.
  14:     /// Otherwise the object returned is a <see cref="System.Collections.Hashtable"/> with all public non-static properties and their respective values
  15:     /// as key-value pairs.
  16:     /// </returns>
  17:     public static IDictionary<string, object> Convert(object dataObject) {
  18:         if (dataObject == null) {
  19:             return null;
  20:         }
  21:         if (dataObject is IDictionary<string, object>) {
  22:             return (IDictionary<string, object>) dataObject;
  23:         }
  24:         return GetObjectToDictionaryConverter(dataObject)(dataObject);
  25:     }
  26:  
  27:     /// <summary>
  28:     /// Handles caching.
  29:     /// </summary>
  30:     /// <param name="item">The item.</param>
  31:     /// <returns></returns>
  32:     private static Func<object, IDictionary<string, object>> GetObjectToDictionaryConverter(object item) {
  33:         rwLock.EnterUpgradeableReadLock();
  34:         try {
  35:             Func<object, IDictionary<string, object>> ft;
  36:             if (!cache.TryGetValue(item.GetType(), out ft)) {
  37:                 rwLock.EnterWriteLock();
  38:                 // double check
  39:                 try {
  40:                     if (!cache.TryGetValue(item.GetType(), out ft)) {
  41:                         ft = CreateObjectToDictionaryConverter(item.GetType());
  42:                         cache[item.GetType()] = ft;
  43:                     }
  44:                 } finally {
  45:                     rwLock.ExitWriteLock();
  46:                 }
  47:             }
  48:             return ft;
  49:         } finally {
  50:             rwLock.ExitUpgradeableReadLock();
  51:         }
  52:     }
  53:  
  54:     private static Func<object, IDictionary<string, object>> CreateObjectToDictionaryConverter(Type itemType) {
  55:         // as seen above
  56:     }
  57: }

There is nothing fancy going on. This implementation already has added synchronization for thread-safety which adds a little complexity. Note by the way that I am using the new ReaderWriterLockSlim. I expect much more reads from the cache than writes to it so this seems to be a reasonable choice. I also added the check to see whether the object provided to Convert already is a IDictionary<string, object> in case of which it is simply cast.

Given this new implementation we change the usage a bit:

   1: // modified version of the MyClass extension method
   2: public static class MyClassExtensions {
   3:     public static void DoStuff(this MyClass client, object data) {
   4:         client.DoStuff(ObjectToDictionaryRegistry.Convert(data));
   5:     }
   6: }
   7:  
   8: // our final testcase using ObjectToDictionaryRegistry
   9: public static void Final(MyClass mc) {
  10:     mc.DoStuff(new {
  11:            date = new DateTime(1970, 6, 12),
  12:            foo = 123,
  13:            name = "Some text"
  14:        });
  15: }

So, let us see how this performs. Adding the Final method to our benchmarking set, we get the following results:

   1: Benchmarking DotNet2                       :41 ms
   2: Benchmarking CollectionInitializer         :33 ms
   3: Benchmarking AnonymousWithReflection2      :1961 ms
   4: Benchmarking Final                         :60 ms
   5: Benchmarking DotNet2                       :27 ms
   6: Benchmarking CollectionInitializer         :28 ms
   7: Benchmarking AnonymousWithReflection2      :1791 ms
   8: Benchmarking Final                         :58 ms
   9: Benchmarking DotNet2                       :31 ms
  10: Benchmarking CollectionInitializer         :29 ms
  11: Benchmarking AnonymousWithReflection2      :1886 ms
  12: Benchmarking Final                         :50 ms
  13: Benchmarking DotNet2                       :30 ms
  14: Benchmarking CollectionInitializer         :28 ms
  15: Benchmarking AnonymousWithReflection2      :1914 ms
  16: Benchmarking Final                         :53 ms
  17: Benchmarking DotNet2                       :30 ms
  18: Benchmarking CollectionInitializer         :28 ms
  19: Benchmarking AnonymousWithReflection2      :1827 ms
  20: Benchmarking Final                         :54 ms

Wow, this is cool: the version using DynamicMethod takes only two times longer than the dictionary version. That's highly acceptable given the fact that:

  • We have to do synchronization at the registry.
  • The method actually being called currently does nothing where in practice it will most likely take more time to execute compared to the cost of conversion. The difference between the DynamicMethod approach and the dictionary approach will very soon shrink a lot.

What we gain and what we lose

Personally I think that the syntax using anonymous types is quite nice. We need less (disturbing) characters like quotes, braces etc. to express what we want. The costs are neglectable if we use DynamicMethod and caching.

However, not everyone is happy with this approach. Jeffrey Palermo for example finds that MS makes "a huge mistake" by providing this kind of API for ASP.NET MVC. I think that this kind of API has its place but should - as always - only be used where appropriate. If you now in advance which keys to expect in the dictionary, you'd most likely be better of using a strongly typed object with an object initializer. When the keys cannot be foreseen the API can gain readability by using anonymous types. You certainly do have a problematic API when the parameters provided are that dynamic, but this is not the fault of the anonymous type. Providing string keys plus object values in a dictionary is not typesafe nor intuitive in the first place. Anonymous types can only help to improve the problematic situation.

Posted in: C# | Snippets

Tags: , ,