Sunday, January 9, 2011

AAPL Part 7: Building a Generic Data Mapper

I got bored with this series a while back and I haven’t been writing on it, mostly because I’ve been using the architecture (Agile ADO.Net Persistence layer) to write a bunch of code.  The basic idea of AAPL was to create an application architecture that embraced code changes instead of resisting them. I took a look back at all of the application architectures I’ve worked with over the years and thought carefully about the parts of those architectures that caused me pain whenever made changes to the application.  I then threw all those things out and tried to put together an application architecture based on ADO.Net (not LINQ, EF, or nHibernate) that was optimized for making code changes easy and reliable.  I’ve been using the AAPL architecture for a while now and I’m happy to report that those decisions have really paid off. 

Why are we doing this again?

Just a a quick reminder.  The point of this architecture is to give us the ability to write persistence code that looks like this.

        // GetFolderByFolderGuid
        public virtual Folder GetFolderByFolderGuid(Guid folderGuid)
        {
            string sql = @"SELECT *
                           FROM [Folder]                            
                           WHERE [FolderGuid] = @FolderGuid";
            SqlDao dao = SharedSqlDao;
            SqlCommand command = dao.GetSqlCommand(sql);
            command.Parameters.Add(dao.CreateParameter("@FolderGuid", folderGuid));
            return dao.GetSingle<Folder>(command);
        }

Whenever we need a new data access method we want to just write the new query, package it up in an ADO.Net SqlCommand object, then pass both the SqlCommand and the desired model class type to a Data Access Object that will then handle all of the plumbing required to execute the command, get the data, package it up as whatever model type we passed in and then give it back. This is a very low friction way to handle data access.  It’s easy to maintain, new data methods take seconds to create, and we never have to touch the plumbing code.   The end result is that maintaining code and making changes to our model should feel like this…

instead of this…

What is a data mapper?

The key piece of plumbing code that makes this architecture so painless is the GenericDataMapper.  As described above, when we read data we execute an SQLCommand, get the resulting IDataReader, then map the data in that IDataReader to a model class (an entity class).  The DataMapper is the class that encapsulates the code that knows how to map the fields from an IDataReader to the properties in a model class.  For example, the UserMapper class contains the logic needed to take an IDataReader and use it’s data to populate a new object of type User.

What does the Generic Data Mapper do?

We could write a separate DataMapper class for each model type.  We could have a UserMapper, a CompanyMapper, a JobMapper, a WorkflowMapper, an ActionMapper, a StepMapper… you get the idea.  But doing it that way would require that 1) We write all those DataMapper  classes, and 2) We modify those DataMapper classes whenever we make a change to one of our models.  No thanks.

It would be better to write a single DataMapper class that could handle the mapping for any model class that we passed to it. Remember that early on we made the decision that the columns in our database would have the exact same names as the data properties in our model classes.  That constraint makes the mapping logic pretty simple.  We can create a GenericDataMapper that uses reflection to find all the properties on our model class and matches them to any fields with the same name in our IDataReader.  No hard coded mapping logic needed.  Just a little bit of reflection and a standard naming convention and we can create a single GenericDataMapper that will handle mapping for all of our model classes.

DataMapperFactory

We use a DataMapperFactory to get the right DataMapper for a model type.  The logic is simple.  We have a GetMapper() method that takes a type as it’s parameter. It checks to see if we have a custom mapper for the type, if not it returns the GenericMapper .  In the example below we have just one custom data mapper, ListItemDTOMapper.  Every other model class is handled by the GenericMapper.

    class DataMapperFactory
    {
        public IDataMapper GetMapper(Type dtoType)
        {
            switch(dtoType.Name)
            {
                case "ListItemDTO":
                    return new ListItemDTOMapper();
                default:
                    return new GenericMapper(dtoType);
            }      
        }
    }

IDataMapper

All of our DataMappers implement a custom IDataMapper interface.

    interface IDataMapper
    {
        // Main method that populates dto with data
        Object GetData(IDataReader reader);
        // Gets the num results returned. Needed for data paging.
        int GetRecordCount(IDataReader reader);
    }

IDataMapper requires that we implement just two methods.  GetData() takes an IDataReader and returns a model class that contains the data for the current record in that reader (note that one call to GetData() returns one model object, lists can be created by calling GetData() multiple times ).  GetRecordCount() will return the total number of records returned by the IDataReader and is used for data paging.  So these are the only two public methods that we’ll need to implement.

GenericMapper

Now we get down to it.  Here is the complete code for the GenericMapper.

    class GenericMapper : IDataMapper
    {
        public System.Type DtoType { get; set; }
        private bool _isInitialized = false;
        private List<PropertyOrdinalMap> PropertyOrdinalMappings;


        public GenericMapper(System.Type type)
        {
            DtoType = type;
        }


        private void InitializeMapper(IDataReader reader)
        {
            PopulatePropertyOrdinalMappings(reader);
            _isInitialized = true;
        }


        public void PopulatePropertyOrdinalMappings(IDataReader reader)
        {
            // Get the PropertyInfo objects for our DTO type and map them to
            // the ordinals for the fields with the same names in our reader. 
            PropertyOrdinalMappings = new List<PropertyOrdinalMap>();
            PropertyInfo[] properties = DtoType.GetProperties();
            foreach (PropertyInfo property in properties)
            {
                PropertyOrdinalMap map = new PropertyOrdinalMap();
                map.Property = property;
                try
                {
                    map.Ordinal = reader.GetOrdinal(property.Name);
                    PropertyOrdinalMappings.Add(map);
                }
                catch { }
            }
        }


        public Object GetData(IDataReader reader)
        {
            if (!_isInitialized) { InitializeMapper(reader); }
            object dto = Activator.CreateInstance(DtoType);
            foreach (PropertyOrdinalMap map in PropertyOrdinalMappings)
            {
                 if (!reader.IsDBNull(map.Ordinal))
                 {
                     map.Property.SetValue(dto,reader.GetValue(map.Ordinal),null);
                 }
            }
            return dto;
        }


        public int GetRecordCount(IDataReader reader)
        {
            Object count = reader["RecordCount"];
            return count == null ? 0 : Convert.ToInt32(count);
        }


        private class PropertyOrdinalMap 
        {
            public PropertyInfo Property { get; set; }
            public int Ordinal { get; set; }
        }

    }

This isn’t very much code, but the logic can be a little difficult to follow.  Let’s break it down and see how it works. 

When the user creates an instance of GenericMapper, the constructor requires them to pass in the model type they want to map to.  That type is assigned to our local DtoType property (ModelType might have been a better name).  Note that we’re creating an instance, not using static methods.  This is important because once the GenericMapper is created it can be used multiple times for multiple records without having to recalculate the mappings.

The user calls the GetData method which takes an IDataReader as a parameter. GetData is the heart of our mapper.

        public Object GetData(IDataReader reader)
        {
            if (!_isInitialized) { InitializeMapper(reader); }
            object dto = Activator.CreateInstance(DtoType);
            foreach (PropertyOrdinalMap map in PropertyOrdinalMappings)
            {
                 if (!reader.IsDBNull(map.Ordinal))
                 {
                     map.Property.SetValue(dto,reader.GetValue(map.Ordinal),null);
                 }
            }
            return dto;
        }

GetData first checks to see if the GenericMapper has been initialized.  If not it calls InitializeMapper() which handles all initialization including a call to PopulatePropertyOrdinalMappings().  This is where the magic happens. 

        public void PopulatePropertyOrdinalMappings(IDataReader reader)
        {
            // Get the PropertyInfo objects for our DTO type and map them to
            // the ordinals for the fields with the same names in our reader. 
            PropertyOrdinalMappings = new List<PropertyOrdinalMap>();
            PropertyInfo[] properties = DtoType.GetProperties();
            foreach (PropertyInfo property in properties)
            {
                PropertyOrdinalMap map = new PropertyOrdinalMap();
                map.Property = property;
                try
                {
                    map.Ordinal = reader.GetOrdinal(property.Name);
                    PropertyOrdinalMappings.Add(map);
                }
                catch { }
            }
        }

We have a private class named PropertyOrdinalMap which is just a simple container for a System.Reflection.PropertyInfo object and an int. The int represents the ordinal (field index) in our IDataReader for the field that matches the name of the property stored in PropertyInfo.  The PopulatePropertyOrdinalMappings method just loops over all of the properties in our DtoType and tries to find an ordinal in IDataReader for a field with the same name.  If a match exists, a new PropertyOrdinalMap is created an added to the PropertyOrdinalMappings List. If a match doesn’t exist, no PropertyOrdinalMap is created.  The result is that we just skip any properties in our DtoType that the reader doesn’t have a value for.  That’s a key design decision.  We don’t throw an error if a mapping doesn’t exist, we just skip that property and get the data that we can map.

Once the mappings are created it’s all pretty straight forward.  We return to GetData().  We use System.Activator() to create a new instance of our DtoType called dto (short for Data Transfer Object).  Then we iterate over our PropertyOrdinalMappings and use reflection to set the value of each dto property that we have a mapping for.  At that point we have a brand new object of whatever type was passed in to the constructor and it’s been populated with the data from the current record in the IDataReader.  We’re done!  We can just return the dto.

So that’s it.  A little object oriented design, some super-simple reflection code, and I have a GenericMapper that allows me to never right another line of data mapping code (as long as I’m willing to use the same name for my model properties and data table columns).  I haven’t spent any time optimizing this.  In fact, I haven’t really looked at it since I first wrote it.  If anyone has suggestions for improvements please let me know.

5 comments:

  1. Great stuff. Have you looked at what the Generic DataMapper would look like using Dynamic in 4.0? The reason I ask is that call-site caching might (???) improve performance over reflection, on subsequent passes.

    ReplyDelete
  2. @Hank, thanks. I'm sad to say I really don't know much about Dynamic in 4.0. It's on my list but I haven't dug into it yet. That's a good thought though. I've also thought about taking a closer look at Jimmy Bogard's AutoMapper. It would be nice to use AutoMapper and get it's rich feature set, but I'm not sure if I could make AutoMapper work with an IDataReader. I'll have to check it out. Might be a nice feature to contribute to AutoMapper.

    ReplyDelete
  3. This is a great design! I like the no-nonsense approach of getting rid of the fact that "we're going to have to code to switch db's from SQL Server to Oracle...". That's never happened for me, esp. working in a Microsoft world.

    Anyhow, if you just cached the property-to-table column information once you reflected it from all your domain objects, you'd get rid of that one bottleneck.

    Besides that, really cool approach to Data Mapper!

    ReplyDelete
  4. Rudy, thanks for taking the time to write up this series of articles. I'm getting back into .NET after 6 yrs of no coding at all, so I'm a little rusty. Is there a reason you have the Persistence classes instead of simply putting the Insert, Update, and Delete methods in Service classes?

    ReplyDelete
  5. Hi Rudy, I really appreciate this entire series, it's been a fantastic example that's helped make a lot of sense of things for me.

    One area that I'm not as clear on when using DTO's and such is how do you handle Parent/Child relationships? Do you hydrate in all of the child records with the initial object, or do you defer them somehow? Can you point me towards any good examples?

    Thanks again!

    ReplyDelete