Byte code instrumentation and the ORM magic

All ORM tools use some kind of byte-code instumentation to do the persistance magic behind the scenes. But as an architect, it is important to understand what Hibernate or any JPA tool does to the entity classes?

Hibernate 'enhances' entity classes at runtime using a byte-code library called Javaassist. For e.g. it adds a '_dirty' flag to each field. It also adds a '_loaded' flag for each field to support lazy loading. A good blog explaining these concepts is here. So Hibernate reads the XML configuration or obtains annotations at runtime using reflection to apply byte-code instumentation.

There are various ways of doing byte-code instumentation using libraries such as CGLib, ASM, Javaassist, etc.
This byte-code enhancement can be done at compile-time or run-time. For Hibernate, besides a few special cases which require compile time 'enhancement' to byte-code; all common scenarios can be satisfied with runtime instrumentation.

The following link gives a good overview of all the enhancement options available in JPA.
http://openjpa.apache.org/builds/1.2.1/apache-openjpa-1.2.1/docs/manual/ref_guide_pc_enhance.html

In the .NET world, NHibernate uses the Linfu or Castle Dynamic Proxies byte-code enhancement providers. 
http://nhforge.org/blogs/nhibernate/archive/2008/11/09/nh2-1-0-bytecode-providers.aspx

Mapping between Entity Objects and DTOs

Very often, we need to map between our Entity objects and DTO's. This mapping code can be quite tedious to write.
There is a lot of hot debate on whether to use DTO's or just pass the entity objects directly to the view or webservices. There are pros and cons of each approach. Some good links on this debate are listed here:

Data Transfer Object - MSDN

http://stackoverflow.com/questions/5216633/jpa-entities-and-vs-dtos

Pros and Cons of Data Transfer Objects 

If you are using popular ORM tools such as Hibernate, iBatis or any other JPA complaint tool, then it may not even be possible to use the Enrity objects directly in your service or presentation tier. This is because these ORM toolkits typically use some kind of byte-code instrumentation to do the persistance magic behind the scenes. A good link explaining this is available here.

To avoid the drudgery of writing the 'Adapter/Mapping' code for each Entity object and DTO object, we can use some cool AutoMapper tools. These AutoMapper tools work on Reflection techniques and automatically map the source and target object properties. Custom mapping is supported using XML configuration or through code.

In the .NET world, there is a popular AutoMapper tool that has become the de-facto standard for a lot of .NET projects. In the Java world, there are 2 popular alternatives - Dozer and ModelMapper.
I found Dozer to be more comprehensive with some pretty good features. The usage is super-simple if you use the Singleton Wrapper and place the custom mapping file in the classpath.

If you are using the Spring Framework, then the 'BeanUtils' class has some simple static methods to copy properties from one object to the other. 

Data Services in the Microsoft world

In my previous blog, I ranted on the concept of Data Services in creating a data virtualization layer. In the .NET world, data services equate to WCF data services (formerly a.k.a ADO.NET data services)

Microsoft is propogating the use of an open standard called OData for building REST style data services. A good article describing OData is available on MSDN. OData essentially leverages JSON/ATOM and HTTP semantics to build a simple data services layer across disparate data sources.
But looks like besides M$, there are no big vendors jumping on the OData bandwagon. Its interesting to note that WebSphere eXtreme Scale Servers also expose a OData service.

Ruminating of Data Virtualization

The industry is flooded with confusing terms when it comes to understanding 'Data Virtualization'. We have IaaS (Information as a service), Data Services, EII (Enterprise Information Integration), Data Federation, etc. and so on! The point is that there are no industry standard definitions for these analyst-coined terms and there is a lot of overlap between them.

Rick Lans tries to clear the cloud with some simple definitions here. Another interesting post by Barry Devlin throws more light on the concept of data virtualization.

The core concept behind data virtualization is to create an abstraction layer (Data Access Layer) that hides the complexities of the underlying disparate data sources and provides a unified view of the enterprise data to the applications. This can be implemented using "SOA style" Data Services or creating a virtual data layer that can be queried using SQL-like semantics. More info can be found at these links: Link1 & Link2

RedHat has a nice whitepaper explaining the concept of Data Services in a SOA environment. This post explains the benefits of data virtualization. Composite Software is a leader in data virtualization techniques and has shared a couple of interesting case studies that demonstrate the use of their data virtualization platform.

One thought that came to my mind was regarding the challenges in accessing NoSQL data from the data virtualization layer. While some type of NoSQL datastores such as XML documents, Key/Value pairs can be exposed as a relational SQL view, it may not be possible to have a uniform query interface for unstructured data. All NoSQL data stores will expose some kind of Java API that can be used for querying. Would it be possible to create a common set of meta-data for both structured and unstructured data?
In such scenarios, IMHO, the only strategy for data virtualization is to use Data Services.