NHibernate vs Entity Framework: a performance test(zhuan)

 

As part of my thesis I measured the performance of some .NET ORM frameworks including NHibernate and Entity Framework. Measuring was done by implementing two simple applications using the same table structure and doing the same operations on the same data.

Note: before reading this article please see my views on comparing ORM tools. Also see the follow up post to this comparison for revised results.

I measured the time it took for each framework to complete these operations:

• store
• read over relations
• read by ID
• update
• delete

The results were somewhat interesting. Here is a short summary of what I’ve found out.

Operation \ Number of operations	NHiberante - 4K	Entity Framework - 4K	NHiberante - 40K	Entity Framework- 40K	Winner
Store	37,37	9,19	1500	98	Entity Framework
Read over relations	1,01	0,54	10,13	4,18	Entity Framework
Read by ID	3,06	25,22	246	230	NHibernate with smaller amount of objects
Update	6,61	7,34	77	72	Both
Delete	3,35	16,76	58	1824	NHibernate

Read on if you’re interested in the details of the performance measurements.

Update: the source code of the program used to measure is nowavailable for download.

How performance was measured

The data model I’ve set up to measure with was pretty simple. I used three entites: a Company entity which had multiple Employees all of which had multiple Reports all with a 1-n relation.

The detailed results for each operation are as follow (note that the graphs use logarithmic scale):

 

Storage

 

Framework/ No. of operations	1K	3K	8K	14 K	23K	33K	40K
NHibernate	4,95	20,64	112	280	575	1107	15000
Entity Framework	2,11	6,18	18,4	31,3	51,7	77	98

Update (2009. 08. 24.):
After NHibernate improvements by Tuna Toksoz (download source) this metric changed the following way:

Framework/ No. of operations	1K	3K	8K	14 K	23K	33K	40K
NHibernate	1,22	3,25	8,98	13,11	28	38	46
Entity Framework	2,11	6,18	18,4	31,3	51,7	77	98

Read over relations

Framework/ No. of operations	1K	3K	8K	14 K	23K	33K	40K
NHibernate	0,21	0,66	1,81	3,31	5,0	8,24	10,13
Entity Framework	0,17	0,4	0,96	1,76	2,47	3,91	4,18

Read by ID

Framework/ No. of operations	1K	3K	8K	14 K	23K	33K	40K
NHibernate	0,28	1,31	13,0	37,6	80	175,5	246
Entity Framework	5,56	17,03	48	83	132	195	230

Update

Framework/ No. of operations	1K	3K	8K	14 K	23K	33K	40K
NHibernate	1,74	4,54	17,1	33,2	43,47	68	77
Entity Framework	1,99	5,29	14,2	24,2	41,9	59	72

Delete

Framework/ No. of operations	1K	3K	8K	14 K	23K	33K	40K
NHibernate	0,78	2,31	8,83	20,6	28	46	58
Entity Framework	1,78	8,19	52,2	133	320	1014	1895

Conclusion

The performance measurements only provided significant differences in two cases.

• When it comes to storing data, Entity Framework proved to be significantly faster than NHibernate
• When deleting data, NHibernate was much faster than Entity Framework

My conclusion however after the test was that in those scenarios when the number of the operations is not too high performance does not differentiate that much. I would choose between these two tools based on which ones provides faster, more efficient and more straightforward development with the current project.

Give me the code

As many of the commenters pointed out a comparison is not really valid without publishing the code enabling validation of this measurement. The program I’ve written to do the test can therefore be downloaded:NHibernate vs Entity Framework source code.

To set up and run the project after downloading an unzipping the package follow these steps:

• Create two MSSQL Databases (e.g. EntityFrameworkTest and NHibernateTest)
• Run CreateTables.sql on both databases to create the database structure
• In the App.config section of the MeasurementController project set the connection strings for NHibernate and Entity Framework (replace the !!! marks)
• Build the solution
• Run the MeasurementController project
• Run times are written to the output and also saved in files in the bin/Data folder

Some additional comments for understanding the source:

• Upon every run test data is created in an XML file. From this XML file Company, Emlpoyee and Report objects are created in-memory as the test structure
• This test structure is passed to the NHibernate and Entity Framework implementations of the Store, Read, Read by ID, Update and Delete tests (in the source additionally search is also measured) on each test run.