Creating a Database in C#: Interviewing the CEO of RavenDB Oren Eini

I had the opportunity to interview Oren Eini , the CEO of RavenDB , a NoSql database created in C#. We had a fascinating talk and you can listen to the full recording here:

Oren certainly makes a unique impression. Having listened to a lot of CEO interviews in countless podcast episodes, I never encountered another CEO more knowledgeable about the intricacies of C# than Oren. As we sidetracked into his many adventures in developing a database, Oren’s talked about .NET garbage collection, backup mechanisms, serialization issues, memory-mapped files, and performance benchmarks. My favorite topics of conversations (yes, I’m great at parties).

RavenDB was created at 2009, around the time NoSQL became a popular concept and new databases sprang to life every other day, including MongoDB , couchbase , Redis , and ElasticSearch . Moving forward to today, RavenDB employs over 50 people, has an impressive feature set, used by thousands of customers, and claims the title of the first document-based database to fully support ACID transactions.

We talked about many issues, challenges, and milestones with RavenDB development. I asked how happy they were choosing C# as the programming language. This decision didn’t turn out all smooth sailing at first, Oren iterates. “going back to that time frame, I think the primary issue with C# was that it limited me to running on Windows, and that was a huge limit for us and probably the number one reason, if I had to do it all over again, I wouldn't choose C#“.

Those were dark times, but it gets much better as you’ll hear in the interview. “Even so, I think that with all of those limitations, the decision to write in C# was correct for one primary reason. The amount of complexity you have to deal with when writing a database engine is huge. So the whole point is that if we can lean on the language, and lean on the runtime to do a lot of the cleanups, then it will be tremendously helpful in actually reaching the end goal of shipping a product“.

About 5 years into RavenDB’s lifetime, the project ran into huge performance problems that turned out to be GC Pressure . As it happens, parsing a ton of JSON documents and allocating memory for them means the garbage collection will be working really hard. The solution turned out to be avoiding allocations altogether by using memory-mapped files and a binary JSON format. If MongoDB’s BSON comes to mind, then this is very much similar. Oren is repeating the mantra that allocating might be cheap, but you’ll be sure to pay for it later. There are a few tricks to it, like keeping track of GC activity, minimizing allocations, and using native code when necessary.

When asked how difficult was it to create the first version of RavenDB, Oren says that creating a database engine core is simple enough to do in a matter of 2-3 months. I have my doubts as to how many teams could accomplish that feat, but the developers at Hibernating Rhinos led by Oren are one of them. The issue is not with the core functionality, says Oren, it’s all the additional features that need 10 more years of development. RavenDB indeed has a full set of features , like RQL, a dedicated query language, clients in six different languages, auto-replication, auto-indexing, ACID transactions, full-text search, and graph API.

When I asked about performance benchmarks, Oren pointed me to a case study where one of his customers migrated from Couchbase to RavenDB. In this particular case, the customer was able to get to more throughput with RavenDB using just 1/6th of the hardware they used with Couchbase.

Why should I choose RavenDB over other databases? I asked. “RavenDB is a grown-up database” says Oren. In the sense that it can monitor and tweak itself to optimize performance without making you do this analysis yourself. For example, when you query a lot over a certain property, RavenDB will create an index automatically. When you search a lot in your queries, RavenDB will know to optimize for a full-text search. These are just two examples out of dozens of things RavenDB does to make development simpler. Other features include a UI studio where you can see important metrics, like memory usage, CPU utilization, I/O, etc. When you’re hitting a bottleneck, RavenDB will alert you automatically, and give you a chance to scale before customer performance is impacted.

Read more about the database in ravendb.net , and check out Oren Eini’s blog .

Listen to the audio recording above for the full interview. Cheers.