His list of "symptoms" that a relational database is not right for you are more often symptoms that your relational database was not well designed (or that the problem you were trying to solve changed along the way) rather than that the relational model itself does not fit your needs.

I think this applies to all his "structural symptoms" but especially with "Do you have tables with lots of columns, only a few of which are actually used by any particular row?" This is more often a sign that the database was not properly normalized than anything else.

This of course is not to say that the relational model is right for everything. In some cases, object oriented databases are the way to go and if you truly need a vast level of scalability and you can afford to relax the ACID standard then it makes sense to look at non-relational options.

There is room in the world for both relational and non-relational models, but I do not think fashion should play a role in choosing a technology personally.

When you mention object databases, can you name any examples? As far as I know object databases where talked about a lot some years ago, but never really gained significant widespread adoption.

Also, the article makes the point that non-relational databases are not just about scalability. In fact, in the case of graph databases, scaling problems are just as pronounced as with relational databases. The different data models can enable you to make good database designs, even if the structure of your data is against you.

PostgreSQL HStore is very handy in situations such as this: http://www.postgresql.org/docs/8.3/static/hstore.html

Frequently you can avoid having object that have a wide variety of attributes most of which are empty in the first place. That often indicates that the objects are not of the same type and that that table should be broken into other tables of objects that are truly alike.

That, though, is not always the case. When you truly have that situation with objects of the same kind, then I do not see why allowing the numerous empty columns (as long as they are all bound to the key and only the key) or the "object-key-valuy" tables are bad relational designs. If I am missing something, please let me know.

As for the object databases, the only one I have really heard of is db4o as another posted mentioned. I do believe that they are used in some areas of physics though. I have never used them personally, but I have heard of people using them in small scale projects just as a way to avoid the object-relational impedence mismatch complications.

To your last paragraph, I agree completely. There is room and a place for both relational and non-relational databases.

It seems that db4o supports some pretty flexible query mechanisms. That leads me to worry about performance. One of my recurring nightmares of relational databases and SQL is optimizing queries on non-trivial constantly-evolving schemas, and I can't help wondering if db4o has the same horrors in store.

HOWEVER, my sense is that bringing a relational mindset to object databases is not going to realize the full potential of them; you have to think about storing the data elements differently.

Object databases seem great for OLTP apps. In particular, ones that require working with individual accounts/patient charts/etc, i.e. query a person's account, modify a data element, and save it back to the db. For something more analytical, like spotting trends in historical data, I'd prefer the full power of an RDBMS and the SQL language that goes with it.

All non-trivial problems change along the way.

Allow me to channel CJ Date and point out current popular databases are not truly relational. A proper relational database built around Tutorial D would pretty much always be the way to go, even if your problem is highly OO.

Basically, I've found that under high read/write loads I get occasional socket timeouts (tested on linux/osx). I think the underlying reason for the timeouts are Berkeley DB locks when data gets flushed to disk. Beyond the hassle of getting intermittent socket timeouts the real problem is that the memcache client API fails gracefully because it was specifically designed to be fault tolerant. You can check for socket errors and retry queries but you'll still get unpredictable query times.

I'm coming to the conclusion that it's an architecture issue - MemcacheDB is a persistent database abstracted behind an interface specifically designed for non-persistence. The abstraction leaks when the database locks.

Anyway, after testing MemcacheDB and Tokyo Tyrant in production my conclusion was to use Tokyo Tyrant instead of MemcacheDB. Tokyo Tyrant implements the memcached protocol and performs really well under load (and has TONS of features such as master-master replication, Lua scripting, different types of engines [hash, b-tree or memory]). You can also check LightCloud, which is a distributed key-value database built on top of Tokyo Tyrant.

I think its a good to go away from relational and look at these things and come back and look at your current practices. Maybe every attribute doesn't need a column and an index. Maybe use a JSON bag to hold the loose parts. Maybe throw away a bunch of indexes; create them on the fly for the monthly report. Maybe you can give away some of your referential integrity.

Actually it can: SQL:2003 includes recursive queries (WITH RECURSIVE), and several popular database systems implement it.

It's trivial to scale out stateless code; databases are the hard part.

Relational databases didn't win because they're fast, they won because they're flexible for querying and language neutral, both features which necessarily slow the database down.

> It's trivial to scale out stateless code; databases are the hard part.

Not really relevant to the conversation. There are databases that can store trees of objects just fine as is, not all databases force you to store everything in tables.

To me, the conversation is about trade-offs, and if the alternative database (for example) forces you to get rid of ACID transactions, then it's not as cut-and-dry as you purport.

Some of these new fangled distributed databases forgo transactions because transactions and distribution don't really go well together, they are opposing forces. Don't confuse issues that have nothing to do with each other, and don't assume that only relational databases support ACID compliant transactions.

Look at a real object database like Gemstone that is directly comparable to your big iron Sql database. These new distributed key/value pair database are little more than distributed persistent hash tables, they're different beasts entirely and aren't directly comparable in features because they're meant to solve different problems.

Is there anything open-source (and language-agnostic) that's comparable to Gemstone?

There's an open source one called Magma in Squeak that's like a Gemstone lite, but I'd take the Gemstone version any day because it'll scale to any level you'll ever need, ever. I'm working on a Gemstone project now, and after having used it, nothing else I've seen comes close to the massive productivity it offers.

By the way, databases shouldn't be language agnostic, this leads to the ever present anti pattern of using the database as an integration point between many programs which turns the database into a giant ball of mud global variable that becomes impossible to change.

Integration databases suck. Application should own their own data and integrate with other application via services. That's how the web became so successful and that's how big ass enterprises should be ran as well, many small apps loosely coupled, not one giant global db where every app is bound to a generic schema that isn't suited for what it actually needs.