Ask HN: Just replace my DBA: painless db admin and scale without the cloud?

xenoterracide · on March 1, 2009

There is no easy out, if you really need to scale, regardless of infrastructure you are going to need someone who understands databases. If you're lucky you might be able to find someone who can do more than one job. e.g. dba+sys admin. Just make sure that they really do know both things.

thwarted · on March 1, 2009

Yeah, this kind of question seems really close to "I'm a business guy, all I need is a tech guy to implement my idea", which implies there's some way to avoid having to know how to do something that is core to the business.

jacobscott · on March 1, 2009

I want to make it very clear that I'm not a business guy.

Think of it in the context of building a database-backed enterprise app that customers run in house. You don't want to ship a DBA to every customer...

thwarted · on March 1, 2009

For that you use an embedded database system. Many enterprise apps come with embedded database systems, or require a major database system to be installed in tandem, and these things are targeted at medium sized companies that don't have DBA specific to the the department that is using the app.

xenoterracide · on March 1, 2009

or explicitly tell them how to configure their system, and what kind of a system to run it on. this doesn't help you avoid a dba, it merely avoids them having one.

jacobscott · on March 1, 2009

It seems like I asked the question in the wrong context. Sorry about that. Given that scale is hard, how about monitoring, or management?

jacobscott · on March 1, 2009

If it helps, I asked a more concrete version of this question focusing on PostgreSQL on stackoverflow: http://stackoverflow.com/questions/556325/postgresql-managem...

iamelgringo · on March 1, 2009

The state of the art _is_ hiring a DBA. They get paid to be up on all the database monitoring, scaling, managing and management tools. This is why DB consultants charge $200+ an hour. It's pretty arcane knowledge, and for a business of any size, the database quickly becomes the lifeblood of the organization.

This is also the reason that databases like Couch DB or Amazon Simple DB or Google's App Engine are so appealing. They all have the promise of reducing db management headaches, at the cost of sacrificing features.

DenisM · on March 1, 2009

Try a commerical database, such as Microsoft SQL Server (FD: I work there as a developer). To bring up a recent example, it can build indexes without locking the whole table so you wouldn't need to jump throught the hoops like the FriendFeed guys just did with MySQL. Unless your time is free it will cost you less to buy SQL Server than to write your own "online index thingie" on top of MySQL.

And if your time is free, then you are probably a startup and you can get SQL Server and bunch of other MS tools for free via http://www.microsoft.com/BizSpark/

MySpace runs on SQL Server: http://www.sqlservercentral.com/blogs/steve_jones/archive/20... and if it's good enough for them it will likely be good enough for your site as well. I'm not sure if the scale-out story is any good (there are "partitioned views" which are basically "sharding" but I never used those), but scale-up story is as good as they come - it can efficiently utilize 64-cores of the HP superdome.

I don't know much about Oracle and IBM DB2, but I know that the latter has a free version without too many restrictions on it, so give them a spin as well. If ease of use is importnat to you, common wisdom has it that Microsoft's is the easiest to use among the top commercial vendors. NB: I did not verify the common wisdom, YMMV :-)

moe · on March 1, 2009

You're suggesting to use a commercial database but don't provide any reasons beyond what free databases can do.

PostgreSQL is pretty much on par in terms of features and performance with SQLServer and Oracle. So if you're looking for a free solution to roll your own then look no further.

The only reason I can see for going with a commercial DB is when you not only want commercial support (you can buy that for Postgres, too) but a shrinkwrapped "don't bother me"- solution with SLAs etc. Be prepared to pay through your nose for that, though.

In terms of usability there is no big difference between any of the commercial or non-commercial offerings. They all have their special peculiarities (which is why you always hire a DBA who is familar with your RDBMS, not just with "any" RDBMS) and the shiny GUIs that some of them provide are not normally of interest in day-to-day work.

charlesju · on March 1, 2009

Managed hosting works for us (Engine Yard), it is significantly cheaper than hiring your own DBA.

xenoterracide · on March 1, 2009

having managed hosting doesn't mean you don't need to a dba. In my experience all the host does (worked for a managed hosting company) is set up the servers and keep them running, their help really can't help you with tweaking queries or tuning the db. I can't speak for on engine yard specifically. But I wouldn't bank on your host knowing anything beyond the basics of databases.

charlesju · on March 1, 2009

Engine Yard has several excellent DBAs on staff. They tell you what queries are slow and tune the db to optimize for your type of traffic. That's why they are worth the 30% premium on top of EC2.

xenoterracide · on March 1, 2009

engine yard may be an exceptional exception to the rule. I was generalizing, and speaking of managed hosting in general. Like I said, I can't speak of engine yard specifically, I haven't worked for or with them.