NewSQL versus traditional optimization/sharding

| | August 6, 2015

We’re a small startup with a write-heavy SAAS app and are (finally!) getting to the point where our usage is presenting scaling issues. We have a small team, so we really appreciate being able to offload sysadmin to Heroku and RDS.

While Heroku is (mostly) fine, we have a couple problems with RDS:

  1. Scaling. This is the biggest concern. We currently run an XL RDS instance. We’ll be able to get by for a while longer with straightforward optimizations, but unless we make some major structural changes to our app, we’ll hit a bottleneck at some point.

Also, the downtime for changing instance size sucks.

  1. Availability. We run a multi-AZ instance, so we should survive a single AZ outage. But RDS is built on EBS, which makes me pretty worried given EBS’s history and design.

  2. Price. Our RDS bill is 4x what we pay Heroku. I don’t mind paying Amazon to save me from hiring a sysadmin, but I would love to find something less expensive.

In my view, we have two options moving forward: the traditional approach (sharding, running a nightly job to move parts of our database to read-only, etc.); or a NewSQL solution (Xeround, VoltDB, NimbusDB, etc).

Traditional pros: It’s been done many times before and there are pretty standard ways to do it.

Traditional cons: It will take a lot of work and introduce significant complexity into the app. It also won’t solve the secondary problems with RDS (availability and price).

NewSQL pros: Supposedly, these solutions will horizontally scale our database without changing application code (subject to a few restrictions on SQL functionality like not using pessimistic locking). This would save us a huge amount of work. It would also improve reliability (no single point of failure) and reduce costs (not having to run an XL instance during off-hours just to provide for peak usage).

NewSQL cons: These solutions are relatively young, and I haven’t been able to find any good reviews or write-ups of people’s experience with them in production apps. I’ve only found one available as a hosted solution (Xeround), so unless we went with that one, we’d have to invest resources in sysadmin.

I’m wondering what opinions are as to what my best option would be.

Xeround is awfully tempting (hosted NewSQL), but I haven’t been able to find any good information use of it in production. The few tweets I’ve seen have been people complaining about it being a bit slow. I’m pretty nervous to move to something that seems so untested.

The conservative side of me says to stick with RDS and use a traditional approach. But it will be really expensive in terms of developer time.

And then part of me wonders if there’s another way, maybe a more battle-tested hosted NewSQL solution I haven’t heard of. Or maybe a NewSQL solution we’d have to host ourselves but that has a really solid history.

Thanks in advance for your thoughts.

3 Responses to “NewSQL versus traditional optimization/sharding”

  1. I hear, too, that NuoDB is interesting. One thing I hear is that Rackspace is coming out with cloud DBaaS sometime soon as well. I don’t know what flavor they’ll use, but you could see how Nuo works as a scalable solution with them. I think it’ll run in conjunction with the Open Stack platform, which, when they open it up, could be more cost and computationally efficient. Just something I’ve been eyeballing myself.

  2. Not sure if you heard about NuoDB yet. But it is a brand new SQL solution that offers the scale-out capabilities of NoSQL and the SQL & ACID compliance capabilities of traditional OLTP. You should take look at the solution.

  3. At Jingit ( we have battle tested VoltDB. It is fantastic on scaling write heavy apps and in AWS cloud. There is no hosted option so our devs own it and they spend < 1 hr a week administering our VoltDB cluster. We actually use both RDS and VoltDB. RDS for our traditional relational workload, and VoltDB for our HIGH VOLUME transaction processing. If you are developing in Java, VoltDB is a great fit as you write all the procedures in Java.

Leave a Reply