Hi. I'm the author of that presentation, and I just got a text message from a friend saying that I'm on the front page of Hacker News...
The one thing in that talk that I was never 100% sure of was whether it was block-mq that provided the performance improvement. It wasn't until about a year later that I came across some articles which confirmed that that it was actually due to the development of, and subsequent fixes to, the Xen persistent-grants feature.
Yes and no. I have all sorts of test results for the 4.x kernel, but they are for i3 instances rather than i2.*, so they wouldn't be directly comparable. Your question kind of makes me think I should put together an updated version of this talk; I've gathered enough material over the last couple of years that would probably be useful to somebody.
This was my first thought, kernel 3.x is more than a little dated now and there is a huge amount of IO performance and latency related changes that have been incorporated since the 3.x days.
Warning: this is from a 2015 talk, but I still found it interesting as it has shown me a very good way to approach the problems described in the early slides.
I find this low-level optimization and performance tuning fascinating. Can anyone recommend any good resources to get started with solving these kinds of problems?
If only there were an AWS service that existed purely for solving these exact problems so that your most technically talented employees could spend time working on your product instead of dicking around with linux kernel settings.
At the time of making this presentation AWS did not have anything in their offer that could match tuned MySQL on i2 instances. Aurora was just getting started.
Not really. The migration of hundreds of terabytes of data from one storage solution to another is never a no-brainer, and it's not just because of technical concerns. RDS is a very good solution for a lot of people, but it's not the right fit for everyone.
OP spent multiple months playing around with the kernel and mysql parameters to solve the issue. What happens when they have to do the same thing for the next version of the kernel, or patches like Meltdown? RDS has entire massive teams dedicated to solving exactly this problem for OP's exact use-case.
The one thing in that talk that I was never 100% sure of was whether it was block-mq that provided the performance improvement. It wasn't until about a year later that I came across some articles which confirmed that that it was actually due to the development of, and subsequent fixes to, the Xen persistent-grants feature.
In other words, ignore slides 15 and 16. =/