I've never understood this "don't obsess about performance at the start" rule.
Is it really that much of a problem spending a few extra hours prototyping different solutions and factoring in performance from the start. Why wouldn't I pick Vertx over Play if it 2-3x faster with similar productivity. That's money not spend on VPSs that can be used for marketing.
Likewise why wouldn't I look at a binary serialization format if it means my iPhone app starts up 5 seconds quicker. Users decide whether or not your app is worth keeping in those few seconds of starting the app. Why not spend time trying to keep them ?
Or as great as PostgreSQL is for single server deployment why wouldn't I look at something easier to manage, trivial to scale and a more agile schema like a DynamoDB, MongoHQ, Cassandra. Especially if I am a single person startup.
I understand about premature optimisation but it seems to make a lot more sense to properly consider your architecture from the start rather than deferring it. Or did we all forget Digg v4 ?
> I've never understood this "don't obsess about performance at the start" rule.
It helps if you've seen, firsthand, a startup wasting time worrying about trying to optimize stuff they don't even know if they're going to need to optimize.
> my iPhone app starts up 5 seconds quicker
That's something you know you need right away, apparently. It's probably a sensible investment.
> Or as great as PostgreSQL is for single server deployment why wouldn't I look at something easier to manage, trivial to scale and a more agile schema like a DynamoDB, MongoHQ, Cassandra.
Because there are some big tradeoffs involved in using those. And because Postgres is not at all hard to manage or scale to a point where you can probably afford to look into other solutions or hire someone to scale it. This article is, after all, written by an Instagram guy, and they seemed to do ok with Postgres in terms of scaling.
That's not to say you shouldn't think about it at all, but there are so many unknowns that it often makes much more sense to spend time doing market experiments rather than technology experiments.
Simply because it's never a case of spending "a few extra hours" when learning and evaluating a new tool. There are many "unseen" hours in troubleshooting, monkey patching, and re-evaluation. These are real costs that can be avoided by defaulting to known best practices (at the expense of ultimate performance).
Worked with friend on a new idea (while also trying new tech across the board), ended up in this:
http://simondlr.com/post/26360955465/dynamodb-is-awesome-but
When we got traction, we wanted to change databases, but because it was just a weekend hack no one wanted to spend time doing it and eventually killed the app. It was our own doing of course (choosing new tech to work with). But yes. Work with what you know when testing ideas, unless you also want to play with new tech.
Right. So then the mantra should be "pick the technology stack you know best" as opposed to "don't obsess about performance". They aren't the same thing.
"don't obsess" is much different than "don't consider".
It's dangerous to prematurely over-optimize, as the OP says, but it's never a bad thing to think about optimization. Otherwise you'll end up with crap.
As another responded mentioned, I think the key here is that word "obsess". I'm personally of the belief that the "premature optimization is the root of all evil" camp goes way overboard. Especially for web sites since getting a "reddit blackout" means thousands, maybe millions of people probably aren't going to bother about your site ever again. However, I personally allocate a certain amount of my time on getting a fast solution but when it gets into serious tweaking, it's time to decide if you're ever going to have a traffic spike.
I'd personally say, go as fast as you can without having to work hard for it. But as others have mentioned, picking something with a lot of options and not coding inefficiently (e.g. don't use associate lists for everything when you know this should be a dictionary).
I agree with you.. something else to keep in mind is that a lot of these startups have millions in funding too. If you're bootstrapping you can't afford to just rely on cloud services like AWS.
Picking the right infrastructure DOES matter if you're hosting it in 1 machine in your basement. It matters less when you got millions in funding, a team of developers, and machines with 32 GB of RAM in your disposal.
> something easier to manage, trivial to scale and a more agile schema [than PostgreSQL]
Not enforcing the schema for your data is one of the riskiest optimizations you can make; it lets any bug start irreversibly poisoning your data for however long it takes you to realize it.
The premature optimization is evil meme has just been pushed beyond reason, that's all. People blindly insist that it is cheaper to just throw hardware at it then waste "precious developer time" on making things fast. But in most cases, a few hours of time spent up front making sure you aren't doing horribly slow things can save you hundreds of thousands of dollars in hosting expenses. I have not met any developers getting paid $100k/hour.
>Or as great as PostgreSQL is for single server deployment why wouldn't I look at something easier to manage, trivial to scale and a more agile schema like a DynamoDB, MongoHQ, Cassandra
That has nothing to do with premature optimization, that is just about using unproven data stores because you didn't bother to read the label.
I would change "Don’t obsess about potential performance" to "Don’t obsess about potential performance except for binary roadblocks".
What's a "binary roadblock"? Something that, when it doesn't run fast, no amount of "tuning" could fix it. You better be obsessing over things like these or you'll be dead in the water as soon as that nasty roadblock manifests itself. Some examples:
- 2 proven technologies whose combination is not proven
- bad database structure that doesn't hurt until a query you never planned for
- bad program structure whose memory leaks don't show without an outlying case
- lazy code that blows up as soon as programmer #2 "enhances" it
- shit that works by accident until it doesn't work anymore
- sleepers than appear to run well until you try to scale
- structures aimed at incorrect user assumptions
Speed to market is important. Careful thought into what can easily become a roadblock is just as important.
If a bad database structure hurts and there's no easy way to finetune it, you can still just denormalize and carry on for quite a while until you figure out a better solution. Bad code leads to code debt, but code debt can be paid off incrementally. Code that looked like it might scale but doesn't will usually be spotted as your user base goes up – unless your business model is predicated on exponential growth there's nothing that can't be fixed in a couple months' time. Etc.
Instead of talking about "binary roadblocks", couldn't you just say "don't obsess about performance, but that doesn't mean don't think about it at all"?
As I like to put it, having the ability to optimize isn't considered premature optimization (IMO). Picking a technology because its fast might be a poor decision early on; picking a technology because it has the flexibility to allow you to make it fast might be a great decision.
To a degree, this common desire to know the amount of headroom available for optimization was an impetus for our web frameworks benchmark project [1].
Some readers of the results have asked why the tests are so trivial (although the multiple query test is anything but; and we are adding more computationally intensive tests for later rounds).
We chose to start with trivial request types to establish the high-water mark that the platform and framework set. Any given web application built on the framework can only expect to perform with higher latency and fewer requests per second than the high-water mark.
In turn, the higher-complexity request types provide some insight into the converge-to-zero point along the complexity axis. Prior to seeing the data, I think some of us intuitively assume the converge-to-zero point is lower than it actually is, using the all too popular refrains such as "well, the app is always waiting on the database anyway." Yes and no. Certainly database queries are generally expensive. But some frameworks can run twenty trivial queries on a gigabit-connected database and serialize the resulting list to JSON faster than others can serialize a single trivial object to JSON.
Choosing a framework that provides headroom for custom application logic--and having a rough idea of how the platform will perform with your custom logic as demonstrated by representative tests--is a good idea. And it doesn't take a whole lot of effort to do that kind of research now. It's easy to avoid premature scale pain.
The subject was 'ability to optimize' and you have changed it to 'how optimized the framework is already'. Most of one's ability to optimize is not dictated by anything about the framework.
Benchmarks which artificially exclude common optimization techniques are avoiding the interesting question of room for optimization in favor of meaningless pissing matches about 'language speed' or 'framework speed'.
Pekk, I appreciate the response, but I don't see it that way. I see an efficient framework as getting out of the way and not consuming too many CPU cycles for its plumbing business, leaving my application with room for me to code quickly and inefficiently then optimize when the time comes.
If I build on an efficient/high-performance framework, I can make sloppy code to start and get the job done fast. When the time comes to optimize, I have a great deal of headroom available. I see this as "ability to optimize."
If on the other hand, the framework (and, as importantly, the platform) is already consuming a great deal of the CPU's clock cycles for things outside of my control, my ability to optimize is greatly diminished. I will run into a brick wall unwittingly erected by the framework and its platform.
If my ability to optimize becomes a matter of replacing core components of the framework with marginally faster alternatives, the experience devolves into a frustrating guessing game wrought with arcane insider's knowledge ("Everyone knows you don't use the standard JSON serializer!") and meandering futile experimentation ("I wonder if using a while loop rather than a for loop would squeeze this request down to 200ms?")
I'd rather know that the framework is designed to give as many as possible of the CPU's cycles to me and my application. I can then be reckless at first and careful when time permits.
Which benchmarks artificially exclude common optimization techniques? If you're referring to ours, please tell us what is on your mind. You brought this complaint up in the comments on our most recent round [1], but didn't follow up to let us know what we did wrong. We are absolutely not interested in artificially excluding common optimization techniques. In fact, we want the benchmarks to be as representative of common production deployments as possible.
When you’ve got an idea, and a team, and you’re raring to get started on hacking for a potential startup, a question that often comes up is, “well, what technology should we use?” (hopefully you’ve answered the necessary, “what problem are we solving and how are going to solve it” questions first).
And hopefully, long before that, you've answered "Who cares about this?", "How will I make money?", "Is there actually a big enough market?", and so on...
Meh - I'd rather make something that's awesome for myself, then figure out marketing it so other people can benefit as well. I believe that thinking about a market and how to make money first is the wrong avenue to go down if you actually want to care about a product you make.
If I want it for myself, I'll make it. If I think other people will want it, I'll find a way to get it there - either by selling or giving it away. Maybe that makes me a bad entrepreneur, but it means I have more fun in the things I build.
You just described the difference between a project and a business. All businesses are projects, but not all projects are businesses. There is nothing wrong with creating projects for the sake of creating projects. But if you are trying to create a business, market and marketing come first and you should consider them way before writing your first line of code.
I still disagree. Just because something isn't business viable doesn't mean you shouldn't do it. You should pretty much always do it - then decide if there's a market/think about how to market for it.
A business for the sake of just making a business feels wrong. A business for the sake of making an awesome product feels right.
He meant that if your goal is to make money with a business, then you should start with businessy stuff like figuring out if there's a market for what you want to build.
You're free to start a business by building the product of your dreams, but no matter how much you personally like it, there's no guarantee people will pay for it. In other words, if you want money, but start with building a product, you're likely to have a bad time.
But if you just want to build an awesome product, and don't care about whether you make any money with it, then there's no problem.
You seem to be implicitly thinking of 'marketing' as pejorative.
I'd suggest instead that identifying an audience for a potential product is actually part of making an awesome product.
There's a pretty wide stretch of territory between not caring whether something is business viable and only caring about building a successful business.
Ultimately, the most important thing to keep in mind is that your “stack” shouldn’t define you.
While I agree with this advice in general, there are cases in which it is advantageous to allow your stack to define you. For example, Jane Street Capital appears to have established themselves as the place to work if you are interested in both functional programming and finance, greatly aiding their recruiting efforts.
Granted, they didn't start out using OCaml; their founders made a switch after the firm had already established itself. However, I wouldn't be surprised if a startup (where the founders have a specific skillset/background) could exploit a similar first-mover advantage.
Not to mention that without a lot of ACTIVE effort to make the stack NOT define you it will. The stack will strongly influence who you hire and who you hire will easily come to define you. A bunch of C/C++/Java/Rust/Go folks are going to have a different value set than a bunch of PHP/Ruby/Perl guys. Not to say their isn't overlap in people or ideas, but people who are attracted to similar technologies are often more self-similar compared to people attracted to other technologies. Depending on the problem space a given technology and the mind-sets that come with it by default may be or less helpful.
That may be more of a case of the stack reflecting you than defining you. The distinction I'm drawing is whether you allow external perceptions to influence your decision.
To try to be more clear, I'll make up an example. Say you as a founder are a Ruby guy, perhaps you value being able to rapid prototype, throw away code, iterate quickly.
Maybe your product needs to be very high performance so you hire a team of C programmers. What you might find is that their idea of iterating quickly is on a different timescale than yours, due to the low-level and unforgiving nature of C perhaps your team tends to be pedantic (by your standards) or correctness/detail oriented (by theirs).
In this situation your team, influenced by your technology choice, is bringing a different value set than your own to the table. BUT this is not (necessarily) a bad thing, if C really is be best tech for your product your team can strive merge both your and their values; however, my point is without you actively trying to do that, your tech just defined the value set, not you. (In this case it also possible that maybe Rust or Go would have been better tech, but that is a different conversation)
Agree. I wouldn't treat it as a rule. Normally it's the right thing to start with solving user problem, but I see many successful businesses built on trying to find application to the "new cool stuff". Github is one more example of the company driven by technology choice first.
Technology is not important for your startup. Choose whatever you know. Then go from there and fix it when you scale because you've found a market.
About time to market: From my experience (and Reinertsen [1]) time to market is not lost during programming but during the decision phase. Optimize your decision speed (reduce meetings, don't postpone decisions, have a clear decider, know what to decide and what not, ...), not your technology stack if you want to beat others to market.
I did have an offer for some years: Anyone beating a Java (the COBOL of the internet world) stack 5x to market, I pay him $1000. No one did take it yet.
I'd add one thing, though about an optimized stack for a startup with a lot of unknowns. I've attended a couple hackathons and such and seeing how much traction and how quick small teams are able to piece stuff together with a JavaScript/Node.JS/MongoDB stack is pretty staggering. Not sure if it scales far down the road or what issues people will run into, but having a single language across the entire stack seems to have some productivity benefits. Especially when you have a developer that has to work in two or three of the layers (UI/Server/DB) all at once...
There was some VC around here who talked about the advantage that startups have because they can choose some of these tools to quickly develop in and such...
Right now I skip from writing heavy JavaScript in the UI (AngularJS) to writing Python on the server. It's a context switch, how I iterate over collections, parse strings, etc. is totally different. I'm not in love with Node.JS, but love the idea of a single language/code constructs throughout the stack. Especially in a small-team startup environment. Maybe someone can come out with a synchronous server-side JS engine? :)
Just realized that it's also a context switch when I go from Python to Mongo, especially when I'm working in the Mongo shell.
I had similar thoughts seven years ago when Ruby on Rails was being sold as 10x faster to develop in....I wondered why no one would build my website for 1/10th the price.
When making decisions for a library, ask yourself:
1) "What kind of problem is this library solving?" "Can I solve the same problem in a simpler way with less code, and later adopt the library if necessary?" -- Dependencies don't always make your life easier.
2) "Does this library make certain problems simple, and other problems no less hard than before? Or maybe SLIGHTLY harder?" -- Last thing you want is to limit yourself to a subset of solutions. Until you know EXACTLY what the long-term goal is, marrying to something that ties your hands is a bad idea.
3) "Is it magic?" -- This is important. In a startup you don't have time time or resources to just sit down and grok a library. If its too magical then its better to avoid it because when something goes wrong, you will have to either sit down and grok it, or rip it out.
Other bonus points for good decisions:
Break everything, but have the most solid data model possible. Test the shit out of it too.
Most new ventures would do well starting with a mature CMS like Drupal, Joomla, etc. They'll get you well down the road to a complete website with the boring stuff like user registration, a built-in blog, templating, SEO, and so on already built in or readily available as an add-on.
I don't think it is so obvious that almost everyone should choose to base their projects on PHP. The only time it should be an obvious choice is if you have lots of PHP talent.
Is it really that much of a problem spending a few extra hours prototyping different solutions and factoring in performance from the start. Why wouldn't I pick Vertx over Play if it 2-3x faster with similar productivity. That's money not spend on VPSs that can be used for marketing.
Likewise why wouldn't I look at a binary serialization format if it means my iPhone app starts up 5 seconds quicker. Users decide whether or not your app is worth keeping in those few seconds of starting the app. Why not spend time trying to keep them ?
Or as great as PostgreSQL is for single server deployment why wouldn't I look at something easier to manage, trivial to scale and a more agile schema like a DynamoDB, MongoHQ, Cassandra. Especially if I am a single person startup.
I understand about premature optimisation but it seems to make a lot more sense to properly consider your architecture from the start rather than deferring it. Or did we all forget Digg v4 ?