The author's acronym is silly, but it's a real problem. Soylent liked to blither about their "infrastructure", for a product that sells a few times per minute. They could be using CGI scripts on a low-end hosting service and it would work fine.
Wikipedia is some MySQL databases with read-only slaves front-ended by Ngnix caches and load balancers. That seems to get the job done. Wikipedia is the fifth busiest web site in the world.
Netflix's web site (not the playout system) was originally a bunch of Python programs.
The article mentions a PostGres query that required a full table scan. If you're doing many queries that require a full table scan, you're doing something wrong. That's what indices are for.
In fairness to Soylent, their marketing revolves entirely around the perception that they're a tech company and not one of many meal replacement shakes. The problem they were solving might have been "Techcrunch hasn't written an article about us recently", which an over-engineered stack does actually fix.
I remember reading a scaling-out article from some startup. Some of the things felt a little over-engineered, some were impressive, some seemed wrong. But then they get to the point where they brag about their scale, and the metric they used was that they can handle thousands of requests.... per day.
This is hilarious and sad at the same time. However, most of these write ups are aimed at attracting talent. Even more, some tech stacks are deliberately built to attract talent when the core domain is just too simple or boring. "We serve user subscriptions and recipe data from an SQL database using Rails" just doesn't sound as snappy as the infra-porn on the blog.
Isn't that kind of thing a real red flag for the kind of talent you'd want to attract? If someone told me they'd built a GPU compute cluster for their phpbb based social club forum, I'd think they were an idiot and not want to work with them.
So you're thinking that kind of technical mark-missing is the startup equivalent of the typos and other glaring errors in email scams? They're to weed out the people smart enough to be a problem?
Absolutely. I'm running into this problem constantly in the front-end dev world. Most developers I talk to are way too eager to list off a million cool techs that they're familiar with, most of which are completely inappropriate for the job at hand.
If I'm interviewing a candidate, I'm not interested in which tools they know how to use. I'm interested in whether they understand the problem those tools solve and when to apply them. Over-engineering is problem #1 with the community at large in front-end dev at the moment and you'll go a long way as an engineering team if you make it your priority to defeat that mentality.
Oh my god. This checks off all the boxes for companies I would never work at.
Starting with this nugget:
>Our gateway is on the frontline of our infrastructure. It >receives thousands of request per day.
LOL! Wat???!!! Thousands of requests per day. Thousands. Not per hour or per second. Per day.
Call out the big guns, people! We need cutting edge enterprise-scale architecture! A boring rails app or a flask app will never be able to handle this kind of scale!!
I don't even need to go farther down the checklist than that. You're putting all of that complexity and infrastructure and problems-waiting-to-happen behind an API that serves thousands of requests per day.
Yeah, I'd rather be homeless than work on that team. Jesus.
Exactly what is wrong with encouraging more women and minorities into the industry and making them feel welcome at a company. I assume the ideal workplace for you is just a group of 20-30 year old, white males.
And in your "expert" opinion where did they go wrong here. What is the correct number of articles they should post on any particular subject in order to get the HN tick of approval ?
What? I was just remarking that their engineering blog seems to be more recruiting focused than focused on the actual technical achievements of the company. I was not at all commenting on whether those posts were good or bad.
I'd wager the vast majority of company tech blogs are maintained with exactly that goal in mind. And in my opinion that's totally reasonable. Done well it helps people understand more about the people, technical philosophy, and problems faced by that team
I'm not the op, but I would charitably assume that they meant that those blog posts demonstrate that the blog's strongly geared toward attracting talent.
How can it be a bad thing for developers to write about what they are working on ? Not all developers are genius experts who suddenly woke up one day and instantly knew how to perfectly architect a system. People learn, people grow, people make mistakes and most importantly people need feedback.
Blogs like this are fantastic to (a) learn what other people are doing and (b) for developers to have an opportunity to show off and be proud of what they are working on. Especially since the rest of the company in most cases isn't going to care less.
ergh, you weren't kidding. I hope that was a typo on their part; even low thousands rpm is mostly not a big deal on modern hardware.
Our gateway is on the frontline of our infrastructure. It receives thousands
of request per day, and for that reason we chose Go when building it,
because of its performance, simplicity, and elegant solution to concurrency.
Then again, if they didn't have that beefy infrastructure, they could get DDOS'd from up to multiple request per minute! I mean you can't be too careful when you are in that league!!1!
Also interesting (in the same way as rubbernecking a car crash) is going to their homepage and viewing source.
More seriously, you can set this up with KPIs in a day with Zabbix. A week, if you've never dealt with Zabbix, which is admittedly somewhat complex compared to some of the others.
But setting up a data warehouse that builds out the KPIs you're looking for along with a dashboard to display them (the easy part, for sure) can certainly take over a month.
Why did you have to type that. Now in my free time there's a 50/50 chance I am going to waste minutes or even hours looking into the viability of running a postgres/sinatra/nginx app on a phone.
Something about that appeals to me in a huge way. Showing people my "server room", etc etc.
I didn't want to share the original article, but you found it :) Then again, Google search doesn't turn up many other results bragging about thousands of requests per day.
At the other end of the scale, an acquaintance of mine is a sysadmin for a government department, and can't get their web framework changed to his frustration (politics...). He gave an example of the websites he has to look after, one being a boring legal website ("so you can imagine how low the traffic is already...") and the software could barely serve one request per second!
They have Kafka, ElasticSearch, some workflow engine thing, and they manage a distributed object store (Swift) deployment.
Apparently in Wikimedia's case they also have a stated company policy / philosophy around self-managed infrastructure that prevents them from using public cloud for a bunch of stuff (I've heard this from a former employee but can't find any further documentation about it).
This happens all the time on StackOverflow/ServerFault. "What sort of server cluster do I need? I have thousands of hits a month!" Usually the answer is "an average smart watch could do it".
It's simply, "think for yourself." Do you ever talk to someone on a topic (in technology or politics) and it feels like you're talking to list of commonly accepted prepackaged talking points?
Working on a migration from a site that modified the WooCommerce core to a new ecommerce platform. Couldn't believe WooCommerce ever did that when I started.
Wikipedia is some MySQL databases with read-only slaves front-ended by Ngnix caches and load balancers. That seems to get the job done. Wikipedia is the fifth busiest web site in the world.
Netflix's web site (not the playout system) was originally a bunch of Python programs.
The article mentions a PostGres query that required a full table scan. If you're doing many queries that require a full table scan, you're doing something wrong. That's what indices are for.