If you're concerned about operational and cost efficiency, keep a thought in the back of your mind that one day your scale would need you to host things yourself. Your design choices should work with you at that point rather than lock you in some proprietary platform. But, the key is to get to the point where that becomes a problem. That day is not today. Azure VMs, AWS something whatever, get to your product features and get yourself to a spot where you need to worry about scaling and the savings it would bring.

As someone who is freelancing (and using Hetzner-cloud + dedicated) and also working as a cloud-ops (using AWS and mainly Azure as cloud providers), I can say that going with a cloud provider like Azure is simpler.

Saving cost is definitely important, however, seeing that there are multiple dev/salaried, I doubt it's really a big concern (you also don't seem to have that many resources).

Most of the time, with resources from Azure, you would get features already completed and existing (I am looking directly at Application Insights for example) - with resources like App services, Web apps and Function apps you get ease of debugging (yes, there is a cost associated, however, in the long term when you scale, you definitely see the advantage).

I've personally suggested my colleagues to try out a dedicated server from either Hetzner or OVH (that specific one needed at least 128GB RAM) so going with Azure would of been quite expensive (and probably less than $200 per month from either OVH or Hetzner) - but they've decided to stay with Azure and I can confirm that they are not paying less than $900 per month.

It seems like cost isn't always the main factor, but rather ease the time it would take to prepare/set up everything.

We're working on a solution to gives you an experience like Heroku on AWS without breaking the bank, and I'm pretty sure we won't be the only one working on such a solution.

Flexibility of cloud and simplicity and convenience of services like what we're building will boost the speed massively, plus you'll never be concerned about outgrowing the solution as everything is built on top of AWS.

Sometimes it does outright, see Heroku, or AWS Lambda.

Sometimes you try to bite more than you feel comfortable to chew, and get bogged down configuring kubernetes or terraform when you don't need as much.

In the simplest case, I think, the cloud is also an inevitability. Can you imagine starting up by buying some used servers, colocating them in some datacenter, calling an operator when they need tweaking a BIOS setting, scheduling a visit and downtime when you need to replace a failing NIC, or a degrading HDD? This is what life without the cloud used to look like. Yes, a EC2 instance is cloud.

Of course you can try a middle ground, dedicated hardware managed by a provider, e.g. what Hetzner offers. If your load is very stable and predictable, your stack is very simple and mature (an RDBMS + Apache + PHP, ore maybe Erlang / Elixir), and you are ready to either keep a hot spare or sustain rare but prolonged downtimes, you'd be fine with a small amount of dedicated hardware.

A large amount of dedicated hardware may be economical, but only at quite large scale (see, well, Facebook).

Also failed PSUs or hard drives don't require downtime, since those are hot-swappable and redundant in servers. If you want to optimize for (physical) hands-off operation you can even add an extra 1-2 hard drives that are configured as hot spare that the RAID (be it hardware, software or ZFS) will automatically use in case one of the active ones fails.

And the performance difference between dedicated hardware (be it dedicated server or your own) can be quite staggering compared to what you actually get from the cloud providers.

Spinning up a new VM takes a few seconds, just like on Azure, and ordering a dedicated server is only a click away if you really need it. So you get the best of both worlds.

I don't believe that you can be "too small" to save on infastructure. Hosting our setup on Google Cloud would probably cost 10x what we currently pay for, maybe more considering that our largest server has 32 physical cores...

[0] Blog article about our setup: https://pirsch.io/blog/techstack/

If you're just running plain Linux VM's running some software without need for additional AWS-like services or fancy orchestration, it's good value for money.

I also have a couple of dedicated Hetzner hosts - also remarkably stable, but it will of course take longer to rebuild if it goes down. You should have a continuity plan in place.

None of your dev team has a background in sys admin or managing a DB? You're going to be in for a period of difficulty while you gain those skills and that's going to be felt quite heavily on a team of 5. Now if only one of you picks up that skillset now you've also got a high Bus Factor (1) in the event something goes wrong with your self-hosted infra.

It's entirely possible that you're right in the long term but in the short term it doesn't make sense because your team is too small and the cost difference, while probably proportionally large, is almost certainly tiny in absolute terms.

I would explore the possibility that you want to be able to easily move off of Azure in the future. This may or may not be worth planning for but it likely makes more sense than going to self-hosted right now. If you can get buy-in that you shouldn't be on Azure long-term then you should avoid Azure specific services that will make it harder to switch. This would mean keeping your stack simple - compute and DB where possible. Avoiding things like Event Grid, their PubSub product and possibly even their CI/deployment offering.

Again - even doing that very possibly does not make business sense and you shouldn't approach that analysis with a specific conclusion in mind. Does your product need something like their AI offerings? If so it's probably a lot faster and more effective to lean into the products they're offering than trying to run an alternative yourself on their compute.

(1) - https://en.wikipedia.org/wiki/Bus_factor

For most B2B businesses offering a SaaS product, I don't think on-demand variable "web scalability" is the decision framework for AWS/Azure/GCP.

Instead, it's really about faster product development iteration. If new SaaS product features can be delivered to market faster because it can leverage many of the higher-level managed services in the AWS/Azure tech stack portfolio like DynamoDB or Cosmos DB -- without your 5 dev team building the equivalent tech stack from scratch, that's when paying the profit margins to AWS/Azure are worth it.

In other words, your CTO and the devs have to look at your future product roadmap and see which features would require extra developer costs in re-inventing aspects of AWS/Azure that may negate the cost savings of a "dumb" IaaS like Hetzner.

If there are actual development costs from moving from Azure to dedicated servers then they've already coupled to Azure and at a very early stage. The point of giving credits paid for for Microsoft. It would also seem they've unknowningly coupled too which should be worrying. I would be very worried about a CTO who unknowningly coupled to anything.

your aws infra: 600-1000 month? hetzner: 200 per month?

Is 800/month gonna make a difference to your organization?

Imagine you launch new feature with performance regressions and suddenly your database is at 100% CPU usage

Managed database service will allow you to scale up the instance in a few clicks, so you can redo the feature with a better plan, instead of rushing to roll back or deploy a hot fix.

in these 258 euros I have 4 dedicated servers that never went down so far. One of the 4 is only for the database while the other 3 are for kubernetes which is managed and deployed using Rancher. Out of the 3, my cpu usage is 4 out of 28 so plenty of room to grow as everything is on overkill.

I do daily backups of the database and upload them to an S3. I know this is not the perfect solution but, for a company that small (mine is similar) $61k are not something to be ignored. I used that money on Adwords which brings much more value than spending huge amounts on something which rarely "shines" for my use case. In fact, I had more downtime in the cloud than I had with Hetzner - mostly because they perform updates (to kubernetes, to database engine etc) or they simply have outages.

Everything is subjective.

not the same as VM to dedicated

In the early stages of product development, how often does this scenario occur where your database is at 100% CPU utilization and the solve is not a new database index but instead a hotfix?

AWS bill is 5k plus per month

One deployment lead to regressions (^2 growth growing along with usage of the new feature)

When it became a problem I just upgraded the instance and made a ticket

the ticket was solved within next sprint, so 2 weeks or so minimum

Hetzner provides cheap and performant VPSses. Depending on the geo, it might be a better fit.

CTO says they have to expertise in "systems administration", yet they're running VPSes, and not managed services. This is exactly where some dev installs some software which is accessible by default. There you have your little dataleak.

So imo, CTO is wrong, but post doesn't have to be right per se.

5 small VMs, 2 big VMs, 1 managed db

To me looks like your are trying to solve a non-problem.

If you don't have sysadmin that could tackle dedicated hosting but you have knowledge to keep things running in cloud - stick with the cloud for now.

If it was my money, yes, dedicated FTW. Egress charges... nien danken.

If it's you boss' money, what's the point arguing it. Just let him have his cloud bills.

Don’t argue with him, provide numbers for the perspective he might not see. Tell him you want to help him make the best decision. He might loosen up on budget for cloud if he see that you are fumbling around to save a few Go of RAM.

Even a spreadsheet with such numbers is primarily speculation and your upper management knows this (they make such things all the time). As such, if they did not want to agree with you before this spreadsheet won't make a difference. If they want to agree but don't want to take the time to present exactly that spreadsheet to their compatriots, it is worth making it. If you don't work for a very large company (essentially a bureaucracy), this situation probably will not occur.

Also, while cloud pricing can be daunting, once you reach a point when Azure costs to much, you can look at optimizing things. There are a lot of techniques to keep cloud cost acceptable. Look up "FinOps" if you haven't already.

Your solution is to spend time and money on reducing costs. This only makes sense when the costs to move outweigh the costs to reduce costs.

- You ignore resource usage and move to dedicated hosting. You are now wasting money on inefficient processes. To remedy this you apply software engineering practices to understand the model and design it correctly.

- You ignore server unit costs and stay on cloud, but improve your runtime characteristics. You are now wasting money on the hardware, but your software is efficient. To remedy this you have to learn how to manage infra and take it on as constant maintenance.

It seems on your team it makes more sense to me to just pay for the cloud and stay in your software wheel-house.

But it sounds like your boss has the idea cloud is where it's at, it's hard to shift. I'd put numbers on it like how many less connections the cloud he wants to pay for can handle, vs the hosted solution for less $$$.

There is a line between premature optimization and total resource ignorance.

Also, you can use vms from other vendors. You don't have to go full dedicated.

You have two small prod servers. Yes, other cloud services would be cheaper but it's probably not worth the migration time. Stepping on dollars to pick up pennies.

In this thread, we have a lot of technical people who would want this to be a technical decision. But it is not. It's a business decision and has long term consquences often lasting longer than the technical staff who made them. The reason cloud providers give credits is for vendor lock-in. It's an entirely apporirate decision to want to avoid vendor lock-in and when people are selling technical products to companies they sell the business value first and the technical value second.

The business decision here is do you want to spend time and money just now on moving or do you want to spend time and money later on. If you want to avoid vendor lock-in go for a cheaper hardware solution then you need to spend time and salary money on setting it up. If you want to stay with Azure then from what I've read in this thread from what you've said you're going to have to spend money later on for resources and you're going to have to spend time and money later on managing the costs. This seems literally like technical debt.

While it may be frustrating to technical people that a business person is making a decision that effects how they work this happens in every other industry. Business people decide the vendors, they decide the materials, etc all based on business reasons.

Another thing to consider, as a developer, is how to make your app as independent from vendor requirements. For example, if you build an app in such a way that it only works using SQS you’ll run into issues down the road.

Also, why y’all using dedicated dev VMs? IMHO it sounds like there are other issues to solve before you migrate the app somewhere else. If I was in your position I would work on making the app easier to setup and deploy. Your CTO has a lot to consider and being a CTO at a startup isn’t an easy thing. How long does it take to deliver a feature to customers? How can you help speed that up? What are the engineering goals of the CTO and how can you help with those goals? Make his or her life easier and work with them. Continuing to push an issue like this will cause you frustration and make your CTO dismissive of your ideas and suggestions in the future.

How many companies actually hit this stage? I can only think of a few, and usually it's because they have very specific hardware requirements (e.g. Dropbox's whole business is file storage, or if you're doing something that requires tons of GPUs).

In practical terms you should expect to never hit it. The point where such scaling (also the main value add of aws/azure) really matters and you start looking at an entire DC to lease, you've arrived in the realm of speculative fiction. You should not plan to get there, just as you should not plan to get a winning lottery ticket.

Also, it’s about priorities and goals of the company. Security and control is the main reasons I see companies migrate to data centers. Generally things like GitHub Enterprise are being used.

For the sake of security and performance you're better off using azure (especially the hosted database).

Otherwise you'll need to configure iptables, backups, software updates, high availability (assuming it's a requirement) all by yourself. If you don't have experience in these things you don't want to be learning how to do them with the thing that generates your revenue.

To be quite frank: Both hosted and cloud have a learning curve. The difficulty is about the same.

Furthermore, many businesses shoot themselves in the foot worrying about scalability too early. Are you really going to jump up to Google scale overnight? (No one does. Chances are your software has bugs that will prevent this, even if you are running "in the cloud.") When your business is well-run, your scalability needs will be predictable enough that you can provision hardware in advance.

That being said: I really like Azure. If you want to "save money for the company," focus on making sure that your software is reasonably efficient. This way you can pay for 10 servers instead of 100 servers.

Ignoring the weird dev VMs. The setup you have at the moment is tiny so cost isn't the issue here. In terms of dedicated you'll likely have to buy that for a least a year and so I don't think fully dedicated is the correct move here.

There are smaller cloud providers that'll give you those 2 prod VMs at a fraction of the Azure price but again price really isn't much of a factor here so I think staying on Azure is the right move.

I think personally you should look at moving those 5 dev VM's locally i.e. not in the cloud. It's really strange to have that many dev VM's.

Make more sense for that to have been 1 per each dev I still read that as 5 for each dev and there are 5 devs.

In that case I would say definitely dump them as they are pretty much useless then you are down to 2 VM's which doesn't really matter how you are hosting to be honest.

For instance, both my postgres master and replica went down one day the other weekend, because another machine in the same rack popped the breaker. In my specific application, I can cope with a once-in-a-blue-moon blip like that -- Hetzner saves me literally thousands of dollars per month in compute and bandwidth, I like doing [a limited amount of] sysadmin, and it's worth the trade-off. But, how do you think your CTO is going to feel about this move when all of your dev VMs go down just before a big customer deployment?

I use Hetzner dedicated servers for some non critical project, I find it fun, but it’s time consuming and requires some knowledge.

Just one example, having an encrypted storage on a dedicated Linux server is not trivial.

If you don't trust your cloud provider enough to let them manage your encryption, you should probably not use their services. If you use an cloud provider from USA, you should trust the USA too as they can access your data without letting you know. It probably applies to other countries too, but the big cloud providers are from USA.

However with dedicated servers the next step is more complicated for both the hoster and the potential other attackers, since the only chance they have here is to catch the key material during reboots, which is quite a bit more noisy than silently extracting the key from ram of a running VM (especially since they would need to mirror the disks or other complications to have access at the same time as you are using the machine).

If it's a colocated machine (or if the hoster trusts you enough with their dedicated machines) one can use TPM/Secure Boot to make the MITM attack very very difficult (since one can use something like dropbear to have an encrypted and authenticated connection to enter the encryption credentials and TPM/Secureboot can prevent manipulation of the bootloader).

The theft scenario is exactly the reason why you wouldn't want to trust Azure to do the encryption for you, since they can just extract the plain text data from the virtual machine. Unless you are saying you don't trust Azure datacenter techs only?

And for preventing the leakage from old storage devices very non-fancy and simplistic encryption setups are good enough (most of the complexity comes from trying to prevent/detect MITM or manipulation by the hosting provider).

Yes, and also the local datacenter people. I understood that Microsoft didn’t build a datacenter in every region and rents some space in existing datacenters managed by other companies.

Everyone seems to be missing this part. This seems nuts. Why is each dev getting a dedicated VM?

I've been working at companies that didn't and people had to run N services, rabbitmq, mongo, postgres locally just to test some stuff.

Add this zoom on top and your CPU will hate you.

The model I tend to favour in general is to have service being edited running locally and let it talk to services running in a dev environment.

https://youtu.be/k3bUvZf89k8?t=697

Keep everything in docker + some cloud agnostic orchestration layer so it's easy to migrate to other clouds (or even to dedicated).

I have experience with all of the clouds and I'm never going to use one. They are ridiculously overpriced and they get way more expensive the bigger you are. What other arguments do you need? I've seen mediocre startups doing the same and ending up with a 1M/y AWS Contract while pulling very little money - and eventually scrambling to move everything to physical servers.

If you're going to be a unicorn, it may be worth to pay lots of money to the cloud because it will be a tiny fraction or your future profits - if you're unlikely to hit that, just roll something up on Hetzner. It's not that hard, and literally what people did for a decade before AWS. I've never been purely a devops guy but I've been rolling my own servers for 15+ years; I understand not wanting to deal with hardware failures (they happen and you'll waste lot of time) but a dedicated / VPS is as hard as running things on AWS EC2.

That said, it's a lost battle.

Most technical leaders are dead on the cloud and some even think they're saving money. It's the same with scrum, microservices, performance not mattering or other nowadays myths.

That's the beauty of containers right there.

Eventually you'll end up with a Kubernetes cluster and a bunch of Helm charts either way.

Having a self-hosted server eliminates a lot of these questions. For hosting in production a managed cloud VM might be sensible as these services often provide additional functions like doing health checks, automatic renewal for tls certs, protection against attacks, etc..... Sure, you can get that on your personal server too, but it does need maintenance.

For development/testing/staging I prefer to have a non-managed server. That said, for most applications it is enough to keep the system of the host up to date and that is easier today than ever before and you might even be faster here than large cloud providers who have a bit of a momentum until problems get addressed.

Also cloud providers do change their infrastructure and might force you to adapt. That can mean extra work out of the blue. They tend to leave your systems as they are, but if you need to update certain components you have to read up on their docs again.

I'd challenge the idea of dedicated infrastructure per developer in the first place. There's little reason why this could not be done locally on a developer machine using Vagrant, local Kubernetes, Docker Compose, whatever.

It makes total sense to use the Cloud then, especially if you already have some experience there.

> Our use case does not really need instant scaling all that much since we're a B2B business and know ahead of time how much compute we will need.

If you had the expertise in the team, going with VMs + dedicated on Hetzner would have been a good idea, though.

I started with a dev team of three: One knew Linux admin stuff pretty well, I knew enough to keep the lights on, neither of us had proper cloud or container experience. We've done pretty fine with our Ansible managed Hetzner setup so far.

Especially if there are few moving parts (E.g. just HaProxy, a Postgres instance + your API) and your dev laptop has the same OS as your servers, I'd say it can be a lot easier to work with than some cloud provider.

A lot of it depends on what you're actually doing, but for a B2B SaaS app you could potentially make an argument around vertical scaling. Nowadays you can spin up a 128 vCPU VM with 2TB memory on EC2 in minutes, so why bother having lots of separate smaller servers? I've seen people making this argument quite well recently, especially in relation to using things like SQLite instead of a networked RDBMS. But.. I can't see you making such an argument very convincingly unless you're heavily into ops or play with this sort of stuff on the regular, so I'd stick with your CTO's approach for now and keep alternative ideas in your pocket for hackathons, spare time, or prototyping sessions.

For 3600$ [0] per month. Spending this kind of money makes sense if plan B is to spend man-years optimizing your app or if you have an insanely expensive per-machine license, but otherwise you are better off with multiple servers - not only for price, but especially for availability.

[0] Assuming a c6a.32xlarge in us-east1, with only 256GB of RAM. Dedicated machines are more expensive.

However, something I didn't learn/internalize until later on in my career was that not everything is your call. In this case, you absolutely should defer to the CTO's judgement because, you guessed it, it _is_ his call. He is the one ultimately responsible for these decisions both in authority and blame. If hes "wrong" and you are spending more than you should then thats on him. But being in his CTO position has earned him that right, you should respect it.

Your CTO is smart.

But if you use the cloud carefully, selectively and avoid vendor lock-in as much as possible, you will be OK and it will be not hard to move to Hetzner (dedicated) incrementally later.

If you have more significant compute needs a dedicated server can start to make sense. Linux admin is becoming a bit of a lost art in this container-everything world but actually it's not that hard. I run a hardware server for a lab and once it's running there's very little work, super reliable, and very cost effective.

If you aren't paying the bills use Azure. I do pay the bills for my product and use Hetzner.

Regarding production VMs: How much do you expect to save by moving to self-managing everything? I’d expect a few dollars saved for the hardware, but a huge amount of time spent setting everything up and documenting it properly. Net loss.

Let me tell you, I host a B2B app on Hetzner because I cannot use typical cloud providers due to GDPR, and I hate it. So much fiddling around to get everything working and I still don’t have high availability because I don’t have the expertise in that area. It’s a huge time sink and takes significant time away from actual development.

Conclusion: Use your time and money wisely. Do what speeds up adding value to the business. Saving a few pennies but missing out on growth and recurring revenue is a bad deal.

If you are selling to enterprise / businesses AWS or Microsoft are much better places to be reputation wise I think.

https://aws.amazon.com/compliance/programs/ To get a feel for the paper pushing exercise they’ve spent time on.

BTW - I put a large memory instance on ECS anywhere, worked great - if you have very high memory needs something to consider - a basic dell gets you 3TB these days - very nice for development