Recent thread but focus on the db side:

https://news.ycombinator.com/item?id=23305111

This is really awesome! thanks for this :D

If you expect to continue to scale, the multi-tenancy should be in software.

One reason is overhead. Each instance of the system will have overhead, and as you scale to more and more customers, that overhead will eat up your profits. If it's a well-engineered multi-tenant system, you won't have this issue.

The problem with multi tenant software is the overall complexity which should also translate to cost . Also releases requires having to keep adding feature flags/config.

I'm a bit afraid that this may lead to a very brittle software/architecture in long term.

Both are valid. It depends on the footprint, type of SaaS (e.g. running complex offline-processing vs running heavily customized web-shops), your PII regulations and your ideal customer type and size.

It can be easier to have custom subdomains running separate instances, since this will limit how outages/upgrades/security affect customers. You also can intentionally keep a customer on an old version, which can be a bug or a feature.

But it can also be easier to have tenants, since you can update all users at the same time and you will pay less overall for infrastructure.

Having done both, I personally prefer tenants since it's easier to write, support and scale. But if you're B2B, you should carefully investigate the other option.

I currently have a mix of both. Mostly shared but some dedicated for only the ones who absolutely needs it (majorly due to regulatory restrictions).

However, like you said separate instances is easier to setup but I'm afraid of the extra management hassle of managing 500 different instances (including different micro-services, DBs, etc)

Multi tenancy has introduced a lot of config flags to enable/disable features for different customers - since they wont all agree to switch at the same time.

Both approaches have their pros/cons. I'd love to find a place to read about people's experience like you with both approaches.

You will need both. Cloud is for on boarding, enterprise is on prem. So build your cloud where ever and how ever, don't expect it to make money, prepare to deploy in all major clouds, and then take in the EE dollars.

Kubernetes is a great choice here.

Even on cloud - What do you think about running shared infrastructure (multi tenancy) vs running every customer on separate instance managed by us?

Multi-tenancy k8s is popular, but requires some protections to be in place and then you have to calc billing. Separate is easier for both security and billing, but requires more management you get in k8s. It also depends on what multi-tenancy actually means to your product, business, and customrts

In my case specifically, the product is logistics operation management software with different types of users like operations team, drivers, customer support etc. The product automates pickup, routing, delivery of packets etc. So it's an operational heavy product.

Is that 100k monthly or annually? Also how much data is that for each customer ?

$100k Annually per customer on avg. Around 3 months of our customer's Data is 200GB.

Just to clarify this is 200GB per customer.

so every 3 months you collect 200GB of data... do you need to keep that data around forever or like 12 months you'll have around 800 GB per customer... Maybe ec2 api to spin up specific customer db then... seems worth it... but also maybe look into ways to reduce that data size...