I'd love to hear both of your thoughts! I'm considering durable execution and DBOS in particular and was pretty happy to see Armin's shot at this.
I'm building/architecting a system which will have to manage many business-critical operations on various schedules. Some will be daily, some bi-weekly, some quarterly, etc. Mostly batch operations and ETL, but they can't fail. I have already designed a semblance of persistent workflow in that any data ingestion and transformation is split into atomic operations whose results are persisted to blob storage and indexed in a database for cataloguing. This means that, for example, network requests can be "replayed", and data transformation can be resumed at any intermediate step. But this is enforced at the design stage, not runtime like other solutions.
My system also needs to be easily auditable and written in Python. There are many, many ways to build this (especially if you include cloud offerings) but, like Armin, I'm trying to find the simplest architecture possible so our very small team can focus on building and not maintaining.
As the CEO of DBOS I'm of course heavily biased, but I think DBOS is the perfect solution for you. It has everything you need (queues, crons, easy to understand pure Python). We'd be happy to help you through it if you get stuck. You can pop into the DBOS discord too (link on our webpage).
We're building a B2B digital quantitative portfolio manager platform for family
offices and hedge funds. Our platform allows portfolio managers to create,
backtest, test and execute systematic quantitative trading strategies. We are
currently closing our seed round.
We are looking for a front-end developer to join us IN PERSON at our brand new
offices in downtown Montréal's FinTech Station. The ideal candidate will have 3+
years of experience in React as well as a strong sense and ability for design.
Good general software engineering knowledge is also a must. The candidate will
work in a small team and own the front-end development, mocking, CI/CD, support
and maintenance. We have a very highly detailed Figma model and we are looking
for the right talent to bring it to life.
Interested applicants should contact me at mabelanger + hn at biasafe dot ai.
Please put our exact company name in the subject line. Please send a CV along
with a short introduction and I will do my best to get back to you ASAP!
I'm building/architecting a system which will have to manage many business-critical operations on various schedules. Some will be daily, some bi-weekly, some quarterly, etc. Mostly batch operations and ETL, but they can't fail. I have already designed a semblance of persistent workflow in that any data ingestion and transformation is split into atomic operations whose results are persisted to blob storage and indexed in a database for cataloguing. This means that, for example, network requests can be "replayed", and data transformation can be resumed at any intermediate step. But this is enforced at the design stage, not runtime like other solutions.
My system also needs to be easily auditable and written in Python. There are many, many ways to build this (especially if you include cloud offerings) but, like Armin, I'm trying to find the simplest architecture possible so our very small team can focus on building and not maintaining.
reply