I completely disagree with you. The fundamental problem with your concept of open source is it goes against what open source really is. The ability for you to completely change what a piece of software can do. IMO, even with LLMs, models are "executables" and weights are "configuration". Yes, of course you can tune the weights by changing the values, but that's the most I can do. Can I actually add "features" to the model? Perhaps you "open-sourced" an LLM model trained on the United States Constitution. Can I change the model to then be a specialist in real estate law? Not with weights. I need it to learn case histories to extend its "feature-set". Without data and the mechanism to reproduce the model, how is this "open-source"?
Yes. You can use a number of libraries to add, mix, merge, etc. layers [1]
> Not with weights. I need it to learn case histories to extend its "feature-set".
Again, yes. You can add attention heads, other features, heck you can even add classification if you want [2]. Because you are working with an open architecture! What you think of weights are not binary blobs. That is a common missconception.
At first glance, that just seems like a bunch of libraries linked together to form a binary. That is not open-source. I completely agree with you that there is just not enough clarity out there.
For my education, following up with my earlier example, can I remove the layers that have references to all chapters / laws in the constitution except for the ones meant for real-estate? How would I do that with the approaches you mentioned here?
Fundamentally, if I have to "reverse-engineer" something, then it's not open-source.
I couldn't bother myself to read the whole article. Got GPT-4 to summarize the main points. Not as much insight as I thought I would get going in.
1. *Testing in Staging vs. Production*:
- Most engineers prefer testing in staging due to a sense of control.
- There's a misconception that it's an either/or situation between staging and production testing. In reality, both are necessary.
2. *Importance of Production Testing*:
- Staging environments can’t replicate all possible real-world scenarios.
- Production testing is essential to identify complex, real-world issues missed in staging.
3. *Uber's Approach to Testing*:
- Uber tests its payment systems in production.
- They have developed tools (Cerberus and Deputy) to facilitate transparent interaction with real systems and gather responses effectively.
4. *Every Deployment as an Experiment*:
- Every deployment is treated as a hypothesis to be validated against business metrics.
- Metrics and monitoring are crucial to determine the success of deployment.
5. *First Rollout Region*:
- Uber chooses a specific first rollout region to minimize risk and impact.
- Initial rollouts are conducted in regions that are small but significant for practical monitoring.
6. *Canary Deployments*:
- Uber conducts canary deployments to a subset of users to detect and mitigate potential issues early.
- This approach helps in identifying and fixing issues with minimal impact.
7. *Examples of Issues Discovered Early*:
- Uber detected significant issues with GooglePay during its cautious rollout in Portugal, which would have been difficult to identify in a staging environment alone.
8. *Philosophy on Software Quality*:
- True robustness and resiliency come from real-world usage and the continuous fixing of encountered issues.
- Only production can provide the real stakes and conditions needed for thorough validation.
9. *Author and Newsletter*:
- Alvaro Duran, author of “The Payments Engineer Playbook”, emphasizes the importance of sharing and learning from real-world experiences in payments systems.
- Encourages readers to engage with the content and share it with colleagues for broader impact.
Fair question. My motivation was mainly to understand if there was something specific that drove that choice. Better question for you maybe: For any new project, which one would you choose and why?
Since I'm not starting a new major project, it's a pointless question to ask as I'm not going to research what I should use. But you completely ignored my question, why do you choose Postgres?
