One of the things I’ve said I would do and never did is create a set of test suites for as many facts about a language as I know and then run it for every new release to see what I need to unlearn.
Most but not all of these were performance related. If it took a few days to run that’s fine. Major versions don’t come out that often.
Hallucinations are now plausibly wrong which is in some ways harder to deal with. GPT4.1 still generates Rust with imaginary crates and says “your tests passed, we can now move on” to a completely failed test run.
The USA hasn’t managed to completely impose their idea of intellectual property on everyone yet. Some countries you can’t sign away authorship even if you can commercial rights.
I am unsure if I fully understand your point, so let me ask a related question to see if I understand.
For many open source projects, there is a CLA (contributor license agreement) that must be signed before contributions can be accepted. The Free Software Foundation (which holds the copyright for most/all? GNU tools) is pretty in/famous for requiring it. Their reasoning: If there are copyright violations, they have the time and financial resources to pursue the violators.
Are you saying that these CLAs and their intended purpose are invalid in some jurisdictions? If yes, please share some examples. To be clear: I'm only interested in "normal/regular" jurisdictions that have at least accepted the Berne Convention.
Yeah the older Pilot Sport tyres wore out quickly. I had PS4 before and now PS5 and they are wearing at something like half to a third as fast? Very happy. Only slightly less grip in cold weather.
My understanding is the way Aurora DSQL distributes data widely makes bulk writes extremely slow/expensive. So no COPY, INSERT with >3k rows, TRUNCATE etc
I've been looking into PgDog for sharding a 40TB Postgres database atm vs building something ourselves. This could be a good opportunity to collaborate because what we need is something more like Vitess for PostgreSQL. The scatter gather stuff is great but what we really need is config management via something like etcd, shard splitting, best-effort transactions for doing schema changes across all shards etc.
Almost totally unrelated but have you had good success using pg_query.rs to re-write queries? Maybe I misunderstood how pg_query.rs works but re-writing an AST seems like a nightmare with how the AST types don't really support mutability or deep cloning. I ended up using the sqlparser crate which supports mutability via Visitors. I have a side project I'm chipping away at to build online schema change for PG using shadow tables and logical replication ala gh-ost.
I would love to collaborate. Email me: lev@pgdog.dev. Config management is a solved problem, we can use K8s or any number of CD tools. PgDog config reloading can be synchronized.
Best effort transactions for schema changes across shards are working today. Ideally, schema changes are idempotent so it's safe to retry in case of failure. Otherwise, we can try 2-phase commit. It'll need a bit of management to make sure they are not left uncommitted (they block vacuum).
Shard splitting can be done with logical replication. I've done this at Instacart with 10TB+ databases. At that scale, you need to snapshot it with a replication slot open, restore to N instances, delete whatever data doesn't match the shard #, and re-sync with logical replication. Another thing I wanted to try was using Pg 17 logical replication from streaming replicas. I feel like it's possible to parallelize resharding with like 16 replicas, without affecting the primary. In that situation, it might be feasible to just COPY tables through foreign tables with postgres_fdw or PgDog (my choice of sharding function was very intentional). Something to consider.
pg_query.rs seems to be mutable now, as far as I can tell. I've been rewriting and generating brand new queries. I like that it's 100% Postgres parser. That's super important. They have the "deparse" method (struct -> SQL) on pretty much every NodeEnum, so I think it's good to go for doing more complex things.
We used it to shard a database at Instacart after your time instead of the “replicate everything and drop the data on the shard approach”. That combined with dropping all the indexes (minus the primary key) worked well for the data copy/initial sharding.
Exactly that. It's an arms race between companies that offer a large number of residential IPs as proxies and companies that run unauthenticated web services trying not to die from denial of service.
