I think Anduril's mission is great but I'd never work for a man who directly funded fascism in my own country. Would be cool if there was a more woke defense contractor that wasn't captured by institutional rot like Raytheon and Lockheed
I chatted with someone on the language side of the project (I believe the same project) and it was fascinating how ambitious the concept was. I do wish it was finished or open sourced though
I’m honestly skeptical of the utility of the Fremont as the specs currently appear. Seems like not enough horsepower compared to the competitor consoles - the Steam Deck was so good because price to performance to battery life is still hard to beat and only now encroached by the Switch 2, it simply didn’t have a console like competitor.
Today, training in "low precision" probably means computing FP8 x FP8 -> FP32. The FP32 accumulation is still important, but otherwise yes this works, especially if we're talking about MXFP8 as supported on Blackwell [0].
What's less proven is a recipe using MXFP4 x MXFP4 -> FP32 compute, e.g. [1], which needs more involved techniques to work. But if you get it to work stably, that pathway is running at full throughput on 5090.
Interesting. My assumption was one of the innovations of DeepSeek and the modern GPT models was performing low precision pretraining rather than just finetuning further. I didn't realize you still need accumulation at a higher precision anyway
Even the large cloud AI services are focusing on this too, because it drives down the average "cost per query", or whatever you want to call it. For inference, arguably more even than training, the smaller and more efficient they can get it, the better their bottom line.
Yeah, I am using my Deck with 512GB sdcard and could never tell it is actually running from sdcard. It does a lot of game updates and always finishes those in reasonable time, at least for me. That card is going strong with all the writes going on on steam deck
UHS-I cards easily go up to 100MB/s. This is the baseline for a modern SD card.
After that you can add more pins for UHS-II. This is used in a bunch of devices and goes up to 300MB/s, but you can't assume anything will have it. And UHS-III was dead on arrival.
Alternatively you can add a different set of more pins for SD Express. It can do gigabytes per second and is probably the future. It's backwards compatible with UHS-I, but not II or III.
And also SanDisk made their own spec for overclocking UHS-I which some things support. It can do about 200MB/s.
The steam deck supports none of those upgrade paths. You get about 100MB/s.
Considering the insane tempuratures the cards reach, and that it destroys brand new SD cards, I don't want them going any faster until it works safely.
Why? The current system graduates kids from poor to not poor. Sure it is a small number, but it is one of the pathways for social mobility in our society.
College in general was meant as a way for people to rise up, and for my generation (early millennials) it worked. My first job in software engineering paid way more than my parents combined income.
Society has managed to mess that pipeline up, first through massive student loans, and now through just general unemployment.
But the system worked for a long time.
The Ivy leagues are something different. Society can only have so many "elites", or else they stop being elite and just start being irritating rich people. There needs to be a path for new blood to enter the elites, so feeder lanes exist.
This all worked rather well for at least half the 20th century, but recently the elites have gone a bit too far into the "eat the poor" territory, and society is starting to crumble around the edges.
>College in general was meant as a way for people to rise up
No, college started out as academic institutions of learning, not instruments of social mobility. It was never intended as a job training program, rather a place of academics to work together on a topic, and was heavily restricted to the aristocrats and elites from the start.
Alot of the problems here is stemming precisely from trying to use higher education for a purpose it was not originally designed to do.
Yes. University needs to be university. It's never been good at being other things.
People have completely forgotten why it mattered on resumes to begin with. Association with a university signaled an appreciation for philosophy, now it signals tolerance for administrative abuse.
Seems fair. Stanford isn't in the business of conferring education. It's in the business of conferring status.
Public funds shouldn't pay any tuition costs for anybody that attends Stanford as an undergraduate. I'll admit for PhD programs the benefit is typically via publicly funded research so I think that can stay. But it's absurd that a California taxpayer would fund elites that consider everyone else beneath them.
reply