> there’s a little, but not a lot of insight into why an LLM does what it does.
That's a "black box" problem, and I think they are some of the most interesting problems the world has.
Outside of technology- the most interesting jobs in the world operate on a "black box". Sales people, psychologists are trying to work on the human mind. Politicians and market makers are trying to predict the behavior of large populations. Doctors are operating on the human body.
Technology has been getting more complicated- and I think that distributed systems and high level frameworks are starting to resemble a "black box" problem. LLMs even more so!
I agree that "prompt engineer" is a silly job title- but not because it's not a learnable skill. It's just not accurate to call yourself an engineer when consuming an LLM.
If your SSD is near its max-capacity, then any extra wear has a bad affect on its longevity. But modern SSD’s handle excess writes very well if they are not near capacity.
A few extra GB written to disk daily is a drop in the bucket in an SSD’d TBW rating, no??
I’d say for a casual user with low storage needs, it’s perfectly fine. Otherwise it’s a bad idea imo.
What's telling for me is that SSDs have been a readily available consumer part for around 15 years now, a default option in PCs for quite a while now, and to my knowledge there hasn't been many tales of SSDs dying (specifically for write endurance or otherwise) beyond occasional bad models like the old OCZ vortex2s. Even early torture tests were finding that you'd need to push around 2PB of writes (on smaller drives than we have now) to get failures, and that was on a sample size of 1 for each model. I wouldn't expect a SSD to die more than any other electronics.
I've got a few dozen tales of SSDs dying in machines I've managed. Some dying slow deaths with lots of bad reads, some locking themselves in a read only mode, some just disappearing from the system.
Wear leveling spreads the wear out. If there is no free space, it can't do that, and you're completely screwed.
The problem with swapping is that SSDs are fast. If you have 8GB of RAM and manage to pick up any workload with a 10GB working set size, you're short 2GB, so the OS will have to put 2GB on the SSD. But your working set is 10GB and now only 8GB is in RAM, so it needs that 2GB back immediately. To do that it has to swap out some other 2GB, which it also needs to have back immediately. The result is that your SSD is the bottleneck and ends up maxed out doing writes.
NVMe SSDs will do something like 4GB/sec. Not a few GB a day, a few GB a second. A 256GB consumer SSD that can handle 100 full drive writes over its lifetime can thereby hit its lifetime wear rating in just two hours. Under ordinary storage use that doesn't happen because you're not maxing out the drive for hours on end -- after all, if you were storing ordinary data, writing at 4GB/s would cause the drive to be completely full after only 64 seconds.
But swap is deleting stuff and overwriting it and deleting it again. In a pathological case it could burn out a brand new drive in an afternoon and in more realistic cases could plausibly do it over a few months.
> But swap is deleting stuff and overwriting it and deleting it again
I'd imagine modern systems wouldn't bother deleting a page in pagefile unless modified, but I don't actually know. Why would one bother deleting pages from the page file unless you have to? After all, if you're swapping a lot, there's a good chance that page will be evicted again and need to be copied back anyways. Leaving it there gives a chance you don't have to actually rewrite the page, assuming it was unmodified. You then also don't have to spend the time doing deletes on all those unmodified pages.
Obviously, your percentage of modified pages to unmodified pages during swapping would be highly dependent on workload. But I imagine a good number of workloads have a lot of stuff in RAM that are somewhat static.
> I'd imagine modern systems wouldn't bother deleting a page in pagefile unless modified, but I don't actually know. Why would one bother deleting pages from the page file unless you have to?
They presumably use TRIM/UNMAP on SSDs when they're done with it because you'd otherwise have pages in the pagefile which are unused but the SSD can't erase to reallocate. Also, to keep track of whether a page has been modified after it has been swapped back in, memory writes would have to generate page faults just to allow the OS to mark the page as dirty, which is slow.
> Obviously, your percentage of modified pages to unmodified pages during swapping would be highly dependent on workload. But I imagine a good number of workloads have a lot of stuff in RAM that are somewhat static.
Memory reads still cause swap writes because a page has to be evicted in order to make room in RAM for the one being swapped back in, even if neither is being modified.
Also consider what would happen if pages were kept in the pagefile even after being swapped back in. You have 64GB of RAM and a 64.2GB working set. If pagefile pages are reused after being swapped back in, your pagefile is 0.2GB. If pages are left in the pagefile after being swapped back in, after one pass over the working set your pagefile is 64.2GB.
> because you'd otherwise have pages in the pagefile which are unused but the SSD can't erase to reallocate
> Also consider what would happen if pages were kept in the pagefile even after being swapped back in
In Windows, the pagefile doesn't grow and shrink based on you reading and writing pages. The pagefile.sys is usually a fixed size managed by the OS, and it will usually be several gigs in size even if you have say 100MB of "active" pages in the pagefile. On this Windows machine I use right now, the pagefile is set to ~25GB. pagefile.sys is this size regardless of if there's only 5MB of active pages in it or 20GB of pages in it. The pagefile.sys is the size it is until I go and modify the setting.
In Linux, swap is often a dedicated partition. It isn't going to shrink or grow based on its usage. And generally, a swapfile cannot be dynamically shrunk while online.
In fact, while there is an option to enable discards on swapon, it seems it doesn't always actually increase performance. I've been seeing a lot of debate with a lot of suggestions to not enable it.
> memory writes would have to generate page faults just to allow the OS to mark the page as dirty
The OS already knows the page is dirty and would need to be rewritten to properly swap out the page. It wouldn't have to always go to update the page in the swap/pagefile immediately on being marked dirty, only when it wants to swap out that page. In Windows, it reports this memory as "Modified", with the description of "memory whose contents must be written to disk before it can be used for another purpose". Meaning, it knows these pages are dirty, but it hasn't bothered touching the pagefile.sys yet to sync the page, largely because it doesn't have to as my machine still has plenty of free memory.
> Memory reads still cause swap writes because a page has to be evicted in order to make room in RAM for the one being swapped back in
Assuming it always deletes when a page gets read in. If both pages have previously been swapped, the page isn't dirty, and it doesn't just automatically delete on paging back in, it wouldn't have to do a write. Once again, significantly increasing performance. This is precisely why you wouldn't want to immediately delete the page from swap, because if the page is going to go back and forth between active memory and swap you might as well just leave it there.
You're also assuming that reading the page back into memory from swap inherently means another page has to be moving to swap. But this isn't always true; maybe some page was swapped out during a period of memory pressure, but now the system has more available memory. In this case (which probably happens often), reading that page back wouldn't require pages moving to swap.
Reading more about swap, it sounds like modern Linux uses an LRU caching strategy for memory pages in the swap cache. It doesn't explicitly go around deleting pages from swap unless it is going to reuse the page, or one has enabled discard=pages.
> When the reference count to the page finally reaches 0, the page is eligible to be dropped from the page cache and the swap map count will have the count of the number of PTEs the on-disk slot belongs to so that the slot will not be freed prematurely. It is laundered and finally dropped with the same LRU aging and logic described in Chapter 10.
Neither of those assertions is correct. You personally may have a workload which requires more RAM, but there are many people – even developers – who have direct experience otherwise. macOS is notably more memory efficient than Windows and the M series hardware has efficient compression, and that configuration holds up fine for the usual browser+editor+Slack+normal app usage which a lot of developers have.
SSD wear is a concern, but they aren’t using low-end components so you’re looking at 5+ years of daily usage. I used an 8GB M1 for years and when I upgraded to an M3 there was no indication of SSD wear either in measured performance or the diagnostic counters.
> You personally may have a workload which requires more RAM, but there are many people – even developers – who have direct experience otherwise. macOS is notably more memory efficient than Windows and the M series hardware has efficient compression, and that configuration holds up fine for the usual browser+editor+Slack+normal app usage which a lot of developers have.
Sure, it's physically possible to use a machine with 8GB of RAM without running out. If all you do is open some terminals and a single-digit number of browser tabs to well-behaved websites, 8GB is an ocean.
But that use case is the exception, not the rule. Worse, ordinary people don't know what causes it. If you're a developer and your machine is sluggish, you know enough to realize it's because it's swapping, and in turn to know that it's swapping because you opened up some ultra-high-res NASA images in an image viewer and forgot to close them, or because you have the tab open for that awful news website that will suck up 20GB of RAM all by itself with its ridiculous JS, or simply because you have 10 different apps running.
For most people, all they know is that their computer is slow -- which it wouldn't be if it had an adequate amount of RAM.
Meanwhile, because they don't know what causes it, they don't know what to do about it, so they just suffer through it. Which has the machine continuously swapping, which is what wears out the SSD.
But in your example the OS should be smart enough to realize "hey these pages related to the image viewer application aren't being messed with much in the last 12 hours, those should be high priority to swap out". So when they switch back to the image viewer app with the 100 gigapixel or whatever image they'll get that slowness hit but otherwise not experience it. Then the machine isn't constantly swapping, it just quietly hides those apps nobody is really touching.
The OS shouldn't be swapping hot pages when there's lots of things sitting around not doing much.
There are probably economies of scale associated with maintaining nuclear weapons. Whether you have 1500 or 3500, you need a “large scale nukes” program. You’re not going to cut costs in half by cutting nukes in half.
You also have to consider that nukes have to be distributed around the world, so that you can target enemies throughout the world, so that enemies don’t know where to target their missile defense systems, and so that you still have adequate threat if sites are attacked.
> And the mainframe giant last week told all US Cloud employees, sales or otherwise, to return to the office at least three days per week at designated "strategic" locations.
I'm afraid I'm only doing 2d ... Yes, GUI related LLM instructions have been exceptionally bad, with multiple prompts me saying "no there is no such thing"... But as I commented earlier, GPT has had it's moments.
I strongly recommend giving Gemini 2.5 Pro a shot. Personally I don't like their bloated UI, but you can set the temperature value, which is especially helpful when you are more certain what and how you want, then just lower that value. If you want to get some wilder ideas, turn it up. Also highly recommend reading the thought process it does! That was actually key in having very complex ideas working. Just spotting couple of lines there, that seem too vague or even just a little bit inaccurate ... then pasting them back, with your own comments, have helped me a ton.
Is there a specific part in which you struggle? And FWIW, I've been on a heavy learning spree for 2 weeks. I feel like I'm starting to see glimbses from the barrel's bottom ... it's not so deep, you just gotta hang in there and bombard different LLMs with different questions, different angles, stripping away most and trying the simplest variation, for both prompt and godot. Or sometimes by asking more general advice "what is current godot best practice in doing x".
And YouTube has also been helpful source, by listening how more experienced users make their stuff. You can mostly skim through the videos with doublespeed and just focus on how they are doing the basics. Best of luck!
The department of education is not a service provider.
It is a conduit through which funding flows and is a standards and enforcement body.
They (or at least, they used to) insure that "state's rights" advocates don't implement curricula that teach children that the world is 6,000 years old and flat. They are in the process of being dismantled.
One's local school district is responsible for a vast majority of one's critical thinking skills and it has been this way in the United States since at least the early 1800s when people realized that only wealthy parents had the time, energy, and money to hire private tutors to impart critical thinking skills on their children.
I imagine that in other countries, especially western countries, the story is the same.
We can look back far into history to see that people have used state-run or sanctioned institutions to teach critical thinking skills since well before the Platonic Academy, from which much of our modern system is derived, based on evidence of organized vocational education ranging from Siberia to Ancient Egypt to city states that dotted the land prior to the Old Babylonian Empire.
The main difference between those ancient systems and today is that, for now, all children get the chance to have a formal, standardized education, instead of just the children of the wealthy, well-connected, or lucky.
> They (or at least, they used to) insure that "state's rights" advocates don't implement curricula that teach children that the world is 6,000 years old and flat. They are in the process of being dismantled.
That is, in my view, a good thing. We should not be a monolithic nation and were never meant to be. If the people of (insert state here) wish to teach their children things I don't agree with, or even things which are outright false, that is their right. Nor does it hurt me in any way.
One of the great problems with our country today is people trying to get the federal government to control more and more things. That is directly responsible for much of the division in our country, as federal elections (especially for president) turn into this big fight over who is going to get to impose their dramatically differing way of life on others for the next 4ish years. To reduce tensions, we need to return to the original design: decisions about government should be made as locally as possible, so that the government can reflect the very diverse needs and cultures that exist across our country.
> That is, in my view, a good thing. We should not be a monolithic nation and were never meant to be.
What we absolutely should be is a nation with a minimum standard of education that all American children capable enough are expected to have by the time they leave school. That standard should include the fact that the world isn't flat.
Providing a minimum standard of quality education is critical to the security and success of the nation because a democracy doesn't function when the population is made up of uneducated people who are easily fooled, can't read, and whose heads are filled with lies that will often conflict with what's been taught to the children one state over.
> If the people of (insert state here) wish to teach their children things I don't agree with, or even things which are outright false, that is their right. Nor does it hurt me in any way
If you don't think that it is possible for you to be harmed by the votes or actions of people who are uneducated, intentionally misinformed, and unable to think critically you obviously still have some learning to do yourself.
Look at what happens in poor areas of less developed countries. Honor killings, deification of dictators, rampant scamming and crime, cartels and gangs... all still things.
Your questions betray an ignorance of how a significant plurality of the world still lives to this day. You need to get out more, and not just at the resort towns.
And new problems are cropping up in the foremost developed nations, like depression due to social media addiction, that we'll also need to think critically about, instead of reverting to medieval religious remedies.
Alternatively, maybe you just think we're better off because we're intrinsically better kinds of humans? Gods chosen few? No doubt many people actually believe that.
To be clear on my intentions- the OP said that conservatives wanted to eliminate critical thought. I assumed this was related to the elimination of DOE. My questions intended to dissect why exactly he thought that the DOE was responsible for critical thought.
I am not a fan of the DOE- I think that the relationship between standardized tests and funding mean that schools prioritize skill development and memorization more than they prioritize critical thought / reasoning.
I am not sure where your line of criticism comes from- it doesn't seem like we're understanding each other
You asked a bunch of specious questions about the DoE. The DoE is part of a complicated and fragile system. It isn't as simple as turning off a light switch for a part you don't like, and chances are you probably don't fully understand the thing you don't like in the first place. Chesterton's fence.
---
Now, to your point about standardized testing of skills vs reasoning, I would love to hear a proposal for ensuring "correct" critical thinking is assimilating into student populations. This doesn't scale and is subject to severe bias. Standardized testing measures more objective traits that are indicators for critical thought. IMO a solution must build on top of that foundation, not throw it away. You add a compass, you don't throw out the map.
They are openly attacking university and public school educators; calling them liberal agitators with agendas which indoctrinate students to ideologies counter to Conservative values.
That's a "black box" problem, and I think they are some of the most interesting problems the world has.
Outside of technology- the most interesting jobs in the world operate on a "black box". Sales people, psychologists are trying to work on the human mind. Politicians and market makers are trying to predict the behavior of large populations. Doctors are operating on the human body.
Technology has been getting more complicated- and I think that distributed systems and high level frameworks are starting to resemble a "black box" problem. LLMs even more so!
I agree that "prompt engineer" is a silly job title- but not because it's not a learnable skill. It's just not accurate to call yourself an engineer when consuming an LLM.
reply