Socrates was correct. In his day memory was treasured. Memory was how ideas were linked, how quotes were attained, and how arguments were made.
Writing leads to the rapid decline in memory function. Brains are lazy.
Ever travel to a new place and the brain pipes up with: ‘this place is just like ___’? That the brain’s laziness showing itself. The brain says: ‘okay I solved that, go back to rest.’ The observation is never true; never accurate.
Pattern recognition saves us time and enables us too survive situations that aren’t readily survivable. Pattern recognition leads to short cuts that do humanity a disservice.
Socrates recognized these traits in our brains and attempted to warn humanity of the damage these shortcuts do to our reasoning and comprehension skills. In Socrates day it was not unheard of for a person to memorize their entire family tree, or memorize an entire treaty and quote from it.
Humanity has -overwhelmingly- lost these abilities. We rely upon our external memories. We forget names. We forget important dates. We forget times and seasons. We forget what we were just doing!!!
Socrates had the right of it. Writing makes humans stupid. Reduces our token limits. Reduces paging table sizes. Reduces overall conversation length.
We may have more learning now, but what have we given up to attain it?
This is an interesting argument. I’m not convinced but I’m open to hearing more. Don’t we only know about Socrates because he was written about? What evidence do we have that writing reduces memory at all? Don’t studies of students show taking notes increases retention? Anecdotally, the writers I know tend to demonstrate the opposite of what you’re saying, they seem to read, think, converse, and remember more than people who aren’t writing regularly. What exactly have we given up to attain more learning? We still have people who can memorize long things today, is it any fewer than in Socrates’ day? How do we know? Do you subscribe to the idea that the printing press accelerated collective memory, which is far more important for technology and industrial development and general information keeping than personal memory? Most people in Socrates’ day, and before, and since, all forgot their family trees, but thankfully some people wrote them down so we still have some of it. Future generations won’t have the gaps in history we have today.
This worked wonders for me as a kid, learning computer programming. There was so much knowledge to be gained by typing in 1000’s of lines of other peoples code, published in magazines.
I think I probably learned so much more in between the lines during that period, than if I’d just read the user manuals.
And the same is true even today - spending a few hours code-reading some wonderful open source project will enlighten you immensely.
Code is a social construct - just like music. It prospers in the space between minds, in my opinion.
Even if the profit margin is driven to zero, that does not mean competitors will cease to offer the models. It just means the models will be bundled with other services. Case in point: Subversion & Git drove VCS margin to zero (remember BitKeeper?), but Bitbucket and Github wound up becoming good businesses. I think Claude Code might be the start of how companies evolve here.
> This reminds me of a paradox: The AI industry is concerned with the alignment problem (how to make a super smart AI adhere to human values and goals) while failing to align between and within organizations and with the broader world. The bar they’ve set for themselves is simply too high for the performance they’re putting out.
I’m still waiting for somebody to explain to me how a model with a million+ parameters can ever be interpretable in a useful way. You can’t actually understand the model state, so you’re just making very coarse statistical associations between some parameters and some kinds of responses. Or relying on another AI (itself not interpretable) to do your interpretation for you. What am I missing?
There is a power law curve to the importance of any particular feature. I work with models with 1000's of features and usually it's only the top 5-10 that really matter. But you don't know until you do it
My take is the model is a matrix (or a thing like a matrix). You can "interpret" it in the context of another matrix that you know (presumably by generating that matrix from known training data, or by looking at the delta between different matrices with different measurable output behavior), you can say how much of your test matrix is present in the target model.
Yeah, I vaguely remember dang saying that they might consider allowing users to unflag posts, but I think he did confirm that currently flagging cannot be undone by non-moderator users. Currently the site errs on the side of flagging for that reason. I think there was a tweet from pg saying that they were talking about the possibility of giving unflagging rights to karma users with dang, but I'm going completely off memory here.
> The argument that persuaded many of us is that people have a lot of desires, i.e., the algorithmic complexity of human desires is at least dozens or hundreds of bits of information
I would really try to disentangle this.
1. I don't know what my desires are.
2. "Desire" itself is a vague word that can't be measured or quantified; where does my desire for "feeling at peace" get encoded in any hypothetical artificial mind?
3. People have different and opposing desires.
Therefore, Coherent Extrapolated Volition is not coherent to me.
This is kind of where I go when I say that any centralized, top-down "grand plan" for AI safety is a folly. On the other hand, we all contribute to Selection.
No need: it would be the AI's job to find out (after it has become very very capable), not your job.
>"Desire" itself is a vague word that can't be measured or quantified
There are certain ways the future might unfold that would revolt you or make you very sad and others that don't have that problem. There is nothing vague or debatable about that fact even if we use vague words to discuss it.
Again, even the author of the CEV plan no longer put any hope in it. My only reason for bringing it up is to flesh out my assertion that there are superalignment plans not vulnerable to Goodhart's Law/Curse, so Goodhart's Law cannot be the core problem with AI: at the very least, the core problem would need to be a combination of Goodhart with some other consideration, and I have been unable to imagine what that other consideration might be unless perhaps it is the fact that all alignment plans I know about not vulnerable to Goodhart would be too hard to implement in the time humanity has left before unaligned AI kills us or at least permanently disempowers us. But even then it strikes me as misleading or outright wrong to describe Goodhart as the core problem just because there probably won't be enough time to implement a plan not vulnerable to Goodhart. It seem much better to describe the core problem as the ease with which an non-superaligned AI can be created relative to how difficult it will be to create a superaligned AI.
Again "superaligned" means the AI stays aligned even if its capabilities grow much greater than human capabilities.
I'm going to need some good citations on that one.
CEV does not resolve Goodhart's Law. I'm really not sure you even can!
Let me give a really basic example to show you how something you might assume is perfectly aligned actually isn't.
Suppose you want to determine how long your pen is. You grab out your ruler and measure it, right? It's 150 mm, right? Well... no... That's at least +/- 1mm. But that's according to the ruler. How good is your ruler? What's the +/- value from an actual meter? Is it consistently spaced along the graduations? Wait, did you mean clicker open or closed? That's at least a few mm difference.
If you doubt me, go grab as many rulers and measuring devices as you can find. I'm sure you'll find differences. I know in my house I have 4 rulers and none of them are identical to 250um. It's easy to even see the differences between them, though they are pretty close and good enough for any task I'm actually using them for. But if you wanted me to maximize the pen's size, you can bet I'm not going to randomly pick a rule... I'm going to pick a very specific one... Because what are my other options? I can't make the pen any bigger without making an entirely new one or without controlling spacetime.
The point is that this is a trivial measurement where we take everything for granted, yet the measurement isn't perfectly aligned with the intent of the measurement. We can't even do this fundamentally with something as well defined as a meter! The physics will get in the way and we'd have to spend exorbitant amounts of money to get down to the nm scale. These are small amounts of misalignment and frankly, they don't matter for most purposes. But they do matter based on the context. It is why when engineers design parts it is critical to include tolerances. Without them, you haven't actually defined a measurement!
So start extrapolating this. How do you measure to determine "what is a cat"? How do you measure happiness? How do you measure any of that stuff? Even the warped wooden meter stick you see in every Elementary School classroom provides a more well defined measurement than any tool we have for these things!
We're not even capable of determining how misaligned we are!
And that was the point of my earlier post. These are the same thing! What do you think the engineering challenges are?! You're talking about a big problem and complaining that we are breaking it down into smaller workable components. How else do you expect us to fix the big problem? It isn't going to happen through magic. It happens by factorizing it into key components, that can be more easily understood by themselves where then we can work back up by adding complexity. We're sure not going to solve the massively complicated problem if we aren't allowed to try to solve the overly simple naive versions first.
All it takes for somebody to nuke Atlanta is an atom bomb and an airplane and somebody willing to fly the plane.
I’m being facetious but there ARE ways to decide/act as a society and as subgroups within society that we want to disallow and punish and select out qualities of AIs that we think are unethical.
A single person acting in isolation (no friends, no colleagues, no customers) has very little agency. While theoretically a single person could release smallpox back into civilization, we have collectively selected it out very effectively.
The question is only relevant if AI is a significant force amplifier. If it is not, it is an unthreatening tool, whose usage should be largely unrestricted, this is the case now.
If that ever changes then there is the question what to do with it and at a certain level of power an individual decision would have impact, if sufficiently amplified.
Which I believe still does have a large grain of truth.
These things can make us simultaneously dumber and smarter, depending on usage.