In fairness, it was a different world back then. There were so few people administering computer networks that you could generally assume someone who was doing so had been thoroughly trained; and the thing about highly trained people is that they tend to view things like failsafes and safeties as pointless time-wasters.
"I know what I'm doing when I hit F7, but the damn system makes me sit there for 30 seconds before it does what I told it to do! Piece of junk."
The result was that software in that era tended to come with a lot more sharp edges. The age of the Recycle Bin that would save you from yourself didn't arrive until administering systems became something the general public was expected to do.
Ahhh....the days of sharp tools, no failsafes, and young programmers or admins.
I recall that time I wrote a batch manager for the VAX 11/780 at Caltech High Energy Physics. It consisted of a program to monitor the batch queue and start jobs as scheduled ("BATch MANager", or "BATMAN"), and a program for users to submit jobs ("Run Overnight Batch INput", or "ROBIN").
The configuration file for BATMAN was stored in /etc/batman.
During development, I occasionally had to "rm /etc/batman". Of course, out of habit, as soon as I typed "/etc/" my fingers would automatically type "passwd", and once I did not catch this in time. Oops. It happened to be a Sunday morning at around 7AM, and I had to call the other admin, who handled backups, to come in and restore that file. He was annoyed.
The second time I did this, he was pretty pissed.
The third time, I fortunately had been working at the terminal we had in the machine room, and managed to shut down power to the machine before the write buffers were flushed, and the file was OK after fsck. I didn't have to deal with an angry co-admininstrator that time. Just angry physicists.
The other admin (Norman Wilson, in case anyone knows him or he reads HN) then made a link named /etc/safe_from_tzs to /etc/passwd to stop my nonsense once and for all.
That worked until the first time I wanted to overwrite /etc/batman instead of rm it.
That led to a cron job that maintained a copy of /etc/passwd in a separate file, and periodically checked to see if it were missing or misformatted, and restored it if so.
One would think after the first two times you'd find a better way to do this, realizing your infrequent but habitual mistake. Why didn't you change any of your practices after the first two screw ups?
Not speaking for tzs, but back in the good ol' days, everybody was pretty busy. A lot of software got written by operators a little bit here and there in between running jobs and moving paper around the building and that sort of thing; paid programmers were frequently dealing with change requests from business departments, all of whom wanted their thing done yesterday; and depending on the size of the organization, there might be a PFY or two, but they generally weren't allowed anywhere near production hardware.
For example, one of my early jobs in IT involved running batch programs that produced reports on a mainframe designed for the punch card era. It had moved on from punch cards, but all of the batch jobs still expected them as input, so they were stored instead as "digital cards" in the job files themselves. The operator -- me -- would be responsible for bringing a job up on the terminal, changing each occurrence of some two-letter code in each card file to some other two-letter code, some date code to some other date code, and so on. Each batch job might be just one step of half a dozen or so required to produce paper printouts from the database. The terminal emulator did not have a find & replace function. Naturally, I screwed up jobs on a regular basis.
This mainframe ran mainly on COBOL74. Over the course of a lot of unpaid overtime, a few hours here and there for several weeks, I gradually wrote a variable interpolator in COBOL that could be called as the first step of a batch file and would replace all occurrences of a variable tag with an input parameter passed to the job. Instead of pulling up a job file and replacing a bunch of two-letter codes, you'd just run the job with the two-letter code as a parameter, and this program would rewrite all of the data cards in the batch file. COBOL has no string operators or a string data type, but I found a way to abuse some system calls to make it work.
So it took weeks to fix the most common operator error in that shop.
IT staff spend more time on Facebook, Reddit, HN, and online gaming now than we ever had available for fixing processes back in the day.
I'm not absolutely sure on the number of times. It is possible that I only rm'ed it twice, and then Norman made the link to end that. This would have been around 1981, so there has been some memory fade.
hahaha :) I didn't mean the comment to sound angry - nope not the co-admin. I do have a running interest in stuff like continuous improvement, organizational excellence etc. so consider this field research!
> you could generally assume someone who was doing so had been thoroughly trained
No amount of training can prevent something like this. It's like today's browsers where the tab can be closed with ctrl+w and the whole window with ctrl+q. It doesn't matter how many times you've done it and how used are you to the position of the 'w'. One day you will close the whole window by accident.
Personally I agree. Mistakes happen, everyone has accidentally hit the wrong key at one point or another in their life. I was pretty surprised how seemingly fine he was with being fired. At the same time, I guess the net result of the mistake was big enough that it did kind of require a response, and it has been about 30 years since it happened.
IMHO, firing someone who owns up to a keystroke mistake like that is wrong. Good managers fix the problem, weak managers fix blame.
Root cause analysis + countermeasure might have boiled down to "operator error due to shitty interface" + "we will tape a guard over F7 key, since it will never ever get fixed in software"
> the net result of the mistake was big enough that it did kind of require a response
That's a dangerous way of thinking. 9/11 was big enough that it required a response. Not sure if we'll ever reverse the airport security stupidity that was such a response.
This is a result of a way of thinking called "Politician Fallacy": "We need to do something. This is something. Therefore, we need to do this". Of course, 9/11 required a response - however it didn't require just any response, it required appropriate response. TSA is not one, and it starts to be more and more clear to more and more people. OTOH, firing somebody who caused the network to go down at the critical moment may be entirely appropriate - one of your responsibilities is not to make such mistakes, you failed at it, you're fired.
I never understood the rationale of "you made a mistake, so you're fired". By making a mistake, the employee has increased her value in that she will never make that mistake again. If you're going to replace the employee you have to pay to hire someone even better (to recoup costs of talent hunt, training) and someone who somehow won't make a typo. It just seems like a situation that is strictly worse than keeping the current employee.
>>> By making a mistake, the employee has increased her value in that she will never make that mistake again
This is a far-reaching conclusion. That assumes that a) no mistakes can be prevented before they happen for the first time and b) every mistake can be prevented after making it. The truth of either far from obvious. Moreover, it is routine in our culture that sever mistakes are punished - e.g. if you make a mistake of driving drunk and cause harm, you'd probably be punished, not lauded as model citizen since you'd never make the mistake again.
Moreover, if no punishment follows the mistake, why the mistake would not be repeated? What would be the motivation to avoid the repetition of the mistake - do you assume the sympathy for the co-workers would be enough? It is not always a sufficient motivator.
>>> It just seems like a situation that is strictly worse than keeping the current employee.
That assumes employees are a fungible commodity, and if you pay the same money you always get the same one. This is not true - you can find employee which would be more attentive, or one with more experience.
If you believe that you can find an employee that is more attentive or with more experience, and you are not laying off your employees right now to find those better employees, what the hell is going on? Are you just hanging out, basically sitting there knowing you have suboptimal employees, and eagerly waiting for them to fuck up so you have an excuse to axe them? You know your employees are (dun dun dun) capable of mistakes but it's expensive to lay them off so you're watching like a hawk for when you get to upgrade them?
The key difference here to me is mistakes vs negligence. Employee makes a typo -> It's a mistake. Not severe, negligent incompetence. It's a learning experience. The company is worse off by firing that person who has experience
If someone is slacking off? Yeah, fire them, that's not a mistake, that's negligence. You email a colleague in another time zone asking for help and they ignore you because you didn't CC their manager? Yeah, fucking fire that person.
I mean, in fact we have an industry based around the fact that people make mistakes: it's called software testing. Should we be firing developers when they make a mistake (i.e. their code has more than zero bugs)? That would be ridiculous. You're not even punishing them in that case - they're going to use their current salary at your shop to leverage a higher salary at the new place they (effortlessly) land at, whereas you're going to spend tens of thousands of dollars to hire that mythical developer that you should have fired this guy for a year ago?
You're right, I shouldn't've used "never". But that mistake is an experience and people learn from mistakes. Now that person is less likely to make the mistake again.
I mean, the goal of the business is to create value / profit and find people who add value to your organization. Not to judge and suss out people who you discover are capable of making a mistake and saying "AHA! I FOUND YOU! You were an imposter all along not worthy of paying! Time to start from scratch again!"
I can't count how many times I accidentally hit the Save Game button instead of the Load Game button in Half-Life. The keys were literally, like, right next to each other.
Thankfully, Chrome has a built-in feature to prevent this from happening (on OSX at least). Just go to Chrome > Warn before Quitting and make sure there's a checkmark next to the option.
Now, if you accidentally press Cmd + Q, it should prompt a "Hold Cmd + Q to Quit" instead of actually quitting.
I disabled that warning because it's annoying every time I want to close the browser, even without the dangerous keyboard shortcut. If it happens, you can go to the menu and find "recent tabs" or just ctrl+shift+t.
Traditionally applications have asked the user if they're sure they want to quit. That's a no-no these days, but it's still a reasonable choice in situations where the cost of quitting might be high (there's unsaved content, or the app takes a long time to start, or it's impossible to persist the current state of the application).
For some time, the Chrome team refused to implement a 'Sure you want to quit?' popup due to a general anti-popups consensus. They also refused to implement a checkbox to enable that behavior due to a general anti-configuration consensus. They've since relented on the latter.
Mozilla Firefox used to have a setting under options/preferences to disable loading images and loading javascript. People complained and said this should be removed as not enough people use it and that people who use it can create/use an add-on to do the same.
It is not easy to find a right balance between providing adequate functionality while avoiding information overload. The web is still evolving. We are learning and we will do better (overall) as time goes by. :)
Yeah, if you don't care about privacy and save such data to disk as active session, active browser tabs, history, cookies, etc. Those should be RAM only for privacy reasons and never stored to any medium which can store those for extended periods.
The power switch on the IBM PCs were way at the back so that people couldn't unintentionally reset the computer. The same thinking went into Ctrl-Alt-Del, which was a combination that people wouldn't accidentally hit.
So having a system where F7 would reboot the entire system was pretty dumb, even in the early 80s.
I can't find information why the design(s) were as they were. The design of Ctrl-Alt-Del was intentionally unintentional.
Gates noted Ctrl-Alt-Del should have been one button, not three [1]. David Bradley, the inventor of the trifecta, did make it deliberately difficult to reboot, however, it was also originally an Easter Egg which made it to production [2].
Ctrl-Alt-Del was an excellent choice for it's intended purpose. It can be easily typed at they keyboard (no need to fiddle with the back of the computer) but it's very unlikely to be something you hit accidentally.
I'm not sure a single key would have been a good idea. "The reboot key" just sounds like a mistake waiting to happen. I've seen enough stories on the 'net of laptops with power buttons in terrible places on the keyboard to get a glimpse.
The mistake was using it for the Windows NT screen lock/unlock. Changing the "reboot your computer" sequence into the "start using my computer" sequence is a rather non-sensicle (ignoring implementation) choice.
> "The reboot key" just sounds like a mistake waiting to happen.
It already did happen: I've used keyboards with shutdown and reboot keys, and yes, it's a terrible idea.
My alma mater used to run programming competitions in a lab where the workstations had the reset button exactly at knee height. This wasn't a problem normally, when you're sitting under the desk next to the computer; but when you have three or four people gathered around the screen, kicking the reset button was a definite possibility. Eventually taping the cover of a calculator over this button became part of our regular routine.
And on the IIe or later you had to press Ctrl+Open Apple+Reset. (Similar to the IBM PC.) On the earlier models, if you mistyped, BEEP, the system rebooted.
As in: I hit F6, "Do you want to reboot this?" dialog pops, I hit 'Y', "Do you really really want to reboot this?", I hit 'Y' again.
Instead of actually reading what it says, you just instead press F6-Y-Y in quick succession.
Modern interfaces sometimes make you type some kind of string to confirm, but most either use a password (like sudo) or some hardcoded string that everyone eventually memorizes.
But even today, Windows 7 only makes you click that one button in UAC, and most people probably do it without even thinking about it.
Whenever I implement a bulk delete feature I tell the user how many records they're about to delete and ask them to type it back in.
If it's possible they're trying to delete data from the wrong place (say, an administrative account that manages many customers) another safeguard is to have them select the name of the context (customer name, etc.) out of a list of four or five nonsense alternatives.
The user experience tends to involve a lot of double takes and rereading, which is precisely what I want.
I have idly considered addressing that problem when it really matters by asking multiple random questions whose answers need to be some combination of "Y" and "N" to proceed. With the result that you simply cannot engage in muscle memory.
When you delete a github repo, you have to retype the fully-qualified name of the repo you want to delete. I think that's exactly the right level of annoyance: it makes sure that if you're mistaken about where you are, you realize it, and that if you're making a typo, you'll have to make it the same way twice.
Not as much as we hate the person who made the decision to prevent phones/computers from turning on immediately when the battery is empty, even if they're plugged in :P
If I ever meet that person IRL... I might even go so far as to make a tasteless joke about committing physical violence in retaliation for the hassle they've caused me.
- When the battery has just started charging, the voltage will not be high enough for the phone to actually work, because the draw from the battery exceeds the plug pack input
- Sometimes when transmitting, the phone uses more power for a fraction of a second than the power pack can deliver. This surge of energy could come from the battery, but the battery is empty so it won't work correctly
- Having some amount of battery means the phone can soft-off correctly when the plug is removed suddenly. The alternative is an un-expected hard off which is usually bad. The user might experience data loss.
There are a bunch of grey areas around low voltage, such as flash writes failing marginally or radio not working correctly or partial saves. Much easier for the engineers and perhaps more reliable for the users to make them wait just a little.
> When the battery has just started charging, the voltage will not be high enough for the phone to actually work, because the draw from the battery exceeds the plug pack input
What type of battery has an innate "draw"? They need a certain voltage and have a certain internal resistance, but it's easy to efficiently increase the effective internal resistance by boosting voltage with switched capacitor circuits (or whatever). If there's a "smart battery manager" you can bet the hardware to do this is already there.
"Draw" would be an excuse if you were hooking things up manually to a car battery. It's not an excuse in the highly integrated environment of a cellphone where corrective circuitry is dirt cheap (and free relative to what's already probably there).
> Sometimes when transmitting, the phone uses more power for a fraction of a second than the power pack can deliver.
That's what capacitors are for. They're almost certainly more efficient, too. Efficiency slumps away from the optimal I,V much faster for batteries than for capacitors.
> Having some amount of battery means the phone can soft-off correctly when the plug is removed suddenly. The alternative is an un-expected hard off which is usually bad. The user might experience data loss.
I'm pretty sure this is the actual reason why it's done. It's an awful reason.
First of all, you claim that "an un-expected hard off is usually bad". WTF? Does your ext4 linux partition usually die when you hard-off it? I've probably hard-offed ext4+linux 1000 times, never had any problems. I would go to great pains to avoid hard-offing a production server but you must acknowledge that in the age of solid journaled filesystems, hard-offs almost never lead to actual bad consequences, especially for light usage patterns. I'm sure it's worse on some hardware configurations but I've never met a system where it got all the way to "usually bad" territory.
On the other hand, having a power manager lock me out of my phone for 15 minutes after I determine I need to use it has led to loss of data. Significant loss of data. And worse. Pictures that were never taken, phone calls that were delayed at significant inconvenience, the inability to look up contacts for others... these are real world negative consequences that are 1000x more important than a .1% chance of filesystem corruption times, say, a 20% chance of actual power failure. It seems hopelessly myopic to suggest that the cost/benefit trivially favors the prevention of uncommon filesystem errors over addressing the immediate and possibly time-sensitive needs of the user.
I think that whichever organizations choose to implement the lockout feature are doing a massive disservice to their customers, foisting significant hassle upon them in order to save a few pennies/customer of repair costs, if that. Your arguments haven't convinced me otherwise.
I believe UAC works well even when a human always hits OK. The point is to make sure a human is there and trying to do something and not some malicious code.
This is very true, which begs the question - why was a 23-year-old summer intern placed in charge of the embassy computer system, or even given access to its central console? He may have have self-taught facility with computers, but I find it hard to believe he'd have much experience at that age with the sort of large, mission-critical institutional computer system described in the article.
I wonder if he had a supervisor sysadmin that he was working under, but given how he described his boss, that seems unlikely as well.
You might be surprised. What I observed during that time was that a lot of, if not most, senior people were clueless about what computers were actually doing. The potential "reach" of that intern might just not have occurred to people.
And we still have national security / diplomacy disasters resulting from relatively low level people having access to vital computer systems beyond anybody's imagination.
You're invoking an annoying and ridiculously overused false dichotomy that is as false today as it was 30 years ago. An interface that has fail-safes does not have to be annoying and clunky. In fact, interfaces that are annoying and clunky are a great contributor to human mistakes, because they require a lot of rote action, which encourages people not to pay attention and work on auto pilot.
If something has changed over these years, it's the overall understanding of design principles and their popularization. (Thanks, Don Norman and other people in the field!)
Training has nothing to do with it. Even if you can train a person to work with a badly designed system without making mistakes (often), designing the system well in the first place is almost always significantly easier and cheaper.
For example, accidental key presses can be easily prevented by requiring the user to type a command of reasonable length. Typing "reboot-all-workstations" is not that difficult, but it would definitely prevent the incident described in the article.
I think he was pointing out why it was built that way, not why it was that way. Today we know it is a false dichotomy, he was pointing out, back then, the world thought it wasn't.
You know, I think that was the point of the comment. I might just be reading it wrong, but it seemed to me that he was decrying such interfaces, using obviously hyperbolic language like "pointless time-wasters" (in the context of something other people believed) and "the damn system makes me sit there for 30 seconds". It sounded to me like he was even slightly making fun of that worldview.
You are correct, making fun of it was my intention. I suppose next time I will have to hang a "WARNING: SARCASM" sign on my comment, to make sure everybody gets it :-D
Even allowing for the era, providing the opportunity for a possibly disastrous effect to take place due to someone dropping the keyboard or knocking it with their elbow is inexcusable. Accidents happen - they should be minimized and steps should be taken towards quick recovery but they should be expected.
It wasn't until Windows 95 (or was it 3.1?) that usability and standardization were taken very seriously in the PC world. I recall vaguely using an 80's-era DOS-based word processor at my father's office, and pressing "F1" for help, and wiping out my work.
If you visit countryside museums with old agricultural or workshop equipment, you can verify they are basically all maiming devices. Things used even in the 1950s. In one machine you push a piece of wood downwards and there's a blade that hacks slices off the bottom. Real tough men pushed right till the end so there was no waste. They often had some missing fingertips. Some really small changes made those accidents avoidable, like using some other pieces to hold the worked piece.
There was probably something like a warning prompt that came up, but he may have been used to disregarding the message because it was always what he intended to do.
Two servers side by side. Broken KVM that only switched the monitor. The screen was Windows NT, but the keyboard in front of the monitor was connected to another server running OS/2.
Ctrl-Alt-Del
The Windows screen did nothing, but the sudden hard drive activity on the OS/2 server told me what I needed to know. Not thirty seconds later, I had visitors. I was around 21 at the time, so yeah... Experience.
You know what's totally plausible? That hitting F6 and F7 prompted for confirmation, but using the same prompt - and he just hit "Y", "Return" as he'd done 1000 times before (just like everybody does with the UAC dialog in Windows 7 today), and that bit didn't make it into the story because it's totally irrelevant to the point he's making. Heck, it probably wasn't even F6 or F7. If he's a normal human, he likely can't remember.
It is possible there was a warning for both functions and the operator just acknowledged the warning assuming it was for the single reset without reading it. This is a common problem with warning too often in interfaces.
Stole the words right out of my keyboard. Who puts the 'reset every workstation in the building' button right next to the 'reset just one workstation' button with no confirmation prompt?
He might have been that times IT Administrator? We've come a long way over these past 30 years, but think about it for a second, if you're an IT Administrator today handling all the office machines, you probably have the power to click 1 button to turn them all off? (Depending on the setup of course..) There's obviously more access rights involved today.
Doesn't answer "who" put the button in there, I'm just thinking out-loud! :)
I don't remember how it is on the newer versions, but try using fdisk in an old linux distro and see how many confirmations you get when deleting a partition.
I was leaving my old job and handing in my MacBook Pro. After getting permission from the network guys (who were going to format it anyway) I ran rm -rf / as root.
It was surprisingly boring, taking a very long time to delete all my files (I should have deleted them first). Eventually it got around to deleting fonts, which caused things to render a little strange, but after 60 minutes nothing much had changed and it was still chugging, so we shut it down and went to the bar for my last "Friday night drinks".
I was trying to move everything out of a folder to tmp (mv ./* /tmp), which was fine. Then I cd to /, and, wanting to rerun the command I'd run right before the mv, pressed up twice and enter quickly...
Well, it failed once it finished moving /bin/mv to tmp. Of course cp, and a lot of other helpful commands come before mv alphabetically.
It wasn't too bad, just needed a boot from a live disc to move everything back, but I still get nervous whenever / and * are in the same command.
I did it to one of my linux boxes once, but caught it before it got everything.
It made for a very trippy and insightful experience. Commands that had already been run (like cp, ls, mv, cd, etc.) worked fine since they were in memory but other ones wouldn't run (/bin was one of the first to go).
I tried for about an hour to bootstrap the system back to normal by copying files of CD/etc but in the end the damage was too bad for my (very amateur) experience and I ended up reinstalling.
It's still the same on some Linux distros at least. I wiped a Slackware install last year carelessly attempting to fdisk an external drive. I caught it a few moments in, but it took a bit of doing to get the data back and I had to re-install.
That was one my my reactions too (yeah it was a different age).
My other reaction was about the total loss due to a restart - in what could have easily happened by faults in many other systems - presumably there was no way to save data as work went no, no journaling or anything. Indeed this was a different age.
It's easy to have this attitude now. But if you ask people who were in IT 30 years ago, they'll tell you that systems in those days had _many_ sharp edges. You were expected to know your way around, and the consequences of mistakes were pretty severe.
Pressing the button would start a timer. While the timer is running, the user would have the opportunity to review their selection (maybe even with a simulation of what effect the selection has) and could undo the request if necessary. Only after the timer expires would the action actually be taken.
This is how the "undo send" feature of Gmail works.
I don't know whether armchair usability trivia helps here. Not every system deals with ephemeral web drivel; some systems interact with the real world and have impact.
Are you saying that usability does not need to be considered in the design of critical systems?
Human factors grew out of the need to build safe and error-resistant weaponry in World War II. Poor attention to human factors and user interface design was a factor in the Three Mile Island disaster.
My thoughts exactly. With that sort of "global" function, you would expect some sort of countdown timer prompt on each terminal that could be cancelled.
agreed 100% (if there truly wasn't a prompt).
OP took the fall because the head administrator was too ashamed to admit that their system was so poorly designed to allow for this to happen.
Fire the idiot who wrote that function.