Hi all! We (@anishathalye, @jjgo, and @jonhoo) have long felt that while university CS classes are great at teaching specific topics, they often leave it to students to figure out a lot of the common knowledge about how to actually use your computer. And in particular, how to use it efficiently.
There’s just no class in the undergrad curriculum that teaches you how to become familiar with the system you’re working with! Students are expected to know about, or figure out, the shell, editors, remote access and file management, version control, debugging and profiling utilities, and all sorts of other useful tools on their own. Often times, they won’t even know that many of these tools exist, and instead do things in roundabout ways or simply be left frustrated about their development environment.
To help mitigate this, we decided to run this short lecture series at MIT during the January Independent Activities Period that we called “Hacker Tools” (in reference to “hacker culture”, not hacking computers). Our hope was that through this class, and the resulting lecture materials and videos, we might be able to bootstrap students’ knowledge about the tools that are available to them, which they can then put to use throughout their time at university, and beyond.
We’ve shared both the lecture notes and the recordings of the lectures in the hopes that people outside of MIT may also find these resources useful in making better use of their tools. If that turns out to be true, we’re also thinking of re-doing the videos in screen-cast style with live chat and a proper microphone when we get the time. If that sounds interesting to you, and if you have ideas about other things you’d like to see us cover, please leave a comment below; we’d love to hear from you!
We’re sure there are also plenty of cool tools that we didn’t get to cover in this series that you all know and love. Please share them below along with a short description so we can all learn something new!
That's funny, I recently had the same revelation (that it's hard to discover these things) with engineering students in mind. I wrote an e-book called Digital Superpowers, covering many of the same topics you did. I added some general interest things on art and publishing too. I think I like your approach better.
Anyway, my list of topics is as follows, maybe you want to consider adding a few of them to your list?
Password Managers • 2-factor authentication • Virtual machines • Package managers • Command line • Routers • QR codes • Port forwarding • Python • Pandas • Onion routing • Ad-blocking • PDF reordering • Flow charts • Regular Expressions • Time-lapses • Encryption • Column edit • Digital photo editing in RAW • Audio noise reduction • Vector graphics • Making websites • DJing • LaTeX publishing • Reference management • pandoc • git version control • Programming • Web apps • Self-hosting • Home automation • Monte Carlo • VPNs, • ImageMagick • Linear regression • ffmpeg • Sphinx • Robotics
As a computer Science graduate with no experience doing images that section alone was worth getting the book for me. I plan on using at as encyclopaedia for future uses.
I am happy you had compatibility in mind as I am not super fussed on which operating system I use.
I would suggest adding a virtual desktop section as that has been the biggest gamechanger for me ( I use windows 10s with virtual desktop enhancer) I dont know any linux or mac equilivants though.
Hopefully you change the recommended tools based on the OS. (i.e. OSX would be slightly different from linux for certain commands). And focus on doing as much as possible from the command line?
I tried to highlight examples that are available on Mac, Linux, and Windows through the package managers, (homebrew, apt, chocolatey) and at least provided system-specific installation lines for each based on those. It is highly command-line and pure-text centric.
You're right. And "covering" is a strong term for the book. It only introduces them, offers a few exercises, and moves on. It's still 200 pages. It's intended to help engineering students (mechanical, civil/structural, nuclear, etc.) discover these wonders and then choose which ones to go deep with based on their needs and interests.
> ...[universities] often leave it to students to figure out a lot of the common knowledge about how to actually use your computer.
Now I'm out in the real world (MIT class of '86!) I think this is more of an MIT thing -- many graduates of other schools get explicit training in things that the MIT faculty, fairly or not, assumes students will simply pick up. And this is despite the heavy emphasis on actually doing stuff, especially in the school of engineering. Actually I've had hit or miss results hiring freshly-graduated MIT grads (maybe less so VI-A) for this reason, though once they are up to speed they've mostly been great.
> many graduates of other schools get explicit training in things that the MIT faculty, fairly or not, assumes students will simply pick up. And this is despite the heavy emphasis on actually doing stuff,
uh...I graduated from a low tier state school that was notable in our state for focusing on practical matters....and we didn't get 'explicitly trained' on this either. Most people do just pick it up.
I got a CS minor from a top-5 program, and while a lot of the core classes definitely covered some of these topics in various degrees, it's really nice to see all of these aggregated in a concise way.
While you definitely do pick stuff up over time, this is arranged in a way that I could see myself preferring to reference this rather than find that one stack overflow answer that clicked with me at the time. Now that I at least have a better understanding of these concepts 3.5 years into real-world web development, this seems like a good resource to cover my bases better now.
Well, I appears from my experience a surprising number of MIT students don’t, making this mini-course popular.
As an MIT grad myself I should be biased to think otherwise, and I certainly hope the credential stands for something good. But I haven’t found much difference in programmers 5 years out of school — it’s been attitude that has mattered more.
>many graduates of other schools get explicit training
From my undergrad programming class, which was a while ago, I remember that the first class had a brief intro to vim. But something like vim is an acquired taste. I never used or appreciated vim until much later when I saw others at work use it. During school, I think I used geany since that's relatively easy to use and install on ubuntu. So maybe there is some merit to spending more time on tools. However, my experience is from before everything was so easily accessible on the internet (i.e., pre github, stackoverflow etc.). So maybe these days it's not such a big deal.
I graduated from a low-tier state school and after reviewing MIT's course materials along the way, I feel the opposite. Sure there wasn't a class, but plenty was covered explicitly in the homework and labs.
University of Washington CSE added a practical course to get students familiar with Linux and C partly because students are expected mostly to come through other departments / disciplines where the only programming may have come from Java or Python. Students were having great difficulty in higher courses like graphics, OS, compilers, and embedded systems because we weren’t teaching programming courses in our curriculum except maybe our programming languages course itself which covers languages like ML, Scheme, Haskell, or Eiffel and OOP languages like Smalltalk.
Early pilots of the course were put together with a set of Linux guides including bash scripts and Makefiles, source control with svn (hopefully hit by now), and moved quickly into practical stuff with bison / yacc to parse complex files and work with GCC output. I never got to see how well it was received by students but I was glad to see students go through the time-honored traditions of learning CLI tools.
Will need to take a deeper look later but from a quick skim I really like the approach taken: a well curated list of time tested tools across each area, going deeper where applicable.
For me the key point above is about a lack of tool knowledge causing "roundabout approaches" and frustration. Knowledge of these tools can do so much to demystify and make the seemingly arcane start to slot into place.
Thanks a lot for this! It appears like the recording doesn't really capture the screen well for multiple videos. I wish it was more professionally done, it could've been useful for years to come.
Sadly, we're missing screen recordings for a couple videos (such as the shell video, where we had some issues with the screen recording software). We're thinking of re-doing some of those videos in a screen-cast style so that they'll be more useful for viewers.
The equivalent UCLA course is CS35L: Software Construction Laboratory (https://web.cs.ucla.edu/classes/winter19/cs35L/). It's taught by Paul Eggert (big open source/coreutils/emacs contributor + author of diff/sort).
Paul Eggert is also the maintainer of the timezone database and designer of the uniform naming convention for time zones, such as America/New_York and Europe/Paris.
CMU did (maybe still does?) have a mandatory, once-a-week course to orient new students to logging in to *nix workstations, setting file permissions and copying files around on AFS, etc. Whether this was just for CIT or university-wide I cannot recall.
But almost all IDEs and editors have a VIM mode - learning VIM is a good way to teach people how to keep their fingers on home row, and the skill is (because of what I stated previously) easily transferred to whatever editor you like.
From my perspective, I tried to learn Emacs, and decided it's a great editor, once you spend weeks to months to get it actually "working good," and learn shitloads of commands (let alone keybindings). Given my initial goal was "keep hands on home row," it was far, far faster/better for me to spend 3 days learning/memorizing VIM commands, installing a vim plugin into VScode, and just doing that.
Would love to use Emacs, but it's a nightmare to set up. I still haven't managed to get it feature parity with vscode.
> Would love to use Emacs, but it's a nightmare to set up.
Maybe you would like spacemacs then? It is a Configuration Bundle for Emacs focusing on "Evil Mode" (aka the Vim emulator in Emacs). It is just a few minutes to install and if you already familiar with vim then there should not be much change what the normal editor usage is concerned. Of course the customization is Emacs lisp which has a bit of a learning curve, but just to get started with Emacs and evil you don't really need to customize anything.
I wouldn't say Emacs is "a nightmare to set up"; rather, it's a continuous process of learning and improving.
Getting Emacs installed and running should hopefully be easy (e.g. via your OS package manager). After that I think it mostly comes down to coming up with workflows that you're comfortable with, building muscle memory and gradually introducing new things to see whether or not they stick.
I’m sure a not-insignificant portion will move on to Evil mode.
Vim -> Emacs is a better direction than Emacs -> Vim, because you’ll learn the more efficient editing style, then have access to the better environment, instead of getting used to the great Emacs environment minus the Vim keybindings, and feeling constrained by losing this when moving on to Vim.
I think this is true. I learned the Emacs keybindings and now they're muscle memory. I can't transfer that to Vim, but I could have transferred the other way and be proficient in two great editors.
Nobody wants to die on Emacs Keybindings Hill, even diehards like me and my pinky.
When I took C at University of New Mexico, and later Data Structures and Algoithms from the same professor, Emacs was required. Period. He even provided a download of his .emacs file (still tucked into my init.el). He also stressed make, gdb and valgrind ("It's not fully clean until it's valgrind clean.")
No emphasis on git, but I used it anyway.
A decade later, and my Emacs easily surpasses most VS Code features.
A decade later. Heh. Co-workers use VS Code OOTB and I'm like, "Yeah, but can you tangle or play Tetris?!"
Truth is, whatever works for you and gets out of your way is the right tool. I'm invested and happy with Emacs, but also I get why it's not the first choice for a lot of devs.
As an MIT alum I was sort of surprised they went with vim over Emacs too. MIT is almost the Vatican of lisp, after all. I sort of felt like an apostate for switching over to vim there.
Can't tell whether this is sarcasm or not. Perhaps the expectation is that those things will be acquired naturally; certainly I was immersed in both within two weeks of arriving at MIT. (Admittedly that was in the latter half of the Carter administration; things may be different these days.)
This is a great series. I'm going to share with my team, just to make sure they can fill in any items they're missing, like using bash functions for shortcuts or aliasing.
BTW, Stanford did an online training course 5 years ago or something called CS 184 Startup Engineering. They had some excellent tutorials on the CLI and Emacs [1]. -I wish I could find the materials, if anyone else can find them.-
So one lecture covers window managers for tiling. I used dwm for years and love it, but am stuck on Windows at work. Is there anything for Windows 10 that has a similar approach of having a stack with your unused applications and a big window for the app that is in focus? And then you can easily move window from stack to focus and vice versa?
I looked at the Wikipedia article [1], but none of the alternatives seem to do that.
You could have a full-screen Xming session connected to dwm running in WSL, if you liked that better. Of course, you would still have to use the standard Windows DE/WM when using Win32 applications, but if most of your work is text editing, web browsing, etc, you might be well served by the full-screen Xming.
It's no where near complete like dwm for that, but it is highly customizable, I use AquaSnap when I have to use windows for work. It is paid but I haven't had any issues with upgrades or redownloading the software when I haven't had to use a windows machine in a while.
I strongly believe man is a product of his tools, my slow but steady increase of their mastery is a source of pride. I've spent nights just mindelessly browsing the web in the search for a tool I didn't know about (it can be a service, a browser extension, a text editor plugin) so just looking at the title of this post made me ready for an injection of dopamine.
I have a branch on my todo list solely focused on my tools, because there are so many things to configure continuously. I schedule these tasks for the occassion I feel like I'll have a couple of minutes to improve and their priority are often labeled as "Compounding Rewards" as opposed to "Critical".
At MIT, the month of January is a reserved time separate from the normal terms called IAP - Independent Activities Period. During IAP, there are many seminars, panels, talks, and short courses (and some long courses taught in an intensive fashion) on a huge variety of topics. Many are taught by students, faculty, staff, and area alumni. While a few classes do give credit, fulfill requirements, and the like, oat exist simply to learn cool, neat, useful, interesting, wacky, etc stuff.
Yeah, sadly the screencasting software malfunctioned on some of the sessions. We're thinking of re-doing some of them with a proper microphone and screencast though (see also https://news.ycombinator.com/item?id=19081669), and will certainly announce when/if that happens! For what it's worth, the lecture notes should capture most of the actual contents, and in the same order.
Literally none of these have anything to do with proper information security/hacking/penetration testing. Will you please consider to stop abusing the term "hacker"?
So, the term "hacker" and "hacking" has traditionally meant someone who is exploring the limits of systems (whether computer-based on otherwise) in a playful manner, often finding "clever" ways to make the system behave in unintended ways. The term is arguably even broader than that — see the Wikipedia entry on Hacker Culture[1] for more. The use of "hacker" to mean the exploitation of computer systems is more recent, and was traditionally referred to as "cracking". The two terms do have a decent amount of overlap, but the latter is much more commonly associated with malicious intent, whereas hacking really just means that a desire to push systems to their limits and see what kind of cleverness you can pull off!
It is you that should consider to stop shoehorning a term with now two common meanings into the one that you prefer. Once upon a time hacker meant nothing of what you are referring to, now it is 'dual use', no need to go overboard in the other direction and forbid the original one. For reference see 'The hacker from MIT'.
"Hacker" and "hacking" at MIT have a couple of meanings, neither of which specifically means trying to "hack" into a system. "Hacking" either means 1) Working hard on a problem set or other project or 2) Pulling off a prank on the MIT campus without permission of the administration, e.g. turning the Great Dome into R2D2's head.
So in the context of some people from MIT putting out a list of resources, originally for others at MIT doing #1 above, "hacker" is the right word to use.
Forget university, most of these this should be the content of K-12 computing/programming classes. It's simpler and has much more real world value than a lot of the crap the tech, like graphical iPad programming or how to use a specific version of word.
The article on editors is basically "here's the names of 5 editors, now learn vim"
I know this will piss off vim users but in my experience vim users are seriously out date when it comes to dev tools. It's like they never left the 70s. vim might be great, it might be available everywhere but when I see a vim user I then see them use something like gdb and have clearly never experienced a modern debugger because if they had they'd be infuriated at how their debugging tools haven't improved in 30 years.
I'm sure others could go into all the features a modern editor or IDE provide. The vim users will say they don't need that crap. They sound like some grandpa saying "back in my we had to walk to 5 miles to school, no one had invented bicycles or cars or buses. Don't need those new fangled things, now Get off my Lawn!"
My experience of IDE users (myself included much of the time) is that they don't know how limited their debugging experience is, just breakpoints and how to step through code. Things like breaking when a variable is set, debugging from a core dump, creating a core dump under certain conditions, viewing the assembly, setting variables on the fly etc, are features of a good debugger (often the one they're using) that are obscured from them.
Exact opposite of my experience. IDEs let me easily add conditional break points, toggle between source, disassembly, source+disassembly, set variables just by clicking on them, view multiple areas of memory in different tabs/windows. Set how that memory is viewed (bytes, words, dwords, floats, as a type of class/struct, array of classes/structs), show different contexts at once, have multiple windows per thread/process, never seen someone use one that didn't know those features exist as they are all obvious to discover in an IDE.
sounds like you've never used one but then that's what I'd guess from watching most vim users. All of those have been standard features of visual IDE debuggers for over 30 years. Mean while instead of seeing every thing update live I watch the vim users type various gdb print and dump commands at the command line and then watch that data scroll off their terminal instead of being updated live as they progress like an IDE debugger.
> sounds like you've never used one but then that's what I'd guess from watching most vim users
You'd be mistaken, I've spent a huge chunk of my adult life inside Visual Studio. IME most devs don't even know you can add a condition to a breakpoint (hiding feature behind right clicks aren't obvious) and anything complicated turns into a very convoluted process very quickly. Even setting conditional breakpoints are basically an input into the command interface, essentially what the windows run menu is to the command line.
Take this short tutorial (https://amazingdim.wordpress.com/2014/02/01/gdb-script/) on gdb scripting and show me how to do similar through an IDE. With Visual Studio at least it is much more complicated, here are a few google results on that path:
You’re assuming that Vim users use only Vim. I use Vim all the time for editing config files, small Ruby scripts, and JavaScript projects without types – for these cases it is among the best tools for the job. And when I work on, for example, big Java projects, I use IntelliJ IDEA plus a Vim keybindings plugin, so I can use the IDE’s debugger and its refactoring support. Don’t make sweeping generalizations about Vim users not knowing when other tools are appropriate.
There’s just no class in the undergrad curriculum that teaches you how to become familiar with the system you’re working with! Students are expected to know about, or figure out, the shell, editors, remote access and file management, version control, debugging and profiling utilities, and all sorts of other useful tools on their own. Often times, they won’t even know that many of these tools exist, and instead do things in roundabout ways or simply be left frustrated about their development environment.
To help mitigate this, we decided to run this short lecture series at MIT during the January Independent Activities Period that we called “Hacker Tools” (in reference to “hacker culture”, not hacking computers). Our hope was that through this class, and the resulting lecture materials and videos, we might be able to bootstrap students’ knowledge about the tools that are available to them, which they can then put to use throughout their time at university, and beyond.
We’ve shared both the lecture notes and the recordings of the lectures in the hopes that people outside of MIT may also find these resources useful in making better use of their tools. If that turns out to be true, we’re also thinking of re-doing the videos in screen-cast style with live chat and a proper microphone when we get the time. If that sounds interesting to you, and if you have ideas about other things you’d like to see us cover, please leave a comment below; we’d love to hear from you!
We’re sure there are also plenty of cool tools that we didn’t get to cover in this series that you all know and love. Please share them below along with a short description so we can all learn something new!
Anish, Jose, and Jon