I feel like at this point we should be teaching History of Computing in CS programs.
The old people cry, "This is all the same (stuff that's been going on forever," the young cry, "This is the future, old man."
The truth is somewhere in the middle.
"The future is already here, it's just unevenly distributed," very much describes the cycles, but I feel like in some ways this was more obvious 15 years ago, when PC evolution was the focus (nowadays it seems like mobility takes up a lot of bandwidth, which is a huge thing that feels small).
Things like GPUs hitting a tipping point where some feature that had been around for 4-8 years was now ubiquitous enough that software would assume that it was available - and fast. UI expectations ratchet up almost overnight even though the underlying technology has been simmering for years. This was quite pronounced in the 90's, but continued well into the 00's.
Everyone I think can see those, but the epicycles take a bit more study or time. Big architectural changes are driven by cost inequalities in our technologies, and those cycle. Eventually the right somebodies gets fed up and we get SSDs, or 10G Ethernet. Each of those makes some previously abandoned solutions viable again, and they sneak back in (often having to relearn old mistakes).
This multiprocessing idea dates back almost to the very beginning of private sector computing. The ILLIAC IV (1975) was intended to scale to 16 cores but some hardware problems capped it at 4 cores. Those processors were 64bit, and connected to ARPAnet a year before the Cray 1 was born.
Sequent revisited this idea in the late 80's, early 90's, using Intel x86 processor arrays.
We now have a handful of programming languages that have features that could be useful in aggressively NUMA systems (in particular,Scala, Elixir, and Rust). We'll probably see single-box clusters coming around again.
There's a quote from Alan Kay (who is full of delightful aphorisms) in an 2002 Dr. Dobbs interview "The lack of interest, the disdain for history is what makes computing not-quite-a-field." that is one of my favorite descriptions of the state of things.
I'm a huge advocate for teaching History of Computing, and try to slip some into my classes (I teach computer engineering, but close enough) - maybe some year I'll manage to sell running a whole elective course.
I strongly agree that we should be teaching a history of computing in CS/CE programs. I fondly remember my intro to computing systems class taught by a greybeard who spent the entire first lecture going over history and then told us to learn emacs or vim for homework.
Have a term where students, familiar with modern programming, have to deal with programming on older (80s?) machines (on VMs, at least). Not only will it force them to deal with constraints they would otherwise be unaware of, they will appreciate what they have available to them today much, much more.
People underestimate how useful it would be, despite it not being "directly applicable".
If I had to pick one class that I would call fundamental to my understanding of CS, it would be CS2110 (Computer Organization and Programming) from Georgia Tech.
It started off with building stuff using logic gates, like APUs. Then it moved onto other stuff. It all culminated into building your own simplistic CPU pipeline (in an emulator) and making a small game for Gameboy Advance. Dealing with hardware limitations of that handheld console, as well as learning some interesting tricks the devs had to employ for it to add stuff like parallax backgrounds, felt eye-opening.
> I feel like at this point we should be teaching History of Computing in CS programs.
I am responding in trepidation, because I am certain you and everyone at HN must know what I am about to say.
Computer Science is not the science of computers, nor is it remotely the history of computers. The "computer" in Computer Science is not a machine... it is a person, "one who computes." Nor is Computer Science programming, not strictly speaking, though programming is often among the tools utilized by a computer scientist. Computer Science is and only is a subset of Mathematics, and properly initially belongs in the Math Department of a university.
The simplest analogy I have heard, which I think most now have, is that a computer is to a computer scientist what a telescope is to an astronomer. Astronomy is not the science of telescopes, nor the history of telescopes, though I would expect most Astronomy curriculums to include some overview of how telescopes work and their history, but not as some core and essential tract within the study. So in that many machines were utilized to forward the pursuit of Computer Science, so long as it is focusing on the computer science and not the nuts and bolts computer, your idea has merit.
If I am not mistaken, Computer Engineering probably doesn't spend much more than a brief overview of the history of the actual hardware. The CE undergraduate degree is overflowing as it is.
IMO, what you are suggesting belongs in the curriculum of the History of Technology, which is a perfectly valid and endlessly fascinating pursuit.
This machine is neat, and I was using computers during this era, so it makes my mouth water, "what if I had access to that?" But unless it was actually used by someone, a computer scientist, for and to advance actual Computer Science (and that can not merely be programming or creating business applications or games, but needs to at least be efforts towards computational systems), it is entirely irrelevant to the field of Computer Science.
Also, in the sense that Computer Science predates hardware by millennia, the History of Computing (i.e. the history of the activity of one who computes) is already covered in C.S.
Suggested to me years ago, which I completely agree with, since "computer" is now an ambiguous term, Computer Science should change its name to avoid the all to common mistake of assuming CS has to do with desktops and servers. Computer Science is really the science of reckoning, so it should be called "Reckoning Science" to avoid further confusion.
You may read too much into the name. CS is called "Informatics" in Germany, was called "Cybernetics" in the USSR etc.
I would agree that a branch of CS deals exactly with the mathy part, e.g. algorithms, graph theory and so on. Then there is the systems branch; the UX/UI branch touching upon psychology etc.; a branch dealing with didactics etc. I don't see why there shouldn't be a history of CS branch under the larger umbrella.
Personally, I think "informatics" is a better name than "computer science", but that's just my biased opinion.
The central locus of study of computer science shifts depending upon what real computers do.
We care about parallel and distributed algorithms more because we have real parallel machines and big distributed systems of many machines. It's not an obscure graduate offshoot about future hypotheticals and exotic high-end machines.
We care about cache efficiency and memory access efficiency a whole lot more, because the ratio of speeds in a typical computer has changed.
And we're not spending so much of our time studying n-way tape merges anymore... even though they may come "back" because of these shifting constraints.
Real changes in what's typically limiting and what we're trying to do with computers shift the emphasis of computer science around.
And the tick-tock cadence between individual resources and centralized resources... across scaling horizontally and scaling vertically... between single fast machines and distributed systems... can be expected to continue.
Astronomy is a curious example since history is often taught along with the science, at least at the introductory level. This history is typically involves the development of ideas rather than the development of telescopes since it is important to convey that it is a science and our understanding of the universe evolves with time. I suspect that something similar would be beneficial in computer science and engineering, particularly to dispel this notion that computation (may it be mechanical or human) predates the history of computers or to give some perspective on what is happening in the industry today.
The physics text that was used in my high school physics class in the 1980s, Project Physics uses this approach to teach physics. I got a hold of a used copy of the book at some point in the last 15 years or so and re-read it (I originally finished high school physics in three weeks because I found the teacher annoying—in retrospect, I would have been better served staying on the self-study plan but going at a more reasonable pace and with a good guide). While at times it was a useful frame, I could also see the time spent on dead ends and incorrect assumptions being confusing to a student reading the book (counterargument: I don't think I ever read anything but the problem sets in my textbooks until grad school).
In Portugal we don't use computer science, rather Informatics Engineering, and one gets to learn everything related to computers not only the theory that Americans see as Computer Science degree.
I got taught history of OSes and programming languages on my degree, during the various lectures on OS architectures and compiler design.
Although OpenGL, Glide and hardware blitters (like Amiga) were all the rage on those days, we also got to learn about PHIGS and other older graphic programming stacks.
This is the beauty of a proper 5 year engineering degree instead of a 6 months bootcamp. Time to learn everything properly.
Some universities, notably in the UK and Canada, use the terms 'computing science' and 'computation' for the reason you describe. I'm not sure it's worth the potential confusion of a weirdly named degree.
Astronomy is not about telescopes, but neither is it entirely about stars ('astra'). The name 'computer science' is not a problem in practice.
> Sequent revisited this idea in the late 80's, early 90's, using Intel x86 processor arrays.
Also, there were NS3200 versions, like the Balance 8000 with six CPUs I remember at UIUC around 1986. I was floored by how effortlessly it handled having just about everyone in the CS class compiling their assignments the night before they were due. Compared to the Pyramid 90x I'd used the year before, it was like night and day.
It took a while, but that eventually percolated down to everyday PCs. It's kind of stunning that I can get 64-core CPUs these days, and even my much more modest first-generation Ryzen is no slouch.
Definitely should be teaching more history. I recall that neural nets were pretty popular in the late 80s into the mid 90s. I recall a local startup that was working on a specialized neural net chip back in the early 90s, but they couldn't keep up with performance/price improvements from Intel & AMD and folded after a few years. Now there must be dozens of companies doing specialized architectures for neural nets.
Not sure I completely agree. I was in the ASIC biz back in the early 90s. I knew about the NN company I described above because they were a customer. Looking at NRE costs now vs then it doesn't seem all that different (considering inflation). (Sure, they can do a lot more gates now than back then)
During the last 20 years the typical x86 box went from single CPU, through multiple CPUs, through multiple CPUs with non-trivial NUMA topology to the current state where not having non-trivial NUMA topology is meaningful point in marketing of the thing. The primary reason is that large class of somewhat interesting workloads do not cope well with NUMA and then obviously running such workloads on some kind of weakly coupled cluster is completely impossible.
“[In the future] a single processor will provide somewhere between a gigaflop and a teraflop of performance. Are there really problems which need such power?”
It goes on to list problems like quantum chemistry simulations and weather forecasting. Turns out the answer was...running a web browser.
I did my Ph.D. thesis on a Sun 4/110 connected to a VME transputer board and from there into a larger transputer array, using T-800s. Amazing and way ahead of the times - what really killed things was a combination of the removal of financial support by the UK Govt, and also an unexpected increase in the clock frequency of single core CPUs, rendering anything which was not on the latest process out of date. Then the UK went into recession, and many good people left. Some went to the west coast US, other took teaching positions in places like Australia. I do always wonder what could have become of the UK computer industry in the early 90's had it been appropriately funded at the right time (via something like DARPA). But instead, the concentration went into turning London into a financial hub.
What killed the transputer, from the technical point of view, was the failure of the T9000. Its target clock speed was, IIRC, 50MHz (it was roughly contemporary with the first Pentium) but Inmos has terrible problems getting it to run at more than about 20MHz
So companies that had been building large multiprocessors using transputer switched to other architectures, eg Meiko who were in the building next door were making machines with SPARC CPUs and their own interconnect.
The T9 was cool, though. The transputer instruction set was a stack-based byte code, very dense but by the 1990s not that fast, because of the growing discrepancy between CPU speed and memory speed. So the T9 had an instruction decoder that would recover risc-style ops from the stack bytecode. It was helped a bit because the transputer had the notion of a “workspace”, a bit of memory (about 16 words) that a lightweight process could access with very short instructions - in the T9 this effectively became the register set. The T9 would have been a very early superscalar CPU.
And the T9’s new fast serial links used a relatively efficient layer 1 signalling scheme that was later reused for IEEE 1344 Firewire.
(I was an intern at Inmos between secondary school and university, 1993-1994, when this was happening.)
> unexpected increase in the clock frequency of single core CPUs
The 90s were brutal for alternative architectures like the Transputer because performance of Intel processors were significantly improving just about yearly. I recall a neural net chip startup company near where I live - they did some cool science Saturday presentations for the public where they explained how neural nets worked (this was the early 90s). But unfortunately, they only lasted a few years - they were only about 25 years ahead of their time. Now here we are in the 2020s and alternative architectures are sprouting like dandelions.
I attended Bristol Uni's Computer Science department in the late 80s. They had a room of Sun 3s which had transputer cards, which were programmed in occam in a weird folding editor. It was clearly the future, and all programming would look like that in the future (hint, not the one I ended up living in).
I also seem to remember seeing a demo of a mandlebrot set being rendered impressively quickly in parallel on a transputer based machine, which I think was a cube shaped machine. A quick look on the web doesn't throw up any obvious hits though.
Just found reference to it on David May's page, which is suitably retro html for the Inmos architect who created occam and did all sorts of interesting things like formally prove their FPU implementation (before formal proofs for that sort of thing were common):
'The B0042 board contains 42 transputers connected via their links into a 2-dimensional array. A number of them were built following a manufacturing error - all of these transputers were inserted into the packages in the wrong orientation so were fully functional but unsaleable. I had them all (around 2000) written off for engineering use and we built the B0042 'evaluation' boards! Many of these were given to Southampton University where they were assembled into a 1260 processor machine and used for experimental scientific computing. Inmos used them in a number of exhibitions (in a box of 10 boards - 420 processors) drawing Mandelbrot sets in real time!'
Sounds like the machine I remember, a 420 processor machine in a box in the late 80s was quite something.
Similar story: large chunk of my PhD at Bath in early 1990s involved porting Helios (which was developed on/for Transputers) to Intel i860. Fun times...
I remember reading about these; I didn't know Atari ever brought it to market.
I did personally see a transputer card in a PC running a Mandelbrot demo in that era. EGA graphics! I don't think the person who had that in his office ever got it to do what he originally bought it for, btw.
Yes, Atari did, and without the ATW, the Falcon and Jaguar wouldn't have happened.
Richard Miller designed the Blossom video card, was hired by Atari to design the Falcon (which contained a cut down blossom), and who in turn hired his ex-Sinclair friends to design the Jaguar.
Transputer (and later Cell) was a bet that SMP with cache coherency would be too difficult to implement. Here’s a long explanation from Linus Torvalds of why we’re not using architectures like Transputer any more: https://www.realworldtech.com/forum/?threadid=200812&curpost...
Some of the folks ended up being the principals at XMOS (a play on words of InMos), which still has a vaguely transputer-ish design.
As well, there is the KROC compiler which allows a variant of the Occam programming language to be run on Linux, OS X and I think Windows. (Mentioning this in case anyone wants to play with the concepts without having a transputer).
This came as a complete surprise to the folks at Atari HQ in Sunnyvale. Well, certainly the engineers who were working on the ST and the 8-bit line. It could be that upper management knew. But us worker-bee types, a truly WTF moment.
"We announced a new computer!"
[collective response:] "WHAT?!"
Never got my hands on one. I regret that a little. The Transputer was a seriously interesting architecture. (I was never impressed with Occam, though).
I saw Mandelbrot sets rendering on these in realtime in the late 1980s or very early 1990s. I knew I was seeing the future. Now it all fits on an M1 Mac.
I saw these too, running on a Meiko Computing Surface containing (IIRC) 64 Transputers when I was at university. We were taught Occam, but weren't allowed to touch the machine itself.
Many years later, I found a T-800 in its storage case abandoned in the drawer of my new (to me) desk at a new job - so I kept it.
I never wrote any nontrivial Occam programs, but I took a class around that time in which we wrote Occam code for a 4-Transputer PC add-in board. I remember being excited to try it, and afterwards feeling like "How could you get this to do something useful?" It seemed ambitious, but to me it felt like the programming model was more about having firm theoretical foundations in CSP, than ease of writing programs.
I was working towards my PhD in machine learning with neural networks in 1990 and I tried to implement backpropagation in Occam in 1989. But gave up and tried using C* on a Connection machine instead. That did not work out well either so I finally wrote a simple C based implementation that ran distributed on a cluster of IBM RS 6000 workstations ( very well I might add) :-)
Parallel computers of the late 80s were a mixed bag - they worked well for a few use cases but in general, they were troublesome and getting access was difficult and expensive.
Back in the mid-90s I was running tech at UKOL in Shepton Mallet; in a converted brewery which we shared with Perihelion Software, which was the team that wrote Helios. They were an interesting bunch of smart people who were using Transputers to run some of the first VR gaming rigs.
Amusingly Perihelion was founded by many of the folk from compiler company Metacomco, who were mainly Cambridge University computer science folk perhaps best known for the Tripos research OS. Tripos, of course, was commercialised as AmigaOS, and was the basis of Helios. So Atari shipped a computer using the same core OS as the Amiga...
Helios ran on a bunch of different systems, 68000, Transputer, i960, and later ARM, I think.
It was a fun system to program: the server protocol is a bit like Plan9's 9P. It was pretty cool at the time to have all your CPUs, and all your services, linked into a distributed namespace along with your filesystems.
The 9P-like part I found particularly interesting after reading over the seemingly excellent documentation for Helios. I would love to ask Rob Pike if perhaps there was inspiration from Helios for Plan 9, or if they’d arrived at a similar abstraction totally independently.
Actually I’ll shoot him an email and see if he answers!
I think it’s likely that they were either independent or the Helios folks had a brief overview of Plan9.
While Helios did a lot of quite cool stuff, I don’t think it was as general as 9P. It felt like it was produced by a small number of smart people under a lot of time pressure, while Plan9 felt more like it had a more leisurely research background.
I recommend the blue book (vs the black one) for detailed documentation. That plus the book of papers from Perihelion was enough to get servers running.
I always wondered if we could achieve greater success in server hardware by having boards with truly massive numbers of tiny CPUs all with their own RAM and maybe even individual storage. Since most web apps are very simple at an individual session level having a 1 to 1 mapping between a CPU and a session could provide for a great developer and user experience when building server apps.
One day I'll design a board with a beefy network switch and a bunch (32, perhaps) of Octavo SiPs (an ARM core, RAM, ethernet, SD reader wires, GPIO in a single BGA package), put a red LED for each module on the edge, stack 32 of those in a translucent box and play with something that, at least, has the same number of blinking lights as a Connection Machine.
I wonder if, rather than just a single LED for each CPU, you could use an old eg. 1280x1024 LCD panel instead, so you could set the display to show different configurations of the inter-CPU links for different algorithms?
> I always wondered if we could achieve greater success in server hardware by having boards with truly massive numbers of tiny CPUs all with their own RAM
Sometimes I miss Byte Magazine - fortunately our University had it in the library - remember reading about some stuff I could never own (NeXt) or some really oddball approaches that never went anywhere.
There was a multi-vendor standard for Transputer modules called TRAMs: they were a bit shorter and a bit wider than a modern DIMM, and used pins on the short ends rather than PCB fingers.
A common TRAM had a CPU and 4MB of RAM. You'd get a bunch of those, maybe a graphics TRAM, a SCSI TRAM, etc, and build a machine using a carrier board with maybe 8 or 10 slots.
The rework on the graphics card is killing me. That it was more economical to patch over circuit board design flaws with manual soldering than to re-spin a second revision of the board surprises me.
interesting. if i'm reading the wikipedia article about this right, they were essentially SoCs with on-die high speed networking and no shared ram? the ring architecture reminds me a bit of multicore x86...
There were 4 x 20Mbps links and you could use them to directly connect to other CPUs (so, a mesh or torus) or to a fabric. The link protocol was very simple.
Each CPU had local DRAM, but there usually wasn't much: 1-16MB was typical.
Unlike most CPUs, there weren't really registers as such: you had a high-speed stack instead (on die). So an add instruction would pop the top two stack locations, and push the sum.
The old people cry, "This is all the same (stuff that's been going on forever," the young cry, "This is the future, old man."
The truth is somewhere in the middle.
"The future is already here, it's just unevenly distributed," very much describes the cycles, but I feel like in some ways this was more obvious 15 years ago, when PC evolution was the focus (nowadays it seems like mobility takes up a lot of bandwidth, which is a huge thing that feels small).
Things like GPUs hitting a tipping point where some feature that had been around for 4-8 years was now ubiquitous enough that software would assume that it was available - and fast. UI expectations ratchet up almost overnight even though the underlying technology has been simmering for years. This was quite pronounced in the 90's, but continued well into the 00's.
Everyone I think can see those, but the epicycles take a bit more study or time. Big architectural changes are driven by cost inequalities in our technologies, and those cycle. Eventually the right somebodies gets fed up and we get SSDs, or 10G Ethernet. Each of those makes some previously abandoned solutions viable again, and they sneak back in (often having to relearn old mistakes).
This multiprocessing idea dates back almost to the very beginning of private sector computing. The ILLIAC IV (1975) was intended to scale to 16 cores but some hardware problems capped it at 4 cores. Those processors were 64bit, and connected to ARPAnet a year before the Cray 1 was born.
Sequent revisited this idea in the late 80's, early 90's, using Intel x86 processor arrays.
We now have a handful of programming languages that have features that could be useful in aggressively NUMA systems (in particular,Scala, Elixir, and Rust). We'll probably see single-box clusters coming around again.