Any of the 1L PCs from Dell, HP, or Lenovo. They sip power (5~10 watts), and take up minimal space. I've got a 6 or 7 VMs running on one, and it barely breaks 5% CPU usage.
This, when I was a student and had to live frugal (2001-2008 or so), I got a second-hand Dell, put it on top of a high cupboard in my dorm room, and installed a bunch of services (e.g. Trac was very popular in the day for hosting projects).
It won't give you 99.999% uptime, but for that stage in my life it was just stellar. I even had an open source project (Slackware fork) where I collaborated with someone else through that little machine.
Second-hand hardware is also a great way to get high-quality enterprise hardware. E.g. during the same time period I had a Dell workstation with two Xeon CPUs (not multi-core, my first SMP machine) and Rambus DRAM (very expensive, but the seller maxed it out).
Agree. If low cost and maximum value is you're goal, grab a used one of these or similar speed laptop (and you sort of get battery back up in that case)
Really, any machine from the last decade will be enough, so if you or someone you know have something lying around, go use that
The two main points to keep in mind are power draw (older things are usually going to be worse here) and storage expandability options (you may not need much storage for your use case though). Worse case you can plug in a USB external drive, but bare in mind that USB connection might be a little flaky
See https://www.servethehome.com/introducing-project-tinyminimic... for a good list of reviews.