I am using a ZimaBlade (2 cores, 16gb DDR3L RAM) as my cloud server, and an old Nvidia Jetson NX (very outdated ARM CPU with negligible GPU power) as my email server. I also run a bunch of frontends and apps on both. They work flawlessly.

Of course is domestic use (indeed, homelab), but I am pretty sure a good NUC (pay attention to thermals!) can be used professionally.

On my old job, we used a raspberry pi 3b+ as a VoIP server and we were in the process of moving a bunch of hosting services on a mid-tower server which actually was less powerful than nowadays NUCs

> I am using a ZimaBlade (2 cores, 16gb DDR3L RAM) as my cloud server, and an old Nvidia Jetson NX (very outdated ARM CPU with negligible GPU power) as my email server. I also run a bunch of frontends and apps on both. They work flawlessly.

How do you like the ZimaBlade and NVDA Jetson? I actually want to get these two, and have been consider going with ZimaBlade or ZimaBoard...

I want to have a Jetson for LLM purpose, but after checking the specs, I don't think it would be powerful enough though...

> ...... we were in the process of moving a bunch of hosting services on a mid-tower server which actually was less powerful than nowadays NUCs

Do you mean you are moving these hosting services to a mid-tower server (English is not my first language, so to is easier for me to understand than on). Would you say that even if they are not more powerful from performance perspective, but they offer some server-specific functionalities that NUCs don't, such as better thermals?

> I want to have a Jetson for LLM purpose, but after checking the specs, I don't think it would be powerful enough though...

Don't waste your money, you can make a way more useful and powerful build for slightly more money. Unfortunately the Jetson Xavier (at least the Xavier, no idea about the Orin but seen the price I'd say you can build a mid tower for the same price) is unable to run any LLM in a decent way ( i got like 0.1 token / s on a 7b model lol)

> How do you like the ZimaBlade and NVDA Jetson? I actually want to get these two, and have been consider going with ZimaBlade or ZimaBoard...

I use the Jetson only because else i would have to throw it away, to be honest I prefer the ZimaBlade hands down even if it is the dual core version.

About that, I am waiting for the quad core version to arrive. I liked it so much I bought another unit that I will use just for Nextcloud in RAID.

I am amazed by the zimablade performance especially as it is a very low specs machine theoretically. Never tried the zimaboard but I have no reasons to do it.

> Do you mean you are moving these hosting services to a mid-tower server (English is not my first language, so to is easier for me to understand than on). Would you say that even if they are not more powerful from performance perspective, but they offer some server-specific functionalities that NUCs don't, such as better thermals?

I am not native too, sorry if something was unclear. Yes, we were moving these hosting services to a mid-tower "server" (actually a normal PC with linux). Thermals surely were better handled due to the size of the build but from my personal experience you can add some thermal solutions to NUCs and get the same / better results. Note that the mid-tower build I am talking about would be considered low spec nowadays (I don't remember exactly the specs but it was surely pre-2019).

In general, I'd suggest trying the ZimaBlade 2 cores which is pretty cheap and can give you an insight of SBCs used as servers.

To give you a practical example, you can try

https://view.tcsenpai.com/feed/popular (invidious instance hosted on the Jetson Xavier)

https://swing.tunnelsenpai.win/#/ (hosted on the ZimaBlade)

In my opinion, while being way cheaper, the ZimaBlade obliterates the Jetson Xavier (which btw is forced to run at 20w vs 6w and must use Ubuntu 18.04 with a 5. something kernel)

> About that, I am waiting for the quad core version to arrive. I liked it so much I bought another unit that I will use just for Nextcloud in RAID. > > I am amazed by the zimablade performance especially as it is a very low specs machine theoretically. Never tried the zimaboard but I have no reasons to do it.

Thanks! Then I will consider buying one for myself!

> Yes, we were moving these hosting services to a mid-tower "server" (actually a normal PC with linux). Thermals surely were better handled due to the size of the build but from my personal experience you can add some thermal solutions to NUCs and get the same / better results. Note that the mid-tower build I am talking about would be considered low spec nowadays (I don't remember exactly the specs but it was surely pre-2019).

No problem, thank you for the clarifications!

> To give you a practical example, you can try > > https://view.tcsenpai.com/feed/popular (invidious instance hosted on the Jetson Xavier) > > https://swing.tunnelsenpai.win/#/ (hosted on the ZimaBlade) > > In my opinion, while being way cheaper, the ZimaBlade obliterates the Jetson Xavier (which btw is forced to run at 20w vs 6w and must use Ubuntu 18.04 with a 5. something kernel)

Fun examples!

The ECC stuff is like asking gearheads about motor oil.

You can absolutely do it. In a large production environment one of my teams used out of service thin clients (t620 iirc) to do some edge computing stuff. We also used t630s in a pinch to run ubiquiti controllers for temporary sites setup during COVID. They last forever and are surprisingly capable. The newer ones can handle 32GB ram and 1TB+ flash. The cpus are weak, but they don’t require cooling and sip power.

Just backup the locally generated work product if there is any.

Remember the “enterprise” nonsense is babble if you have a team who can think and operate outside the box. If you’re placing 1000 servers in dental offices nationwide, buy the HP server with the nationwide 24x7 support network. If you don’t need that, save the money.

> Remember the “enterprise” nonsense is babble if you have a team who can think and operate outside the box.

I am quite new to sysadmin, security, servers, networking, and so on.

Do you have any recommendations of materials (like books or videos) for these kinds of topics, e.g. how to optimize hardware

Sans the HP, they lack ECC support which automatically puts them in the 'home computer' market.

Why would you use home-grade "servers" when a startup has access to a variety of cheap cloud solutions? What business market segment in a server space would these mini PCs occupy?

> Why would you use home-grade "servers" when a startup has access to a variety of cheap cloud solutions? What business market segment in a server space would these mini PCs occupy?

This is more of a hypothetical question. From simple use, the Beelink SER5 I have seems to have better performance than my 13-year-old Macbook Pro, so I was curious about the upper bound / limits of little machines like it.

May I ask what you mean by "a variety of cheap cloud solutions", do you refer to services such as DigitalOcean's Droplet and/or similar products?

> From simple use, the Beelink SER5 I have seems to have better performance than my 13-year-old Macbook Pro, so I was curious about the upper bound / limits of little machines like it.

Beware of Beelink. They are known to suffer from weird hardware design issues (i.e., mouse/keyboard activity required to wake up from sleep mode but their USB stack is disabled when entering sleep mode) and are riddled with other hardware issues. I bought one and for example video randomly shuts down a few seconds after booting in without a way to recover.

Their hardware issues are surprisingly not the problem. The problem is that their support is limited to their message board, and it's heavily censored to hide all issues. I bothered to go through their china social credit score process to register a user account to report the issues I experienced and all posts were heavily censored. In fact, the first post I noticed being censored out was a reply to a boilerplate post with troubleshooting tips where I simply said that I tried out the suggestions but they weren't effective at fixing or mitigating the problem. This left their board with a thread where a problem is reported, they chime in with a list of fixes that they automatically post to all problems, and by magic no one bothered to reply.

Azure, AWS, GCP, DO as you noted all have viable alternatives, from IaaS to PaaS to SaaS.

Why run a web server when you can run an Azure Web Site? Why distribute servers around the world to serve as CDNs at POPs when you can put Cloudflare, Azure Front door, etc. etc. in front of it on the cheap?

Anyway, these small machines don't have a place in the modern server room. They don't fit in racks, they lack management features, expandability, replacement parts, warranty, and so on and so forth.

There is more to a server than "CPU goes faster". If a hypothetical server had the fastest CPU available but only provided a single NVMe interface (like many of these small PCs do), it'd be a poor candidate for a critical RDMBS, for example.

Why use your own computer when you could use someone else's computer.

Any computer, even an Android phone, can be used as a server. The issue is MTBF and maintenance in numbers. ECC might be not that important if your workload is not critical and you have a correct reboot policy.

Lack of BMC like IPMI is much worse, meaning you will need to hop on a trip or ask someone at the installation site to check your HW.

So what makes a server server is probably a mix of remote maintenance capabilities and redundancies to minimize said maintenance.

Just get a

  Gigabyte MC12-LE0
  Ryzen Pro 5650
  4x32GB DDR4 ECC RAM 3200 Modules
  

Much more of a server system with ECC Support, IPMI. With the newest BIOS there is Bifurcation Support and iGPU passthrough.

This board seems really cheap to get in Germany but seems extremely overpriced for what I can seem to get it for in the US.

EDIT: I did find the same listing on Ebay with similar pricing, so disregard my earlier quip about price

Quite a bit of issues regarding dead boards too, though that might be related to certain BIOS versions

https://forums.servethehome.com/index.php?threads/gigabyte-m...

My home server is a little square PC and it has done a marvelous job. It's massively overkill for my use, but it converts full movies to H264 quickly.

I would use an actual small server, with ECC RAM, and IPM. It would ideally be flexible so I can add/remove storage, RAM etc.

Do you have any recommendations and/or suggestions, homelab subreddit seems to recommend things like Dell R720.

Take a look at HL15 from 45Drives. Many will rush to tell you it’s expensive for homelabers. Sure, but not that much. The retail price of the motherboard is 800 USD, CPU, PSU, NVME RAM around 400-500. That leaves you with the case which is 800 USD. This is a premium case, but probably 200 USD more than I would like it to be.

You will get a motherboard up to 1TB of ECC RAM, 10G networking, space for 15 HDDs, IPMI, a 4U case that will last forever, space for GPU, and a lot of expansion and upgradability possibilities via PCIe.

Dell, HP, supermicro, … all sell entry level servers.

absolutely!

Checkout these links

* https://blog.codinghorror.com/the-cloud-is-just-someone-else...

* https://blog.codinghorror.com/the-scooter-computer/

The people screaming about ECC - meh, honestly you can grab some ecc ram and build a couple of small boxes if you want, but there's no reason you can't host a whole lot on these mini-pcs without ECC. Google had plenty of production going without ECC -- eventually they grew up though.

* https://blog.codinghorror.com/to-ecc-or-not-to-ecc/

> In the end, we decided the non-ECC RAM risk was acceptable for every tier of service except our databases.

Jeff recognizes that using ECC is critical where criticality matters.

> we burn in every server we build with a complete run of memtestx86 and overnight prime95/mprime

I'm unable to grok whether Jeff said to do this in a response to not using ECC, or if he's just stating this is good practice regardless of the type of memory, which it is. It's like fixing all compile-time errors only to ignore run-time errors.

> I find it very, very suspicious that ECC – if it is so critical to preventing these random

Jeff decided ECC was critical enough for his RDBMS. This statement seems to be at odds with the previous paragraph.

> memory corrupting bit flips – has not already been built into every type of RAM that we ship in the ubiquitous computing devices all around the world as a cost of doing business.

We can't see "ECC everywhere" because Intel prevented ECC from being everywhere via their market muscle.

As for the studies he's quoted, great, I have no reason to doubt them from a 10,000' view. If ECC is only good for 5%, 10%, 20%, or 25% corrections (or uncorrected but reported errors), that's better than non-ECC RAM which may have let the bitflip silently persist into your Active Directory database, nuking the CEO's account while IT runs around for hours or days struggling to figure out what happened, why, and how to prevent it from happening again.

It's an easy choice to make when the cost of ECC today is low.

Thank you for the recommendations! In fact I also read his article [0] yesterday, which was somewhat related to my questions.

[0]: https://blog.codinghorror.com/building-a-computer-the-google...

Lack of SATA ports, and sometimes lack of a PCI/PCIe slot which you can put a standard SATA card in mean you often are restricted to 2-disk models of RAID/Mirror filestore.

zimablade, for all its faults (USB-C power FFS. there are pins on the mobo for 12V direct in, if you want to risk it) and the low power budget to actually run drives.. has the PCIe slot. It means its just a tiny PC, intel arch, quad core, 16GB memory and .. a real PCIe slot which means you can run a real HBA.

I get round the power budget using an old ATX psu with the pins set to always-on, in the 24 pin ATX mobo connector.