That's fine for the main article but I think there should be a way to get higher quality images should the reader request them. If power is a concern those can be hosted elsewhere.
I think it's acceptable for the drawings to be compressed this way but the photographs are very unclear.
The issue is that it's hacky, and in that case I'd rather go with a Intel or AMD x86 system with more or less out of the box Linux support. What we're looking for is a performant ARM system where Linux is a first class citizen.
It might be fun to have so many machines, but in reality it's simpler and cheaper to virtualize everything on two or three powerful hosts. Considering that you're using a soundproof rack already you might as well go with used rack servers with lots of memory and compute. Those also come with goodies like BMCs, dual PSUs, and ECC.
Personally I have two Xeon rack servers running in a Proxmox cluster with a SBC Qdevice. It has more than enough memory and compute for my needs and it also serves as my virtualized router and NAS. The whole setup only takes up 4U of space (servers + switch + modem/qdevice) with a single UPS on the floor, and idle power is around 150W.
We may be optimizing for different needs. For instance, while I was able to get a significant amount of extra height, I didn't have a lot of cabinet depth to work with, which is somewhat limiting for traditional server hardware. There are short-depth options out there, but I also wanted at lease some GPU capability. The integrated GPUs in the SER9s are not top of the line by any means, but they're more than capable for what I want to be working on.
9 degrees. arcsin(arccos(arctan(tan(cos(sin(9)))))) basically makes a set of sin-cos-tan layers that arctan-arccos-arcsin unwrap one-by-one, which should result in nothing having changed, unless the functions used weren't accurate.
There is no choice here - each inverse is uniquely determined. That's similar to how 3 and -3 are both square roots of 9 (i.e., solutions to x^2=9), but sqrt(9)=3 as it denotes the principal square root, which by convention is always the non-negative value. Of course, in a different context we might design functions to have multi-valued properties, like atan2(x,y) != atan(y/x) in general (atan2 takes quadrant in account and returns full range [-pi, pi], atan only returns principal values in [-pi/2, pi/2]) as practical applications benefit from preserving quadrant beyond just the principal inverse (or not failing when x=0!)
The inverse branches are not unique, you might think there is no choice being made but picking the standard branch is a choice b/c I can always shift the result by 2π by picking a different branch of the inverse. The answer is not unique & the assumption is that the calculators are using the standard branch.
Of course, but the choice is standard and thus the answer is 9. I can define a non-standard sqrt(x) which sometimes gives the positive root and sometimes the negative one, and then sqrt(sqrt(16)) could be -2 or undefined (if I defined sqrt(16)=-4) but that's just silly - the only reasonable interpretation for what the calculator should show for sqrt(sqrt(16)) is simply 2.
You can assume that sin(9) is within the range of all the functions that are post-composed w/ it so what you end up w/ in the end is arcsin(sin(9)). Naively you might think that's 9 but you have to be careful b/c the standard inverse branch of sin is defined to be [-1, 1] → [-π/2, π/2].
Edit: The assumption is that the calculators are using specific branches of the inverse functions but that's still a choice being made b/c the functions are periodic there are no unique choices of inverse functions. You have to pick a branch that is within the domain/range of periodicity.
arcsin(arccos(arctan(tan(cos(sin(9)))))) = 9 (in degrees mode - when regular trig functions output pure numbers, those numbers get interpreted as degrees for the next function and similar for inverses - calculator style), because each intermediate lands in the principal-value domain of the next inverse (e.g., arctan(tan(x)) = x when x \in (-90°, 90°) and the intermediates happen to be in those ranges). Specifically, sin(9°) ≈ 0.156434, cos(0.156434°) ≈ 0.999996, arctan(tan(0.999996°)) = 0.999996°, arccos(0.999996)≈0.156434°, arcsin(0.156434)≈9°.
Hold on, you're only getting 45 tokens/sec with Mistral 7B on a 5090 of all things? That gets ~240 tokens/sec with Llama 7B quantized to 4 bits on llama.cpp [1] and those models should be pretty similar architecturally.
I don't know exactly how the scaling works here but considering how LLM inference is memory bandwidth limited you should go beyond 100 tokens/sec with the same model and a 8 bit quantization.
My understanding is that quantizing lowers memory usage but increases compute usage because it still needs to convert the weights to fp16 on the fly at inference time.
Clearly I'm doing something wrong if it's a net loss in performance for me. I might have to look more into this.
Yes it increases compute usage but your 5090 has a hell of a lot of compute and the decompression algorithms are pretty simple. Memory is the bottleneck here and unless you have a strange GPU which has lots of fast memory but very weak compute a quantized model should always run faster.
If you're using llama.cpp run the benchmark in the link I posted earlier and see what you get; I think there's something like it for vllm as well.
In my case yes, because I did not care about breaking email delivery to that domain (it's a novelty domain pointing to my residential IP address, (surname-home.tld) which I use exclusively for my selfhosted
Services
I know the feeling. Ive already configured almost all services to be header auth or disabled auth entirely if possible, and just put them all behind a SSO forward proxy (nginx + authentik)
I also played around with injecting a tiny script into the proxied response to just add a small drop down menu with all services I've got available. .. while that worked, finding a good place to inject that menu was a chore so it's currently disabled again :)
I've hosted at home for years and if you have it properly setup it's not any more risky than using a VPS. I have 443 open on my router and basically all web traffic is routed to a container on my server. The container is on an isolated vlan and basically runs nginx as a ssl reverse proxy.
The actual web services behind the proxy run in their own containers and with proper isolation and firewall rules the effects of a security compromise are limited. At most an attacker will be able to take over the containers with an exploit (and they could do that with a VPS as well) but they won't be able to access the rest of the network or my secure internal systems.
If I was this guy and wanted to let people connect directly to my vapeserver I would simply host it on another vlan and port forward the HTTP connection. Even if someone manages to take over such an obscure system they're not going to be able to do much.
The problem with custom ROMs is that many government, banking, and similar apps don't run on them without workarounds. Some of those apps also consider this as a TOS violation as well.
When Microsoft first proposed a remote attestation scheme for PCs under the name Palladium, it was widely seen as a nightmare scenario. Even the mainstream press was critical[0]. There was barely a whimper when Google introduced Safetynet a decade later.
It wasn't OK in 2003. It wasn't OK in 2014. It isn't OK now. I'm just not sure what anybody can do about it.
There are many third-party money apps that login to your online banking that are a violation of ToS. That doesn't stop people using them. In fact, when they get really big, they can be legitimised by banks. For example, to get my mortgage, I had to use a third party service that logs in to my online banking account and ingests all my transactions to show that I saved for my deposit legitimately.
Then I won't run those apps. Seriously. I know not everyone has this option, but it's been my experience that a lot of processes do in fact have workarounds when you show them the cryptic error their poorly behaved app throws.
GrapheneOS has offical support for hardware attestation[0].
It does require the developer to make minor adjustments, and most banks are simply too risk averse to agree to doing that (I would know, used to be a senior android app dev at a bank).
I don’t use any utility apps (identity, banking, services etc) on my phone and stick to the desktop web. And don’t use services that do require me to have a Google or apple account and phone. (Spoiler: I do)
I hope my tiny datapoint shows up in some aggregated stats somewhere.
I think it's acceptable for the drawings to be compressed this way but the photographs are very unclear.
reply