> If you're doing AI stuff, any Python framework will be best.
I understand that AI stuff now is mainly Python. But is it possible to connect Rails with some Python program/service, is this one of the purpose of microservices?
Thank you for your reply. In your other comment, you mention that you know Ruby relatively well. Would you recommend learning Ruby along side with Rails (i.e., learning Ruby in/with Rails), if I have some experience with other languages, such as Java, Python, C/C++, Lisp, and so on?
Also are there any Ruby AI libraries that you would recommend? Thanks!
Yes, of course. One of the Ruby shops I worked for did exactly this.
But why add the extra complexity? Ruby and Python are similar enough in terms of productivity and processing time that there's no reason to do this IMHO.
Congrats OP and it looks really nice! Out of curiosity, may I ask what’s ur tech stack for developing this software? I’m also an aspiring app developer :) Thank you!
May I ask how small? From what I see on r/localllama, it seems that at least 3B and 7B ones are starting to be useful? Also I cannot use CUDA or modern AMD cards..
My YOShInOn RSS reader uses BERT-family embeddings to capture the "gist" of text the way the neural network for that search engine did, then I treat content-based recommendation as a classification problem. (e.g. "Will the user like this?")
My system is quite a bit different from others because, like TikTok but even more so, it demands a thumbs up/thumbs down judgement for every article so I get a set of negative samples that are really reliable.
There are numerous frontiers of improving this system. One of them is that there are certain things, like "roundup" articles that cover a wide range of topics (say https://www.theguardian.com/world/live/2024/jun/03/russia-uk...) that the embedding doesn't capture well, adding some new features could clear out maybe 10% of articles I'd rather not see but I am not in such a rush because overall the system is very satisfying to me and I am already blending in more random articles than that to get samples to keep it calibrated and also sometimes discover new topic areas I find interesting.
Another interesting frontier is sequential recommendation
but I'm not sure if I really want to take an ML approach to this because I'm not sure there is enough training data for one person's content-based recommendation. I'm not sure exactly how I want to do it, but when I post to a place like HN I do not want to post a stick of five articles from phys.org, rather I want there to be some diversity in my feed not just over the course of a 300 article batch but on the scale of individual posts. Items can be hung up in queues for several days in this process; most "news" on HN is fairly evergreen and it doesn't matter if it is delayed a week but articles about sports have a short shelf life as you look like a fool if you post an article about what happened in week 3 of the NFL after the games of week 4 have played. So I need some way for sports articles to "jump the line" ahead of other articles but I don't want that to privilege sports over everything else.
Similarly there is "the probability that article A is relevant" but there is also "Is A or B a higher quality article?" One Google innovation was using a document quality score (PageRank) asides a normal document-query ranking which is tricky because now you're not optimizing for one thing but trying to optimize for two things that could compete with each other. I am thinking about switching that system from a batch to a streaming mode and need some answers for that.
Take a look at HL15 from 45Drives. Many will rush to tell you it’s expensive for homelabers. Sure, but not that much. The retail price of the motherboard is 800 USD, CPU, PSU, NVME RAM around 400-500. That leaves you with the case which is 800 USD. This is a premium case, but probably 200 USD more than I would like it to be.
You will get a motherboard up to 1TB of ECC RAM, 10G networking, space for 15 HDDs, IPMI, a 4U case that will last forever, space for GPU, and a lot of expansion and upgradability possibilities via PCIe.
Dell, HP, supermicro, … all sell entry level servers.
> Why would you use home-grade "servers" when a startup has access to a variety of cheap cloud solutions? What business market segment in a server space would these mini PCs occupy?
This is more of a hypothetical question. From simple use, the Beelink SER5 I have seems to have better performance than my 13-year-old Macbook Pro, so I was curious about the upper bound / limits of little machines like it.
May I ask what you mean by "a variety of cheap cloud solutions", do you refer to services such as DigitalOcean's Droplet and/or similar products?
> From simple use, the Beelink SER5 I have seems to have better performance than my 13-year-old Macbook Pro, so I was curious about the upper bound / limits of little machines like it.
Beware of Beelink. They are known to suffer from weird hardware design issues (i.e., mouse/keyboard activity required to wake up from sleep mode but their USB stack is disabled when entering sleep mode) and are riddled with other hardware issues. I bought one and for example video randomly shuts down a few seconds after booting in without a way to recover.
Their hardware issues are surprisingly not the problem. The problem is that their support is limited to their message board, and it's heavily censored to hide all issues. I bothered to go through their china social credit score process to register a user account to report the issues I experienced and all posts were heavily censored. In fact, the first post I noticed being censored out was a reply to a boilerplate post with troubleshooting tips where I simply said that I tried out the suggestions but they weren't effective at fixing or mitigating the problem. This left their board with a thread where a problem is reported, they chime in with a list of fixes that they automatically post to all problems, and by magic no one bothered to reply.
Azure, AWS, GCP, DO as you noted all have viable alternatives, from IaaS to PaaS to SaaS.
Why run a web server when you can run an Azure Web Site? Why distribute servers around the world to serve as CDNs at POPs when you can put Cloudflare, Azure Front door, etc. etc. in front of it on the cheap?
Anyway, these small machines don't have a place in the modern server room. They don't fit in racks, they lack management features, expandability, replacement parts, warranty, and so on and so forth.
There is more to a server than "CPU goes faster". If a hypothetical server had the fastest CPU available but only provided a single NVMe interface (like many of these small PCs do), it'd be a poor candidate for a critical RDMBS, for example.
I understand that AI stuff now is mainly Python. But is it possible to connect Rails with some Python program/service, is this one of the purpose of microservices?
reply