I've been closely following the discussion on k3s and Kubernetes in general. I recently acquired an M1 Mac Ultra, and I'm curious about the best options available for running Kubernetes locally on it.
Does anyone have experience or recommendations in this area? I've heard about a few tools but wanted to gather insights from this knowledgeable community.
I switched to it completely, it’s very convenient to have both fast (-est on Mac) Docker support and a really smooth VM setup for running occasional Linux tools (such as Yocto in my case).
Edit: added some background info to my recommendation.
If you go the kind route, be sure to not use Docker for mac and instead opt for podman which is much more resource efficient. Now that I've switched over to podman, my computer doesn't sound like it's about to blast off when I'm running clusters.
The tools are a bit rough around the edges if you try and do something outside of the happy path with them. Nothing bad as such, just the user experience isn't as seamless when say running the VMs on custom, addressable host networks or managing vms with launchd.
Is this possible to fine tune llama-2 locally on M1 Ultra 64GB, I would like to know or any pointer would be good. Most of them are on Cloud or using Nvidia Cuda on linux.
I don't think so. I have M1 Max 64GB and it works okay for some inference. I'm buying a few credits from RunPod. It will be a few 10's of dollars to get it trained.
Coqui-TTS with vtck/vits is very good right now. Not as good as eleven labs or coqui studio, but for fast open TTS it's pretty good, in case you're not familiar with it.
It will be great when there's eventually something open that competes with the closed models out there.
And there are open-source alternatives but I don’t think the quality is super good.
There’s also enough information out there to do this yourself with a bunch of GPU time, I have some ideas I want to try out but don’t have the (GPU) time.
Going untraceable is the stuff of novels. Pulling that off in the real world is almost certainly well beyond the abilities of any of these individuals.
I highly recommend ClearML for effortless experiment that just works. It does a lot more of MLOps besides experiment tracking but I haven’t used those functionalities
I had researched and spent time with several other tools including DVC, GuildAI and MLFlow but finally settled on ClearML. WandB pricing is too aggressive for my liking (they force an annual subscription of $600 last I checked)
There are a lot of tools in this space. Shameless plug to follow.
I helped build and use Disdat, which is a simple data versioning tool. It notably doesn't have the metadata capture libraries MLFlow has for different model libs, but it's meant to a lower-layer on which that can be built. Thus you won't see particulars about tracking "models" or "experiments", because models/experiments/features/intermediates are all just data thingies (or bundles in Disdat parlance). For the last 2+ years we've used Disdat to track runs and outputs of a custom distributed planning tool, and used Disdat-Luigi (an integration of Disdat with Luigi to automatically consume/produce versioned data) to manage model training and prediction pipelines (some with 10ks of artifacts). https://disdat.gitbook.io/disdat-documentation
Does anyone have experience or recommendations in this area? I've heard about a few tools but wanted to gather insights from this knowledgeable community.