I've been using this or something similar internally for months and love it. The thing that gets downright spooky is the comments believe it or not. I'll have some method with a short variable name in a larger program and not only does it often suggest a pretty good snippet of code the comments will be correct and explain what the intent behind the code is. It's just a LLM but you really start to get the feeling the whole is greater than the sum of the parts.
I just don’t understand how anyone is making practical use of local code completion models. Is there a VS Code extension that I’ve been unable to find? HuggingFace released one that is meant to use their service for inference, not your local GPU.
The instruct version of code llama could certainly be run locally without trouble, and that’s interesting too, but I keep wanting to test out a local CoPilot alternative that uses these nice, new completion models.
There are a bunch of VSCode extensions that make use of local models. Tabby seems to be the most friendly right now, but I admittedly haven't tried it myself: https://tabbyml.github.io/tabby/
