One thing that has worked for me when I have a long list of requirements / standards I want an LLM agent to stick to while executing a series of 5 instructions is to add extra steps at the end of the instructions like "6. check if any of the code standards are not met - if not, fix them and return to step 5" / "7. verify that no forbidden patterns from <list of things like no-op unit tests, n+1 query patterns, etc> exist in added code - if you find any, fix them and return to step 5" etc.
Often they're better at recognizing failures to stick to the rules and fixing the problems than they are at consistently following the rules in a single shot.
This does mean that often having an LLM agents so a thing works but is slower than just doing it myself. Still, I can sometimes kick off a workflow before joining a meeting, so maybe the hours I've spent playing with these tools will eventually pay for themselves in improved future productivity.
There are things it’s great at and things it deceives you with. In many things I needed it to check something for me I knew was a problem, o3 kept insisting it were possible due to reasons a,b,c, and thankfully gave me links. I knew it used to be a problem so surprised I followed the links only to read black on white it still wasn’t. So I explained to o3 that it’s wrong. Two messages later we were back at square one. One week later it didn’t update its knowledge. Months later it’s still the same.
But at things I have no idea about like medicine it feels very convincing. Am I in hazard?
People don’t understand Dunning-Kruger. People are prone to biases and fallacies. Likely all LLMs are inept at objectivity.
My instructions to LLMs are always strictness, no false claims, Bayesian likelihoods on every claim. Some modes ignore the instructions voluntarily, while others stick strictly to them. In the end it doesn’t matter when they insist on 99% confidence on refuted fantasies.
The problem is that all current mainstream LLMs are autoregressive decoder-only, mostly but not exclusively transformers.
Their math can't apply modifiers like "this example/attempt there is wrong due to X,Y,Z" to anything that came before the modifier clause in the prompt.
Despite how enticing these models are to train, these limitations are inherent. (For this specific situation people recommend going back to just before the wrong output and editing the message to reflect this understanding, as the confidently wrong output with no advisory/correcting pre-clause will "pollute the context": the model will look at the context for some aspects coded into high(-er)-layer token embeddings, inherently can't include the correct/wrong aspect because we couldn't apply the "wrong"/correction to the confidently-wrong tokens, thus retrieves the confidently-wrong tokens, and subsequently spews even more BS.
Similar to how telling a GPT2/GPT3 model it's an expert on $topic made it actually be better on said topic, this affirmation of that the model made an error will prime the model to behave in a way that it gets yelled at again... sadly.)
> solo dev/no name company is going to suddenly drop a product
A developer/company with an opaque background that you're to trust to give access to backend systems using passwordless embedded SSH (no keys needed!).
That's a big NOPE.
(Also, even the answers OP has provided really give an AI bot vibe)
Octelium's author here. You don't give me access to anything. The project is 100% open source and designed specifically for self-hosting. I don't even know whether you're using the project or not since there isn't usage telemetry to begin with. As for the SSH part of your weird comment, I wonder whether you even understand what embedded passwordless SSH means in the first place.
For me it highlights the issue of how easily nefarious/misleading information will be able to be injected into responses to suit the AI service provider's position (as desired/purchased/dictated by some 3rd party) in the future.
It may respond 99.99% of the time without any influence, but you will have no idea when it isn't.
No mention of privacy (or on prem) - so assumed it's 100% cloud.
Non-starter for me. Accuracy is important, but privacy is more so.
Hopefully a service with these capabilities will be available where the first step has the user complete a brief training session, sends that to the cloud to tailor the recognition parameters for their voice and mannerisms... then loads that locally.
I would hazard to guess that Google classroom (starting at Kindergarten and continuing through post secondary) software is mostly installed via next-next-finish (i.e. whatever the defaults set by Google are). I'd also assume that these defaults are set to very minimal privacy protection for students.
Having this digital record entrusted to any company that is not under strong privacy controls should be frightening to parents.
School administrators figure the low-cost low-barrier-to-entry is well worth the long term privacy risk to children.
* Fortunately my children were out of school when this became common place - so kindly correct me if I'm mistaken.
You are correct but it's worse than that. School admins are full blow ignorant to technological privacy risks for children and themselves. Same goes for teachers if I'm being honest. Just assume their level of understanding is equivalent to the general population.
They are ambivalent/confused when you try and explain it.
I've assumed that when AI becomes much more mainstream we'll see multiple levels of services.
The cheapest (free or cash strapped services) will implement several (hidden/opaque) ways to reduce the cost of answering a query by limiting the depth and breadth of its analysis.
Not knowing any better you likely won't realize that a much more complete, thoroughly considered answer was even available.
> The cheapest (free or cash strapped services) will implement several (hidden/opaque) ways to reduce the cost of answering a query by limiting the depth and breadth of its analysis.
This already happens. Many of the cheap API providers aggressively quantize the weights and KV cache without making clear that they do.
But... I wouldn't want to be an outlier i.e. serious injuries. That would require suing the driver that has few/no assets.
Uber/Lyft sure as hell ain't going to let you sue them for a dime.