That's using https://shot-scraper.datasette.io/ to get just the document.body.innerText as a raw string, then piping that to gpt-3.5-turbo with a system prompt.
In terms of retaining context, I added a feature to my strip-tags tool where you can ask it to NOT strip specific tags - e.g.:
First, love your work. Sadly, I don't think this path would work for me as for what I do, I need the selectors as part of the workflow.
Roughly, my end goal is to do a single or multi-shot with the following information HTML differential (could be selectors, xpaths, data regions, differentials of any of the above, etc...), code stacktrace, related code, and prompt.
For this example, let's consider that the flow involves the bot to login to a website. I have selectors for the `.username` and `.password` inputs and then a selector for the login button as `.login-btn`.
1. The site updates their page and changes up all their IDs, but keeps the same structure.
2. The site updates their page and changes up all their IDs, but changes the structure and the form is named something different and is somewhere else in the DOM.
3. many... many other examples.
Trying to figure out how to minimize the tokens, but keep the needed context to regenerate the selectors that are needed to maintain the workflow.
Yeah, I've been thinking a bit about that kind of problem too.
My hunch is you could do it with a much more complex setup involving OpenAI functions - by trying different things (like "list just input elements with their names and associated labels") in a loop with the LLM where it gets to keep asking follow-up questions of the DOM until it finds the right combination.
I have a strip-tags CLI tool which I can pipe HTML through on its way to an LLM, described here: https://simonwillison.net/2023/May/18/cli-tools-for-llms/
I also do things like this:
Output here: https://gist.github.com/simonw/3fbfa44f83e12f9451b58b5954514...That's using https://shot-scraper.datasette.io/ to get just the document.body.innerText as a raw string, then piping that to gpt-3.5-turbo with a system prompt.
In terms of retaining context, I added a feature to my strip-tags tool where you can ask it to NOT strip specific tags - e.g.:
That strips all HTML tags except for h1, h2 and h3 - output here: https://gist.github.com/simonw/fefb92c6aba79f247dd4f8d5ecd88...Full documentation here: https://github.com/simonw/strip-tags/blob/main/README.md