There's a chrome extension that records all your steps and then generates a puppeteer/playwright script for you. Then you can just edit bits of that script and run it. This is my go-to method presently.
I feel like you could incorporate some of those features into this and eliminate some of the manual entry of css selectors.
UI Vision is unique because it supports visual automation with image recognition and OCR. A bit like Sikuli but inside the browser. This avoids dealing with CSS selectors and XPaths.
We are using Automa chrome extension for this kind of use cases. We really like it. It is free and open source. If you are thinking of building a product, you can make use of it.
You can do it in the chrome browser tools! There's a record section. Note if you're doing this for automation testing it's going to be pretty flakey and you're better off following best practices and using custom testing selectors instead :)
Why is it flakey for automation testing though? In my experience that's basically targeting page elements and interacting with them. Doesn't this do the same thing?
It uses a combination of xpaths that are like body[1]div[2]div[1]p[2]button[1] so if you edit the UI it'll break, or worst case they do X,Y coordinates of clicks on a page so if the UI changes it'll break. Sometimes it'll get your ID or Name or whatever but some frameworks randomize these on compilation and some devs change them as part of refactoring.
Best practice is to add test attributes to the HTML elements (such as data-testid="SubmitButton") so if the page is moved around or the button is edited the tests remain stable. There's a lot of good online sources on making UI testing somewhat more reliable :)
That’s good to know, I had bad luck with it then. But even so, you also lost the ability to use testing patterns like the page object model where you can update changing IDs or flows more easily - if something changes drastically early in the recording, wouldn’t it need to be re-recorded for all test cases?
Selenium is more widespread but I recommend Playwright as the most modern automation library. Comes with plugins for making your automations more... nimble.
Selenium is the most popular tool in the industry but mostly for legacy reasons, its real competitor is Cypress. If you've just started a project I'd recommend checking it out, a huge time saver is that you install it as an npm package with a headless browser out the box without any webdriver/infra stuff slowing you down.
I feel like you could incorporate some of those features into this and eliminate some of the manual entry of css selectors.