If you're trying to make a good onboarding user experience then you should do your onboarding testing with people who've never seen the product before, not devs or QA. Once people are familiar with the product (devs, QA, and anyone who has used it before) then they're "tainted". They'll remember the weird way that they had to work around an issue, and that'll just end up being "the way it is" rather than something to fix.
I've read that a strategy for this is create an ad and pay people $50 to come in and try to use your software. Tell them to do something in your software and see what they get hung up on. The worst UX problems will be hit by nearly every user.
As simple as that is, none of my employers have ever done this. The closest was one of the bosses asking his wife to try out the software.
You can go one simpler if you need, you can use something like https://www.usertesting.com/ which will do the ad side of it for you. I used to do testing for them for some beer money. You'd use something for the first time and try and achieve some task, often talking through it, and it's all either screen recorded or it used to be videoed for phones.
Having someone in person can be extremely valuable for other reasons, but this can be a quick approach.
For larger customers, going onsite and watching them use your tools is so valuable.
> Once people are familiar with the product (devs, QA, and anyone who has used it before) then they're "tainted".
I broadly agree, though this is where I'd split out really good QA people I've worked with. The added advantage is they can also explain the change required that would get some user X to have a better experience (e.g. how your autistic users may get more stuck at a certain place, or how to change the flow such that a 3-4 year old can navigate a UI).
I've read that a strategy for this is create an ad and pay people $50 to come in and try to use your software. Tell them to do something in your software and see what they get hung up on. The worst UX problems will be hit by nearly every user.
As simple as that is, none of my employers have ever done this. The closest was one of the bosses asking his wife to try out the software.