If I'm a large institution buying floor cleaner, I'll probably wash a floor or two before buying. If I notice holes being burned in my floor, maybe I'll go with an alternate vendor.
For this device, it's pretty clear how to beware of it for anyone who took stats 101. Put a grenade or a soda can (depending on a coin flip) into a bag. Point the detector at the bag, record the # of times the detector goes off when grenade vs soda can. Do this double blind.
The most important requirement for the testing methodology is double-blindness. Only someone familiar with proper scientific testing procedures would know that, and even scientists who should know better have gotten this wrong. Most people would probably think that single-blind testing would be enough to confirm whether this works or not (assuming you couldn't judge its plausibility on pure physics principles), but you would be amazed at the ability of humans to pick up informational cues unconsciously. The way dowsing works, in general, is through ideomotor responses[1]. The stick moves due to unconscious muscle movements that are amplified by the instability of the rod itself. The muscles move unconsciously based on what the person believes.
Even if the person who is operating the device doesn't know if the grenade is present, they may be able to pick up unconscious physical tells from the person who does (if he is present during the test). When the person who knows sees the tester approaching, they may unconsciously reveal this knowledge through body language that could be picked up by the tester. A similar phenomena occurred with a horse[2].
This is all true. But on the other hand, this is all knowledge accessible to any college graduate who majored in a STEM field. Any army or other large organization should have at least one or two such people on hand.
For this device, it's pretty clear how to beware of it for anyone who took stats 101. Put a grenade or a soda can (depending on a coin flip) into a bag. Point the detector at the bag, record the # of times the detector goes off when grenade vs soda can. Do this double blind.
Use this calculator to determine how many times to do this: http://www.evanmiller.org/ab-testing/sample-size.html
Then type the results into this calculator:
http://www.evanmiller.org/ab-testing/chi-squared.html
Don't buy if it says "No Significant Difference".
Total resources needed: 2 soldiers, 1 grenade, 1 coin, internet.