A fun trick is that if you're sure the batteries are in series you can sometimes swap out a pair of AA batteries for a 3.7V 14500 Lithium rechargeable and a fake AA battery (just a wire in an AA-sized plastic shell). Drill a hole in the battery-cover of the device where the 14500's USB port is and now your device is plug-in rechargeable, assuming it can handle 3.7V instead of the normal ~3.1V that comes from a pair of AAs (which isn't a huge stretch). Worked for my old Logitech game controllers.
Downside is that you've got parts in your bin that are dangerous because they look like AA batteries but could cause damage or even fires if somebody put them into the wrong AA thing. Make sure to mark the batteries loudly. I've electrical-taped the pairs together to prevent this problem.
I have a headlamp for hiking with a slightly different take on this.
It will take 3x ordinary or rechargeable cells, or it will take it's Li-Ion pack that is the same size as the 3 cells side by side. Designed in from the start so there are no dangerous bits. Hiking headlamps are something where you do *not* want to be left with a dead battery!
A headlamp that can run on either AA or 14500 tends to be a better approach than this. Those proprietary batteries are usually pretty expensive compared to a 14500, and three AAAs as spares aren't nearly as convenient as one AA.
There are a handful of these on the market and they're not common in retail stores. I'm fond of the Skilhunt H150.
Follow up on this: from conversation elsewhere, another important safety tip is to get the 14500s with fancy electronics incl a USB port, because without that the battery will be a fire-hazard in this kind use.
Downside is that you've got parts in your bin that are dangerous because they look like AA batteries but could cause damage or even fires if somebody put them into the wrong AA thing. Make sure to mark the batteries loudly. I've electrical-taped the pairs together to prevent this problem.