> Ideally, the games themselves would have some sort of API, or some other way to either communicate what’s happening in the game so I can map colors to it
A number of games implement Mumble Link (https://wiki.mumble.info/wiki/Link) which provides a memory-mapped file that includes positional data used to provide directional sound by voice-comms software (so when a teammate speaks you hear their voice as if you were both in your relevant locations in the shared virtual space). Some games also expose other data through this including health/mana/ammo levels which could be used to control lighting based on game state.
Although this might be much, much harder because you'd have to trick each SDK into thinking it's talking to the devices it expects. A good example of this is a nice hack that takes Logitech LED commands and sends them to a Corsair keyboard instead - http://forum.corsair.com/v3/showthread.php?t=140755
This is one place where some common standards / APIs would have really helped. Instead, the community is left to do all the work.
That's another great idea. I'm looking for a Razer Chroma API, not much luck yet, but their site[1] seems to have some info. Of course, they're going to make it much easier for you to tell their keyboard what the color should be than to read that color...
There's been quite a few people exploring logitech gaming keyboards with LCD (G19 etc), adding pidgin plugins, games widgits etc. Maybe you can get enough from github to figure out what you need?
Or there's Roccat's power grid which interfaces games to a phone app. No idea how open or investigated that is though.
Aha! That's very interesting, thank you. I had no idea this was a thing, but it looks like I can use it for some interactions. I'll explore further, thanks again.
Next time, you should try some addressable LEDs. Neopixels, aka WS2811/12/12B, let you set the colors of individual bulbs instead of the entire strand. They work great with ESP8266!
https://www.amazon.com/ALITOVE-WS2812B-Individually-Addressa... are perfectly cromulent LEDs and less expensive than Neopixels. Not as cheap as the RGB strips, but if you pick off more points for colour analysis, you could adjust them along the edges, if you really wanted to.
Nice work on the project!
It wasn't actually that hard (or at least the learning curve was smooth), I only have a few months' experience with hardware. Plus, it was extremely fun!
neat! i was just looking at the dreamscreen a few days ago (https://www.touchofmodern.com/sales/dreamscreen/dreamscreen?...) but the lightpack2 looks even more elegant/smooth in the way it reacts. it even has the "pixels" that add an interesting element...
Would it be possible to implement some sort of "HDMI proxy" using something like the raspberry pi? You would plug the HDMI input (e.g. from xbox) into the raspberry pi, and then another HDMI cable from the raspberry pi to the TV. The raspberry pi would then somehow read the input stream, parse the bits to get the color you want, and forward the input stream unmodified to the output.
I have no idea how HDMI works, so I don't know if this is even possible. Would you see a degradation in quality even though you are not re-rendering anything? i.e. Can you do this without a high powered graphics card acting as the HDMI proxy? Is it even possible to intercept HDMI stream, or is it a proprietary protocol?
EDIT: Some good info here [0]. tl;dr, seems possible with < $200 hardware.
I'm building a 100w RGB LED light show that will also be controlled with the ESP8266. The software will just receive UDP packets and maybe do a bit of interpolation if necessary.
It's a bit trickier since I need to drive them at ~30v and 1 amp per channel :)
Hmm, probably not too much trickier, my controller was designed for 1 amp per channel as well, although at 12V... I'm guessing your leds will be individually addressable, though? Or maybe you'll just have more controllers. Either way, it sounds like a lot of fun!
For reading directly from a program's memory, have you tried using Cheat Engine/scanmem to find the location of the variables you're looking for? I think that'd be easier than using IDA.
Yes, I tried that too, but it's hard for me to find the location of something like a color, which changes every frame. I know that people have done it, I'm just not aware of an efficient way to find something that you don't know the value of or how it changed.
If you have any tips, please share, because that would be very useful for this project.
Cool project! As many games lack APIs, you could potentially analyze the raw audio and the graphics in real time, and use machine learning for detecting certain events, such as goals in rocket league
I've been working on a weekend project to make lights that react meaningfully to music, by taking into account beats and chords. Here's a video of it: https://www.youtube.com/watch?v=h5sRgSmRGpg
Yes Chordify has time information - it's pretty accurate too, and works for folk, rock, classical, whatever.
It's just a matter of timing the lights with the audio output. My setup is a Raspberry Pi as a master, and an Arduino to talk to the lights (which is necessary because the RPi's pins are too slow to send signals to an addressable LED strip). The RPi sends chord and beats in real time as they happen, over USB. Every few seconds it syncs the light show clock with the audio player clock.
Hmmm, good idea, I didn't think of analyzing the sound. You can pretty easily tell when the "boom" goal sound plays, but not who scored. Maybe you can tell the latter by reading the screen, though. I will have a play when I get back home, although I still need to find a quick method to take screenshots in Windows.
This one is actually way easier than Super Hexagon! You can use a memory cheat tool to look for the current scores, then watch for changes in those addresses. I'd be happy to hunt that down and write up an implementation if you want; I'd love to have the Hue lights in my theater change for Rocket League.
A number of games implement Mumble Link (https://wiki.mumble.info/wiki/Link) which provides a memory-mapped file that includes positional data used to provide directional sound by voice-comms software (so when a teammate speaks you hear their voice as if you were both in your relevant locations in the shared virtual space). Some games also expose other data through this including health/mana/ammo levels which could be used to control lighting based on game state.