Have an NES game you're particularly fond of? Load it up in the latest fceuxdsp and start playing around with the debugger, RAM/ROM/PPU viewers, trace logger, etc. Try to figure out where the level data is stored in the ROM, how it's loaded into RAM and converted into name tables, how the game logic works, etc.
The NES uses basically a 6502, which is 8-bit and has a limited instruction set, so it's totally grokkable. It's fun since it's a real game, too, instead of some toy example from the web.
I've been spending a few weekend dissecting MCKids[1], trying to dump level images for TCRF, with great results. It's my first time doing this, so it's hard, slow work, but the breakthrough moments are absolutely worth it.
I'll bet the details took a lot of work but conceptually it isn't hard to understand at all. There is a bug which causes the game to jump into a bad address. The goal then becomes to manipulate the game so that this bad address happens to contain valid code, then trigger the bug.
You don't really need to know how RAM works to understand that. How assembly programming or Von Neumann architecture works perhaps. Or basic knowledge of buffer overflows and similar concepts.
Yeah sure. I guess what I really meant was I don't know how to even start to debug the ram to understand where the addresses are, and what each part does.