There's a lot of affordable choices depending on what you want, but everything has tradeoffs. I love my Microfreak, but it only has 4 voices of polyphony, a weird keybed and no onboard effects. That said it makes amazing sounds and has a very discoverable set of controls, and since I play guitar I have external effect pedals at home anyway.

If you're not committed to hardware, a nice midi controller and a software synth will let you learn without spending big bucks. And even then the midi controller is optional.