Totally agree. As long as it's not just a "higher" offer, going with the company you really think is better for you (both financially and technically), is always the right move. Even if you accepted the offer, it's OK to rescind, like Steve said above, just be honest and direct, yes it's awkward, but it's definitely more awkward for you than the company that made you the offer. If Company A was a really small company, be extra communicative as recruiting is hard for small teams :)