I highly recommend you play Kerbal Space Program if you want to get an intuitive understanding for how it works. But in short, yes, and also if you accelerate towards another orbiting object your own orbit becomes more elliptical such that over sufficient distance you will actually move away from it rather than toward it.