wouldn't call it quits but would do a serious review of your goals, product, and current user acquisition methods.
It sounds like you are looking for more growth and more users (duh).
After 2 years you have a solid baseline to draw conclusions from and now need to review usage stats - talk to users (pick up the phone)- find out what they love and what they don't love.
Make a plan for updates - relaunch- and set a time line for this second stage - if your at the same spot in terms of users then that is a good signal for moving on.