Words can definitely be the actus reus of a robbery. You also need the mens rea.

There are very few actions that are always criminal. Sex with a minor is the only major one. Things like speeding if you count small fines.