In a controlled environment - where the object to pick up, for example, is always in exactly the same location - you could do that. If there is any variation in the location of the object, you need vision to localize it each time. You need a camera, maybe two, and probably some kind of 3d perception, which is an unsolved problem at the moment (well, not solved in a general way, there are some solutions for specific objects).