The number of visitors you have is irrelevant.

If you have any visitors you are doing them a huge disservice if you do not have SSL session caching.

You would use external SSL caching like this if you have more than one SSL termination point (typically a webserver like nginx/Apache) behind a load-balancer.