Scaling horizontally is basically "we can add machines to make it go faster" and "scaling vertically" means "getting a faster or more disk/memory/CPU/etc. makes it go faster". In horizontal scaling you're usually talking about adding "commodity" hardware, though that's not actually that true any longer (I believe AWS at least uses customized boards, form factors, etc. to achieve better economy of scale.)

Vertical scaling is where you add more resources to a machine. E.g; more RAM or processing power.

Horizontal scaling is where you add more machines to your pool of resources (servers).

Examples;

Horizontal scaling would be adding more web servers behind the load balancer to facilitate more traffic.

Vertical scaling would be adding more RAM to your database server to keep all the data (or just the indexes if your database is that big) in memory.

The short version: scaling horizontally means getting many more boxes scaling vertically means getting a bigger box (replacing the one you have)