CUDA was released in 2007 and the development of it started even earlier - possibly even in the 90s. Back then nobody else cared about GPU compute. OpenCL came out 2 years after that.
Everything has old beginnings that the specialists will remember, but GPU compute really reached mass popularity and became a large selling point for Nvidia in the 2010s.
