It shows up to the OS as two 12GB cards still. So it kind of depends on whether the higher cost is worth the extra slot (in my case it isn't). Total power is less than 2X on the card versus one K40, although I'm not sure offhand if that indicates less performance or just power savings from shared components.
As mentioned in a peer comment, K40 aren't the fastest cards either, but the 12GB is really nice for some use cases.
I have one K40 as well; you should be fine, even Haas F1 team built a CFD supercomputer full of them fairly recently ;-) Kepler is pretty good in FP64.
The article link mentions an NVIDEO card, and an NVIDIA GeForce GTX 1080 8GB sells for ~ $750 new. I don't know how this $400 card compares for this use case, but it is relatively cheap.
K80 is about as fast as 1080 in FP32 (8.2 vs 8.8TFlops) but vastly faster in FP64 (2.7 vs 0.28TFlops; FP64 supercomputers are still being built with it). It has also 3x the memory, so fitting BERT_large NLP model might be possible. 1080 is pretty much outdated at this point as not many state-of-art models can fit inside 8GB. Disadvantage of K80 are older CUDA kernels versions so customized kernels for new CUDA versions might not work, but most models don't touch CUDA directly anyway.
Being a dual-GPU model, it really can only fit half the total memory in the majority of cases, because sharing memory across GPUs is far slower and more difficult to implement.
