> AWS spot instances for gpu can have availability issues as well or sometimes be priced higher than on demand (still trying to understand that one).
You always pay the spot market price, not your bid. Your instance gets killed if somebody outbids you. A higher bid increases the probability that your instances don't get killed while at the same time letting you pay spot market price.
By bidding above on-demand price, you are speculating that for the majority of the time, nobody else will bid more than the on-demand price. If you're not the only one doing that, spot price can rise above on-demand price.
Being a dual-GPU model, it really can only fit half the total memory in the majority of cases, because sharing memory across GPUs is far slower and more difficult to implement.
You always pay the spot market price, not your bid. Your instance gets killed if somebody outbids you. A higher bid increases the probability that your instances don't get killed while at the same time letting you pay spot market price.
By bidding above on-demand price, you are speculating that for the majority of the time, nobody else will bid more than the on-demand price. If you're not the only one doing that, spot price can rise above on-demand price.