Being denied privileges if one does not do something is not the same as forcing someone to do something. I get many privileges for being employed, I'm not forced to be employed.
I also have to work to afford soap. That does not make me a slave.
If you're going to contend that solitary confinement is "denial of privileges" rather than a punishment designed to compel labor, then we just aren't going to agree and I don't see much point continuing. "Do this work, or I will take an action that is costly to me in order to worsen your life" is forced labor in any definition I can come up with.
You are not required to work one, specific job that can be changed at a moment's notice without your consent to afford soap. Almost any work one might do in the market economy is sufficient to pay for basic hygiene products for one person.
Solitary confinement is used in many other cases where people are not even convicted yet. If it was considered a right for the incarcerated to not be in solitary confinement, then it would be illegal to put someone in solitary confinement for refusing to work.
I can't have any job I want either, nobody can. Criminals are not entitled to free room and board just because they have injured society. Criminals are not the victims, they are the criminals. They should pay for room and board and make restitution to society.
If you're going to go to the "legal therefore moral" argument, you could've just started there. There are people who contend that the way some prisons are operated is actually illegal, but I haven't seen any of them here. This discussion is about whether requiring labor with threat of an explicit punishment -- not just "I will withhold something that is mine", but "I will go out of my way to make your life worse" -- is morally and practically comparable to (though clearly a lesser evil than) slavery. There are pretty good arguments that it is not (e.g. that chattel slavery would often extend to children of slaves, or include the right to capriciously murder the slave), but you haven't made them.
You can't have any job you want in the sense that you can't e.g. be President of the United States just because you want to. You likely (unless you happen to be in prison right now?) have a choice between at least two at any given time, though.
I suppose exactly zero of (to try to cover all political bases): Chelsea Manning, Julian Assange, the various January 6 convicts, the various people jailed after the George Floyd riots, or even those convicted of things they literally did not do are victims then. There are, always have been, and always will be "convicted criminals" that definitely don't deserve their fate -- though, to be clear, I believe some (not all) of the ones I mentioned do (and you'd probably be wrong if you guessed which were which). They are in the minority, but I'm not willing to deliberately hurt that minority just to also hurt the majority who arguably deserve it.
I also have to work to afford soap. That does not make me a slave.