Yes, "[r]emaining silent is (in the USA) evidence of nothing", but only regarding the facts of the case. But it can be evidence of contempt, and that can put you in jail. And because each refusal to answer is an additional act of contempt, there's effectively no limit to jail time. You're stuck in a contempt-of-court loop.