I have no low level experience with android but we have fought this issue on our analytical servers. One culprit was java code (hadoop) that read small amounts of entropy with buffered io essentially filling and discarding a large unneeded buffer every time it needed a small unique id. It caused all sorts of weird lags especially with ssh and was a pain to debug.