In the worst case you might have to run a kernel built for Android instead of the default debian kernel (e.g. lifted off the android OS for that type of device), depending on how it was packaged up. Hopefully those aren't built with certain useful features turned off.

For devices that support a vanilla android, I imagine the driver issue is already worked out and there would be no problem getting them to work in a kernel that you can compile yourself, even if not the default debian kernel.