Compare the addresses at http://www.realsimple.com/home-organizing/decorating/eye-cat... and http://mandydouglass.blogspot.com/2010/10/addressing-envelop... . Those are two representative images I picked from the first google hits for "house number" and "handwritten address" respectively - all the others were comparable. Are you seriously going to claim that the house number is harder to recognize than the handwriting?

I do have an idea (literally, even) that there are additional problems having to do with extracting the house number images themselves from full-motion video, but that's an image registration problem and not an object recognition problem.