Nitpicking, but you can have a uniform distribution on the unit interval, which is infinite :). What you cannot have is a uniform distribution on an unbounded set.
Edit: As pointed out, the correct assumption is not unboundedness, but having infinite measure (since then no normalization constant exists). I thought they were equivalent, but a simple counterexample is [0,1] \cup Z.
Other answers failed to give examples of a set with positive finite measure, which is what you really need. What we want is a set that has no upper bound, but isn't really weird or tricky—if it's too tricky, then the uniform continuous distribution doesn't make any sense—and an easy way to do this is take a union of a countable set of intervals.
For example, consider a set that contains (0,1/2), (1,1+1/2), (2,2+1/4), ... etc, so interval i has measure 2^-i. It should be obvious that there is no largest element, and if you sum the measures of all the sets you will find that the measure is 1. A big chunk of "doing mathematics" is having a library of techniques on tap to come up with weird objects like this to attack assumptions.
Thanks. That seems obvious now that you point it out. Just like Zeno's paradoxes, which are also about stretching a finite measure over a countable infinity.
Okay, again, I was being imprecise. All continuous uniform distributions have support with nonzero finite measure. The discrete uniform distribution is often thought of as a different distribution than the continuous uniform distribution.
This just goes to show how much math relies on people knowing which definitions you happen to be using at the moment you say something.
The point is that not all sets with nonzero finite measure are bounded, therefore you can have a uniform distribution whose support is equal to such a set.
I agree. I am of course nitpicking, but it is exactly the same mistake I made in the first post (but I was assuming a discrete distribution instead of a continuous distribution).
It's an interesting exercise in measure to figure out what falls apart between a uniform distribution on [0,1] and the lack of one on [0, inf]. (Adding a point at infinity to compactify the set, which makes the two intervals topologically equivalent.)
To elaborate, the reason topological equivalence doesn't help here is because probability is defined in terms of measure spaces, not topological spaces.
In any probability class, you will learn that you can't have an uniform distribution on an infinite set.