When you use this method you are implicitly making the function periodic. I can give you any function on some interval (sufficiently well behaved) and you can compute the Fourier series of it. Even though it's only defined on the interval, if you plotted the Fourier series you would still find it to be periodic. The same idea carries over to the spectral derivative. Even if the function isn't periodic, the method still works, it's just that the numerics near the discontinuity are going to be crap and can affect things further away. But if you are sufficiently far away, things are kosher. Note there are ways of dealing with this using Chebyshev interpolation (see http://math.mit.edu/~stevenj/fft-deriv.pdf section 6) which Trefethen's book also discusses.

And no FFTs are the only transformations I am aware of. I've never heard of anyone using other transformations in a general numerical context, outside of specialized problems.

Given the local nature of the problem and the potential for using O(n) operations (as opposed n log n) my first suspect was wavelet transforms and sure enough people have done that but that body of work seems more recent than the use of FFT.