If I were to drop CSRF tokens (which work reliably) I would go directly to Origin verification. No Referer. Something in the middle is worst of both worlds.
I'm curious about your comment. We (attempted) to address cases where we needed to infer the Origin from the Referer due to incomplete browser support. What about using both makes this necessarily worse, when the use of the Referer is really only a temporary bandage for said incomplete support?
Worst because it is neither proven as tokens which are used for decades already nor as convenient as simply whitelisted Origin and you even offer an extra dependency. Are those scenarios without Origin important? Like ff for form request - you can use xhr there and drop referer support entirely.
You're right, support for cross-origin form POSTs isn't important for us. What is important, however, is same-origin requests, and Firefox doesn't send the Origin header with those. For us, the Referer serves its most important role by indicating where those requests originate from.
EDIT: when Firefox _does_ start sending the Origin header, we may drop the Referer - we'll see.