You wouldn't place the "binary" output in the repository, just the whatever source format you're using to compile the rest. Say (for the sake of the example) LaTeX or, heck, plain UTF-8 text. Only that is stored in the git repo, the rest are generated on the fly with a continuous build system.

I'm not sure how big that would be, but it's not going to be a lot. Currently, they have 38,000 titles so that's 38,000 files of, I dunno, 50,000 words on average? Probably less, as a lot of the PG works are rather short (novellas and short stories), though some are also equally monstrously long.