Hacker News new | past | comments | ask | show | jobs | submit login

It seems pretty easy to build big files variant of sqlar; instead of a single sqlar table, have index and filedata tables; the index has all the elements of the normal sqlar table except data, and adds a counter for the number of chunks the file is stored in (a directory is zero chunks, an empty file is zero-length with one, empty, chunk), the filedata has composite PK consisting of an FK to the index and an integer sequence number in the file, and a data blob. The code changes to store/extract from it should be straightforward, and given the size of SQLite integers it would in theory structurally handle file sizes (as stored, which may be compressed) of up to 2^63-1 bytes; but the SQLite 281 TB database size limit would be reached well before this.



Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: