Hacker News new | past | comments | ask | show | jobs | submit login

Text files are very hard to efficiently parse in parallel because there are so many variable-length and conditionally present fields. That necessarily leads to branches in your code which GPUs are not designed for.



It depends on the format of the text, the parser for that format, the language of the parser, and many other factors. Such a broad affirmative statement that text files are inefficient is tenuous at best.


We need some technology that turns text files into binary. To the startupmobile!


Hmm sounds like a compiler


Then pad fields and require them, even if the value is Null/None/Nil, and the field names too.

  USERNAME:dotancohen\0\0\0\0\0\0
  FOOBAR\0\0:\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0
This might look ugly if you open it in Notepad, but so do XML and Org mode files. However an Emacs/VIM/VsCode plugin could make this seamless for the user.


They might not look as pleasant as Markdown, but Org mode files still look good in a plain text editor like Notepad.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: