LLMs could count the number of letters in a sentence if you stopped tokenizing t... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		astrange on May 15, 2024 \| parent \| context \| favorite \| on: Ilya Sutskever to leave OpenAI LLMs could count the number of letters in a sentence if you stopped tokenizing them first.

HarHarVeryFunny on May 15, 2024 [–]

tokenization is not the issue - these LLMs can all break a word into letters if you ask them.

Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact