Hacker News new | past | comments | ask | show | jobs | submit login

Of all the off the shelf text splitters I have tried, the recursive character splitter actually performs really well. Especially if the chunk size is so large you will likely have more than the actual needed context in a chunk anyway.

Regarding the index usually a mix of BM25 and vector index seems to perform best for most generic data.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: