Hacker News new | past | comments | ask | show | jobs | submit login

I think they messed up the templates.



Here's the template I'm using - https://github.com/BerriAI/litellm/blob/5ca8b23e22139a4f49bd...

Anything I'm doing incorrectly?


The format is really weird for chat models. In your code post message is `<</SYS>>\n [/INST]` which is wrong. Instead the overall format is `[INST] <<SYS>>\n{system}\n<</SYS>>\n\n{instruction} [/INST]`

So both the system message and first instruction are in same [INST] block.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: