ONNX is horrible for anything that has variable input shapes and that is why nob... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		imtringued 4 months ago \| parent \| context \| favorite \| on: Run LLMs on Apple Neural Engine (ANE) ONNX is horrible for anything that has variable input shapes and that is why nobody uses it for LLMs. It fundamentally is poorly designed for anything that doesn't take a fixed size image.

zozbot234 4 months ago [–]

ANE itself is also limited to fixed computation "shapes" so I'm not sure how much that would matter practically.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact