Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Am I correct in understanding that SWE-bench is limited to python?


The core benchmark is only Python, but there is also SWE-bench Multimodal which uses JavaScript: https://arxiv.org/abs/2410.03859

And the new SWE-bench Multilingual (released a couple of weeks ago) which covers 9 programming languages - C, C++, Go, Java, JavaScript, TypeScript, PHP, Ruby and Rust: https://www.swebench.com/multilingual.html


yeah, they're all python at the moment.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: