I started down that route as well but was running into the same gcc issues as when compiling manually. I found this issue https://github.com/pyenv/pyenv/issues/2046 but the workaround didn't immediately work for me and I didn't dig any deeper.