The app got into a state where new fsi.exe processes wouldn't start up properly - they'd raise an unhandled exception and hang. I started being more diligent about cleaning up old processes.