I've actually found it helpful as an API autocomplete, but... also not helpful at the same time.
So for example I was working with processing an image to extract features and a few variants of docstrings for the method got me a pretty close to working function which converted the image to gray scale, detected edges, and computed the result I wanted.
The helpful thing here was that there were certain APIs that were useful as a part of doing this that it knew but which I would have to do look up. I had to go through and modify the proposed solution: it got the conditional in the right place, but I wanted a broader classification so switched from a (255, 255, 255) check to a nearBlack(pixel) function which it then autocompleted successfully. I also had to modify the cropping.
When doing a similar task in the past I spent a lot more time on it, because I went down a route in which I was doing color classification based on the k nearest neighbors. Later I found that the AI I was working on was learning to exploit the opacity of the section of the screen I was extracting a feature from in order to maximize its reward, because it kept finding edge cases in the color classifier. I ended up switching to a different color space to make color difference distance functions more meaningful, but it wasn't good enough to beat the RL agent that was trying to exploit mistakes in the classifier.
Anyway, what I'm getting at here is that it is pretty easy to spend a lot of time doing similar things to what I'm doing and not get a great solution at the end. In this case though it only took a few minutes to get a working solution. CoPilot didn't code the solution for me, but it helped me get the coding done faster because it knew the APIs and the basic structure of what I needed to do. To be clear, its solutions were all broken in a ton of ways, but it didn't matter it still saved me time.
To give another example let's say you have a keyboard key event press and you weren't sure about how to translate that into the key that was pressed. key.char? key.key? str(key)? key.symbol? A former method of figuring out what the right key might be is looking up the code, but with CoPilot you type '# Get the key associated with the key press' then hit tab and it gives you code that is broken but looks perfect and you gain a false sense of confidence that you actually know the API. You later realize after being amazed that it knew the API so well that you didn't have to look it up that actually the key press event handles symbols differently and so it errors on anything that is used as a modifier key.
My general impression is something like: Wow, this is amazing. It understood exactly what I wanted and it knew the APIs and coded up the solution... Wait, no. Hold on a second. This and that are wrong.
I am in the same boat with you. I am simultaneously wowed and underwhelmed to some degree.
Yes it is amazing when it gets right, it feels like cheating. But at the same time, it many times, does ... too much? To read a huge chuck of code and figuring out where it goes wrong is not a thing for me. Also Copilot doesn't really know the API, so yes, the amount of mental tax isn't less to make sure your program really behaves.
But again, I see the idea of Copilot is already huge win. I hate writing those manual scripts just offer people an entrance to some utility behind. Copilot does those things, surprisingly well and with accuracy.
Let it improve in the future, and we will see changes that quite fundamental to the idea of programming itself.
So for example I was working with processing an image to extract features and a few variants of docstrings for the method got me a pretty close to working function which converted the image to gray scale, detected edges, and computed the result I wanted.
The helpful thing here was that there were certain APIs that were useful as a part of doing this that it knew but which I would have to do look up. I had to go through and modify the proposed solution: it got the conditional in the right place, but I wanted a broader classification so switched from a (255, 255, 255) check to a nearBlack(pixel) function which it then autocompleted successfully. I also had to modify the cropping.
When doing a similar task in the past I spent a lot more time on it, because I went down a route in which I was doing color classification based on the k nearest neighbors. Later I found that the AI I was working on was learning to exploit the opacity of the section of the screen I was extracting a feature from in order to maximize its reward, because it kept finding edge cases in the color classifier. I ended up switching to a different color space to make color difference distance functions more meaningful, but it wasn't good enough to beat the RL agent that was trying to exploit mistakes in the classifier.
Anyway, what I'm getting at here is that it is pretty easy to spend a lot of time doing similar things to what I'm doing and not get a great solution at the end. In this case though it only took a few minutes to get a working solution. CoPilot didn't code the solution for me, but it helped me get the coding done faster because it knew the APIs and the basic structure of what I needed to do. To be clear, its solutions were all broken in a ton of ways, but it didn't matter it still saved me time.
To give another example let's say you have a keyboard key event press and you weren't sure about how to translate that into the key that was pressed. key.char? key.key? str(key)? key.symbol? A former method of figuring out what the right key might be is looking up the code, but with CoPilot you type '# Get the key associated with the key press' then hit tab and it gives you code that is broken but looks perfect and you gain a false sense of confidence that you actually know the API. You later realize after being amazed that it knew the API so well that you didn't have to look it up that actually the key press event handles symbols differently and so it errors on anything that is used as a modifier key.
My general impression is something like: Wow, this is amazing. It understood exactly what I wanted and it knew the APIs and coded up the solution... Wait, no. Hold on a second. This and that are wrong.