Main impression - it does hallucinate like crazy. I asked "How does authorization of HTTP request work?" and it started spitting explanation of how user bcrypt hash is stored in SQlite database and token is stored in Redis cache. There are no signs of SQLite or Redis whatsover on this project.
In other query it started confidently explaining how `getTeam` and `createTeam` functions work. There are no such entity or a word "team" in the entire codebase. To add to the insult, it said that this whole team handling logic is stored in `/assets/sbadmin2/scss/_mixins.scss`.
Other time it offered extremely detailed explanation of some business-logic related question, linking to a lot of existing files from the project, but that was completely off.
Sometimes it offered meaningful explanations, but was ignoring the question. Like I ask to explain relation between two entities and it started showing how to display that entity in a HTML template.
But I guess it's just a question of time when tools like this become a daily assistant. Seems invaluable for the newcomers to the codebase.
Wrong documentation is even worse than no documentation. Without it your at least force to look at the code and validate your assumptions. Get a feel for the code base. Wrong docs pointing you to tech that’s not even used is going to be a mess.
Furthermore, software is essentially versioned by its documentation first. If it says it does something, people will depend on that, and it not doing that is a bug.
I wasted two hours yesterday just to track down a bug that was in the documentation, not the code. It's so frustrating. Maintaining good documentation is incredibly hard especially when you're trying to document how your program interacts with other software because now you have to change your documentation when a third party changes their code.
Alternatively, it's hyped up like crazy, the tech is inherently bad at information retrieval, and most of the people hyping it are trying to get in on a gold rush.
Main impression - it does hallucinate like crazy. I asked "How does authorization of HTTP request work?" and it started spitting explanation of how user bcrypt hash is stored in SQlite database and token is stored in Redis cache. There are no signs of SQLite or Redis whatsover on this project.
In other query it started confidently explaining how `getTeam` and `createTeam` functions work. There are no such entity or a word "team" in the entire codebase. To add to the insult, it said that this whole team handling logic is stored in `/assets/sbadmin2/scss/_mixins.scss`.
Other time it offered extremely detailed explanation of some business-logic related question, linking to a lot of existing files from the project, but that was completely off.
Sometimes it offered meaningful explanations, but was ignoring the question. Like I ask to explain relation between two entities and it started showing how to display that entity in a HTML template.
But I guess it's just a question of time when tools like this become a daily assistant. Seems invaluable for the newcomers to the codebase.