It's also not ideal to have a different query language for different Grafana datastores (LogQL, PromQL, TraceQL). Are there any plans on making a unified Grafana query language?
There is an effort in OpenTelemetry to create a standard query language for observability. There were a lot of discussions with a lot of opinions; there were even several talks during KubeConEU about that:
Why not just use SQL? With LLMs evolving to do sophisticated text-to-SQL, the case for a custom language for the sake of simplicity is diminishing.
I think that expressiveness, performance and level of fluency by base language models (i.e. the amount of examples in training set) are the key differentiators for query languages in the future. SQL ticks all those boxes.
You are right. SQL is the best language, but it likely needs some extensions. See SQL with pipe syntax. Read Google paper or try it out in Big Query.
There are a lot of fundamentals in observability, but there are very verbose in SQL:
- rate operator, which translates absolute value to rate, possible with SQL and window functions, but takes many lines of code
- pivot, where you like to see the top 5 counts of errors of most hit-by-error microservices plus others over time
- sampling is frequent in observability and will be useful for LLMs, it is a one-liner in SQL with pipe syntax, even customizing specific
I actually believe LLM gen AI plays extremely well with pipe syntax. It allows us to feed partial results to LLM, sampling as well as show how LLM is evolving queries over time. SQL troubleshooting is not a single query but a series of them.
Still, SQL with pipe syntax is just syntactical sugar on SQL. It let's you use all SQL features as well as compiles to SQL.
I think I’m probably not interested in this. PromQL is already relatively dense to learn, but reasonably well fit to the domain model and internally consistent, unlike most other metric querying tools I’ve tried over the years.
Maybe that would work as well with traces and logs but IMO the problem space is quite different and not sure how much value we’d get from a unified language where some subsets only apply to parts, ie traces and logs and metric, as opposed to spiritually similar but distinct languages.
