fix(expressions): allow NOT expressions in SQL transformations
Add *sqlparser.NotExpr to the allowlist in allowedNode() function,
enabling queries like `WHERE NOT column` and `WHERE NOT (value > 5)`.
Fixes#116616
Return the SQL schema for all DS queries in request (to provide information to AI / Autocomplete for SQL expressions).
All DS queries are treated as if they were inputs to SQL expressions in terms of conversion, regardless if they are selected in a query or not.
Requires feature toggle queryService = true
Endpoint is apis/query.grafana.app/v0alpha1/namespaces/default/sqlschemas
---------
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
Use fork that does not have cgo as default, had to revert build tag method attempt since it broke things like running go test on macs (without the tag) #112289.
- Added gms_pure_go build tags to disable cgo - (cgo was added to GMS since we last updated it)
- Docs note on regex limitations
---------
Co-authored-by: Matheus Macabu <macabu.matheus@gmail.com>
This is because MySQL doesn't support storing of NaN valuels, and therefore go-mysql-server isn't going to either.
Float fields/columns are always mapped to be nullable now, otherwise we would have to replace NaN/Inf with 0.
sql_expression_query_length_limit
Set the maximum length of a SQL query that can be used in a SQL expression. Default is 10000 characters. A setting of 0 means no limit.
* Capture error_type label on metrics/traces
* Make error messages more helpful to user
* Use errutil, categorized errors, and tie them to error_type (category in code)
* Misc trace fixes
* Add metric to track SQL input conversion
Adds settings for SQL expressions:
sql_expression_cell_output_limit
Set the maximum number of cells that can be returned from a SQL expression. Default is 100000.
sql_expression_timeout
The duration a SQL expression will run before being cancelled. The default is 10s.
* SQL Expressions: Bump GMS for STDDEV(), VARIANCE()
Bump Go MySQL Server to the latest version, to get support for STDDEV()
and VARIANCE() functions, and their aliases.
See https://github.com/dolthub/go-mysql-server/pull/2928
* Update app-code to comply with latest GMS version
* Run `go mod tidy`
* Update test which had depended on STDDEV being missing
* Run `make update-workspace`
* SQL Expressions: Allow many more functions and nodes
* Also allow the `REGEXP_SUBSTR` function
* Add window functions
* add more JSON support, remove now and current time (for now)
---------
Co-authored-by: Kyle Brandt <kyle@grafana.com>
* SQL Expressions: Add Null-literal node
* Retain some order in the code - put NullVal with BoolVal
* Add support for `IN` keyword
* Add GROUP_CONCAT keyword
* Add COLLATE keyword
From Claude:
The test case demonstrates a simple use of COLLATE with a string
literal, but in real MySQL queries, COLLATE is often used in more
complex scenarios like:
1. String comparisons:
`SELECT * FROM table WHERE name COLLATE utf8mb4_bin = 'John'`
2. Sorting:
`SELECT * FROM table ORDER BY name COLLATE utf8mb4_unicode_ci`
3. JOINs:
`SELECT * FROM table1 JOIN table2 ON table1.name COLLATE utf8mb4_bin = table2.name`
The COLLATE clause is particularly useful when you need case-sensitive
comparisons (utf8mb4_bin) or specific language-aware sorting rules.
* SQL Expressions: Add CASE/WHEN SQL nodes to allowlist
* Fixed and test for functions added in #102011
* Add remaining functions to the test-case
These are mostly aliases, so the LLM chose to omit them originally.
But adding now for completeness
* Fix ordering of allowed nodes
- use types.Convert to get a more normalized type returned from GMS. After syncing with GMS maintainers, the underlying type of the row could be different things (e.g. int when unit32, so this calls the Convert method from the GMS types library to normalize the interface.
- Clean up fieldValFromRow more
- Use IsText to capture different string types
- Add more types to test, also update test to use same cmp.Diff method as others
---------
Co-authored-by: Sam Jewell <2903904+samjewell@users.noreply.github.com>
* Under feature flag `sqlExpressions` and is experimental
* Excluded from arm32
* Will not work with the Query Service yet
* Does not have limits in place yet
* Does not working with alerting yet
* Currently requires "prepare time series" Transform for time series viz
---------
Co-authored-by: Sam Jewell <sam.jewell@grafana.com>
