fix(expressions): allow NOT expressions in SQL transformations
Add *sqlparser.NotExpr to the allowlist in allowedNode() function,
enabling queries like `WHERE NOT column` and `WHERE NOT (value > 5)`.
Fixes#116616
* Deprecating features.IsEnabled
* add one more nolint
* add one more nolint
* Give better hints to devs in the deprecation message of IsEnabledGlobally
* adding more doc strings
* fix linter after rebase
* Extend deprecation message
Return the SQL schema for all DS queries in request (to provide information to AI / Autocomplete for SQL expressions).
All DS queries are treated as if they were inputs to SQL expressions in terms of conversion, regardless if they are selected in a query or not.
Requires feature toggle queryService = true
Endpoint is apis/query.grafana.app/v0alpha1/namespaces/default/sqlschemas
---------
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
Use fork that does not have cgo as default, had to revert build tag method attempt since it broke things like running go test on macs (without the tag) #112289.
- Added gms_pure_go build tags to disable cgo - (cgo was added to GMS since we last updated it)
- Docs note on regex limitations
---------
Co-authored-by: Matheus Macabu <macabu.matheus@gmail.com>
This is because MySQL doesn't support storing of NaN valuels, and therefore go-mysql-server isn't going to either.
Float fields/columns are always mapped to be nullable now, otherwise we would have to replace NaN/Inf with 0.
s/time_series/timeseries and change from underscores to hyphens in numeric and ts types
This makes it more consistent with the other timeseries kind strings
Fixup on a misleading error being returned due to a missing return statement in the code. Was returning the error "conversion succeeded but no frames" even though there was an error.
sql_expression_query_length_limit
Set the maximum length of a SQL query that can be used in a SQL expression. Default is 10000 characters. A setting of 0 means no limit.
* Capture error_type label on metrics/traces
* Make error messages more helpful to user
* Use errutil, categorized errors, and tie them to error_type (category in code)
* Misc trace fixes
* Add metric to track SQL input conversion
* parse via sse
I need to figure out how to handle the pipeline.execute with our own
client. I think this is important for MT reasons, just like using our
own cache (via legacy) is important.
parsing is done though!
* WIP nonsense
* horrible code but i think it works
* Add support for sql expressions config settings
* Cleanup:
- remove spew from nodes.go
- uncomment out plugin context and use in single tenant flow
- make code more readable and add comments
* Cleanup:
- create separate file for mt ds client builder
- ensure error handling is the same for both expressions and regular queries
- other cleanup
* not working but good thoughts
* WIP, vector not working for non sse
* super hacky but i think vectors work now
* delete delete delete
* Comments for future ref
* break out query handling and start test
* add prom debugger
* clean up: remove comments and commented out bits
* fix query_test
* add prom debugger
* create table-driven tests with testsdata files
* Fix test
* Add test
* go mod??
* idk
* Remove comment
* go enterprise issue maybe
* Fix codeowners
* Delete
* Remove test data
* Clean up
* logger
* Remove go changes hopefully
* idk go man
* sad
* idk i ran go mod tidy and this is what it wants
* Fix readme, with much help from adam
* some linting and testing errors
* lint
* fix lint
* fix lint register.go
* another lint
* address lint in test
* fix dead code and linters for query_test
* Go mod?
* Struggling with go mod
* Fix test
* Fix another test
* Revert headers change
* Its difficult to test this in OSS as it depends on functionality defined in enterprise, let's bring these tests back in some form in enterprise
* Fix codeowners
---------
Co-authored-by: Adam Simpson <adam@adamsimpson.net>
* capture errors on metrics
* rename seconds to sql_command_duration_milliseconds to match unit that has been captured, and update buckets
* rename sql_command_errors_total to sql_command_count
Adds settings for SQL expressions:
sql_expression_cell_output_limit
Set the maximum number of cells that can be returned from a SQL expression. Default is 100000.
sql_expression_timeout
The duration a SQL expression will run before being cancelled. The default is 10s.
* SQL Expressions: Bump GMS for STDDEV(), VARIANCE()
Bump Go MySQL Server to the latest version, to get support for STDDEV()
and VARIANCE() functions, and their aliases.
See https://github.com/dolthub/go-mysql-server/pull/2928
* Update app-code to comply with latest GMS version
* Run `go mod tidy`
* Update test which had depended on STDDEV being missing
* Run `make update-workspace`
* wip
* Add prom flavor support for data source variables and export/import dashboards (#103321)
* add dashboard and data source var selection
* use match plugin id instead
* use updated matchpluginid
* formatting
* cleanup
* regex anchor
* update error msg
* Alerting: Clean up prometheus-flavored types and functions (#103703)
* clean up types and utility functions for dealing with
prometheus-flavored data sources
* Refactor alerting datasource types to use constants as source of truth
* Alerting: Clean up prometheus-flavored types and functions on the bac… (#103716)
Alerting: Clean up prometheus-flavored types and functions on the backend
* add matchPluginId tests
* Update matchPluginId func to bidirectional (#103746)
* update matchpluginid func to bidirectional
* lint
* formatting
* use actual isSupportedExternalRulesSourceType in test
* add tests in datasource_srv
* betterer
* remove type assertion
* remove unnecessary case
* use satisifies to not have to convert tuple to an array of string
* add prometheus_flavor test
---------
Co-authored-by: Andrew Hackmann <5140848+bossinc@users.noreply.github.com>
Co-authored-by: Gilles De Mey <gilles.de.mey@gmail.com>
Co-authored-by: Alexander Akhmetov <me@alx.cx>
* SQL Expressions: Allow many more functions and nodes
* Also allow the `REGEXP_SUBSTR` function
* Add window functions
* add more JSON support, remove now and current time (for now)
---------
Co-authored-by: Kyle Brandt <kyle@grafana.com>
Initial support for alerting with SQL expressions
- When `format` is set to `alerting`, SQL expressions output in a format suitable for alerting evaluation.
- Outstanding TODOs:
- Deduplicate output rows
- Add more tests
- Fix broken alerting UI rendering (likely due to shape change to undocumented full-long format)
- Basic usage:
- SQL must return one numeric column and one or more string columns.
- Each row may become an alert.
- The alert fires if the numeric value is non-zero.
- String columns are treated as labels.
---------
Co-authored-by: Konrad Lalik <konradlalik@gmail.com>
Co-authored-by: Tom Ratcliffe <tom.ratcliffe@grafana.com>
Co-authored-by: Sam Jewell <sam.jewell@grafana.com>
