Possible to group by a user metadata label that includes dashes, in MQL?
I am trying to use MQL queries to export data from Google Cloud Monitoring to BigQuery for long-term archiving. I've found that queries that group-by user generated metadata labels including a dash character (e.g "wdl-call-alias") do not seem to be compatible with MQL. I rely on a third-party system that generates these labels and they are not easy to change.
Is there a way to use MQL to group-by metadata labels that include dashes? I've included two cases below, for reference.
Working query using "sample" label:
fetch gce_instance
| metric 'compute.googleapis.com/instance/cpu/reserved_cores'
| group_by 1m, [value_reserved_cores_mean: mean(value.reserved_cores)]
| every 1m
| group_by [metadata.user.sample: metadata.user_labels.sample],
[value_reserved_cores_mean_aggregate: aggregate(value_reserved_cores_mean)]
Broken query using "wdl-call-alias" label:
fetch gce_instance
| metric 'compute.googleapis.com/instance/cpu/reserved_cores'
| group_by 1m, [value_reserved_cores_mean: mean(value.reserved_cores)]
| every 1m
| group_by [metadata.user.wdl-call-alias: metadata.user_labels.wdl-call-alias],
[value_reserved_cores_mean_aggregate: aggregate(value_reserved_cores_mean)]
Solution 1:
Use the syntax: metadata.user_labels.c'wdl-call-alias'
Metadata uses the same syntax as column labels (it is kind of a pseudo-column), and column label components that are not identifiers need to be quoted using the c'....'
or c"...."
syntax.
So:
| group_by [metadata.user.wdl-call-alias: metadata.user_labels.c'wdl-call-alias'],
[value_reserved_cores_mean_aggregate: aggregate(value_reserved_cores_mean)]
Or more compactly:
| group_by [metadata.user_labels.c'wdl-call-alias'], .aggregate
Solution 2:
You can rename your rows in a separate map
call, or inline in the group_by
:
From https://cloud.google.com/monitoring/mql/examples#qlx-groupby-aggr
The
group_by
operation takes two arguments, separated by a comma,
. These arguments determine the precise grouping behavior.The first argument controls the grouping of time series. This is a map expression, [...] The map expression can do much more than list labels; for more information, see the
map
reference page.
You can read the full doc there, but TLDR is:
The syntax of map
is map : [ modifier ] '[' [ maplet { ',' maplet } ] ']'
, the "modifier" you need is rename
, and the "maplet" should be:
maplet : [ column_name ':' ] expr .
column_name : ID | QUOTED_COLUMN .
ie: "column_name" which can be quoted!
so,
| map rename ['metadata.user.wdl-call-alias': 'metadata.user.wdl_call_alias']
should give you the result you want.
But i don't have a GCP account right now to test it.