Data to External InfluxDB?

I’d really like to get my data into an instance of InfluxDB. I’m currently doing so with the HomeAssistant service but obviously that’s a limited data set. Any chance of re-incarnating the previous InFlux connector to optionally send data to an external InfluxDB in addition to what goes to VictoriaMetrics for the core functionality?

This is not something we’d want to officially support, but the actual conversion of Brewblox history data → Influx Line Protocol is trivial, to the point where I may as well put it in a script. Depending on preference, this can be done in Python or Javascript (for Node-RED).

Just getting back to this after a busy summer. I haven’t really looked at how data ingress works with InfluxDB besides a general understanding of configuring Telegraf for various sources. This does indeed look trivial and I think I might take a shot at it myself in Javascript.

Looking into this over the weekend I think the best path forward for me is to configure Telegraf to use the history data from MQTT. I think I’ll probably do that by deploying a Telegraf docker container. Other than the the Telegraf configuration file, nothing custom will be required. That gives me a chance to learn some about consuming data from MQTT into InfluxDB which will likely come in handy later.

Skimming through the Telegraf documentation, this appears very doable. The topic for Spark history data includes the service name, and the field path within the JSON object is deterministic (no lists).

Please let us know how it works out. I suspect more people would be interested in possibly taking this path.

Got it working!

# Telegraf Configuration

  interval = "10s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = "0s"
  hostname = ""
  omit_hostname = false

   urls = ["http://<influxdb servername>:8086"]
   token = "<secret>"
   organization = "<org>"
   bucket = "brewblox"

  servers = ["wss://brewblox:443/eventbus"]
  topics = ["brewcast/history/+",]
  insecure_skip_verify = true
  qos = 0
  connection_timeout = "30s"
  persistent_session = true
  client_id = "telegraftest"
  data_format = "xpath_json" 
  xpath_native_types = true
    topic = "brewcast/history/+"
    measurement = "measurement/_/_"
    tags = "_/_/service"
      metric_name = "'measurment1'"
      metric_selection = "//data/*"
      field_selection = "*"
      field_name = "replace(name(.), ' ', '_')"

          block = "replace(name(.), ' ', '_')"
1 Like

Doesn’t this clash with the metrics name parsed from topic?

I missed that but it would appear that the parent one takes precedence since _measurement is brewcast in my data. I would guess that line isn’t needed.

1 Like