Telegraf works manually but not the service - Run Telegraf in background

I have a virtual machine running on Ubuntu 20.04, I installed Telegraf, InfluxDB and Grafana for monitoring VMs and cluster's nodes.

Everythings work except when I try to make telegraf working in background.

The command to start manually Telegraf which is working :

telegraf --config http://XXX.XX.X.X:8086/api/v2/telegrafs/XXXXXXXXXXX

returns me :

2021-03-14T00:02:27Z I! Starting Telegraf 1.17.3
2021-03-14T00:02:27Z I! Loaded inputs: cpu disk diskio mem net processes swap system
2021-03-14T00:02:27Z I! Loaded aggregators:
2021-03-14T00:02:27Z I! Loaded processors:
2021-03-14T00:02:27Z I! Loaded outputs: influxdb_v2
2021-03-14T00:02:27Z I! Tags enabled: host=XXXXX
2021-03-14T00:02:27Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"XXXXX", Flush Interval:10s

So everythings good so far.

But impossible to run this service (/lib/systemd/system/telegraf.service):

[Unit]
Description=The plugin-driven server agent for reporting metrics into InfluxDB
Documentation=https://github.com/influxdata/telegraf
After=network.target

[Service]
EnvironmentFile=-/etc/default/telegraf
User=telegraf
ExecStart=/usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d $TELEGRAF_OP>
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartForceExitStatus=SIGPIPE
KillMode=control-group

[Install]
WantedBy=multi-user.target

It's starting well but after checking the service status I'm getting :

14T00:10:46Z E! [agent] Error writing to outputs.influxdb: could not write any address
14T00:10:56Z W! [outputs.influxdb] Metric buffer overflow; 25 metrics have been dropped
14T00:10:56Z E! [outputs.influxdb] When writing to [http://XXX.XX.X.X:8086]: 401 Unauthorized
14T00:10:56Z E! [agent] Error writing to outputs.influxdb: could not write any address
14T00:11:06Z W! [outputs.influxdb] Metric buffer overflow; 25 metrics have been dropped
14T00:11:06Z E! [outputs.influxdb] When writing to [http://XXX.XX.X.X:8086]: 401 Unauthorized
14T00:11:06Z E! [agent] Error writing to outputs.influxdb: could not write any address
14T00:11:16Z W! [outputs.influxdb] Metric buffer overflow; 25 metrics have been dropped
14T00:11:16Z E! [outputs.influxdb] When writing to [http://XXX.XX.X.X:8086]: 401 Unauthorized
14T00:11:16Z E! [agent] Error writing to outputs.influxdb: could not write any address

And InfluxDB does not get any data and so Grafana.

Can anyone help me solve this service problem and get Telegraf working in background pls ?


This information was hard to find! I had exactly the same problem and I found the solution here: https://github.com/influxdata/telegraf/issues/8395.

So all you have to do is to add the INFLUX_TOKEN variable to /etc/default/telegraf such as:

INFLUX_TOKEN=mytoken