Influxdb is restarting constantly since my last reboot
It looks like /usr/lib/influxdb/scripts/influxd-systemd-start.sh
is trying to do a health check:
while [ "$result" != "200" ]; do
sleep 1
result=$(curl -s -o /dev/null http://$HOST:$PORT/health -w %{http_code})
done
this is failing. From the file date, the start wrapper was only created on 21 July, so it looks like the start check is new.
If I manually try I get:
root@monitor$ curl https://127.0.0.1:8088/health
curl: (35) OpenSSL SSL_connect: Connection reset by peer in connection to 127.0.0.1:8088
It fails for several reasons.
- Because I have configured TLS it needs to be https
- Because I have not explicitly defined the Bind Port, because I am using the default, the script gets the wrong port.
- because TLS is enabled, it needs the FQDN, not localhost or the cert validation check fails.
- the perms were also wrong on the default startup script
To resolve it I edited the /lib/systemd/system/influxdb.service
file and
- change Type=forking to Type=simple
- change ExecStart to :
ExecStart=/usr/bin/influxd -config /etc/influxdb/influxdb.conf --pidfile /var/lib/influxdb/influxd.pid $INFLUXD_OPTS
This is a bug introduced in Influxdb v1.8.7. Github Issue.
There's a variety of ways of fixing this, your solution being one of the ways. In our case Influx took a bit longer to startup than the 10 second window the startup script allows, so I simply changed the line sleep 1
in the file /usr/lib/influxdb/scripts/influxd-systemd-start.sh
to sleep 2
to give Influx more time to startup.