Why is logrotate causing Apache to seg fault each time?

Every time logrotate runs Apache/2.4.7 (Ubuntu) experiences a seg fault and does not restart:

[Wed Sep 10 06:35:54.266018 2014] [mpm_event:notice] [pid 20599:tid 140630283466624] AH00493: SIGUSR1 received.  Doing graceful restart
[Wed Sep 10 06:35:54.885118 2014] [core:notice] [pid 20599] AH00060: seg fault or similar nasty error detected in the parent process

My apache logrotate script looks like:

/var/log/apache2/*.log {
    daily
    missingok
    rotate 52
    compress
    delaycompress
    notifempty
    create 640 root adm
    sharedscripts
    postrotate
        /usr/sbin/apachectl graceful
    endscript
    prerotate
        if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
            run-parts /etc/logrotate.d/httpd-prerotate; \
        fi; \
    endscript
}

/srv/apache/log/*.log {
    daily
    missingok
    rotate 52
    compress
    delaycompress
    notifempty
    create 640 root adm
    sharedscripts
    postrotate
        /usr/sbin/apachectl graceful
    endscript
    prerotate
        if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
            run-parts /etc/logrotate.d/httpd-prerotate; \
        fi; \
    endscript
}

I keep the default logs in /var/log/apache2, but I keep vhost specific logs (for the three different vhosts hosted on this server) in the /srv/apache/log directory (e.g. mysite1_error.log, mysite1_access.log, mysite2_error.log, mysite2_access.log....).

The relevant parts of /etc/apache2/apache.conf:

#/etc/apache2/apache.conf
ErrorLog ${APACHE_LOG_DIR}/error.log
LogLevel trace8   # I set to try and get more info about this problem.
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent

where export APACHE_LOG_DIR=/var/log/apache2$SUFFIX. In my vhost conf files, such as /etc/apache2/sites-enabled/mysite1.conf I have:

#/etc/apache2/sites-enabled/mysite1.conf
LogLevel debug
ErrorLog  /srv/apache/log/mysite1_error.log
CustomLog /srv/apache/log/mysite2_access.log combined

and similar for the other sites.

Anyone know why logrotate is causing this crash?

One more thing:

Manually forcing the logrotate as root:

logrotate -v -f /etc/logrotate.d/apache2

does not cause the seg fault, but I know it's logrotate as I've tried playing with the times (weekly, daily) and the seg fault always occurs exactly as the logs are being rotated only.

How to reproduce on demand

# Set a crontab to run each minute (simulating cron.daily run of logrotate)
crontab -e
*/1 * * * * root /usr/sbin/logrotate -v -f /etc/logrotate.d/apache2 > /home/myuser/logrotate.log 2>&1

Finally

If I comment out the /srv/apache/log/*.log rotation block in /etc/logrotate.d/apache the seg fault does not occur either.

It seems that if I just change the logrotate script to be a single block, i.e. /var/log/apache2/*.log, /srv/apache/logs/*.log {....} I don't get the segfault. So it was just the two blocks were causing a second restart attempt during the first restart...

If I enter apache2ctl graceful & apache2ctl graceful & apache2ctl graceful & apache2ctl graceful & on the cmd line my system DOES segfault, which seems to confirm

Why is logrotate causing Apache to seg fault each time?

Related

Recent Posts