[DevOps] Use logrotate to automatic rotate, compress, remove and mail log files

Posted on Edited on In , Views: Word count in article: 4.1k Reading time ≈ 4 mins.

logrotate

The logrotate utility is designed to simplify the administration of log files on a system which generates a lot of log files. Logrotate allows for the automatic rotation compression, removal and mailing of log files. Logrotate can be set to handle a log file hourly, daily, weekly, monthly or when the log file gets to a certain size.

Configuration

The Linux system installs logrotate by default.

The primary configuration file for logrotate which sets default parameters is /etc/logrotate.conf; additional application-specific configuration files are included from the /etc/logrotate.d directory. Values set in application-specific configuration files override those same parameters in the primary configuration file. See logrotate.conf(5) — Arch manual pages - https://man.archlinux.org/man/logrotate.conf.5 for configuration examples and a reference of available directives.

1
2
3
4
5
6
7
8
9
10
11
$ cat /etc/logrotate.d/mylog

/app/mylog/*.log {
        daily
        missingok
        copytruncate
        rotate 365
        compress
        delaycompress
        notifempty
}

There are some frequent directives:

  • compress

    Old versions of log files are compressed with gzip(1) by default. See also nocompress.

  • copytruncate

    Truncate the original log file in place after creating a copy, instead of moving the old log file and optionally creating a new one. It can be used when some program cannot be told to close its logfile and thus might continue writing (appending) to the previous log file forever. Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost. When this option is used, the create option will have no effect, as the old log file stays in place.

  • daily

    Log files are rotated every day.

  • delaycompress

    Postpone compression of the previous log file to the next rotation cycle. This only has effect when used in combination with compress. It can be used when some program cannot be told to close its logfile and thus might continue writing to the previous log file for some time.

  • missingok

    If the log file is missing, go on to the next one without issuing an error message. See also nomissingok.

  • notifempty

    Do not rotate the log if it is empty (this overrides the ifempty option).

  • rotate count

    Log files are rotated count times before being removed or mailed to the address specified in a mail directive. If count is 0, old versions are removed rather than rotated.

Usage

logrotate is usually run through the systemd service: logrotate.service.

To run logrotate manually:

1
# logrotate /etc/logrotate.conf

To rotate a single log file:

1
# logrotate /etc/logrotate.d/mylog

See logrotate(8) - Linux man page - https://linux.die.net/man/8/logrotate for more details.

Debug or Verbose logrotate

To simulate running your configuration file (dry run):

1
# logrotate --debug /etc/logrotate.d/mylog

To force running rotations even when conditions are not met, run:

1
# logrotate -vf /etc/logrotate.d/mylog

Check logrotate cron

Normally, logrotate is run as a daily cron job by default.

1
2
3
4
5
6
7
8
9
$ cat /etc/cron.daily/logrotate
#!/bin/sh

/usr/sbin/logrotate -s /var/lib/logrotate/logrotate.status /etc/logrotate.conf
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
    /usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"
fi
exit 0

It will not modify a log multiple times in one day unless the criterion for that log is based on the log’s size and logrotate is being run multiple times each day, or unless the -f or --force option is used.

Check logrotate status

Logrotate rotations are usually logged to /var/lib/logrotate/logrotate.status or /var/lib/logrotate.status (according to the operating system or the -s option allows you to specify another state file):

1
2
3
4
5
6
7
8
9
10
11
12
$ cat /var/lib/logrotate/logrotate.status
logrotate state -- version 2
"/var/log/yum.log" 2020-11-29-3:26:1
"/var/log/boot.log" 2020-11-29-3:29:1
"/var/log/chrony/*.log" 2020-11-29-3:0:0
"/var/log/spooler" 2020-11-29-3:11:1
"/var/log/btmp" 2020-11-29-3:50:1
"/var/log/maillog" 2020-11-29-3:11:1
"/var/log/wpa_supplicant.log" 2020-11-29-3:0:0
"/var/log/secure" 2020-11-29-3:11:1
"/var/log/messages" 2020-11-29-3:11:1
"/var/log/cron" 2020-11-29-3:11:1

References

[1] logrotate(8) - Linux man page - https://linux.die.net/man/8/logrotate

[2] logrotate.conf(5) — Arch manual pages - https://man.archlinux.org/man/logrotate.conf.5

[3] Logrotate - ArchWiki - https://wiki.archlinux.org/index.php/logrotate

[4] logrotate/logrotate: The logrotate utility is designed to simplify the administration of log files on a system which generates a lot of log files. - https://github.com/logrotate/logrotate