Don’t Forget to Rotate Logs

Log rotation is an essential task for system administrators and developers. It ensures that log files don’t become too large and consume too much disk space. Additionally, log rotation can also be used for archiving and compression. In this guide, we will discuss how to automate log rotation using a bash script.

Use-case 1: Rotating Apache Logs

One of the most common use-cases for log rotation is rotating Apache logs. Apache, by default, writes log files in the /var/log/httpd directory. These logs can become quite large and consume a significant amount of disk space. To rotate these logs, we can use the logrotate utility. However, we can also create a bash script to automate this process. Here is an example of a bash script that rotates Apache logs:

#!/bin/bash

LOG_DIR="/var/log/httpd"
DATE=$(date +"%Y-%m-%d")

# Rotate access logs
mv ${LOG_DIR}/access_log ${LOG_DIR}/access_log-${DATE}
gzip ${LOG_DIR}/access_log-${DATE}

# Rotate error logs
mv ${LOG_DIR}/error_log ${LOG_DIR}/error_log-${DATE}
gzip ${LOG_DIR}/error_log-${DATE}

# Restart Apache
service httpd restart

This script first sets the LOG_DIR variable to the location of the Apache logs. It then sets the DATE variable to the current date. The script then uses the “mv” command to move the access and error logs to a new file with the current date appended to the file name. The script then uses the “gzip” command to compress the rotated logs. Finally, the script restarts the Apache service to start writing to new log files.

Use-case 2: Rotating Nginx Logs

Another common use-case for log rotation is rotating Nginx logs. Like Apache, Nginx writes log files in a default location, in this case, /var/log/nginx. To rotate these logs, we can create a similar bash script as the one for Apache. Here is an example of a bash script that rotates Nginx logs:

#!/bin/bash

LOG_DIR="/var/log/nginx"
DATE=$(date +"%Y-%m-%d")

# Rotate access logs
mv ${LOG_DIR}/access.log ${LOG_DIR}/access.log-${DATE}
gzip ${LOG_DIR}/access.log-${DATE}

# Rotate error logs
mv ${LOG_DIR}/error.log ${LOG_DIR}/error.log-${DATE}
gzip ${LOG_DIR}/error.log-${DATE}

# Reload Nginx
service nginx reload

This script is similar to the one for Apache. It sets the LOG_DIR variable to the location of the Nginx logs and sets the DATE variable to the current date. The script then uses the “mv” command to move the access and error logs to a new file with the current date appended to the file name. The script then uses the “gzip” command to compress the rotated logs. Finally, the script reloads the Nginx service to start writing to new log files.

In the examples provided, Nginx was reloaded and Apache was restarted. This is because reloading Nginx is a less disruptive way to apply changes to the configuration and rotate logs. Reloading Nginx allows the server to re-read its configuration files and apply any changes without losing existing connections or interrupting service. On the other hand, Restarting Apache stops and starts the Apache service, which closes all existing connections and may cause a brief interruption of service. In general, it’s best practice to use the least disruptive method possible to apply changes to a service.

Tips for Automating Log Rotation

1. Schedule the script to run at a specific time: You can use cron to schedule the script to run at a specific time, such as daily or weekly. This way, you don’t have to manually run the script every time.
2. Use the logrotate utility: While creating a bash script can automate log rotation, the logrotate utility is a more robust and feature-rich option. It allows for more advanced options such as compression and archiving.
3. Use unique file names: When rotating logs, it’s important to use unique file names to avoid overwriting any previous log files. You can use the date or a timestamp in the file name to make it unique.
4. Rotate logs for all services: Make sure to rotate logs for all services running on your system, not just Apache or Nginx. Services such as MySQL, PostgreSQL, and others also write log files that should be rotated.
5. Monitor disk space: Keep an eye on the disk space usage on your system, especially after rotating logs. If disk space is running low, you may need to increase the frequency of log rotation or adjust the retention period of rotated logs.

Challenge

Create a bash script to rotate log files for a service called “myapp”. The log files are located in /var/log/myapp and the script should gzip the rotated log files and move them to a directory called /var/log/myapp_old.