Actively Monitor Server Performance with a Bash Script

As an admin, you may want to keep an eye on your system’s performance to ensure that it runs smoothly and efficiently. One way to do this is by creating a Bash script that logs key performance metrics. In this blog post, we’ll cover how to create such a script, as well as some useful tips to help you get the most out of it.

Use-cases

• Monitoring server performance: If you manage a server, it’s crucial to keep track of its performance to ensure that it’s running smoothly and to identify any issues that may arise. By logging key metrics such as CPU usage, memory usage, and disk usage, you can quickly identify any bottlenecks and take action to address them.
• Troubleshooting performance issues: If you’re experiencing performance issues on your system, a performance log can help you identify the cause. By logging metrics such as CPU usage, memory usage, and disk usage over time, you can pinpoint the exact moment when the problem began and take steps to resolve it.
• Tracking system health over time: By regularly logging performance metrics, you can track the health of your system over time. This can help you identify trends, such as a gradual increase in CPU usage or a gradual decrease in free memory, that might indicate an underlying issue.

Creating the script

Creating a Bash script to log system performance is relatively straightforward. Here’s an example of a basic script that logs CPU usage, memory usage, and disk usage every minute:

#!/bin/bash

while true; do
    echo "--- $(date) ---"
    echo "CPU usage: $(top -bn1 | grep "Cpu(s)" | awk '{print $2 + $4}')%"
    echo "Memory usage: $(free -m | awk 'NR==2{printf "%.2f%%\\t", $3*100/$2 }')"
    echo "Disk usage: $(df -h | awk '$NF=="/"{printf "%s\\t", $5}')"
    sleep 60
done

This script uses the top, free, and df commands to retrieve the performance metrics. It then uses echo to print the metrics to the console, along with the current date and time. The script runs in an infinite loop, using the sleep command to pause for 60 seconds between each iteration.

5 Tips

1. Customize the script to suit your needs: The example script above logs basic performance metrics, but you can customize it to include additional metrics or exclude ones you don’t need. You can also adjust the frequency at which the script logs metrics by modifying the sleep time.
2. Save the logs to a file: Instead of printing the metrics to the console, you can save them to a file by redirecting the output of the echo command. For example, echo "CPU usage: $(top -bn1 | grep "Cpu(s)" | awk '{print $2 + $4}')%" >> performance.log
3. Use cron to schedule the script: The example script runs continuously, but you can use the cron utility to schedule the script to run at specific intervals. This allows you to run the script automatically in the background without needing to manually start it.
4. Parse the log files: Once you’ve collected performance logs, you can use tools like awk and grep to parse and analyze the data. For example, you can use grep to search for specific patterns in the logs, or use awk to calculate the average CPU usage over a period of time.
5. Set up alerts: You can set up alerts to notify you if certain performance thresholds are met. For example, you can set up an alert to notify you if CPU usage exceeds 90% for an extended period of time. This can be done by using if statements in the script and sending an email or a text message using command line utilities like mail or curl.

Challenge

Your challenge is to take the script above and customize it to suit your needs. Add a few more performance metrics that you think would be useful to log, such as disk I/O, network usage, and processes. Then, set up the script to run automatically using cron and save the logs to a file. Finally, try parsing the logs to analyze the data and set up an alert to notify you when a certain threshold is met. Let us know how it goes and if you have any questions. Happy scripting!

Answer

Here is an example of few performance metrics to add:

echo "Disk I/O: $(iostat -d -x 1 2 | awk 'FNR==4{print $6}')"
echo "Network usage: $(ifstat -i en0 1 2 | awk 'FNR==3{print "In: " $1 " Out: " $2}')"
echo "Processes: $(ps aux | wc -l)"

The ifstat command might not be installed on your system, so install it using your package manager, for example sudo apt install ifstat and the interface you use with ifstat needs to match your active interface. So you might need to change it to eth0 or something else. View your interfaces if ip a command.