CPU load average is a metric used to determine CPU usage in Linux. CPU load is a measure of the number of processes that are running as well processes that are waiting for CPU access. Tracking CPU load averages over a period of time can help system administrators diagnose the cause of server slowdowns.

Understanding CPU load

Ideally you want your CPU load to remain equal to or less than the number of CPU cores in your server. So if you have a single core server a 1.0 load average will indicate almost complete utilization. If you have quad core system then a 4.0 or less load average will be ideal.

However, under Linux you can generally get away with at least 2x the core count as load average during peak times. That is because Linux is very efficient at CPU scheduling and on a web server network delays have a bigger impact than processor delays.

Tracking CPU load average

To keep track of your system’s CPU load averages you will need to install the sar program. This program is usually found in the sysstat package.

To install sysstat on Centos or Fedora type the following command as root:

yum install sysstat

On OpenSUSE you would do:

zypper install sysstat

Once installed sysstat will automatically run at periodic intervals via a cronjob. Each time it runs it will record statistics about the server to its log files. So before you can invoke sar for the first time you will need to wait for the cronjob to run at least once.

Viewing statistics

Type the following command to get today’s CPU load average statistics:

sar -q | less

The output of sar is piped to the text viewing program or pager called ‘less’ for easy viewing. Press q to quit less when you are done viewing the stats.

sar stores its statistics in log files. On CentOS and SUSE these files are stored in the /var/log/sa directory. Each file in that directory is named after a different date of the month. So if you want to view the load averages for the 10th you would do:

cd /var/log/sa
sar -q -f sa10 | less

You can also narrow down the time period of the statistics using the -s and -e arguments. The -s argument is for the starting time and -e for the ending time. In both cases you specify the time in 24 hour format:

sar -q -s 22:00:00 -e 00:00:00 -f /var/log/sa/sa10

Example output:

abdussamad@homebase:/var/log/sa> sar -q -s 22:00:00 -e 00:00:00 -f /var/log/sa/sa10
Linux 2.6.34.7-0.3 (homebase)   11/10/2010      _x86_64_        (2 CPU)

10:20:01 PM   runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15
10:30:01 PM         2       271      0.34      0.21      0.13
10:40:01 PM         1       266      0.09      0.08      0.09
10:50:01 PM         1       263      0.12      0.12      0.09
11:00:01 PM         2       266      0.00      0.01      0.03
11:10:01 PM         1       288      0.68      0.29      0.11
11:20:01 PM         1       284      0.02      0.10      0.09
11:30:01 PM         3       292      0.02      0.05      0.07
11:40:01 PM         1       288      0.00      0.00      0.02
11:50:01 PM         1       284      0.03      0.09      0.04
Average:            1       278      0.14      0.11      0.07

In the above example the columns ldavg-1, ldavg-5 and ldavg-15 represent load averages 1 minute, 5 minutes and 15 minutes in the past respectively. The idea is to show a rising or falling trend in CPU load. In this case you can see that the system is under very little load. Its a 2 core system yet load is less than 1 so that means there are no processes waiting for CPU time.

Final thoughts

Using sysstat you can record statistics about your system including CPU load averages over a period of time. However, keep in mind that CPU load figures are just part of the picture and it is important to consider other metrics as well for a complete picture. Try the command man sar to learn how to view other statistics about your system.