Every system admin normally wants to be in full control of his box. One part of the “full control”-thing is to monitor your Linux system. You usually want to know what happens right now and how it affects the system. There are tools like iotop, cbm, htop, top, vmstat and so on, but none of them shows you all of the wanted information in one single place and none of them is flexible enough to allow external plugins.
And this is the spot where dstat comes in.
Being written in 2004 and heaving heavily evolved ever since, dstat has become a replacement for most of the tools being mentioned above (at least for me). dstat is very flexible and shows you the information you want in real time, e.g. RAM and CPU usage, bandwith, I/O in total, MySQL-stats, interrupts and performance bottlenecks.
Introduction to dstat
In this blog post, the latest version of dstat (0.7.2, 03/09/2010) on Debian 6 will be covered. Please notice that I will only cover the real-time monitoring features of this tool, but not the “export data to a csv-file and use it for my own charts”-stuff.
For the start let’s simply install and call dstat:
apt-get install dstat
As you can see, dstat starts with some default parameters and shows the CPU, net, disk and paging usage. Furthermore it indicates the current amount of hardware interrupts and context switches. If you already looked at man dstat, you might guess that calling “dstat” equals to “dstat -c -d -n -g -y”:
-c= total CPU usage
-d= disk total
-n= net total
-g= paging system
-y= system stats, such as the hw interrupts and the context switches
For most ambitious sys admins, this is not enough. So let’s take a look at some of the most useful parameters:
-l= shows load statistics
-m= shows the memory usage (used, buffer, cache, free)
-r= displays I/O statistics,
-s= shows the swap usage
-t= puts the current time in the first column
--fs= displays file system stats (includes amount of files and used inodes)
--nocolor= sometimes very useful…
--socket= shows interesting network statistics
--tcp= displays common TCP stats
--udp= shows you the listen and active figures for the UDP usage
The parameters mentioned above already make dstat a very powerful monitoring tool, and we haven’t covered the external plugins yet Have a look at /usr/share/dstat in order to view them. Some of them are:
--disk-util= shows how much the disks are busy at the moment
--freespace= shows the current disk usage
--proc-count= displays the number of running processes
--top-bio= points to the most expensive block I/O process
--top-cpu= draws the attention on the most expensive CPU process
--top-io= shows the most expensive “normal” I/O process
--top-mem= displays the process using the most memory
Simply call “dstat –plugin name” in order to use them.
There are also many MySQL and NFS plugins, simply have a look at them and try them out (do “dstat –list” to view the list of available plugins) In addition there is an wifi extension which displays the signal power and noise.
Some useful examples
After having looked at dstat’s potential, we are now ready to use it.
In the first example we want to cover everything which is related to the disk:
dstat -d --disk-util --freespace
In the second example we simply want to see who is eating all the memory:
dstat -g -l -m -s --top-mem
The third example shows some stats to the CPU resource consumption:
dstat -c -y -l --proc-count --top-cpu
Example number four will show some beautiful network statistics:
dstat -n --socket --tcp --udp
As you can see, there are many things you can observe with dstat. I highly recommend to have a look at dstat’s manpage and get in touch with all the other parameters.
Writing your own dstat plugin
Writing a new dstat plugin should be easy. I have not tried it yet, but /usr/share/dstat/dstat_helloworld.py shows how simple this should be:
### Author: Dag Wieers
Example "Hello world!" output plugin for aspiring Dstat developers.
self.name = 'plugin title'
self.nick = ('counter',)
self.vars = ('text',)
self.type = 's'
self.width = 12
self.scale = 0
self.val['text'] = 'Hello world!'