I wrote a little script which does exactly that: After you provided the login credentials, the script connects to your inbox and displays the subjects and sender names of the messages.

You can find the new tool here.

Related posts:

1. Simple SQL Injection Vulnerability Scanner 0.3 released
2. Simple LAN Scanner 1.0 released
3. Simple SQL Injection Vulnerability Scanner 0.5 released

Let a simple script do the work

body common control {
version => "1.0";
inputs => { "cfengine_stdlib.cf" };
bundlesequence => { "mount_nfs_share" };

}

bundle agent mount_nfs_share {
storage:
"/mnt" mount => nfs("localhost", "/blubb");
}

Save the file and check if the syntax is correct:

cf-promises -f /var/lib/cfengine/inputs/example4.cf

You should not receive any errors, so let’s run cf-agent for executing your new script. I assume that you have a NFS server installed on localhost and created a directory called /blubb. Please adjust the script accordingly if this is not the case for your test setup.

cf-agent -f /var/lib/cfengine3/inputs/example4.cf

Running the command mount now shows that the NFS share is mounted:

mintbox inputs # mount
/dev/sda1 on / type ext4 (rw,errors=remount-ro,commit=0)
[...]
localhost:/blubb on /mnt type nfs (rw,vers=4,addr=127.0.0.1,clientaddr=127.0.0.1)

The script explained
I will not talk about the first section “body common control” since you already know what happens there.
But the second part of example4.cf is very interesting:

bundle agent mount_nfs_share {
storage:
"/mnt" mount => nfs("localhost", "/blubb");
}

The bundle “mount_nfs_share” contains a promise of the type “storage” which executes the NFS mount. The important parameters are provided within the round brackets.
Well, that’s already it! This code sample shows that Cfengine 3 takes its promise of providing simplicity very serious.

Adding a bonus

Cfengine 3 also has the power to add an entry to the /etc/fstab file. I find this to be very useful, that’s why I will show you how to do that in one of my next blog posts.

By the way, you can download today’s Cfengine3 code snippet here.

Related posts:

1. Taking a quick look at the configuration management system Cfengine 3
2. Installing packages with Cfengine 3
3. Cfengine3: Check for running services

Writing the script

body common control {
version => "1.0";
inputs => { "cfengine_stdlib.cf" };
bundlesequence => { "manage_packages" };
}

bundle agent manage_packages {
vars:
"package_list" slist => { "slapd", "ldap-utils" };

packages:
"$(package_list)"
package_policy => "add",
package_method => generic;
}

Save the file and check if the syntax is correct:

cf-promises -f /var/lib/cfengine/inputs/example3.cf

No errors? Nah, of course not! Time to run cf-agent:

cf-agent -f /var/lib/cfengine3/inputs/example3.cf

In my case, cf-agent gives some output and shows me that the cache of the apt repos are renewed. A few seconds later, aptitude –assume-yes install ldap-utils slapd gets called (excerpt):

cf3> ?? Already have a (cached) package list for this manager
cf3> -> Package version was not specified
cf3> -> Looking for (slapd,*,*)
cf3> No installed packages matched (slapd,*,*)
cf3> -> Looking for (slapd,*,*)
cf3> No installed packages matched (slapd,*,*)
cf3> Checking if package (slapd,*,*) is at the desired state (installed=0,matched=0)
cf3> -> Package promises to refer to itself as "slapd" to the manager
cf3> -> Package version seems to match criteria
cf3> -> Schedule package for addition
cf3> -> Package (slapd,any,any) found
cf3>
cf3> .........................................................
cf3> Promise handle:
cf3> Promise made by: ldap-utils
cf3> .........................................................
cf3>
cf3> -> This promise has already been verified
cf3> ?? Already have a package list for this manager
cf3> -> Package version was not specified
cf3> -> Looking for (ldap-utils,*,*)
cf3> No installed packages matched (ldap-utils,*,*)
cf3> -> Looking for (ldap-utils,*,*)
cf3> No installed packages matched (ldap-utils,*,*)
cf3> Checking if package (ldap-utils,*,*) is at the desired state (installed=0,matched=0)
cf3> -> Package promises to refer to itself as "ldap-utils" to the manager
cf3> -> Package version seems to match criteria
cf3> -> Schedule package for addition
cf3> -> Package (ldap-utils,any,any) found
cf3> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
cf3> Offering these package-promise suggestions to the managers
cf3> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
cf3> -> Deletion schedule...
cf3> -> Addition schedule...
cf3> Execute scheduled package addition
cf3> Command prefix: /usr/bin/aptitude --assume-yes install
cf3> Executing /usr/bin/aptitude --assume-yes install ldap-utils slapd ...
cf3> Q:aptitude --assume-ye ...:Reading package lists...
cf3> Q:aptitude --assume-ye ...:Building dependency tree...
cf3> Q:aptitude --assume-ye ...:Reading state information...
cf3> Q:aptitude --assume-ye ...:Reading extended state information...
cf3> Q:aptitude --assume-ye ...:Initializing package states...
cf3> Q:aptitude --assume-ye ...:Writing extended state information...
cf3> Q:aptitude --assume-ye ...:The following NEW packages will be installed:
cf3> Q:aptitude --assume-ye ...: ldap-utils slapd
cf3> Q:aptitude --assume-ye ...:0 packages upgraded, 2 newly installed, 0 to remove and 348 not upgraded.
[...]

Some details of the Cfengine 3 script
Since I assume that you have read my previous blog posts about Cfengine 3 I am also assuming that you are already familiar with the first section of the script (“body common control”).

Hence we should directly look at the next part:

bundle agent manage_packages {
vars:
"package_list" slist => { "slapd", "ldap-utils" };

Here we define a promise of the type “vars”. The promise contains a variable with the name “package_list” and the type “slist”. The “package_list” contains a list of packages which should be always installed. If they are not, Cfengine 3 will keep the promise and install them for you..

… which leads us to the last part:

packages:
"$(package_list)"
package_policy => "add",
package_method => generic;
}

“packages” is a promise type, like “vars”, “commands” or “interfaces”. The “package_policy” basically defines what happens next and could also be “addupdate” in this example. If the packages would have already been installed on our system they simply would be updated (if even possible).

Now the “package_method” is very interesting. There are several package methods and they are all defined in the file “cfengine_stdlib.cf” which comes with Cfengine 3 by default. The method “generic” is valid for supported operating systems. This means that you could run this Cfengine3 script on Debian, RedHat and other distros and it would still be work. If you wanted some more distro-specific, you could use “package_method => apt” or “package_method => yum” for example.

Well, that’s already it! Today’s example showed how to perform a simple package management with the help of Cfengine 3. I hope this code snipped might help you in some way. If not, you maybe want to contribute a new blog entry to my blog?

Btw, you can download the Cfengine 3 script of this blog post here.

Related posts:

1. Mounting NFS-Shares with Cfengine 3
2. Taking a quick look at the configuration management system Cfengine 3
3. Cfengine3: Check for running services

Writing the script
Connect to your Linux box and create the following file: /var/lib/cfengine3/inputs/example2.cf

body common control {
version => "1.0";
bundlesequence => { "check_services" };
}

bundle agent check_services {
vars:
"services" slist => { "apache2", "mysql" };
"init_scripts_path" string => "/etc/init.d";

processes:
"$(services)"
comment => "Check if the processes for '$(services)'",
restart_class => "restart_$(services)";

commands:
"${init_scripts_path}/${services} start"
comment => "Restarting the service",
ifvarclass => "restart_${services}";
}

Save the file and check if the syntax is correct:

cf-promises -f /var/lib/cfengine/inputs/example2.cf

The output should not contain any errors. In this example, I have installed Apache2 and MySQL 5 on my Linux Mint VM. The script above checks if both daemons are running – if they aren’t, they will be started.
Of course you should stop those services before running the script and you can replace the service names with your own ones.

Now let the cf-agent start Apache2 and MySQL5:

cf-agent -f /var/lib/cfengine3/inputs/example2.cf

Running pstree, I now see that both services are up again and running. The output of cf-agent acknowledges this fact:

cf3> Initiate variable convergence...
cf3> -> Immunizing against parental death
cf3> -> Bundlesequence => {'check_services'}
cf3>
cf3> *****************************************************************
cf3> BUNDLE check_services
cf3> *****************************************************************
cf3>
cf3>
cf3> =========================================================
cf3> vars in bundle check_services (1)
cf3> =========================================================
cf3>
cf3>
cf3> + Private classes augmented:
cf3>
cf3> - Private classes diminished:
cf3>
cf3>
cf3>
cf3> =========================================================
cf3> processes in bundle check_services (1)
cf3> =========================================================
cf3>
cf3> Observe process table with /bin/ps -eo user,pid,ppid,pgid,pcpu,pmem,vsz,pri,rss,nlwp,stime,time,args
cf3>
cf3> .........................................................
cf3> Promise handle:
cf3> Promise made by: apache2
cf3>
cf3> Comment: Check if the processes for 'apache2'
cf3> .........................................................
cf3>
cf3> -> Making a one-time restart promise for apache2
cf3>
cf3> .........................................................
cf3> Promise handle:
cf3> Promise made by: mysql
cf3>
cf3> Comment: Check if the processes for 'mysql'
cf3> .........................................................
cf3>
cf3> -> Making a one-time restart promise for mysql
cf3>
cf3> =========================================================
cf3> commands in bundle check_services (1)
cf3> =========================================================
cf3>
cf3> -> Promiser string contains a valid executable (/etc/init.d/apache2) - ok
cf3>
cf3> .........................................................
cf3> Promise handle:
cf3> Promise made by: /etc/init.d/apache2 start
cf3>
cf3> Comment: Restarting the service
cf3> .........................................................
cf3>
cf3> -> Executing '/etc/init.d/apache2 start' ...(timeout=-678,owner=-1,group=-1)
cf3> -> (Setting umask to 77)
cf3> -> Finished command related to promiser "/etc/init.d/apache2 start" -- succeeded
cf3> Q: "....d/apache2 star": * Starting web server apache2
Q: "....d/apache2 star": apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName
Q: "....d/apache2 star": ...done.
cf3> I: Last 3 quoted lines were generated by promiser "/etc/init.d/apache2 start"
cf3> -> Completed execution of /etc/init.d/apache2 start
cf3> -> Promiser string contains a valid executable (/etc/init.d/mysql) - ok
cf3>
cf3> .........................................................
cf3> Promise handle:
cf3> Promise made by: /etc/init.d/mysql start
cf3>
cf3> Comment: Restarting the service
cf3> .........................................................
cf3>
cf3> -> Executing '/etc/init.d/mysql start' ...(timeout=-678,owner=-1,group=-1)
cf3> -> (Setting umask to 77)
cf3> -> Finished command related to promiser "/etc/init.d/mysql start" -- succeeded
cf3> Q: "...it.d/mysql star": Rather than invoking init scripts through /etc/init.d, use the service(8)
Q: "...it.d/mysql star": utility, e.g. service mysql start
Q: "...it.d/mysql star": Since the script you are attempting to invoke has been converted to an
Q: "...it.d/mysql star": Upstart job, you may also use the start(8) utility, e.g. start mysql
Q: "...it.d/mysql star": mysql start/running, process 4235
cf3> I: Last 5 quoted lines were generated by promiser "/etc/init.d/mysql start"
cf3> -> Completed execution of /etc/init.d/mysql start
cf3>
cf3> =========================================================
cf3> vars in bundle check_services (2)
cf3> =========================================================
cf3>
cf3>
cf3> + Private classes augmented:
cf3>
cf3> - Private classes diminished:
cf3>
cf3>
cf3>
cf3> =========================================================
cf3> processes in bundle check_services (2)
cf3> =========================================================
cf3>
cf3> -> Reusing cached process state
cf3>
cf3> =========================================================
cf3> commands in bundle check_services (2)
cf3> =========================================================
cf3>
cf3>
cf3> =========================================================
cf3> vars in bundle check_services (3)
cf3> =========================================================
cf3>
cf3>
cf3> + Private classes augmented:
cf3>
cf3> - Private classes diminished:
cf3>
cf3>
cf3>
cf3> =========================================================
cf3> processes in bundle check_services (3)
cf3> =========================================================
cf3>
cf3> -> Reusing cached process state
cf3>
cf3> =========================================================
cf3> commands in bundle check_services (3)
cf3> =========================================================
cf3>
cf3> Outcome of version 1.0 (agent-0): Promises observed to be kept 0%, Promises repaired 100%, Promises not repaired 0%
cf3> Estimated system complexity as touched objects = 256, for 10 promises

Taking a closer look at the script

body common control {
version => "1.0";
bundlesequence => { "check_services" };
}

The first section begins with “body common control” and defines a version for this document. This string has no big meaning and is only used within reports and error messages. The term “bundlesequence” points to the promise “check_services”. Of course you can provide multiple promises and their order here.

bundle agent check_services {
vars:
"services" slist => { "apache2", "mysql" };
"init_scripts_path" string => "/etc/init.d";

The next section is starting with a bundle with the type “agent” and the name “check_services”. The type has to be one of the pre-defined types of Cfengine3. “agent” seems to be one of the most used types and is often called the “executive bundle”. With the help of agent bundles the wanted (configuration) changes are implemented in the system (e.g. editing config files). This is also where the free and commercial version of Cfengine3 differ. While the free edition allows interacting with commands, files, methods, packages, processes, storage and devices within agent bundles, the commercial edition lets the script also interact with databases, virtual environments, Windows services and gives control about the log level.

With “vars” we tell Cfengine3 that we want to define promises of the type “vars” (which obviously stands for variables). These vars can be included in every type of bundle and are defined similar to programming languages: “type” => “value”.
The quoted string before the variable type is provided as an attribute and tells the reader what is stored inside the variable.
In our example we defined two promises of the type “vars”. One contains the name of the services to be looked for and the other one will lead Cfengine3 to the path of the init scripts later.

processes:
"$(services)"
comment => "Check if the processes for '$(services)'",
restart_class => "restart_$(services)";

The next section defines promises of the type “processes”. For each list item within the defined “services” the promise will be executed. In this case, a restart class is defined.

commands:
"${init_scripts_path}/${services} start"
comment => "Restarting the service",
ifvarclass => "restart_${services}";
}

The last section defines promises of the type “commands”. The two variables are concatenated and serve with the full path of the “ifvarclass” which will be responsible for restarting the service(s).

That’s it! I hope that I made no mistakes while presenting this simple Cfengine3 script example to you. You can download today’s script here.

Related posts:

1. Installing packages with Cfengine 3
2. Mounting NFS-Shares with Cfengine 3
3. Taking a quick look at the configuration management system Cfengine 3

What the hell is Cfengine?
Cfengine can be considered as a free management tool for servers and clients. With the help of an agent and configuration scripts, a system can be fully configured without editing any file manually. The agent does all the work and you only provide the necessary script once. Many data centers use Cfengine to take care of the configuration of thousands of servers. Having this in mind, it is easy to imagine that suddenly one single guy can actually administrate so many servers on his own.

What we will do 
Without any talking and diving into the materia, we will take a quick look at Cfengine 3. If you are interested in more details or a longer description about the tool please visit the official website.
Now access your Linux box and install the stuff. I use Linux Mint, that is why I assume you you apt for package management. Please note that the paths mentioned here are default paths in Debian and Ubuntu. They might differ if you are using another Linux distribution.

apt-get install cfengine3

Before we can get started, we need to copy a few files to the right place:

cp /usr/share/doc/cfengine3/examples/promises.cf /etc/cfengine3/
cp /usr/share/doc/cfengine3/examples/update.cf /etc/cfengine3/
cp /usr/share/doc/cfengine3/examples/failsafe.cf /etc/cfengine3/
cp /usr/share/doc/cfengine3/examples/site.cf /etc/cfengine3/
cp /usr/share/doc/cfengine3/examples/cfengine_stdlib.cf /etc/cfengine3/

These files are basic input files for Cfengine and are needed for basic operations.
If you want to edit /etc/cfengine/promises.cf with an editor of your choice and replace the default mail address with yours.

Having done that, we can “enable” Cfengine 3 by editing the file /etc/defaults/cfengine:

RUN_CFMONITORD=1
RUN_CFSERVERD=1
RUN_CFEXECD=1

Now let’s go!

/etc/init.d/cfengine3 start

When running ps aux and grepping for cf you should now see the following daemons:

• /usr/sbin/cf-monitord
• /usr/sbin/cf-serverd
• /usr/sbin/cf-execd

A very simple example for editing text files
Create the file /var/lib/cfengine3/inputs/example.cf and fill it with the following content:

body common control

{
any::

bundlesequence => {
editexample
};
}

bundle agent editexample

{
files:

"/etc/example"

edit_line => addline,
create => true;
}

bundle edit_line addline

{
insert_lines:

"Test.";

}

Close and save the file.
Cfengine provides you with a tool to check the syntax of a .cf-script, so let’s use it:

cf-promises -f /var/lib/cfengine/inputs/example.cf

Ok, no errors, so let’s execute the script with the help of the Cfengine agent:

cf-agent -f /var/lib/cfengine3/inputs/example.cf

Now have a look at your file which was auto-created by the “create => true”-statement:

cat /etc/example
Test.

Woho!
You will realize that Cfengine created a backup of the original file before editing it (here: /etc/example.cf-before-edit).

Some final words
Of course this is a very simple example and only demonstrates a tiny part of Cfengine’s abilities! But you now should be able to get the idea behind Cfengine. Cfengine also allows connections over network, meaning that you can run a configuration server with “clients”. Furthermore Cfengine logs most of the stuff in the /var/log/ folder.

I hope you liked the short tour! Hopefully I will have the time and motivation to write more blog posts about Cfengine and show you some complex examples.

Update on 2012-05-07: Jon posted some feedback, I updated this blog post accordingly.

You can download the sample Cfengine3 script described in this post here: example.cf

Related posts:

1. Mounting NFS-Shares with Cfengine 3
2. Installing packages with Cfengine 3
3. Cfengine3: Check for running services

You can view the simple awk cheat sheet here.

Most of my blog readers will already be familiar with awk. Well, I am not and I am still learning. The awk cheat sheet therefore helps to keep track of the important and cool features I am discovering during my learning process.

As it turns out, I needed awk not that often in the past although I am working as a Linux professional. But well, awk is really useful and it is worth that one dives in deeper.

Please note that the simple awk cheat sheet is still work in progress.

No related posts.

The first issue is about 10 new features in PostgreSQL 9.1 and contains a very interesting interview with “the admin” begind postgresql.org.

I highly recommend to read this magazine if you are a fan of the MySQL alternative and want to dive in deeper.

No related posts.

No related posts.

Linux Magazin 05/2012

Diese Ausgabe beinhaltet einen Artikel von Thilo und mir: “Log-Wellenreiter – Logfiles überwachen und Aktionen anstoßen“. Dieser Artikel behandelt die automatisierte Auswertung von Log-Dateien und das Reagieren auf vordefinierte Events.

Zusätzlich durfte ich den Artikel “Linux-Multimeter – Systemdiagnose von Vmstat über Netstat bis Dstat” beisteuern. Dieser Beitrag behandelt das Tool dstat und erklärt, warum man bei dessen Einsatz auf viele andere Analyse-Tools verzichten kann.

Beide Artikel können online “gepreviewt” und jeweils einzeln für 1 Euro gekauft werden.

Related posts:

1. New articles published in the German Linux Magazine and Android User
2. Real-time system monitoring with Dag Wieers’ dstat

Furthermore, they describe what actions should be taken in order to improve the WHOIS service (and data accuracy).

The WPRT even quotes one of my suggestions (see page 49). You can view the latest draft here:

http://www.icann.org/en/reviews/affirmation/whois-rt-draft-final-report-05dec11-en.pdf

Related posts:

1. The future of the WHOIS protocol