Fixed LDAP after upgrading from CentOS 5.4 to 5.5

Some months ago I upgraded my CentOS servers from version 5.4 to 5.5. One of these servers were running LDAP Master and LDAP Slave as playground. Although after the upgrade to CentOS 5.5 it was broken, but due to other priorities I didn’t had a change to fix it. 

On my systems I enabled TLS to communicate to LDAP-servers and also enabled kerberos. So this results in a modified /etc/sysconfig/ldap:

# Enable Kerberos
export KRB5_KTNAME=”FILE:/etc/openldap/ldap.keytab”

But I noticed that the RPM installed a new version of that, although with the extension .rpmnew. So after applying the changes that were in the .rpmnew file and when I set SLAPD_LDAPS and SLAPD_LDAPI to “yes” I end up with the following content:

# Parameters to ulimit called right before starting slapd
# – use this to change system limits for slapd
ULIMIT_SETTINGS=

# How long to wait between sending slapd TERM and KILL
# signals when stopping slapd by init script
# – format is the same as used when calling sleep
STOP_DELAY=3s

# By default only listening on ldap:/// is turned on.
# If you want to change listening options for slapd,
# set following three variables to yes or no
SLAPD_LDAP=yes
SLAPD_LDAPS=yes
SLAPD_LDAPI=yes
export KRB5_KTNAME=”FILE:/etc/openldap/ldap.keytab”

And guess what… It works again :-)

use subversion to publish websites

Sometimes I’m really surprised about myself… especially how lazy I am. :-)

I’m currently playing around with one of my private websites, and to improve developing I decided to use subversion. So far so good, but I wanted that the committed subversion code was automatically online on the webserver. So I did the following very simple trick.

First I check out the code (subtree) from the subversion server (which is using https):

$ cd /sites
$ mv dev.adslweb.net{,-backup}
$ svn co https://svn.adslweb.net/svn/websites/dev.adslweb.net

Next step was to commit the current content of the website into subversion:

$ cd /sites/dev.adslweb.net
$ cp -Rv /sites/dev.adslweb.net-backup/*  ./
$ svn add *
$ svn commit -m “Initial commit of ADSLWEB.net dev env”

Now download the simple script I created for making sure that subversion doesn’t fire off twice for updating the same tree.

Download svn-update.sh via this link.

So something like this:

$ mkdir ~/scripts/
$ cd ~/scripts
$ wget http://www.xs4all.nl/~paderijk/pics/svn-update.sh
$ chmod 700 svn-update.sh

Now… the last step… create a crontab entry with the following content:

*/1 * * * * /home/pieter/scripts/svn-update.sh /sites/dev.adslweb.net 2>&1 > /dev/null

 And guess… and it works like a charm, on every new commit done by whoever… you get your online site updated within 1 minute without the need log in into the website/webserver using ftp/ssh/whatever.

More flexible yum-repo sync script

In the past I started syncing every night the Updates-repositories from Fedora and CentOS on a local server, just to speed up updates/kick starts et cetera… The first version of the script was very quick and dirty, now I’ve a more decent script that allow you to add/remove very quick new versions of CentOS and Fedora.

 You can find the script here.

How full are your snapshot volumes in LVM?

As I mentioned in my previous post, which is already 2 months old :(, I’m using snapshots for data retention.

Now I run up in the situation, that I wanted to know how full the snapshots are. A ‘normal’ df will not work… but I figured it out! The command lvs is willing to do the work:

# lvs –aligned –separator | vol_backup
  LV                |VG        |Attr  |LSize |Origin|Snap% |Move|Log|Copy% |Convert
  lvm0              |vol_backup|owi-ao|40.00G|      |      |    |   |      |      
  snap-20100412_2350|vol_backup|swi-a-| 4.00G|lvm0  | 23.71|    |   |      |      
  snap-20100413_2350|vol_backup|swi-a-| 4.00G|lvm0  | 21.70|    |   |      |      
  snap-20100414_2350|vol_backup|swi-a-| 4.00G|lvm0  | 19.52|    |   |      |      
  snap-20100415_2350|vol_backup|swi-a-| 4.00G|lvm0  | 17.53|    |   |      |      
  snap-20100416_2350|vol_backup|swi-a-| 4.00G|lvm0  | 15.54|    |   |      |      
  snap-20100417_2350|vol_backup|swi-a-| 4.00G|lvm0  | 13.56|    |   |      |      
  snap-20100418_2350|vol_backup|swi-a-| 4.00G|lvm0  | 11.56|    |   |      |      
  snap-20100419_2353|vol_backup|swi-a-| 4.00G|lvm0  |  9.02|    |   |      |      
  snap-20100420_2353|vol_backup|swi-a-| 4.00G|lvm0  |  6.76|    |   |      |      
  snap-20100421_2350|vol_backup|swi-a-| 4.00G|lvm0  |  2.79|    |   |      | 

In the ‘Snap%‘ column you can see how full your snapshot volume is!

Creating Snapshots of a backup using LVM snapshot

Normally I used to have a backup-retention-script in place that will create a TAR-ball of the backup data (using Herakles). But this way I was not able to have a retention of longer then 3 days :-(

So I had to look into another solution, I could add a new harddrive in the server… but there should be something else possible. So I ended up by using LVM snapshots. So I created a Volume group of about 100GB. In that volume group I created a logical volume of about 30GB, which is enough (and if not, we can ‘grow’ the Filesystem thanks to LVM :-) )

After having all that done, I’ve created a script located in /root/scripts/lvm-snapshot. This script runs every midnight and creates a snapshot.

#!/bin/bash
#
# Create LVM Snapshots
#
#
#—————————————————————————————————————
CURRENT_SNAPNAME=”snap-“$(date “+%Y%m%d%H%M%S”)
VOLUME2SNAPSHOT=”/dev/vol_backup/lvm0″
LVMSNAPSHOTCMD=”/usr/sbin/lvcreate -L 2G -s -n $CURRENT_SNAPNAME $VOLUME2SNAPSHOT”
LINE=”———————————————————————————————————————“

echo $LINE
df -h /mnt/data
echo $LINE
$LVMSNAPSHOTCMD 2> /dev/null
#—————————————————————————————————————
SNAPSHOT_RETENTION=15
CURRENT_SNAPSHOT_COUNT=$(lvdisplay | grep “^  LV Name                /dev/vol_backup/snap” | sort | awk ‘{ print $3 }’ | wc -l)

OVERFLOW=$(echo $CURRENT_SNAPSHOT_COUNT – $SNAPSHOT_RETENTION | bc)
if [ $OVERFLOW -gt 0 ];
then
        echo $LINE
        for files in  $(lvdisplay | grep “^  LV Name                /dev/vol_backup/snap” | sort | awk ‘{ print $3 }’ | head -n$OVERFLOW);
        do
                 /usr/sbin/lvremove -f $files 2> /dev/null
        done
fi
#—————————————————————————————————————
echo $LINE
/usr/sbin/vgdisplay vol_backup
echo $LINE
/usr/sbin/lvdisplay $VOLUME2SNAPSHOT

And the crontab entry is:

# crontab -l
0 0 * /root/scripts/lvm-snapshot

Import private key and (signed) certificate into Java keystore (JKS)

Last monday, I had to ‘secure’ the smcwebserver from Sun (or should I say Oracle?), that is used by ARCo. But I run into a few issues:

  1. My lack of knowledge about Java;
  2. Keytool doesn’t allow you to import keys generated by tools like openssl :-(

But… I was able to handle them both and know I have an smcwebserver (which is using Java-keystores) running with a key that was generated by openssl and a certificated signed by our enterprise CA.

There for I had to do some Java ‘hacking’. After some hours spending on Google-searches, I landed on a posting on the website of ‘Agent Bob‘. He has some Java-program that allows you to ‘import’ keys and certificates that were generated outside keytool :-)

Although, I had to perform some minor modification on the Java-code, to set the password of the new JKS to ‘changeit‘, because that is what smcwebserver will try to open the keystore. So, you need to make sure that line 87 is:

String keypass = “changeit”;

For your convenience you can download the modified version here.

Now, create a Java class with the command (please note, I’m not a Java-specialist, so something else will work as well… but this worked for me ;-) ):
$ javac ImportKey.java

Having this done, you must make sure, your key-file and (signed) certificate are in the DER format. If they are not, you can convert them using the following commands:
$ openssl pkcs -topk8
               -nocrypt
               -in server.key
               -out server.key.der
               -outform der

$ openssl x509 -in server.crt
               -out server.crt.der
               -outform der

We can import the keys with the Java-program:

$ java ImportKey server.key.der server.crt.der webconsole

And last, but not least, put the keystore in place (and of course we make sure we’ve a backup of the old one):

# cp /var/opt/webconsole/domains/console/conf/keystore.jks{,.backup}
# cp $HOME/keystore.ImportKey /var/opt/webconsole/domains/console/conf/keystore.jks

Now we have to restart the smcwebserver:

# smcwebserver stop
# smcwebserver start

That’s all :-)

Testing a kernel and initrd with qemu

In my previous post I wrote how to add a module to the initrd.img file. Although, some testing might be nice.This testing can be done using qemu.So for example:# qemu-system-x86_64 -kernel /scratch/blah/isolinux/vmlinuz -initrd /tmp/initrd.img-mod…

Adding additional/new modules to the initial ramdisk on Linux.

Recently I needed to add the support for a new network-interface to a initial ramdisk (initrd) on Linux (RHEL 5u3).

After some h4ck1ng and a number of hours on Google I was able to do add the module to the initrd. :-)

The steps I did are:

Extract initrd

  1. Get an original initrd.img from a boot ISO and put it into /tmp/initrd.img-original
  2. Create a temp environment where we can extract the initrd:

    $ mkdir -p /scratch/initrd-mod/{initrd,modules}

  3. Extract the /tmp/initrd.img-original to the temp-environment:

    $ cd /scratch/initrd-mod/initrd
    $ zcat /tmp/initrd.img-original | cpio -i

Add the module to initrd.img

Extract the modules.cgz file

The modules (.ko files) are located in the (container) modules/modules.cgz in the initrd.

Now you need to extract the modules.cgz file:

$ cd /scratch/initrd-mod/modules
$ zcat /scratch/initrd-mod/initrd/modules/modules.cgz | cpio -idvm

Add the module

Now you’ve to make sure you’ve an module compiled using the right kernel version and architecture. Copy the new .ko file into the extracted modules.cgz tree

$ cp /tmp/new-module.ko /scratch/initrd-mod/modules/{VERSION}/{ARCH}

So in my case with RHEL5u3 it is the location:

$ cp /tmp/igb.ko /scratch/initrd-mod/modules/2.6.18-128.el5/x86_64/

Repack the modules.cgz

Now we need to repack the modules.cgz file:

$ cd /scratch/initrd-mod/modules
$ find . -type f | cpio -o -H crc | gzip -n9 > /scratch/initrd-mod/initrd/modules/modules.cgz

Modify the modules.alias file

Now you need to modify the modules.alias file in order to get the module loaded properly.

The aliases can be find using modinfo:

$ /sbin/modinfo /tmp/igb.ko | grep ^alias
alias: pci:v00008086d000010D6sv*sd*bc*sc*i*
alias: pci:v00008086d000010A9sv*sd*bc*sc*i*
alias: pci:v00008086d000010A7sv*sd*bc*sc*i*
alias: pci:v00008086d000010E8sv*sd*bc*sc*i*
alias: pci:v00008086d0000150Dsv*sd*bc*sc*i*
alias: pci:v00008086d000010E7sv*sd*bc*sc*i*
alias: pci:v00008086d000010E6sv*sd*bc*sc*i*
alias: pci:v00008086d0000150Asv*sd*bc*sc*i*
alias: pci:v00008086d000010C9sv*sd*bc*sc*i*

The correct entries can be create using the following one liner (in this case for the igb module):

$ /sbin/modinfo /tmp/igb.ko | grep ^alias | awk ‘{ print “alias ” $2 ” igb” }’
alias pci:v00008086d000010D6sv*sd*bc*sc*i* igb
alias pci:v00008086d000010A9sv*sd*bc*sc*i* igb
alias pci:v00008086d000010A7sv*sd*bc*sc*i* igb
alias pci:v00008086d000010E8sv*sd*bc*sc*i* igb
alias pci:v00008086d0000150Dsv*sd*bc*sc*i* igb
alias pci:v00008086d000010E7sv*sd*bc*sc*i* igb
alias pci:v00008086d000010E6sv*sd*bc*sc*i* igb
alias pci:v00008086d0000150Asv*sd*bc*sc*i* igb
alias pci:v00008086d000010C9sv*sd*bc*sc*i* igb

Make sure you remove duplicates in the modules.aliases file.

Repack the initrd.img

Now it’s time to repack the intird.img file with the new module:

$ cd /scratch/initrd-mod/initrd
$ find ./ | cpio -H newc -o | gzip -n9 > /tmp/initrd.img-modded

And put the /tmp/initrd.img-modded onto your boot disk.