Weblog moved to VPS and uses CloudFlare’s CDN

Maybe you’ve noticed… maybe you did not… but the weblog got a new URL… instead of blog.adslweb.net it’s now blog.angits.net.

The reason for the new URL is the fact that the weblog now uses CloudFlare’s Content Delivery Network and with the free plan from CloudFlare it’s not possible to use ‘subdomains’ and since I had angits.net ‘laying around as spare’ I started using that one.

Besides using a CDN, the weblog physically moved from my server at home to a VPS in the DataCenter of NedZone.

Network overview of CloudFlare’s CDN

Investigate disk usage

Recently I had to investigate the usage of a very big volume… with a lot of data-files, owned by several users.

I started with “agedu”, but somehow I was not able to get the information I needed.. so I started using find with stat and put everything into MySQL.

So the first step was to do the following find command:

# find /nfs/bigfiler -exec stat –format=”%F:%n:%s:%U:%u:%G:%g:%X:%Y:%Z” {} ; > /scratch/big-filer.info

Create a table in MySQL:

CREATE TABLE `pieter.bigfiler_content` (
`fileid` int(10) unsigned NOT NULL AUTO_INCREMENT,
`file_type` char(32) NOT NULL,
`filename` varchar(512) NOT NULL,
`size` int(11) NOT NULL,
`user` char(16) DEFAULT NULL,
`uid` char(16) DEFAULT NULL,
`groupname` char(16) DEFAULT NULL,
`gid` char(16) DEFAULT NULL,
`time_access` datetime DEFAULT NULL,
`time_mod` datetime DEFAULT NULL,
`time_change` datetime DEFAULT NULL,
PRIMARY KEY (`fileid`),
KEY `idx_file_type` (`file_type`) USING BTREE,
KEY `idx_user` (`user`) USING BTREE
)

Load the data into MySQL:

LOAD DATA INFILE '/scratch/big-filer.info'
INTO TABLE pieter.bigfiler_content
FIELDS TERMINATED BY ':'
LINES TERMINATED BY 'n'
(file_type, filename, size, user, uid, groupname, gid, @time_access, @time_mod, @time_changed)
SET time_access = FROM_UNIXTIME(@time_access),
time_mod = FROM_UNIXTIME(@time_mod),
time_change = FROM_UNIXTIME(@time_change);

And now you can run nice queries to analyse the data 😀 

YUM repo fails with koji/mock when base URL is used

Recently I started playing around with Koji for package building.

Everything was set-up pretty fast… but the first attempts to build a build-root failed…

After some troubleshooting I found the cause…

If you use the repo-data as available on an installation ISO (served via a webserver) and use mergerepos the location of an RPM will look like:

<location xml:base="CentOS" href="pam-0.99.6.2-6.el5_5.2.x86_64.rpm"/>

The entry as available on the ISO is:

<location xml:base="media://1330913492.861127#1" href="CentOS/pam-0.99.6.2-6.el5_5.2.x86_64.rpm"/>

The way to workaround this issue, is be recreated the repodata using createrepo:

# createrepo -u http://172.16.3.240/repo/centos/5.8/base/x86_64/-o ${WEBSERVERPATH}/new-repo/ /media/cd/

And then the entry will look like:

<location xml:base=”http://172.16.3.240/repo/centos/5.8/base/x86_64/CentOS” href=”pam-0.99.6.2-6.el5_5.2.x86_64.rpm”/>

The bug can be found in bugzilla.

SSL Chain

I’ve ordered via http://www.cheapssls.com/ a simple SSL-certificate signed by Comodo for use with apache… although a lot of browsers (Firefox on Mac OS X, all browsers in Linux) didn’t accepted it (CA was not know…)
After some discussion with …

Block mail from certain countries with sendmail

If you have your own MTA running… you are probably known with the spam-problems… Once you’ve tuned the filters, you have to do it again… because a new spam-run comes in. I also blocked whole /8 subnets in different countries (India/China/…)… but that is not a “real” solution… aka I want to block the whole country…

 The “DNSBL” countries.nerd.dk  allows you to do so… the map ip-adresses to countries based on whois-information… so on my MTAs I added the following lines to the mc sendmail file:

FEATURE(dnsbl,`br.countries.nerd.dk', `554 - Rejected - SPAM from Brazil:$&{client_addr} rejected')dnl
FEATURE(dnsbl,`in.countries.nerd.dk', `554 - Rejected - SPAM from India:$&{client_addr} rejected')dnl
FEATURE(dnsbl,`kr.countries.nerd.dk', `554 - Rejected - SPAM from Korea:$&{client_addr} rejected')dnl
FEATURE(dnsbl,`cn.countries.nerd.dk', `554 - Rejected - SPAM from China:$&{client_addr} rejected')dnl
FEATURE(dnsbl,`ro.countries.nerd.dk', `554 - Rejected - SPAM from Romenia:$&{client_addr} rejected')dnl
FEATURE(dnsbl,`co.countries.nerd.dk', `554 - Rejected - SPAM from Colombia:$&{client_addr} rejected')dnl
FEATURE(dnsbl,`mk.countries.nerd.dk', `554 - Rejected - SPAM from Macedonia:$&{client_addr} rejected')dnl
FEATURE(dnsbl,`vn.countries.nerd.dk', `554 - Rejected - SPAM from Vietnam:$&{client_addr} rejected')dnl
FEATURE(dnsbl,`ru.countries.nerd.dk', `554 - Rejected - SPAM from Russia:$&{client_addr} rejected')dnl

And within a few hours the first are already blocked… I hope this will reduce the amount of incomming spam at the “front door”. Because simply… I don’t know people in these countries…

How to update Python bindings to subversion.

Recently I run into the problem that a team had a requirement for subversion 1.6.6 (while CentOS 5u3 was not supporting this… but the vendor didn’t provide a newer release). This team also had a requirement to have TRAC… TRAC is depended on Python… but I was not allowed to update the subversion bindings for python by updating the it on the whole system… so… this is what I did:

  • Installed a number of devel packages:
       # yum install apr-devel neon{,-devel} apr-util-devel

  • Compiled sqlite version 3.6.13 and installed it on NFS:
      $ ./configure --prefix=/nfs/apps/webservices/trac-parent/sqlite/3.6.13
    ...
    $ make ; make install
    ...

  • Compiled subversion 1.6.6 and installed it on NFS:
    $ make clean; ./configure 
    --prefix=/
    nfs/apps/webservices/trac-parent/subversion/1.6.6
    --with-sqlite=
    /nfs/apps/webservices/trac-parent/sqlite/3.6.13
    --without-neon

    ...
    $ make -j8 ; make install ; make swig-py ; make install-swig-py

  • Added the following line to /etc/sysconfig/httpd:
    export LD_LIBRARY_PATH=/nfs/apps/webservices/trac-parent/sqlite/3.6.13/lib/

  • Modified /etc/httpd/conf.d/trac.conf by adding a ‘PythonPath’ to the location-directive:
    <Location /projects>
    ...
    PythonPath "['/nfs/apps/webservices/trac-parent/subversion/1.6.6/lib/svn-python'] + sys.path"
    </Location>

  • Restart the trac daemon:
    # service httpd stop
    # service httpd start

  • Now you’ve to resync the trac-instance with Subversion (the
    repository_dir value in the trac.ini of the instance).. but make sure
    you use the correct bindings in Python:

    # export LD_LIBRARY_PATH=/nfs/apps/webservices/trac-parent/sqlite/3.6.13/lib/
    # export PYTHONPATH=/nfs/apps/webservices/trac-parent/subversion/1.6.6/lib/svn-python
    # trac-admin ${TRAC_INSTANCE_PATH} repository resync "*"

CentOS 5 enabling Two-factor SSH authentication via Google

Today I noticed a very nice article about enabling Google’s two-factor authentication for Linux SSH.

After reading it… I found some time to play with it… so I enabled it within 10 minutes on my CentOS 5 64bit play-ground server… but there are some small ‘caveats’.

hg – Command

To checkout the code, you must make install the mercurial RPM… this one is available via the EPEL repositories.

So after having the EPEL repositories enabled, run as root:

yum -y install mercurial

Compiling the PAM module

When you checked out the code.

hg clone https://google-authenticator.googlecode.com/hg/ google-authenticator/

You cannot compile directly the module… therefor you must apply a small change to the Makefile.

Change where /usr/lib/libdl.so is stated to /usr/lib64/libdl.so (3 occurrences)

$ make
$ sudo make install

Now you’ve to update the /etc/pam.d/sshd so it contains:

#%PAM-1.0
auth       required     pam_google_authenticator.so
auth       include      system-auth
account    required     pam_nologin.so
account    include      system-auth
password   include      system-auth
session    optional     pam_keyinit.so force revoke
session    include      system-auth
session    required     pam_loginuid.so

Configure SSH

You also have to make sure that in /etc/ssh/sshd_config the following settings are set on yes:

ChallengeResponseAuthentication yes
UsePAM yes

And restart the SSH-daemon

Set up your smartphone/credentials on the system

$ google-authenticator
https://www.google.com/chart?chs=200×200&chld=M|0&cht=qr&chl=otpauth://totp/user@server%3Fsecret%3DSAEP64T5VZAVWAFB
Your new secret key is: SAEP64T5VZAVWAFB
Your verification code is 376046
Your emergency scratch codes are:
  67868696
  26247332
  54815527
  54336661
  71083816
Do you want me to update your “~/.google_authenticator” file (y/n) y
Do you want to disallow multiple uses of the same authentication
token? This restricts you to one login about every 30s, but it increases
your chances to notice or even prevent man-in-the-middle attacks (y/n) y
By default, tokens are good for 30 seconds and in order to compensate for
possible time-skew between the client and the server, we allow an extra
token before and after the current time. If you experience problems with poor
time synchronization, you can increase the window from its default
size of 1:30min to about 4min. Do you want to do so (y/n) n
If the computer that you are logging into isn’t hardened against brute-force
login attempts, you can enable rate-limiting for the authentication module.
By default, this limits attackers to no more than 3 login attempts every 30s.
Do you want to enable rate-limiting (y/n) y

And you’re done :-D

Give it a try to SSH to that box…

 TIP: Make sure you’ve an SSH session still open… or you might lock yourself out of the system…

WordPress template with jQuery flippage

Recently I’ve been working on creating a template for WordPress for my brother in law’s company. My brother in law is photographer so I had also had to implement albums/galleries using “jQuery jFlip“. So I decided to use the “NextGEN Gallery” plugin for WordPress.

The benefit of NextGEN Gallery is that it allows you to add custom gallery templates to your WordPress template/theme by having in your theme-folder a nggallery folder and files named gallery-{template_name}.php.

To enable jQuery jFlip with NextGEN Gallery I had to do the following modifications:

Add to $TEMPLATE_PATH/header.php the following lines in the head section:

<!–[if IE]><script src=”<?php bloginfo(‘template_url’); ?>/js/excanvasX.js” type=”text/javascript”></script><![endif]–>
<script src=”<?php bloginfo(‘template_url’); ?>/js/jquery-1.6.1.min.js” type=”text/javascript”></script>
<script src=”<?php bloginfo(‘template_url’); ?>/js/jquery.jflip-0.4.min.js” type=”text/javascript”></script>

Make sure you put jquery-1.6.1.min.js and jquery-jflip-0.4.min.js and excanvasX.js (for IE support) in your template, or deep link to the developer sites.

And now create a NextGEN template $TEMPLATE_PATH/nggallery/gallery-flippage.php:

<?php if (!defined (‘ABSPATH’)) die (‘No direct access allowed’); ?><?php if (!empty ($gallery)) : ?>
<script type=”text/javascript”>
  jQuery.noConflict();
  $(function(){
    $(“#gallery1″).jFlip(600,300,{background:”transparent”,cornersTop:false,scale:”fit”});
   })
</script>
<p>&nbsp;</p>
<center>
<ul id=”gallery1″>
   <?php foreach ( $images as $image ) : ?>
     <li><img src=”<?php echo $image->imageURL ?>” /></li>
    <?php endforeach; ?>
</ul>
</center>
<?php endif; ?>

Please note the ‘jQuery.noConflict()’… please make sure it’s there other wise it will drive you crazy :-(

Now make sure the NextGen gallery plugin is active and make a page in WordPress with the following content:

 

[nggallery id=5 template=flippage]

 That’s all :-)

And the results can be checked here.