All posts in Uncategorized

On one of the servers with GlusterFS, I use USB-Disks in a replicated GlusterFS volume. Although once in a while a disk just “dies” and using “replace brick” command is not an option.

So I figured out the following steps:

1. Stop the GlusterFS Daemon
2. Remove the faulty brick
3. Configure the new brick to mount on the same mountpoint as the old one
4. Start GlusterFS Daemon
5. Trigger a self-heal on the GlusterFS Volume
6. Get coffee/..
7. The new brick will contain the data

Please note, this only works for replicated volumes!

 

At home I set up a small 3 node GlusterFS Cluster based on CentOS 6.4 with one client with Fedora 16.

On the client and the nodes iptables and SELinux is disabled.

The GlusterFS package is installed using the EPEL repository.

Preparing the nodes

The nodes have been kick-started using the following %packages in the kickstart:

%packages
@core
@server-policy

After enabling the EPEL repository you run the following commands on all nodes:

yum -y install glusterfs glusterfs-server
mkdir -p /mnt/glusterfs/test/{balanced,replicated}
service glusterd start
chkconfig glusterd on

Now on one of the nodes we bind all the systems to the “cluster”:

[root@node01 ~]# gluster peer status
No peers present
[root@node01 ~]# gluster peer probe node01
Probe on localhost not needed
[root@node01 ~]# gluster peer probe node02
Probe successful
[root@node01 ~]# gluster peer probe node03
Probe successful
[root@node01 ~]# gluster peer status
Number of Peers: 2

Hostname: node02
Uuid: 6d5d5101-5a8e-4057-a854-2aac49ce22ca
State: Peer in Cluster (Connected)

Hostname: node03
Uuid: f651c972-7519-4b74-b610-f1f2f4db31ef
State: Peer in Cluster (Connected)

Create the volumes

We create two volumes, one replicated and one distributed.

Replicated volume

root@node01 ~]# gluster volume create VolReplica replica 3 transport tcp
node01:/mnt/glusterfs/test/replicated
node02:/mnt/glusterfs/test/replicated node03:/mnt/glusterfs/test/replicated
Creation of volume VolReplica has been successful. Please start the volume to access data.
[root@node01 ~]# gluster volume start VolReplica
Starting volume VolReplica has been successful
[root@node01 ~]# gluster volume info VolReplica

Volume Name: VolReplica
Type: Replicate
Status: Started
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: node01:/mnt/glusterfs/test/replicated
Brick2: node02:/mnt/glusterfs/test/replicated
Brick3: node03:/mnt/glusterfs/test/replicated

Balanced (distributed) volume

[root@node01 ~]# gluster volume create VolBalanced  transport tcp
node01:/mnt/glusterfs/test/balanced
node02:/mnt/glusterfs/test/balanced node03:/mnt/glusterfs/test/balanced
[root@node01 ~]# gluster volume start VolBalanced
[root@node01 ~]# gluster volume info VolBalanced

Volume Name: VolBalanced
Type: Distribute
Status: Started
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: node01:/mnt/glusterfs/test/balanced
Brick2: node02:/mnt/glusterfs/test/balanced
Brick3: node03:/mnt/glusterfs/test/balanced

Configure the client(s)

You can easily use one of the nodes also as client, but I nominated my (old) laptop with FC 16 as GlusterFS client.

First ensure the EPEL repositories are enabled, and then install the packages:

[root@laptoppie ~]# yum -y install glusterfs-fuse

create the (target) mount points:

[root@laptoppie ~]# mkdir -p /mnt/glusterclient/{replicated,balanced}

And mount the filesystems:

[root@laptoppie ~]# mount -t glusterfs node01:VolReplica /mnt/glusterclient/replicated
[root@laptoppie ~]# df -h /mnt/glusterclient/replicated
Filesystem         Size  Used Avail Use% Mounted on
node01:VolReplica   37G  924M   35G   3% /mnt/glusterclient/replicated
[root@laptoppie ~]# mount -t glusterfs node01:VolBalanced /mnt/glusterclient/balanced/
[root@laptoppie ~]# df -h /mnt/glusterclient/balanced/
Filesystem          Size  Used Avail Use% Mounted on
node01:VolBalanced  111G  2.8G  103G   3% /mnt/glusterclient/balanced

 

A long time I was enthusiastic about the fact that with Lotus Notes you were able to disable “reply-to-all” in a mail. After some search on Google, I noticed, that is also possible with MS Outlook. Scott Hanselman wrote a nice article on his weblog:

http://www.hanselman.com/blog/HowToEasilyDisableReplyToAllAndForwardInOutlook.aspx

To make it more user friendly, I modified the code a little bit:

Sub NoReplyAll()
‘——————————————————————–
‘Disable reply to all/forward options from a mail
‘This code is taken from
‘
‘ http://www.hanselman.com/blog/HowToEasilyDisableReplyToAllAndForwardInOutlook.aspx
‘
‘and some small modifications are applied by Pieter de Rijk
‘
‘ http://blog.adslweb.net
‘——————————————————————–

If ActiveInspector.CurrentItem.Actions(“Reply to All”).Enabled = True Then

‘Check the status if Reply-to-All is enabled, so we disable it
ActiveInspector.CurrentItem.Actions(“Reply to All”).Enabled = False
ActiveInspector.CurrentItem.Actions(“Forward”).Enabled = False

‘Give a prompt that it is disabled
MsgBox “Reply to all/Forward is disabled”, vbInformation, “Reply to all/Forward”
Else
‘The Reply-to-All is disabled, so enable it again
ActiveInspector.CurrentItem.Actions(“Reply to All”).Enabled = True
ActiveInspector.CurrentItem.Actions(“Forward”).Enabled = True
‘Prompt that we enabled it again
MsgBox “Reply to all/Forward is enabled”, vbExclamation, “Reply to all/Forward”
End If

End Sub

To offload my RaspBerry Pi with RaspBMC, I move away from the internal SQLite database to my MySQL server (as described on the XBMC Wiki).

After adding new sources and scraping the content… nothing showed up in the Movies overview… while the tables were populated…

In the xbmc.log file I found the following clue:

21:06:59 T:2928669760   ERROR: SQL: The table does not exist
                                      Query: select * from movieview
21:06:59 T:2928669760   ERROR: GetMoviesByWhere failed

 And after investigation, I noticed that there were no views defined in the schema.

To do a quick analysis on the sqllite database, I installed the SQLite Manager plugin for Firefox and found 4 views… and there I found the create statement for these views… so I simply copy-and-paste it into mysql… and it worked…

Unfortunately I was not able to insert the triggers that way… still have to find it out..

According to Bug ticket #13959 on the XBMC-Trac site, this was caused because I was a MySQL version < 5.16…. which was true… it was 5.0.something, so I upgraded my MySQL to version 5.1.67, and guess what… that did not work either…

So something goes “weird”, but looking into the source code didn’t gave me a clue so far… so up for further investigation…

---

The code to create the views:

CREATE VIEW episodeview AS SELECT   episode.*,  files.strFileName AS
strFileName,  path.strPath AS strPath,  files.playCount AS playCount, 
files.lastPlayed AS lastPlayed,  files.dateAdded AS dateAdded, 
tvshow.c00 AS strTitle,  tvshow.c14 AS strStudio,  tvshow.c05 AS
premiered,  tvshow.c13 AS mpaa,  tvshow.c16 AS strShowPath,  
bookmark.timeInSeconds AS resumeTimeInSeconds,  
bookmark.totalTimeInSeconds AS totalTimeInSeconds,   seasons.idSeason AS
idSeason FROM episode  JOIN files ON    files.idFile=episode.idFile 
JOIN tvshow ON    tvshow.idShow=episode.idShow  LEFT JOIN seasons ON   
seasons.idShow=episode.idShow AND seasons.season=episode.c12  JOIN path
ON    files.idPath=path.idPath  LEFT JOIN bookmark ON   
bookmark.idFile=episode.idFile AND bookmark.type=1;

CREATE VIEW
movieview AS SELECT  movie.*,  sets.strSet AS strSet,  files.strFileName
AS strFileName,  path.strPath AS strPath,  files.playCount AS
playCount,  files.lastPlayed AS lastPlayed,   files.dateAdded AS
dateAdded,   bookmark.timeInSeconds AS resumeTimeInSeconds,  
bookmark.totalTimeInSeconds AS totalTimeInSeconds FROM movie  LEFT JOIN
sets ON    sets.idSet = movie.idSet  JOIN files ON   
files.idFile=movie.idFile  JOIN path ON    path.idPath=files.idPath 
LEFT JOIN bookmark ON    bookmark.idFile=movie.idFile AND
bookmark.type=1;

CREATE VIEW movieview AS SELECT  movie.*, 
sets.strSet AS strSet,  files.strFileName AS strFileName,  path.strPath
AS strPath,  files.playCount AS playCount,  files.lastPlayed AS
lastPlayed,   files.dateAdded AS dateAdded,   bookmark.timeInSeconds AS
resumeTimeInSeconds,   bookmark.totalTimeInSeconds AS totalTimeInSeconds
FROM movie  LEFT JOIN sets ON    sets.idSet = movie.idSet  JOIN files
ON    files.idFile=movie.idFile  JOIN path ON   
path.idPath=files.idPath  LEFT JOIN bookmark ON   
bookmark.idFile=movie.idFile AND bookmark.type=1;

CREATE VIEW
tvshowview AS SELECT   tvshow.*,  path.strPath AS strPath, 
path.dateAdded AS dateAdded,  MAX(files.lastPlayed) AS lastPlayed, 
NULLIF(COUNT(episode.c12), 0) AS totalCount,  COUNT(files.playCount) AS
watchedcount,  NULLIF(COUNT(DISTINCT(episode.c12)), 0) AS totalSeasons
FROM tvshow  LEFT JOIN tvshowlinkpath ON   
tvshowlinkpath.idShow=tvshow.idShow  LEFT JOIN path ON   
path.idPath=tvshowlinkpath.idPath  LEFT JOIN episode ON   
episode.idShow=tvshow.idShow  LEFT JOIN files ON   
files.idFile=episode.idFile GROUP BY tvshow.idShow;

As mentioned before, I am playing around with Splunk’s authentication scripts with Centrify DC.
While playing I ran into a performance issue with the AuthenticationScripts implementation of getUsers.
According to the documentation the output is…

Recently I have been playing with Splunk’s Authenticationscript to retrieve information via Centrify DC, alhtough I noticed that an item for a user which is available in the webinterface for new users, is not available for users “created” via an authentication script.

If you read the specification the output of getUserInfo is:

--status=success|fail --userInfo=<userId>;<username>;<realname>;<roles> 

So for example:

--status=success|fail --userInfo=;pieter;Pieter de Rijk;admin

While in the webinterface you can also enter an e-mail address, so I raised a request with Splunk to get the e-mail address also populated via the authenticationscript-feature and it is picked up as an enhancement request.

I proposed the following output, so the authenticationscript stays backwards compatible:

--status=success|fail --userInfo=<userId>;<username>;<realname>:<mail-addres>;<roles>

So some output like (no comma, because that is used within a lot of companies with the DisplayName attribute like: “Rijk, Pieter de”):

--status=success|fail --userInfo=;pieter;Pieter de Rijk:[email protected];admin

I currently have a GlusterFS setup where the bricks are replicated, although brick 8 was having bad-sector issues, so should get replaced asap. After some searching I did not find how to do it… but is it pretty simple…

 gluster volume replace-brick GlusterVolume node1:/usr/glusterfs/brick_1_8 node1:/usr/glusterfs/brick_1_13 start

Depending on the speed of the disks (2TB USB disks in my case) it takes some while, you can monitor the status via:

gluster volume replace-brick GlusterVolume node1:/usr/glusterfs/brick_1_8 node1:/usr/glusterfs/brick_1_13 status

Once the replacing is “done” you simply do a commit to finalize the replacing:

gluster volume replace-brick GlusterVolume node1:/usr/glusterfs/brick_1_8 node1:/usr/glusterfs/brick_1_13 commit

That’s all

Recently I had to upgrade MySQL and PHP to enable the latest and greatest version of WordPress on a server.

The server is currently still running CentOS 4, but didn’t want to upgrade it to a newer major version. Thanks to the builds of Jason Litka for RHEL4/CentOS 4, I was able to upgrade the packages…

Finally I was able to upgrade the following packages:

• MySQL from version 4.1.22 to 5.1.58
• PHP from version 4.3.9 to 5.2.17
• Apache from version 2.0.52 to 2.2.21

You can easily enable the repo by adding a .repo file to /etc/yum.repos.d/ on your box with the following content:

[utterramblings]
name=Jason's Utter Ramblings Repo
baseurl=http://www.jasonlitka.com/media/EL$releasever/$basearch/
enabled=1
gpgcheck=1
gpgkey=http://www.jasonlitka.com/media/RPM-GPG-KEY-jlitka

Please note that some modules are changed if you upgrade Apache from 2.0 to 2.2, for example mod_access functionality moved into mod_authz_host.

Another “weird” thing was that the dovecot package had a dependency on /usr/lib/mysql/libmysqlclient.so.14,  this can be solved by installing the mysqlclient14.i386 rpm.

Sometimes I’m really surprised about myself… especially how lazy I am.

I’m currently playing around with one of my private websites, and to improve developing I decided to use subversion. So far so good, but I wanted that the committed subversion code was automatically online on the webserver. So I did the following very simple trick.

First I check out the code (subtree) from the subversion server (which is using https):

$ cd /sites
$ mv dev.adslweb.net{,-backup}
$ svn co https://svn.adslweb.net/svn/websites/dev.adslweb.net

Next step was to commit the current content of the website into subversion:

$ cd /sites/dev.adslweb.net
$ cp -Rv /sites/dev.adslweb.net-backup/*  ./
$ svn add *
$ svn commit -m “Initial commit of ADSLWEB.net dev env”

Now download the simple script I created for making sure that subversion doesn’t fire off twice for updating the same tree.

Download svn-update.sh via this link.

So something like this:

$ mkdir ~/scripts/
$ cd ~/scripts
$ wget http://www.xs4all.nl/~paderijk/pics/svn-update.sh
$ chmod 700 svn-update.sh

Now… the last step… create a crontab entry with the following content:

*/1 * * * * /home/pieter/scripts/svn-update.sh /sites/dev.adslweb.net 2>&1 > /dev/null

 And guess… and it works like a charm, on every new commit done by whoever… you get your online site updated within 1 minute without the need log in into the website/webserver using ftp/ssh/whatever.

In the past I started syncing every night the Updates-repositories from Fedora and CentOS on a local server, just to speed up updates/kick starts et cetera… The first version of the script was very quick and dirty, now I’ve a more decent script that allow you to add/remove very quick new versions of CentOS and Fedora.

 You can find the script here.