GlusterFS 10 minute guide on CentOS/RHEL/Fedora

At home I set up a small 3 node GlusterFS Cluster based on CentOS 6.4 with one client with Fedora 16.

On the client and the nodes iptables and SELinux is disabled.

The GlusterFS package is installed using the EPEL repository.

Preparing the nodes

The nodes have been kick-started using the following %packages in the kickstart:


After enabling the EPEL repository you run the following commands on all nodes:

yum -y install glusterfs glusterfs-server
mkdir -p /mnt/glusterfs/test/{balanced,replicated}
service glusterd start
chkconfig glusterd on

Now on one of the nodes we bind all the systems to the "cluster":

[root@node01 ~]# gluster peer status
No peers present
[root@node01 ~]# gluster peer probe node01
Probe on localhost not needed
[root@node01 ~]# gluster peer probe node02
Probe successful
[root@node01 ~]# gluster peer probe node03
Probe successful
[root@node01 ~]# gluster peer status
Number of Peers: 2

Hostname: node02
Uuid: 6d5d5101-5a8e-4057-a854-2aac49ce22ca
State: Peer in Cluster (Connected)

Hostname: node03
Uuid: f651c972-7519-4b74-b610-f1f2f4db31ef
State: Peer in Cluster (Connected)

Create the volumes

We create two volumes, one replicated and one distributed.

Replicated volume

root@node01 ~]# gluster volume create VolReplica replica 3 transport tcp
node02:/mnt/glusterfs/test/replicated node03:/mnt/glusterfs/test/replicated

Creation of volume VolReplica has been successful. Please start the volume to access data.
[root@node01 ~]# gluster volume start VolReplica
Starting volume VolReplica has been successful
[root@node01 ~]# gluster volume info VolReplica

Volume Name: VolReplica
Type: Replicate
Status: Started
Number of Bricks: 3
Transport-type: tcp
Brick1: node01:/mnt/glusterfs/test/replicated
Brick2: node02:/mnt/glusterfs/test/replicated
Brick3: node03:/mnt/glusterfs/test/replicated

Balanced (distributed) volume

[root@node01 ~]# gluster volume create VolBalanced  transport tcp
node02:/mnt/glusterfs/test/balanced node03:/mnt/glusterfs/test/balanced

[root@node01 ~]# gluster volume start VolBalanced
[root@node01 ~]# gluster volume info VolBalanced

Volume Name: VolBalanced
Type: Distribute
Status: Started
Number of Bricks: 3
Transport-type: tcp
Brick1: node01:/mnt/glusterfs/test/balanced
Brick2: node02:/mnt/glusterfs/test/balanced
Brick3: node03:/mnt/glusterfs/test/balanced

Configure the client(s)

You can easily use one of the nodes also as client, but I nominated my (old) laptop with FC 16 as GlusterFS client.

First ensure the EPEL repositories are enabled, and then install the packages:

[root@laptoppie ~]# yum -y install glusterfs-fuse

create the (target) mount points:

[root@laptoppie ~]# mkdir -p /mnt/glusterclient/{replicated,balanced}

And mount the filesystems:

[root@laptoppie ~]# mount -t glusterfs node01:VolReplica /mnt/glusterclient/replicated
[root@laptoppie ~]# df -h /mnt/glusterclient/replicated
Filesystem         Size  Used Avail Use% Mounted on
node01:VolReplica   37G  924M   35G   3% /mnt/glusterclient/replicated
[root@laptoppie ~]# mount -t glusterfs node01:VolBalanced /mnt/glusterclient/balanced/
[root@laptoppie ~]# df -h /mnt/glusterclient/balanced/
Filesystem          Size  Used Avail Use% Mounted on
node01:VolBalanced  111G  2.8G  103G   3% /mnt/glusterclient/balanced