GlusterFS 10 minute guide on CentOS/RHEL/Fedora

At home I set up a small 3 node GlusterFS Cluster based on CentOS 6.4 with one client with Fedora 16.

On the client and the nodes iptables and SELinux is disabled.

The GlusterFS package is installed using the EPEL repository.

Preparing the nodes

The nodes have been kick-started using the following %packages in the kickstart:


After enabling the EPEL repository you run the following commands on all nodes:

yum -y install glusterfs glusterfs-server
mkdir -p /mnt/glusterfs/test/{balanced,replicated}
service glusterd start
chkconfig glusterd on

Now on one of the nodes we bind all the systems to the "cluster":

[[email protected] ~]# gluster peer status
No peers present
[[email protected] ~]# gluster peer probe node01
Probe on localhost not needed
[[email protected] ~]# gluster peer probe node02
Probe successful
[[email protected] ~]# gluster peer probe node03
Probe successful
[[email protected] ~]# gluster peer status
Number of Peers: 2

Hostname: node02
Uuid: 6d5d5101-5a8e-4057-a854-2aac49ce22ca
State: Peer in Cluster (Connected)

Hostname: node03
Uuid: f651c972-7519-4b74-b610-f1f2f4db31ef
State: Peer in Cluster (Connected)

Create the volumes

We create two volumes, one replicated and one distributed.

Replicated volume

[email protected] ~]# gluster volume create VolReplica replica 3 transport tcp
node02:/mnt/glusterfs/test/replicated node03:/mnt/glusterfs/test/replicated

Creation of volume VolReplica has been successful. Please start the volume to access data.
[[email protected] ~]# gluster volume start VolReplica
Starting volume VolReplica has been successful
[[email protected] ~]# gluster volume info VolReplica

Volume Name: VolReplica
Type: Replicate
Status: Started
Number of Bricks: 3
Transport-type: tcp
Brick1: node01:/mnt/glusterfs/test/replicated
Brick2: node02:/mnt/glusterfs/test/replicated
Brick3: node03:/mnt/glusterfs/test/replicated

Balanced (distributed) volume

[[email protected] ~]# gluster volume create VolBalanced  transport tcp
node02:/mnt/glusterfs/test/balanced node03:/mnt/glusterfs/test/balanced

[[email protected] ~]# gluster volume start VolBalanced
[[email protected] ~]# gluster volume info VolBalanced

Volume Name: VolBalanced
Type: Distribute
Status: Started
Number of Bricks: 3
Transport-type: tcp
Brick1: node01:/mnt/glusterfs/test/balanced
Brick2: node02:/mnt/glusterfs/test/balanced
Brick3: node03:/mnt/glusterfs/test/balanced

Configure the client(s)

You can easily use one of the nodes also as client, but I nominated my (old) laptop with FC 16 as GlusterFS client.

First ensure the EPEL repositories are enabled, and then install the packages:

[[email protected] ~]# yum -y install glusterfs-fuse

create the (target) mount points:

[[email protected] ~]# mkdir -p /mnt/glusterclient/{replicated,balanced}

And mount the filesystems:

[[email protected] ~]# mount -t glusterfs node01:VolReplica /mnt/glusterclient/replicated
[[email protected] ~]# df -h /mnt/glusterclient/replicated
Filesystem         Size  Used Avail Use% Mounted on
node01:VolReplica   37G  924M   35G   3% /mnt/glusterclient/replicated
[[email protected] ~]# mount -t glusterfs node01:VolBalanced /mnt/glusterclient/balanced/
[[email protected] ~]# df -h /mnt/glusterclient/balanced/
Filesystem          Size  Used Avail Use% Mounted on
node01:VolBalanced  111G  2.8G  103G   3% /mnt/glusterclient/balanced