Tuesday, May 17, 2011

GlusterFS: replace-brick

GlusterFS has a volume command called replace-brick, intuitively it replaces one brick with another. However, the way it works is not quite intuitive but requires some understanding before actually trying it out.



Let us say the cluster looks like this:

# gluster volume info

Volume Name: rb-test
Type: Distribute
Status: Started
Number of Bricks: 4
Transport-type: tcp
Bricks:
Brick1: 192.168.1.95:/data/distributestore/store-1
Brick2: 192.168.1.95:/data/distributestore/store-2
Brick3: 192.168.1.96:/data/distributestore/store-1
Brick4: 192.168.1.96:/data/distributestore/store-2

To replace the brick 192.168.1.95:/data/distributestore/store-2 with 192.168.1.77:/data/distributestore/store-2 we run the following command.

# gluster volume replace-brick rb-test \ 192.168.1.95:/data/distributestore/store-2 \ 192.168.1.77:/data/distributestore/store-2 start

This command basically migrates the data from
192.168.1.95:/data/distributestore/store-2 to 192.168.1.77:/data/distributestore/store-2

But actual brick replacement is not done on the volume yet. After the completion of the above command the data is present both on 192.168.1.95:/data/distributestore/store-2 and
192.168.1.77:/data/distributestore/store-2.

To include the brick 192.168.1.77:/data/distributestore/store-2 to the cluster the command,

# gluster volume replace-brick rb-test \ 192.168.1.95:/data/distributestore/store-2 \
192.168.1.77:/data/distributestore/store-2 commit

has to be run. This attaches the brick to the cluster. After the above command volume info looks like this:

[root@centos5 store-2]# gluster volume info rb-test

Volume Name: rb-test
Type: Distribute
Status: Started
Number of Bricks: 4
Transport-type: tcp
Bricks:
Brick1: 192.168.1.95:/data/distributestore/store-1
Brick2: 192.168.1.77:/data/distributestore/store-2
Brick3: 192.168.1.96:/data/distributestore/store-1
Brick4: 192.168.1.96:/data/distributestore/store-2

If you need to just create a backup of a brick just run `gluster volume replace-brick <VOLNAME> <BRICK> <NEW-BRICK> start' ;-).

The volume command `replace-brick' has other sub-commands viz pause, abort, status which are pretty much intuitive. See http://goo.gl/F3Lfw for more details on them.