Test Ceph Persistant Rbd Device

laurentbarbe

Create persistant rbd device

Create block device and map it with /etc/ceph/rbdmap

1
2
3
4
5

$ rbd create rbd/myrbd --size=1024
$ echo "rbd/myrbd" >> /etc/ceph/rbdmap
$ service rbdmap reload
[ ok ] Starting RBD Mapping: rbd/myrbd.
[ ok ] Mounting all filesystems...done.

View rbd mapped :

1
2
3

$ rbd showmapped
id pool image snap device    
1  rbd  myrbd - /dev/rbd1

Create FS and mount :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

$ mkfs.xfs /dev/rbd/rbd/myrbd 
log stripe unit (4194304 bytes) is too large (maximum is 256KiB)
log stripe unit adjusted to 32KiB
meta-data=/dev/rbd/rbd/myrbd     isize=256    agcount=9, agsize=31744 blks
         =                       sectsz=512   attr=2, projid32bit=0
data     =                       bsize=4096   blocks=262144, imaxpct=25
         =                       sunit=1024   swidth=1024 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=8 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

$ mkdir -p /mnt/myrbd
$ blkid | grep rbd1
/dev/rbd1: UUID="a07e969e-bb1a-4921-9171-82cf7a737a69" TYPE="xfs"
$ echo "UUID=a07e969e-bb1a-4921-9171-82cf7a737a69 /mnt/myrbd xfs defaults 0 0" >> /etc/fstab
$ mount -a

Check :

1
2

$ mount | grep rbd1
/dev/rbd1 on /mnt/myrbd type xfs (rw,relatime,attr2,inode64,sunit=8192,swidth=8192,noquota)

Test snapshot 

1

$ touch /mnt/myrbd/v1

Make snapshot :

1
2
3

$ sync && xfs_freeze -f /mnt/
$ rbd snap create rbd/myrbd@snap1
$ xfs_freeze -u /mnt/

Change a file :

1

$ mv /mnt/myrbd/v1 /mnt/myrbd/v2

Mount snapshot in RO :

1
2
3

$ mkdir -p /mnt/myrbd@snap1
$ rbd map rbd/myrbd@snap1
$ mount -t xfs -o ro,norecovery,nouuid "/dev/rbd/rbd/myrbd@snap1" "/mnt/myrbd@snap1"
1
2
3

$ ls "/mnt/myrbd"
total 0
v2

OK.

1
2

$ ls "/mnt/myrbd@snap1"
total 0

Nothing ??? Something went wrong with the sync ?

Try again :

1
2
3
4
5
6

$ sync && xfs_freeze -f /mnt/
$ rbd snap create rbd/myrbd@snap2
$ xfs_freeze -u /mnt/
$ mkdir -p /mnt/myrbd@snap2
$ rbd map rbd/myrbd@snap2
$ mount -t xfs -o ro,norecovery,nouuid "/dev/rbd/rbd/myrbd@snap2" "/mnt/myrbd@snap2"

Move again the file.

1

$ mv /mnt/myrbd/v2 /mnt/myrbd/v3
1
2
3
4
5
6

$ ls /mnt/myrbd@snap2
total 0
v2
$ ls /mnt/myrbd
total 0
v3

All right.

Stop rbdmap (will remove all rbd mapped device)

1

$ service rbdmap remove

Remove line added in /etc/ceph/rbdmap

Remove myrbd :

1
2
3
4

$ rbd snap purge rbd/myrbd
Removing all snapshots: 100% complete...done.
$ rbd rm rbd/myrbd
Removing image: 100% complete...done.