Ceph: how to test if your SSD is suitable as a journal device?

A simple benchmark job to determine if your SSD is suitable to act as a journal device for your OSDs.

To give you a little bit of background when the OSD writes into his journal it uses D_SYNC and O_DIRECT. Writing with O_DIRECT bypasses the Kernel page cache, while D_SYNC ensures that the command won't return until every single write is complete. So yes, basically the OSD forces all the writes to be flushed prior to start the next IO.

First disable the write cache on the disk:

bash $ sudo hdparm -W 0 /dev/hda 0

Now you can start benchmarking your SSD correctly using two different methods. The FIO way:

bash $ sudo fio --filename=/dev/sda --direct=1 --sync=1 --rw=write --bs=4k --numjobs=2 --group_reporting --invalidate=0 --name=journal-test

The dd way:

bash $ sudo dd if=randfile of=/dev/sda bs=4k count=100000 oflag=direct,dsync

What matters the most here is to find how the SSD is performing while using D_SYNC. At some point users reported some SSD misbehaving with DSYNC. Then you better always test your SSD prior to go in production.

Data agregation table ¶

Gathering all the comments in a table: