Ceph: validate that the RBD cache is active
Quick and simple test to validate if the RBD cache is enabled on your client.
Simple thing first, if you are running a Ceph version newer or equal than 0.87 the cache is enabled by default. If not then you can simply enable the cache via the
[client] rbd cache = true rbd cache writethrough until flush = true
Then you need to activate two flags in your
ceph.conf within the
[client] admin socket = /var/run/ceph/$cluster-$type.$id.$pid.$cctid.asok log file = /var/log/ceph/
Both paths must be writable by the user running using the RBD library and the security context (SELinux or AppArmor) must be configured properly.
Once this is done, run your application that is supposed to use
librbd (a virtual machine or something else) and simply request the admin daemon from a socket:
$ sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.66606.140190886662256.asok config show | grep rbd_cache "rbd_cache": "true", "rbd_cache_writethrough_until_flush": "true", "rbd_cache_size": "33554432", "rbd_cache_max_dirty": "25165824", "rbd_cache_target_dirty": "16777216", "rbd_cache_max_dirty_age": "1", "rbd_cache_max_dirty_object": "0", "rbd_cache_block_writes_upfront": "false",
Verify the cache behaviour
If you want to go further and test the performance enhancement brought by the cache you can do the following turn it off from the
[client] section in your
ceph.conf like this
rbd cache = false. Then a benchmark like so using the following command (assuming the RBD pool exists):
$ rbd -p rbd bench-write fio —io-size 4096 —io-threads 256 —io-total 1024000000 —io-pattern seq
Eventually run this test with and without the cache section should bring a significant difference :).