Feature Set Mismatch Error on Ceph Kernel Client

laurentbarbe

Depending on the kernel version you may be missing some features required by the cluster (or vice versa, required by client but not available on the cluster). Hence the errors “feature set mismatch”.

Some examples of errors that can be encountered :

mon0 192.168.0.1:6789 feature set mismatch, my XXXXXX < server's XXXXXX, missing 2040000

—> Upgrade kernel client up to 3.9 or set tunables to legacy : ceph osd crush tunables legacy

mon0 192.168.0.1:6789 feature set mismatch, my XXXXXX < server's XXXXXX, missing 40000000

—> Upgrade kernel client up to 3.9 or unset hashpspool : ceph osd pool set rbd hashpspool false

mon0 192.168.0.1:6789 feature set mismatch, my XXXXXX < server's XXXXXX, missing 800000000

—> Remove cache pool and reload monitors or upgrade kernel client up to 3.14

mon0 192.168.0.1:6789 feature set mismatch, my XXXXXX < server's XXXXXX, missing 1000000000

—> Upgrade kernel client up to 3.14

mon0 192.168.0.1:6789 feature set mismatch, my 4a042a42 < server's 2004a042a42, missing 20000000000

—> Upgrade kernel client up to 3.15 or disable tunable 3 features

CEPH_FEATURE Table and Kernel Version

You can find the feature missing in that table :

For exemple, missing 2040000 means that CEPH_FEATURE_CRUSH_TUNABLES (40000) and CEPH_FEATURE_CRUSH_TUNABLES2 (2000000) is missing on kernel client.

‘R’:required, ’S’:support, ‘-X-’ feature is new since this version

FeatureBITOCT3.83.93.103.143.153.18
CEPH_FEATURE_UID01
CEPH_FEATURE_NOSRCADDR12RRRRRR
CEPH_FEATURE_MONCLOCKCHECK24
CEPH_FEATURE_FLOCK38
CEPH_FEATURE_SUBSCRIBE2410
CEPH_FEATURE_MONNAMES520
CEPH_FEATURE_RECONNECT_SEQ640-R-RRR
CEPH_FEATURE_DIRLAYOUTHASH780
CEPH_FEATURE_OBJECTLOCATOR8100
CEPH_FEATURE_PGID649200RRRRR
CEPH_FEATURE_INCSUBOSDMAP10400
CEPH_FEATURE_PGPOOL311800RRRRR
CEPH_FEATURE_OSDREPLYMUX121000
CEPH_FEATURE_OSDENC132000RRRRR
CEPH_FEATURE_OMAP144000
CEPH_FEATURE_MONENC158000
CEPH_FEATURE_QUERY_T1610000
CEPH_FEATURE_INDEP_PG_MAP1720000
CEPH_FEATURE_CRUSH_TUNABLES1840000SSSSSS
CEPH_FEATURE_CHUNKY_SCRUB1980000
CEPH_FEATURE_MON_NULLROUTE20100000
CEPH_FEATURE_MON_GV21200000
CEPH_FEATURE_BACKFILL_RESERVATION22400000
CEPH_FEATURE_MSG_AUTH23800000-S-
CEPH_FEATURE_RECOVERY_RESERVATION241000000
CEPH_FEATURE_CRUSH_TUNABLES2252000000SSSSS
CEPH_FEATURE_CREATEPOOLID264000000
CEPH_FEATURE_REPLY_CREATE_INODE278000000SSSSS
CEPH_FEATURE_OSD_HBMSGS2810000000
CEPH_FEATURE_MDSENC2920000000
CEPH_FEATURE_OSDHASHPSPOOL3040000000SSSSS
CEPH_FEATURE_MON_SINGLE_PAXOS3180000000
CEPH_FEATURE_OSD_SNAPMAPPER32100000000
CEPH_FEATURE_MON_SCRUB33200000000
CEPH_FEATURE_OSD_PACKED_RECOVERY34400000000
CEPH_FEATURE_OSD_CACHEPOOL35800000000-S-SS
CEPH_FEATURE_CRUSH_V2361000000000-S-SS
CEPH_FEATURE_EXPORT_PEER372000000000-S-SS
CEPH_FEATURE_OSD_ERASURE_CODES384000000000
CEPH_FEATURE_OSD_TMAP2OMAP38*4000000000
CEPH_FEATURE_OSDMAP_ENC398000000000-S-S
CEPH_FEATURE_MDS_INLINE_DATA4010000000000
CEPH_FEATURE_CRUSH_TUNABLES34120000000000-S-S
CEPH_FEATURE_OSD_PRIMARY_AFFINITY41*20000000000-S-S

(Note that CEPH_FEATURE_OSD_ERASURE_CODES is no more required for client since firefly)