Ceph Days India 2024

  IBM Ceph

  IBM - ISL CTO

IBM

In this talk, we would investigate the scalability of Ceph RGW by uploading a billion objects to a single bucket and synchronizing them across a multisite setup. The test involves setting up the environment, implementing dynamic resharding of the bucket, and monitoring system behaviour during the resharding and IO operations.

The talk would provide valuable considerations for managing large-scale data in Ceph RGW clusters. Moreover, it discusses the challenges and limitations encountered in optimizing the performance of dynamic resharding, offering essential insights for addressing such issues for feature enhancements in the future.

IBM

We want to present on how we use ceph s3-select to query on the log data generated for every build of every release

IBM

D4N is an intelligent caching framework within Ceph RGW. This project is a collaboration between MOC team at Boston University and Ceph RGW team. The ongoing effort to upstream D4N into Ceph is mainly going to target accelerating analytics workload.

IBM

NFS-Ganesha operates as a user-space NFS server and provides support for CephFS FSAL via libcephfs. This presentation suggests a comprehensive examination of the integration between NFS Ganesha and Ceph, a distributed storage system acclaimed for its scalability and reliability. We will commence by delving into the foundational architectures of NFS Ganesha and Ceph, elucidating how they synergize to furnish a resilient storage solution. Subsequently, we will delve into the technical nuances of merging NFS-Ganesha with CephFS, encompassing aspects such as setup, deployment, and operational considerations

IBM

  IBM

Panel discussion lead by our Experts around -
* Getting users feedback about Ceph
* Sharing about Ceph User Council Iinitiative
* Improvements going in Community Ceph releases
* Ceph release updates

Efficiently migrating petabytes of object storage data between two production Ceph clusters posed a significant challenge with live data being written to both clusters, necessitating a seamless process to minimize disruptions. The migration strategy involved extracting user accounts, including access and secret keys, from the old cluster and seamlessly transferring them to the new one. The synchronization of buckets and live data has been improved by extending and enhancing powerful tools such as rclone, executed in parallel. This migration endeavor not only resulted in the successful transfer of vast amounts of data but also paved the way for the creation of a robust tool named Chorus. Chorus, specifically designed for synchronizing S3 data, emerged as a versatile solution capable of harmonizing data seamlessly across multiple cloud storage backends. This innovative tool is helpful in effective bridging of data between Ceph clusters, demonstrating the adaptability and scalability required for modern data management challenges. Key highlights of Chorus include persistence of migration, execution of migration on multiple machines, rate limiting RAM/network usage during migration.

Clyso GmBH

This presentation sheds light on an essential tuning enhancement in Ceph, specifically the RocksDB compression, now enabled by default in versions Reef and above. Through this presentation, our aim is to familiarize a broader audience with rocksDB compression’s significance, advantages and usage in Ceph, particularly in non-collocated deployments where NVMe drives store WAL/DB data. By exercising this optimization, we achieve nearly identical performance while considerably reducing space utilization for storing metadata. In addition to explaining rocksDB compression, we delve into the testing workflow employed during the performance evaluation and highlight the enhancements made to MinIO Warp to cater to our requirements. Furthermore, we emphasize the newfound flexibility in S3 benchmarking facilitated by Warp, providing a comprehensive overview of its implications and benefits.

IBM

Rook.io

This presentation outlines the configuration of a virtual hosting style for Ceph Rados Gateway (RGW) using Ingress and Rook for AL/ML applications like Milvus. The s3 buckets can be accessed either path based or host style. Path is deprecated by S3 protocol and lot of application defaults with vhost style. Ingress provides the wildcard support for endpoint and Rook can deploy RGW with vhost style access.

IBM

In an Octopus release, the Cephadm utility was introduced to manage a single Ceph cluster which does not rely on external tools like rook & ansible. However, due to its complex nature it is difficult to troubleshoot cephadm issues for operations such as OSD redeployment or deployment of any new service. Understanding Cephadm's internal working concepts is necessary for troubleshooting issues, as it will help us identify which component needs to be troubleshooted. Along with the internals working concepts, an approach/guidance for troubleshooting common issues related to Cephadm makes it easier & quicker to find resolutions.

IBM