Ceph Days India 2024

D4N is an intelligent caching framework within Ceph RGW. This project is a collaboration between MOC team at Boston University and Ceph RGW team. The ongoing effort to upstream D4N into Ceph is mainly going to target accelerating analytics workload.

IBM

"The aim of this talk is to give real experience while I recently used the technology to build highly available ceph cluster, specially when it comes to use dashboard , block or object from anywhere.

Using ceph-ansible and haproxy with some automation build to prepare and to give real exposure to deployed this in any industry."

Clear-Trail

In this talk, we would investigate the scalability of Ceph RGW by uploading a billion objects to a single bucket and synchronizing them across a multisite setup. The test involves setting up the environment, implementing dynamic resharding of the bucket, and monitoring system behaviour during the resharding and IO operations.

The talk would provide valuable considerations for managing large-scale data in Ceph RGW clusters. Moreover, it discusses the challenges and limitations encountered in optimizing the performance of dynamic resharding, offering essential insights for addressing such issues for feature enhancements in the future.

IBM

This presentation outlines the configuration of a virtual hosting style for Ceph Rados Gateway (RGW) using Ingress and Rook for AL/ML applications like Milvus. The s3 buckets can be accessed either path based or host style. Path is deprecated by S3 protocol and lot of application defaults with vhost style. Ingress provides the wildcard support for endpoint and Rook can deploy RGW with vhost style access.

IBM

Efficiently migrating petabytes of object storage data between two production Ceph clusters posed a significant challenge with live data being written to both clusters, necessitating a seamless process to minimize disruptions. The migration strategy involved extracting user accounts, including access and secret keys, from the old cluster and seamlessly transferring them to the new one. The synchronization of buckets and live data has been improved by extending and enhancing powerful tools such as rclone, executed in parallel. This migration endeavor not only resulted in the successful transfer of vast amounts of data but also paved the way for the creation of a robust tool named Chorus. Chorus, specifically designed for synchronizing S3 data, emerged as a versatile solution capable of harmonizing data seamlessly across multiple cloud storage backends. This innovative tool is helpful in effective bridging of data between Ceph clusters, demonstrating the adaptability and scalability required for modern data management challenges. Key highlights of Chorus include persistence of migration, execution of migration on multiple machines, rate limiting RAM/network usage during migration.

Clyso GmBH

Rook.io

This presentation sheds light on an essential tuning enhancement in Ceph, specifically the RocksDB compression, now enabled by default in versions Reef and above. Through this presentation, our aim is to familiarize a broader audience with rocksDB compression’s significance, advantages and usage in Ceph, particularly in non-collocated deployments where NVMe drives store WAL/DB data. By exercising this optimization, we achieve nearly identical performance while considerably reducing space utilization for storing metadata. In addition to explaining rocksDB compression, we delve into the testing workflow employed during the performance evaluation and highlight the enhancements made to MinIO Warp to cater to our requirements. Furthermore, we emphasize the newfound flexibility in S3 benchmarking facilitated by Warp, providing a comprehensive overview of its implications and benefits.

IBM

In an Octopus release, the Cephadm utility was introduced to manage a single Ceph cluster which does not rely on external tools like rook & ansible. However, due to its complex nature it is difficult to troubleshoot cephadm issues for operations such as OSD redeployment or deployment of any new service. Understanding Cephadm's internal working concepts is necessary for troubleshooting issues, as it will help us identify which component needs to be troubleshooted. Along with the internals working concepts, an approach/guidance for troubleshooting common issues related to Cephadm makes it easier & quicker to find resolutions.

IBM

We want to present on how we use ceph s3-select to query on the log data generated for every build of every release

IBM

In the dynamic world of storage optimization, fine-tuning Ceph RGW (RADOS Gateway) configurations can significantly impact performance and efficiency. However, navigating the multitude of tuning options can be daunting. This lightning talk delves into various RGW tuning strategies, offering insights into both the benefits and potential pitfalls. From adjusting thread pools to optimizing objector and frontend parameters, we explore practical use cases, scenarios, and considerations for standalone and multisite clusters. Drawing from real-world experiences and community insights, we highlight the importance of testing, caution against over-tuning, and emphasize the need for informed decision-making. Attendees will gain actionable insights into RGW tuning, enabling them to optimize their Ceph deployments effectively.

IBM

Panel discussion lead by our Experts around -
* Getting users feedback about Ceph
* Sharing about Ceph User Council Iinitiative
* Improvements going in Community Ceph releases
* Ceph release updates