Loading…
Welcome to Percona Live Online 2021
Online Open Source Database Conference
REGISTER HERE!
Back To Schedule
Thursday, May 13 • 11:00 - 12:00
Scaling Out Distributed Storage Fabric with RocksDB

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.


Engineers at Nutanix have been working on the challenge of building a next-generation architecture for its distributed storage fabric. Scaling this architecture to the needs of the future required three primary objectives: significant improvements in sustained random write performance, support for large-capacity deep storage nodes for multi-petabyte scale and reducing storage latency by a significant magnitude.
 
These goals required re-imagining the core approach to how metadata is stored in the fabric management system and move the metadata closer to where is the data is stored.
 
After extensive research and testing, RocksDB was chosen as the core component for this project, based on its open-source pedigree and proven reliability and industry adoption. Within a few months, the engineering team was able to ramp up expertise, build confidence with the open-source technology and eventually grow its adoption into several core products at Nutanix.
 
In this technical talk, we will share the new architecture, deployment mode and some of the early lessons learned in adopting RocksDB and discuss some innovative enhancements we were able to make to fit our performance goals and objectives.
 
One of the significant improvements has been the addition of async read/write support to RocksDB. Currently, the open source RocksDB exposes blocking I/O APIs which can limit overall system throughput under resource constraints. We developed a Fibers/Co-routine based non-blocking I/O solution for RocksDB.
 
In addition to this, we plan to talk about topics and projects that have been built on this enhanced RocksDB implementation.
These projects will become the foundation for the Nutanix future products.

Speakers
YK

Yasaswi Kishore

Senior member of Technical Staff, Nutanix
Yasaswi is a senior member of technical staff in the metadata subsystem for Nutanix distributed filesystem. Prior to Nutanix, Yasaswi completed his undergraduate program in Computer Science at PES University, Bangalore, India.
avatar for Sandeep Madanala

Sandeep Madanala

Nutanix
Sandeep is a Senior technical manager in the metadata subsystem for Nutanix distributed filesystem. He leads and manages the ChakrDB team, a scale out KV Store built on top of RocksDB. Prior to Nutanix, Sandeep worked at VMWare and graduated from Indian Institute of Technology, M... Read More →
avatar for Raghav Tulshibagwale

Raghav Tulshibagwale

Staff Engineer, Core Data Path, Nutanix Inc.
Raghav is a Staff engineer and technical lead in the metadata subsystem for Nutanix distributed filesystem. Prior to Nutanix, Raghav worked on Database Kernels and filesystems. Raghav completed his Masters in Computer Science from University of Southern California, Los Angeles.
avatar for Pulkit Kapoor

Pulkit Kapoor

MTS, Core Data Path, Nutanix, Inc.
Pulkit is a member of technical staff in the metadata subsystem for Nutanix distributed filesystem. Prior to Nutanix, Pulkit completed his Masters in Computer Science at University of Wisconsin, Madison.


Thursday May 13, 2021 11:00 - 12:00 EDT
Room #4