Percona Live Online

Over the past twelve years, we've seen a "third revolution" in database systems. The one size fits all RDBMS has given way to an explosion of diverse data management technologies. In this talk, we'll discover these new database technologies, consider their utility in leveraging existing data assets and speculate on how these will evolve to meet tomorrow's data needs.

The relational model dominated for a generation of computer professionals and represents a triumph of software architecture. However, today we are clearly in the midst of the third database revolution as the demands of an increasingly information-centric economy and global always-on applications have led to the emergence of new database architectures. In this third wave, multiple and diverse database technologies co-operate to accomplish the disparate business challenges provided by the migration to cloud computing, web-scale applications with social and mobile contexts, the promise of big data analytics, and the emerging challenge presented by the Internet of Things.

In this presentation, we’ll look at the key technologies and imperatives driving the third wave of databases and dive into specific facets of the revolution such as Big Data technologies, NoSQL, NewSQL, and graph technologies. In particular, we'll review important new database systems such as MongoDB, Cassandra, CockroachDB, SnowflakeDB and Neo4J. Finally, we’ll contemplate what might be in store for database management systems of the future.

Some NoSQL databases popularized the notion of “loose schema”, often misunderstood as “schemaless” - but there is always a data model, in the DB, the application or the mind of the developer. However, NoSQL schemas are designed with very different goals in mind than SQL schemas: where SQL normalizes, NoSQL denormalizes; Where SQL joins ad-hoc, NoSQL pre-joins; where SQL tries to push performance to the runtime, NoSQL bakes performance into the schema. Adding to the confusion, various NoSQL databases have different ideas on what schemas should enforce. This talk aims to introduce the core concepts of NoSQL schema design, using Scylla as an example explaining tradeoffs and rationale.

If we look at the data trend and how things are changing as far as the data generation, processing and consumption are concerned, we see that there is a convergence of different problem spaces happening at the core. For example, to do even a simple job of monitoring an ongoing operation, we need various data to be structured, ingested, integrated and processed in real-time (or quasi, streaming) manner. Further training of models or prediction on streaming data is required for it to be predictive in nature, both at the local (edge or within the device) or at cloud level. The speed and scale at which this takes place, it becomes almost infeasible to use siloed or “stitched together” kind of a platform, which simply doesn’t seem to scale anymore.

As a philosophical shift, we must converge all participating dimensions from solution space as well in order to counter this fusion of different problems or challenges that we face at the moment, which will grow only bigger and become tougher to handle. We must break the silos and create a converged architectural space which then should linearly grow in order to tackle the velocity, variety, and volume of data.

This fusion of different dimensions from the solution space would provide ways to natively integrate and support different flavors of data without having to upfront structure the data. The convergence of streaming and AI will allow continuous processing of data in both absolute and predictive manner. The stream processing will ensure continuous aggregation, running statistics, complex event processing, predictions and relevant actions in real-time basis.

The native integration at the buffer pool or IO layer will give the user full control of every single byte being ingested and processed by the system, which will reduce the latency to allow high-speed precision processing. Further siloed (semi siloed) architecture forces too many network hops along with too many copies of data. In this scenario, even with a very high processing efficiency, low latency (or high speed) is not possible with this architecture. We need to minimize network hops and copy of data as much as possible. With convergence, we minimize both the network hops and data copy, thereby improving the performance.

This converge first approach would also allow true linear scaling of the system. With siloed architecture we find it always extremely hard to scale different verticals together. Further complete utilization of resources is also not possible. But with convergence, we need to bother about scaling single dimension and high resource utilization is definitely the by-product.

Therefore a NoSQL database which converges different entities such as ML and streaming and which works within a device connected with the local or cloud instances of itself could possibly offer some relief by reducing the pain of operation and maintenance.

BangDB is a converged NoSQL Database, designed to handle the emerging use cases with ease at scale.

In the last 20 years our data systems have been growing both in terms of data and throughput. However, common database design is based on architectures dating 30 years ago and did not keep up with the changes in modern hardware. The open source Seastar framework has been used by Scylla and other projects to squeeze every last bit of performance from modern hardware unlocking unprecedented vertical scalability. This talk showcases the unique architecture used and its implications for modern database design.