Percona Live Online

If we look at the data trend and how things are changing as far as the data generation, processing and consumption are concerned, we see that there is a convergence of different problem spaces happening at the core. For example, to do even a simple job of monitoring an ongoing operation, we need various data to be structured, ingested, integrated and processed in real-time (or quasi, streaming) manner. Further training of models or prediction on streaming data is required for it to be predictive in nature, both at the local (edge or within the device) or at cloud level. The speed and scale at which this takes place, it becomes almost infeasible to use siloed or “stitched together” kind of a platform, which simply doesn’t seem to scale anymore.

As a philosophical shift, we must converge all participating dimensions from solution space as well in order to counter this fusion of different problems or challenges that we face at the moment, which will grow only bigger and become tougher to handle. We must break the silos and create a converged architectural space which then should linearly grow in order to tackle the velocity, variety, and volume of data.

This fusion of different dimensions from the solution space would provide ways to natively integrate and support different flavors of data without having to upfront structure the data. The convergence of streaming and AI will allow continuous processing of data in both absolute and predictive manner. The stream processing will ensure continuous aggregation, running statistics, complex event processing, predictions and relevant actions in real-time basis.

The native integration at the buffer pool or IO layer will give the user full control of every single byte being ingested and processed by the system, which will reduce the latency to allow high-speed precision processing. Further siloed (semi siloed) architecture forces too many network hops along with too many copies of data. In this scenario, even with a very high processing efficiency, low latency (or high speed) is not possible with this architecture. We need to minimize network hops and copy of data as much as possible. With convergence, we minimize both the network hops and data copy, thereby improving the performance.

This converge first approach would also allow true linear scaling of the system. With siloed architecture we find it always extremely hard to scale different verticals together. Further complete utilization of resources is also not possible. But with convergence, we need to bother about scaling single dimension and high resource utilization is definitely the by-product.

Therefore a NoSQL database which converges different entities such as ML and streaming and which works within a device connected with the local or cloud instances of itself could possibly offer some relief by reducing the pain of operation and maintenance.

BangDB is a converged NoSQL Database, designed to handle the emerging use cases with ease at scale.