In-memory databases for IOT
The use of
memory in computing is not new. But while memory is faster than disk by an
order of magnitude, it is also an order of magnitude more expensive. That has
for the most part left memory relegated to acting as a caching layer, while
nearly all of the data is stored on disk. However in recent years, the cost of
memory has been falling, making it possible to put far larger datasets in
memory for data processing tasks, rather than use it simply as a cache.
It’s not
just that it is now possible to store larger datasets in memory for rapid
analytics; it is also that it is highly desirable. In the era of IoT, data
often streams into the data centre or the cloud – the likes of sensor data from
anything from a production line to an oilrig. The faster the organization is
able to spot anomalies in that data, the better the quality of predictive
maintenance. In-memory technologies are helping firms see those anomalies close
to, or in, real-time. Certainly much faster than storing data in a disk-based
database and having to move packets of data to a cache for analytics.
It is
expected of data processing to accelerate dramatically, as companies come to
grips with their data challenges and move beyond more traditional data
analytics in the era of IoT. In-memory databases are 10 to 100 times faster
than traditional databases, depending on the exact use case. When one considers
that some IoT use cases involve the collection, processing and analysis of
millions of events per second, you can see why in-memory becomes so much more
appealing.
There’s
another big advantage with in-memory databases. Traditionally, databases have
been geared toward one of two main uses: handling transactions, or enabling
rapid analysis of those transactions – analytics. The I/O limitations of
disk-based databases meant that those handling transactions would slow down
considerably when also being asked to return the results of data queries.
That’s why data was often exported from the transactional database into another
platform – a data warehouse – where it could more rapidly be analyzed without
impacting the performance of the system.
Renu Yadav
Assistant Professor (BCA Dept.)
Comments
Post a Comment