Inside the new ScyllaDB feature toolbox
ScyllaDB is the firm behind the Scylla NoSQL database claims to be thinking big — and so, the organisation has used its annual Scylla Summit conference to detail a whole selection box of new features designed to serve real-time big data applications.
Scylla has unveiled new capabilities including Lightweight Transactions, Change Data Capture (CDC) and an Incremental Compaction Strategy.
CEO and cofounder of ScyllaDB Dor Laor explains that he knows his firm has become known for its approach to speed and reliability for latency-sensitive big data applications… and for its ability to help reduce storage costs.
Latency-sensitive applications are (perhaps obviously) chunks of enterprise software that can not work effectively with any extended period latency (or wait time), typically because they serve a real-time data need in some live operational/transactional deployment.
“But performance [for low-latency] is just part of what makes Scylla so powerful. With these latest features, we’re extending Scylla’s benefits to exciting new use cases and opening the door to a wide range of new functionality,” said Laor.
Lightweight Transactions
Among the new features is Lightweight Transactions (LWT), a development that is already committed into Scylla’s main source code branch.
LWT works to deliver ‘transaction isolation’ similar to that of traditional relational databases, which the company says will help bring Scylla to a new range of business-critical use cases. In database systems, isolation determines how transaction integrity is visible to other users and other systems… so it’s a good form of lock down and control where needed.
Going deeper here… LWT ensures, with a reasonable degree of certainty, that whenever you [i.e. your database] reads a record, you see the version that was last written. Without LWT, you might read a record (call it Record A) off one node of the cluster just as someone was updating the record on another node. With LWT, the database updates Record A on all clusters at the same time, so the nodes don’t (or very rarely) disagree, and two applications querying the same record are much less likely to see two different versions.
The company will also soon release a Scylla Open Source version that includes this LWT feature.
Change Data Capture (CDC)
Scylla reminds us that the modern application stack is no longer a monolith.
Because of this core truth, we see microservices that need to constantly push and pull data from their persistence layer. CDC enables Scylla users to stream updates to datasets with external systems, such as analytics or ETL tools.
Scylla CDC identifies and exports only new or modified data, instead of a full database scan. Beyond its efficiency, CDC allows organisations to use Scylla tables interchangeably.
“Opening new possibilities for users to consume data. CDC is already committed into Scylla’s main source code branch. We will soon release a Scylla Open Source version that includes this feature,” noted the company, in a press statement.
Incremental Compaction Strategy (ICS)
Reducing storage costs by what are said to be up to 40%, Incremental Compaction will soon be available with Scylla Enterprise and Scylla Cloud.
While compaction reduces disk space usage by removing unused and old data from database files, the operation itself typically suffers from space amplification. ICS lowers costs significantly by improving this operation.
Finally here there’s DynamoDB-compatible API for Scylla Cloud: Project Alternator, which presents an alternative to Amazon DynamoDB… and the technology is now available in beta for Scylla Cloud, the company’s powerful database as a service (DBaaS).
Applications written for DynamoDB can now run on Scylla Cloud without requiring code changes. This enables DynamoDB users to quickly transition to Scylla Cloud to significantly reduce costs, improve performance, and take advantage of Scylla’s cloud and hybrid topologies.