Confluent: Current 2022 user conference - live report day #1
Hosted by Confluent, Current 2022 is billed as the ‘next generation of Kafka Summit’ and the Computer Weekly Developer Network team is in Austin, Texas for the showdown.
As an aide memoir, Confluent is a full-scale data streaming platform that enables users to access, store and manage data as continuous, real-time streams – built by the original creators of Apache Kafka, Confluent expands the benefits of Kafka with enterprise-grade features, while removing the burden of Kafka management or monitoring.
As TechTarget reminds us that Apache Kafka is a distributed publish-subscribe messaging system that receives data from disparate source systems and makes the data available to target systems in real-time.
“Kafka is written in Scala and Java and is often associated with real-time event stream processing for big data. Like other message broker systems, Kafka facilitates the asynchronous data exchange between processes, applications and servers. Unlike other messaging systems, however, Kafka has very low overhead because it does not track consumer behaviour and delete messages that have been read,” notes the above TechTarget link.
Current 2022 – live report day #1
So to the event itself then, data at rest has had its day says Confluent – streaming data, therefore, is the here and now.
“We’ve seen this type of growth [as in data streaming] before, where exponential growth looks like a flat line… until it’s not and we really start getting to the hockey stick [upward] part of the curve,” said Confluent CEO and co-founder Jay Kreps – speaking on stage at the event keynote.
Suggesting that we’re at a point where a whole new raft of adoption is about to happen, Kreps pointed to the early development of electricity (and noted the point at which it became ubiquitous)… and then looked to the rise of early databases, the rise of open source, the arrival of Apache Kafka (and the connector ecosystem that has grown up around Apache Kafka that helped popularise it) and the way the demand use cases for streaming data.
“We are witnessing the rise of a major new data platform,” said Kreps.
He noted that the small ‘bits’ (i.e. small amounts, not computing bits) of data streaming happening in use cases inside live working organisations today are now coming together and evolving as a category.
5 capabilities for data streaming
Kreps says that the five key capabilities needed for a data streaming platform to succeed — Store, Connect, Process, Govern, Shape.
With so many companies now running on 1) off-the-shelf operational databases, 2) SaaS applications and 3) analytics stores – these three core layers for data all essentially point to data that is at rest. Kreps suggests that there is a fourth layer – and that is data streaming platform use. It’s all about taking data that was previously at rest and starting to create and use a first-class interface for data streaming that ultimately ends up connecting those three well-known pre-existing layers.
Confluent is now promoting data streaming for start-ups to make the whole set of technologies being showcased here more accessible.
Data streaming tablestakes
Previously, prior to the more sophisticated use of data streaming at this level (suggested the customer speakers showcased in a promotional video shown halfway through the opening keynote), a lof of data developer scenarios saw engineers having to accept tradeoffs of workarounds. Before now, developers would have to think more about the way they were attempting to engineer streaming data technology from a systems operations point of view – and so, importantly and logically, less about the semantics of the queries they were trying to build.
“Today, data streaming is tablestakes for us,” enthused one of the customers in customary gung-ho upbeat tones (tablestakes meaning the minimum anyone has to lay down at a casino in order to be able to enter and engage in any given game) designed to create entusiasm.
Moving from batch data to streaming data platform progression is as big of a deal as the move to mobile technology ubiquity, as big as the move to cloud, as big as the move to adopt the new breed of Artificial Intelligence (AI)-fuelled automation — this is the core suggestion being tabled here.
Is it all true – and is streaming data the next killer app [ingredient] today?
We perhaps will know for sure if we start to see the wider tech vendor community starting to tack this onto their stacks in much the same way that they all added AI, RPA, ML, CI/CD, low-code/no-code (LC/NC) and much besides.
Confluent CEO & co-founder Jay Kreps.
