Applying blockchain, Kafka and MongoDB in green energy trading

Energy trading platform Powerledger is among a growing number of blockchain technology providers that have been riding on the renewable energy wave.

The Western Australian company provides various services that are built on its blockchain-based energy trading platform, facilitating peer-to-peer (P2P) trading of excess electricity from rooftop solar power installations, the assembly of multiple household batteries into virtual powerplants and the purchase of renewable electricity from a particular provider.

It even allows a participant to give their excess solar electricity to a customer in another country on a mutual basis between related parties in opposite hemispheres, or as a gift to family members living in a country where electricity prices are high relative to incomes.

As with any blockchain platform, it provides traceability with non-fungible tokens (NFTs) representing renewable energy certificates (RECs), so participants can be assured that the electricity they are buying is indeed from a renewable source.

The company works in cooperation with electricity retailers but provides a decentralised rather than the conventional unidirectional market. The benefits include reduced customer acquisition costs, increased customer satisfaction, better prices for buyers and sellers (compared with feed-in and supply tariffs), and provision for cross-retailer trading.

Vivek Bhandari, Powerledger’s chief technology officer, said although there are existing markets for RECs, they are opaque with prices set by brokers. An organisation might be committed to buying 100% of its electricity from renewable sources, but there may be times when it has no choice other than conventionally generated electricity. When that happens, it will need to buy RECs to compensate.

Powerledger’s market is transparent, and is much faster than transactions conducted through brokers, Bhandari said. Furthermore, it can cope with situations where supply data is subsequently updated, and offsets need to be bought or sold to balance the customer’s consumption.

To date, Powerledger has been operating on a relatively small scale, but there are billions of users in its addressable market, so the company had to identify technology that would be reliable at hyperscale.

It was already using Apache Kafka as a core piece of infrastructure, specifically to ingest data from smart electricity meters and feed it into the trading system. To cope with the anticipated scale, and to help with the adoption of an event-driven architecture, it turned to Kafka specialist Confluent.

Confluent was founded by the creators of Kafka and is used as the basis of scalable real-time applications at companies such as Netflix and around 80% of the Fortune 100, said Confluent’s vice-president in Australia and New Zealand, Gavin Jones.

Examples of such applications include the detection of possibly fraudulent credit card transactions, automatically blocking access to systems when a user’s behaviour resembles that of a breach in progress and determining that a piece of equipment requires preventative maintenance.

Confluent offers the most complete edition of Kafka, Jones said, with the scalability, security and high availability needed for important applications. Furthermore, it can be used on-premise, in the cloud, or as a fully managed cloud service on Amazon Web Services, Microsoft Azure, or Google Cloud.

The managed service means customers spend less time on the backend systems, so they can more quickly gain insights into their data.

What Powerledger is doing “is not trivial” due to the combination of internet of things (IoT), network monitoring, blockchain and more, said Jones, suggesting that there are few energy providers that could achieve something similar.

MongoDB is another important part of Powerledger’s infrastructure. The company provided Powerledger with a proof-of-concept showing that the combination of MongoDB and Confluent could handle 100 million electricity meters with the ability to scale even further, said MongoDB regional vice-president Anoop Dhankhar.

Bhandari claimed that the system has been resilient so far, with no data lost when it was tested to the point of failover. He also described the combination of the two products as “very powerful” when it comes to security. And with the need to store data onshore to meet regulations such as the General Data Protection Regulation (GDPR), “the integrity of the data becomes very important”, he added.

Both Confluent and MongoDB are certified to a level that allows their use by security-conscious government agencies, Jones said, so Powerledger did not need to worry about that, and instead focus on its business model and applications.

Dhankhar picked up this theme, noting that even the GDPR’s “right to be forgotten” provision is accommodated by MongoDB, so Powerledger and other users can be confident that they can comply with the rules and regulations applicable in various countries.

The relationship between Confluent and MongoDB extends to reference architectures addressing a range of situations including cloud migration, application modernisation and IoT, said Jones, and the integration provided by the two companies “is a huge benefit” to customers, allowing them to leverage the power of real-time technology.

In Powerledger’s case, it provided not only the scalability required, but also simplified the integration of multiple applications, reducing both time-to-market and operational overheads