How to build your own IoT with Azure

The Azure IoT Suite: connect, analyse, integrate

In September 2015, Microsoft made its Azure IoT suite available for purchase. This stitches together several services for connecting devices, analysing the data they produce, and integrating with customers’ own systems or those from third-party providers.

One of the most important components of this suite is the IoT Hub, which lagged behind the Suite, finally becoming available quite recently, in February 2016. The Hub is the interface between IoT devices and the Azure cloud.

Think of it as the on-ramp for millions of industrial and consumer devices, enabling them to talk to applications running on the Azure infrastructure.

The Hub is designed to scale up to millions of devices while monitoring them collectively and optimizing their resource use. It includes a device registry and data storage specifically for IoT services, along with security features that work with individual IoT devices.

Developers provision their own instances of the IoT Hub, which can then be used to provision each device and present it with its own unique communication endpoint. That can be done using several languages.

“Most of Microsoft’s IoT focus tends to point towards the IoT gateway, and within that there are several types of protocols that call out reliably to these various end devices,” says Ian Hughes, analyst in the IoT practice at 451 Research.

In addition to providing devices with REST application programming interfaces (APIs) access to the IoT Hub, Microsoft also provides the Advanced Message Queuing Protocol (AMQP), which is a rich language designed for business messaging that is finding traction in the IoT space.

This was Microsoft’s mechanism of choice, says Hughes. It also added supported for MQTT (MQ Telemetry Transport), which is a lightweight messaging protocol from IBM designed for small footprint machine-to-machine (M2M) communications.

Devices connect to the cloud directly, but they may also connect via field gateway hardware that aggregates multiple devices into a single group. “Edge of network computing is a way to be more efficient about IoT, to say locally to a nest of sensors ‘what are you doing’ and then aggregate that and then send it on,” says Hughes.

Harvesting data

Many companies are still grappling with this connectivity stage at the moment, points out Michele Pelino, a principal analyst in infrastructure and operations at Forrester. “A lot of the focus for the initial IoT has been around connecting,” she says.

Companies are busy trying to plug their devices into the cloud in secure and scalable ways.

Having connected their devices into Azure’s IoT infrastructure, the next data step for many companies is to begin harvesting data, says Pelino, adding that what “these platforms help to simplify is not just the connecting and managing of the devices, but bringing to light the information and insight from those connected devices”.

“That’s what is particularly relevant to Microsoft because that’s what the Azure piece of the platform does,” she says.

Analytics is a key part of Azure’s toolset for the IoT, and it starts with telemetry garnered from connected IoT devices. This can be fed from the IoT Hub into Azure’s other services.

These include Event Hub, an event processing service in Azure that hoovers up large numbers of events and uses them to trigger other business logic where appropriate.

Events may also be fed into Stream Analytics, Azure’s real-time event streaming and processing service, which can analyse the data and look for certain trigger events.

Potential challenges for developers

Herein lies one of the potential challenges for IoT developers: data management. Put enough devices out there in the field and they will spit out incredible amounts of information, points out Patrick McFadin, chief Cassandra evangelist at DataStax, which provides databases for cloud applications.

“I want to be able to store it in a location that makes sense, but I also need access to storage at the processing stage to get back-filled information,” he says. For example, an application might need to scan a day’s entire information collected from a thermostat to find the highest temperature in the past 24 hours. That requires mining and assessing stored information.

Companies will compress data as a matter of course, but they should also take other steps to store their information as efficiently as possible.

After all, storage may be cheap but it still incurs a business overhead, and price shock is still a danger if you aren’t tracking your use of cloud resources. To avoid this, use tiering strategies, and summarise data appropriately before it is even stored, added McFadin.

Machine learning

The real power for IoT in Azure really comes with the backend services that Microsoft has made available. Developers can access services such as machine learning to pull more value from the potentially huge mounds of data that devices in the field will generate.

Because cloud solutions are designed to be exposed as services, these become relatively simple to access.

Harman Connected Services, an IoT consulting firm and winner of Microsoft’s 2015 IoT Partner of the Year award, has used the IoT Suite to build solutions including a smart helmet for NFL players that can help coaches and medical staff monitor players, as well as smart building solutions that can monitor the usage of conference rooms.

The company frequently uses Cortana Analytics, Microsoft’s range of analytics functions, exposed as services for Azure developers. Developers can access the machine learning service within Azure in different ways to process that IoT data.

If real-time analytics are necessary, then you could expose the machine learning module to your application as a service, says Sriram Jayaraman, vice-president of consulting at Harman Connected Services.

“If you’re solving a segmentation of data problem such as ‘what kind of events are happening’ then you’re using ‘cold’ analytics where we don’t use it as a web service,” he adds. In that case, the machine learning model effectively delivers a batch job.

When the data from IoT devices is inside Azure, developers are able to build applications such as dashboards for equipment monitoring. Developers at Redmond are making it easier for companies to get to this point by creating solution templates in the IoT Suite.

There are two at the time of writing, for remote device monitoring and predictive maintenance. The code is on Github so companies can deploy it generically or customize it for their own needs.

This is an area upon which the company will likely expand. At the beginning of May it announced its acquisition of Italian firm Solair, which creates IoT-based services for specific industries, including manufacturing, retail, and transportation.

Harman used Azure’s Service Fabric microservices platform – only officially out of preview mode as of March 2016 ­– to build a collection of small, independent functions that combined to form an IoT application. These services run on a dense pool of machines. The idea here is to scale different parts of the application depending on the business’s needs.

This is something which comes in handy when dealing with large numbers of connected devices.  It also enables developers to tweak particular functions quickly without breaking the rest of the application via unknown dependencies.

Azure developers can also create two-way communications with devices, controlling them in the field via a so-called cloud-to-device ‘send endpoint’ within IoT Hub.

These instructions would come from business logic running on Azure, and could be used to adjust the temperature on a thermostat, for example, or simply to ping a device and make sure that it is still operational and sending information.

The remote device monitoring template application in Azure already contains that functionality, and it’s a sign of things to come as organizations get to grips with what IoT can do.

Integration

Companies will also begin moving into more advanced applications, where they integrate with their own back-end systems, explains Dave Bartoletti, Pelino’s colleague at Forrester and a principal analyst in infrastructure and operations.

Analytics in Azure may be able to predict when a part is going to fail, but what then? “Well, that means we can connect it to our service system and notify service three weeks in advance and so potentially have more people available in this area,” he says.

This integration process is the third part of Azure’s IoT story. Microsoft provides interfaces to products from firms such as Oracle, Salesforce and Informix, so that companies can exchange data between Azure and their own systems rather than having to host everything in Azure.

Companies are still mostly at the early stages of their IoT projects, analysts told Computer Weekly. The majority of projects outside of the industrial sectors are still proofs of concept. Nevertheless, there have been some big wins.

These include projects such as Harman’s and others Rockwell Automation’s Azure IoT-based monitoring of oil and gas-based control systems. As companies tackle the intricacies of managing vast arrays of sensors and other connected devices, the proven use cases for IoT on Azure will continue to expand.