Feature

Portrait of the software developer as an artist

Many programmers see themselves as artists – of a sort. Computer Weekly investigates what it means to be a modern software developer

This article can also be found in the Premium Editorial Download: Computer Weekly: Freezing digital history in the Arctic Circle
By
Published: 10 Aug 2020

Creating applications, data engineering and services across PCs, mobile devices, the cloud backbone and throughout the internet of things (IoT), the developer is the artist, the keyboard is the colour palette and the command line is the painter’s easel and canvas.

Art is categorised into a number of genres, such as classical, abstract, post-modern and renaissance. Similarly, software programming falls into various fields, categories and sub-genres, such as waterfall, agile, scrum or pair-based, rapid application development (RAD) and now low-code and no-code (for business people) in the wider spectrum of code creation.

Going deeper, some people would classify different programming types by the various language types, such as declarative, concurrent, data flow, functional, and so on. Others separate them out by target device as desktop, mobile, embedded firmware software development.

To add to the confusion of how to categorise software development, modern development practices are often associated with cloud-native, cloud-first and multicloud approaches.

There are those who will argue that modern software development embraces data-driven, big data insight and makes inherent use of artificial intelligence (AI) and machine learning. Computer Weekly asked a number of industry experts to define what it means to be a modern software developer.

There is a school of thought that categorises modern software development as primarily services-centric and web-scalable, which enables the developer to create code that is capable of being deployed across connected services backbones. Other software developers strive to build reusable components and frameworks that form the fundamental building blocks of new applications.

Today, there is a specialist segment of the enterprise software market that defines itself as application modernisation specialists. Often focused on the migration from legacy, mainframe and pre-cloud applications, the thrust from software tools providers in this space is towards microservices, virtual machines, containers and Kubernetes.

A modern software architecture

Volterra provides a distributed cloud platform to deploy, connect, secure and operate applications and data across multicloud and edge sites. Its CEO and founder, Ankur Singla, thinks microservices will have an increasingly important role to play in the immediate future shape of software application development. Singla says the surge seen with Kubernetes adoption and a selection of other factors are the reasons microservices will become more mainstream in 2020 and onwards.

Read more about modern software development

GitLab looks at what does not rank as a measure of modern development for contemporary forward-thing programmers and code engineers

How should we think about the cloud-native, open-compliant, mobile-first, Agile-enriched, AI-fuelled, bot-filled world of coding and how do these forces now come together to create the new world of modern programming?

“Microservices is a part of Kubernetes’ DNA – it is the primary method by which apps are developed and deployed when using Kubernetes,” he says. “With the rise of Kubernetes, tech players are releasing open source toolkits and frameworks that address microservice challenges and ultimately allow other organisations to adopt them properly.”

As an example, Singla says Microsoft recently launched the open source Dapr project. Microsoft describes Dapr as a portable, event-driven runtime that makes it easy for developers to build resilient, microservice stateless and stateful applications that run on the cloud and edge and embrace the diversity of languages and developer frameworks.  Singla  says a number of startups are also ramping up efforts to address these issues.

Write once, deploy on any device

According to open source web server company Nginx, a modern application is one that supports multiple clients – whether the client is a user interface based on the React JavaScript library, a mobile app running on Android or iOS, or a downstream application that connects to a back-end application through an application programming interface (API).

“Modern applications are expected to have an undefined number of clients consuming the data and services they provide,” says Chris Stetson, chief architect and senior director of microservices engineering at Nginx. “A modern application provides an API for accessing that data and those services. The API is consistent, rather than bespoke to different clients accessing the application. The API is available over HTTP(S) and provides access to all the features and functionality available through the graphical user interface (GUI), or a command-line interface (CLI).”

Looking at the ability of web-scale businesses to develop new software-powered products and services suggests that modern development is typified by a high degree of experimentation and iteration. So-called old world IT does not map to modern businesses that want to emulate the success of the web giants. This means there is now a need for increased customer and user feedback in the software development process, which suggests that applications are being created with user experience (UX) sensitivity plugged into their core DNA.

The goals of modern software development practices are increasingly focused on time to value. Chris Bailey, chief architect for cloud-native solutions, IBM Cloud Paks for Applications at IBM, argues that these practices should not only focus on the ability to deliver software rapidly but, crucially, they need to ensure that software delivers real user and business value.

Multidisciplinary teams

Bailey believes software development teams need to become multidisciplinary and more self-contained, reducing handovers and scheduling dependencies on other teams. He says they should also adopt behaviour-driven development (BDD) and test-driven development (TDD) so that software is based on meeting user needs and quality requirements.

Bailey says software development teams are utilising continuous integration to increase velocity and ensure continuous quality checking occurs as part of the development process. He says they also tend to use continuous delivery with capabilities such as canary releases in order to limit risk and continually validate and enforce resilience.

Bailey believes modern software development practices involve building software in a way that makes the code easy to manage. This means adding consistent health-checking, observability and operational controls, which, he says, makes it easy to manage and operate applications once they are in production.

Managing a service mesh – the experience of Bloomberg’s DevX team

Peter Wainwright is a senior engineer and author with the developer experience (DevX) team at Bloomberg, which develops the tools and processes used by the company’s 6,000-plus software engineers to manage its codebase.

The core back-end has traditionally been composed of a few monolithic binaries, some of which are still in use. This means the development team has taken a hybrid approach to integrating service meshes into its infrastructure, says Wainwright.

“The mega-monoliths are mostly C++ and code from thousands of developers is pulled into them,” he says. “They have a weekly cycle, with release branches that spawn at set times, each the start of a new release cycle – you might call this a ‘monoschedule.’ It is, quite deliberately, not very flexible. Deployment is slow but simple. Everyone knows the social contract to get changes submitted, testing completed and any needed fixes in. It works well, because it is predictable.”

To counter the inflexibility of the “monoschedule”, the Bloomberg team built in feature toggles that allow changes to be flipped without interrupting users’ sessions – per customer, if required.

Describing how the service mesh idea at Bloomberg evolved, Wainwright says: “We first started building service meshes almost as an afterthought. Our services scale very well, up to the hardware’s limits. But they can’t scale further unless we deploy new hardware, and you can’t do that at a moment’s notice. Our customers need us most when market activity peaks.

“Monoliths mean that high-traffic functions are deployed with the same mechanisms and priority as low-traffic ones. As we added more features, the different priorities among teams meant an increasing demand for special cases and exceptions, all of which had to be tracked.

“We found that some analytics scaled better if we extracted them into services and just had the monolith route traffic to them. This solved the impedance mismatch that the monolith created between teams and it brought the added advantages that we could deploy changes more rapidly. Because analyses often query one another, a mesh naturally started to develop.”

Wainwright added: “You might think the biggest challenge here is the migration from monolithic applications to isolated containerised microservices, but it really isn’t. Migrating a routine to a service often was easy. However, as more moving pieces were introduced, we had to invest in our tooling and our culture, too.”

From a technical perspective, Wainwright says that replacing in-process calls with service calls introduces potential latency, timeouts and queueing issues. But there are other, more subtle challenges too, he says. “Service meshes are harder for engineers to reason about. These were problems they’d never had to consider before, let alone solve.”

Software development practices go hand-in-hand with change management and having diverse thinking from a broad base of people. There was a time when software was developed to serve generic business functions, all within a single, monolithic application that ran on a central system and could be accessed via a dumb terminal. These days, the preferred architecture for new software development projects is often highly componentised, where individual building blocks may run on different servers, containers or even split across different clouds.

The front-end application not only needs to run on any device, but developers are encouraged to create user interfaces that engage end-users. From an end-user perspective, the front- and back-end applications need to create good user experience.

Any company can assemble the best development tools and services, but that will not necessarily guarantee success, says Catherine Wong, chief product officer and executive vice-president of engineering at Domo, which specialises in cloud-based business intelligence tools, data visualisation and data integration.

“If it were that straightforward, we’d see a lot more startups succeed,” she says. Wong believes there are, of course, a multitude of reasons why building and scaling software is so challenging. While writing millions of lines of code and distributed bits and bytes are absolutely important, success requires a team effort.

“For the majority of us, software development is a team sport,” she says. “Our teams have long had crisply defined roles, like engineers and all their specialisations, architects, product managers, quality assurers, designers, project managers and technical writers. Those are still relevant functions, but how we enable diversity of thought and experience, as well as how we cross-train for increased empathy and better communication among the team, has become more critical than any one job title.”

According to Wong, this focus on diversity dramatically influences the business impact of the software that is being developed, and, more importantly, on a human level, it stretches project and product managers to cultivate an environment of inclusion, innovation and growth. “Over the years, I’ve seen countless examples of how the art of software development and the human elements of diversity and collaboration are what really differentiate a product and its speed of response to the market,” she says.

Read more on Software development tools