Infrastructure-as-Code series - Shipa: Bridging the IaC–app disconnect
The Computer Weekly Developer Network (CWDN) continues its Infrastructure-as-Code (IaC) series of technical analysis discussions to uncover what this layer of the global IT fabric really means, how it integrates with the current push to orchestrate increasingly cloud-native systems more efficiently and what it means for software application development professionals now looking to take advantage of its core technology proposition.
This piece is written Ravi Lachhman in his capacity as field CTO at Shipa – the company is an Application-as-Code specialist with a corresponding and appropriate knowledge of IaC.
Lachhman writes as follows…
Infrastructure-as-Code has advanced significantly and (it has to be said) rather thoroughly, over the past decade.
It’s fair to say that IaC processes and tools are now part and parcel within many – if not most – enterprises. This codification of infrastructure has unlocked the ability for elastic and ephemeral infrastructure to be provisioned at deployment time. Scaling events for applications and services becomes a fast and relatively pain-free process, a far cry from the weeks’ worth of lead time required in the decades gone by.
For applications and services, there clearly needs to be infrastructure powering those applications in the traditional sense, such as networking, storage and compute. But as you march closer to the application itself, application infrastructure starts to appear – such as a message broker/stream processor, web/application containers, a container orchestrator, etc.
As you get closer still to the application, the requirements and order of operations become more bespoke to the application/service and its functionality itself. There is still a disconnect between applications and IaC that needs to be addressed…applications to-be-deployed typically have a pipeline that goes from code to deployment on infrastructure, requiring several tools and syntaxes.
The modern application pipeline
Looking back at a containerised application that is deployed to Kubernetes… we need to stop and ask ourselves what exactly are all the pieces that have to be put in place i.e. from source code to running the application in production?
The source code has to be built into some sort of artifact, then the artifact needs to be stored in a registry, then all of the steps that the application needs to be installed in a package manager like Helm or a configuration management solution like Kustomize. The Kubernetes clusters themselves can, potentially, be automated up to a point with IaC tools. But there are still quite a few moving parts as to what is prudent to the cluster (e.g node groups) and what is prudent to the application (e.g custom resource definitions and deployment manifests). Every time there is a new tool or new paradigm, another syntax gets thrown into the mix and has to be learned. The ability to enforce becomes challenging.
With the lines increasingly blurred between application infrastructure and the application these days, providing IaC coverage can be complicated for developers who are trying to focus in on applications. More often than not, if a team is providing archetypes to application teams via IaC, the process requires a learning curve and extensive templating.
How to describe & enforce
Lachhman: The road is long & complex, but we have tools to overcome.
Even the definition of what an application is comes into question. Having the ability to balance innovation with control when providing templating leads to abstraction creation by DevOps and/or platform engineering teams – and creating, administering and enforcing abstractions for development resources is a complex endeavor.
To add to the complexity, IaC tools themselves are changing also. As teams who leverage established IaC tools such as Terraform look for new ways of describing infrastructure, such as Crossplane for a very Kubernetes-centric approach, the IaC syntaxes change and tooling has to be re-learned anew.
Brass tacks for applications
From a development perspective, developers know that their applications and services have traffic coming in and traffic coming out. The nuances of VPCs, subnets, load balancer implementations and whatnot are typically too low-level details for the average feature developer. IaC tools would focus on spinning those pieces of infrastructure up. Similarly, developers can describe their applications easily for what they are able to create locally, or in a lower environment such as ‘my application is version 1.5.1 of this image’.
The application-specific dependencies to get the application installed/deployed on something like Kubernetes is a long road and that’s especially the case for those green in Kubernetes.
Bridge the IaC–application disconnect
The goal of platform engineering is to meet your internal customers where they are. Focusing on the feature developer – providing the functionality that matches their application’s needs in a generic sense – is key.
Also, as the implementation of IaC solutions change and expand and contract (e.g using Terraform today but coupling with cloud-specific IaC tools such as CloudFormation or newer Kubernetes-centric approaches such as Crossplane) re-writing the underlying abstractions is a time-consuming process. Upping the abstraction to an Application-as-Code (AaC) approach is one mechanism to accomplish this.
A running joke is that all applications are code. But using open source projects like Ketch, teams can create a more common-sense-based abstraction that aligns with the needs of feature developers and also enforce and monitor what, well, needs to be enforced and monitored. The end result is a strategy for overcoming one of the biggest hurdles with IaC adoption – the application disconnect.
