GitOps gets the grit in the composable stack
As we take a momentary breather after a truly larger than expected number of commentaries on the nature of the ephemeral composable IT stacks that we are all building now, some people still have opinions that they want to be voiced.
One such ‘no hang on there a minute, have you thought about xyz’ person is Nima Badiey, who works as VP of alliances at GitLab.
Nima (he/him), who Tweets under @badnima has said that Git-based source code tools such as GitLab and GitHub (long popular with developers who use the open interfaces to build their applications) are gaining popularity to store infrastructure configuration and state for composable infrastructure.
Compute, storage and network services are provisioned through declarative YAML (a data serialisation language) manifests and act as the source of truth when DevOps teams need to recover from entropic drift, patch CVEs or rotate security credentials (repair, rotate, repave).
This amalgamation from infrastructure through the application tier is organic, propelled by the need for different system elements across the stack to communicate and share state. Common configuration settings can be updated in the same manner as application code is through a merge request.
So, what aspects of the Git-based tool universe do we need to think about when we move forwards up the composable stack? Badiey writes as follows to tell all…
12 Factor Apps
Adam Wiggins’ 12 Factor App guidelines became a manifesto for cloud-native application developers. Adopted by Heroku, Cloud Foundry and OpenShift developers and platform builders recognised that to be able to build modern, scalable, maintainable software-as-a-service (SaaS) apps, they needed to externalise app configurations.
By following the 12-factor model, applications and their deployment topologies were now composable – the app developer could describe in simple language how multi-element apps (e.g. web, app, data layers) should be deployed and the underlying deployers could rely on the source Git config files as the source of truth should any app component need to be recovered or re-deployed.
Containerisation & infrastructure orchestration
Google created Borg in 2003 as a flexible, scalable scheduler to address its rapidly growing estate of workloads across tens of thousands of machines. As a proprietary cluster management tool, it was part of Google’s ‘secret sauce’ which used containerisation to maximise the utility of every layer of infrastructure. The composability of Borg’s declarative manifest found purchase in later open source infrastructure technologies including Docker and Kubernetes, the latter of which experienced a Cambrian explosion with 130,000 contributors and 630+ member companies across 70+ cloud-native projects in the Cloud Native Computing Foundation.
The launch of vSphere, VMware’s cloud computing virtualisation platform, was a watershed for ops teams because immutable machines (servers) could now be virtualized and partitioned as defined by the vCenter API.
Any combination of compute, memory and storage could be requested by users on-demand and released just as easily. As composability moves up the stack into the application tier, the Kubernetes ecosystem is advocating for similar open interfaces to promote control and visibility via Container Storage Interface (CSI) and Container Networking Interface (CNI).
Declarative configuration files expressed in a Kubernetes-readable format called the Kubernetes Resource Model (KRM) state what you want to run on your cluster, expressed as configuration code. Even bare-metal servers are becoming configurable on demand with bare metal-as-a-service (BMaaS) solutions including Canonical MaaS, OpenStack Ironic, Equinix Tinkerbell, Cobbler and Foreman.
APIs & modernising legacy
Many digital transformation and enterprise modernisation projects are hamstrung by the pervasive friction of legacy COTS (commercial off-the-shelf) apps and mainframe systems.
Badiey: A command line guru with an appreciation for composable constructs.
These aging monuments limit the aspirations of many CIO/CTOs seeking to modernise their IT infrastructure so they can build and release new products at cloud native speeds. Fortunately, legacy mainframe systems that cannot be retired are being de-risked using API frameworks which, enable developers to interact with legacy systems via modern JSON-based API end-points instead of legacy SOAP/XML/SOA-based architectures.
Solutions like Mulesoft Anypoint, Dell Boomi, Apigee and Kong can even automate caching, security and payload adjustments in addition to enabling communication through composable APIs.
For example, according to Google Cloud’s ‘State of the API Economy, Telecom Report’, 70% of telecoms companies use APIs to speed up new application development and 50% do so for connecting internal applications. These composable architectures can also promote partnerships across industries – a key benefit for telecom providers exploring new revenue drivers in manufacturing, financial services, retail, healthcare and more.
GitOps is an emerging operational framework that takes DevOps best practices used for application development such as version control, collaboration, compliance and CI/CD and applies them to infrastructure automation. Through Git, infrastructure can be version controlled, audited and even updated using the same push, pull and merge verbs used by developers writing apps.
Emerging GitOps goodness
Since Kubernetes uses YAML/JSON files for specifications, it becomes easy to combine these artifacts with code. Git tools such as GitLab and GitHub and service-specific solutions such as Google’s Anthos Config Manager (ACM), WeaveWorks and Spring Cloud Config are just a few examples of emerging GitOps solutions.
To support this practice, we are adding Protected Variables and Protected Environments to extend GitLab’s concept of Protected Branches for use in role-based deployment security and quality gating.
These composability patterns are transforming infrastructure and systems of record/engagement through GitOps frameworks. Bare metal has APIs for control; VM across all providers are controlled via APIs; services atop machines (Kubernetes, Istio, Knative) can be defined, provisioned and managed via YAML; and applications comport to 12-factor principles which include decoupled state and data services defined by config and environment variables.
Organisations focusing on DevOps competencies should know what and how they need to deliver products and services to market and what processes and tools enable them to meet their business needs.
This is particularly cogent for CIOs concerned about the escalating cost of not aligning with GitOps and DevOps principles. A good first step is to assess your existing DevOps capabilities with any of the available DORA frameworks to identify improvements for your organisation.