Infrastructure-as-Code series - GitLab: Control, efficiency & paths to GitOps
The Computer Weekly Developer Network (CWDN) continues its Infrastructure-as-Code (IaC) series of technical analysis discussions to uncover what this layer of the global IT fabric really means, how it integrates with the current push to orchestrate increasingly cloud-native systems more efficiently and what it means for software application development professionals now looking to take advantage of its core technology proposition.
This piece is written by Abubakar Siddiq Ango in his role as developer evangelism programme manager for GitLab.
The exponential growth in our dependence on online services also translates into a constantly increasing need for infrastructure resources to power these services.
Back in the day, technology companies would host their services on-premises or rent space in a datacentre. Spinning up a server is a signup form and a few clicks away with the explosion in cloud services. But in an ever-expanding and demanding world, the constant need to scale and manage infrastructure scalability can spin out of control and become costly if not well thought out.
That is where Infrastructure as Code (IaC) comes in.
Cloud console control
Managing infrastructure in the cloud often involves accessing the console of your cloud provider to provision resources, define permissions, create roles and users and monitor how everything works. But things can get harder to manage with growth… and questions like who, when and why things were done become challenging to answer.
IaC involves adhering to best practices in defining and versioning your entire infrastructure. The most effective strategy is using proven coding techniques that define what resources to provision and how they should interact. Each file should additionally be stored with version control so that every change can be tracked.
The latest DevOps platforms, like GitLab, integrate tightly with best-of-the-breed IaC tools. Prominent among them is HashiCorp Terraform, which enables organisations of different sizes, from single developer shops to Fortune 100 companies, to foster collaboration across teams, ranging from operations to quality assurance, security, finance and top-level management.
Everyone works together to ensure the infrastructure runs smoothly, safely, cost-effectively and brings maximum ROI. This has given rise to Policy as Code tools which automatically ensure that every change introduced meets policies set by the organisation. An example is HashiCorp’s Sentinel framework.
Usable reusability reality
DevOps platforms seek to provide a smooth experience with IaC, providing features like Terraform CI templates and managed Terraform state with advanced state management that provides a Terraform backend that is integrated with CI pipelines, all without the need for configuration or complicated tooling.
Reusability is also a crucial part of IaC and private Terraform module registries are also a popular feature of DevOps platforms, enabling organisations to follow best practices in how they re-use modules across different projects, which is especially valuable in large organisations. For example, government and financial institutions are using IaC strategies to reduce complexity and ensure the same configurations across their operations and improve testing.
Security is also at the heart of IaC workflows: every change introduced can be scanned with security scanners in the CI pipeline to find vulnerabilities, compliance issues, and infrastructure misconfigurations, using tools like KICS from the team at Checkmarx.
Abubakar Siddiq Ango: DevOps platforms integrate with best-of-the-breed IaC tools.
A central location where every stakeholder can collaborate is also important to the IaC workflow.
DevOps platforms provide features that put all information needed for everyone to act in a single interface, enabling collaboration and further strengthening Policy as Code initiatives because it is easy to see what changes need more work to meet set standards.
While Terraform has become an industry-standard in IaC workflows, other infrastructure definition and configuration management tools like AWS Cloud formation, Chef, Puppet, Ansible are also configurable with CI pipelines. The possibilities are limitless.
Automated (advantageous) operations
Extending the power of IaC is GitOps, which applies automated operations to versioned definition files. Take a Kubernetes cluster, for example; you can create a Kubernetes manifest defining how your application should be deployed and the sizes of resources it should consume.
With GitOps, your changes are automatically applied by a Kubernetes agent running. The Kubernetes agent runs within the cluster, constantly monitors your environment and application definitions for changes and automatically applies them using the Pull Continuous Deployment strategy.
With such an approach, approved changes are built and pushed to the environment and application registries, from where the Kubernetes agent regularly checks for updates, pulls the changes to the cluster and applies them. This allows the cluster to be secured behind a firewall and its credentials are never shared.
This is a popular use case in regulated environments, with large financial institutions having been able to cut overall time spent on updates and boost their security using pull strategies.
Get the gist of drift
Another crucial function that the Kubernetes agent performs is drift management.
The agent constantly monitors how the infrastructure environment behaves. When the actual state of the environment has drifted from how it is defined, the agent automatically pulls the most updated definitions from the environment and application repositories to bring everything back to normal, drastically reducing TTR (time to recovery) when incidents occur.
IaC has revolutionised how efficient and fast organisations can grow so it is now easier to manage scale and document how organisations grow over time. Introducing changes and operating disaster recovery policies also takes less effort, because organisations can quickly identify what went wrong and revert to a stable state.
With the rise of DevOps platforms, tools will go more into the background with interfaces allowing developers, DevOps teams and other stakeholders to worry less about the tools they use and focus more on delivering value.