Modern development - Redgate: A contemporary validation for dedicated (database dev models)

The new world of data…

Even before the pandemic, software development teams were under pressure to be faster and more agile. As a result, teams are being encouraged to embrace DevOps to break down barriers and be more responsive to dynamic, even volatile, market and business demands for software updates and improvements.

With many full-stack developers in DevOps teams making changes to databases as well as applications, the shared development model seems a natural choice when collaboration lies at the heart of this software development approach and culture.

Yet, the pressure to deliver more innovative software quickly and successfully suggests this model doesn’t align well with DevOps. According to our own State of Database DevOps Report, about 70% of organisations are sticking to a shared approach. But it’s not the best option and teams should explore a dedicated model if they want to make the most impact on their business-critical databases.

Understanding the differences

It’s good to be clear about definitions. With the shared development model, everyone uses the same copy of the production database on the same server. By contrast, under the dedicated model, each developer has their own copy of the database running locally

There are good reasons why the shared development environment has been so prevalent. Firstly, sharing one database means less set up time and lower operating costs. The differences between this and a dedicated approach are significant, one example being that a team of 10 developers all work on the same database. For the same sized team using a dedicated model, the disk space required would need to increase tenfold, and it would take considerable time and effort to provision database copies for everyone. Justifying that kind of infrastructure spend and extra work is extremely difficult.

Secondly, there’s the benefit of control and visibility that the shared model offers. A DBA can be responsible for every configuration change including those that most developers are not even aware of, such as security and filegroups.

This does come at the cost of limiting the freedom to experiment, conflicts when developers work on the same object at the same time… and the database becoming unpredictable very quickly as change after change is made for different reasons by different developers. Many, though, would conclude that if this model has worked for years, why replace it?

However, as teams look to deliver more value to their customers, faster and through ever-shortening release cycles, the dedicated model becomes much more vital.

Why dedicated is more ‘modern’

So can we really justify why the dedicated approach can be more forward-thinking and (dare we say it) more modern?

The dedicated approach gives teams the freedom to experiment, which is a critical part of DevOps. By working with an isolated copy of the database, a team member can push the limits and even break things to uncover solutions, discover innovations and do a lot more useful work. Quite simply if the data is local, you can mess up without crashing the whole live database, while also learning what will or won’t work in the real world.

For developers under pressure to innovate, the dedicated model is increasingly attractive because it gives them independence. There are a tick list of other benefits ranging from being able to develop application and database code side by side, providing support for branching strategies and version controlling changes that enable developers to work in parallel without any problems. Yet, less than one in three organisations (30%; Source: 2020 State of Database DevOps Report) are applying the dedicated model today. 

While there are a few challenges that go with the dedicated model (like possible security concerns about keeping personal data in multiple locations) it’s important to weigh all the options in transitioning to the dedicated model.

When you explore what types of organisations are migrating, it’s also interesting that the most likely to have dedicated development environments for high value databases are media and retail (36%), IT and technology (35%) and financial services (29%) (Source: State of Database DevOps 2020). These aren’t industries that take risks with personal data and they have found solutions to ensure dedicated environments are efficient and secure.

One obvious step is to use virtualisation technology to avoid the high infrastructure costs that would otherwise be needed to support the multiple database copies in a dedicated development environment. By using such technology, an organisation can minimise the storage footprint and costs significantly. Modern cloning solutions can create copies, or clones, that act like full databases for developers to work on their own sandbox environments, and are only a fraction of the size of the source database. Importantly, they can also be provisioned in a matter of minutes rather than the many hours it typically takes.

Another is to automate the provisioning of database copies so that they are regularly refreshed to reflect the latest production data, and so that developers can self-serve new copies when they make a breaking change during development.

How data masking can help

The real value of a dedicated development environment is how it enables developers to work with up-to-date production data to test the impact of their changes and capture any errors. While using live data has obvious risks when it comes to the protection of personal or confidential information, this can also be mitigated by using data masking.

Data masking allows developers to test their changes with realistic data without the risks of non-compliance with data protection regulations, or worse, introducing cybersecurity vulnerabilities. The dedicated model also opens the door to version control, which by default creates a history of changes that can be used as an audit trail to demonstrate compliance.

Perhaps most importantly, at a time when many developers are working remotely, it removes the biggest issue with the shared development model, often called the last one wins scenario. Quite simply, the latest change a developer makes will overwrite the changes other developers have made. If developers are working on the same object at the same time, or if it involves an action like renaming a column, this inevitably causes conflicts. The normal workaround is for developers to constantly communicate with each other, but if they’re no longer in the same workspace, this will fail.

The trend toward dedicated development environments is likely to grow as more technology teams find themselves being asked to be more agile, more flexible and more innovative. Clearly, organisations need to choose the database development environment that’s right for them, but as the obstacles to adopting the dedicated approach begin to fade away, it’s becoming much more of a frontrunner. 

Image source: Redgate Software