Infrastructure Changes: Modularize & Version Your Infrastructure Code

Refactoring infrastructure code is unavoidable and can be uniquely risky. This ignite talk from devopsdaysNYC 2020 outlines practices that yield more maintainable infrastructure code that is less risky to change.

[Transcript]

My name is Michael Wytock and I am a senior consultant with Stride Consulting and this talk is about how we can make our infrastructure code maintainable, resilient when changed, and supportive of future changes. Oftentimes when we're writing code of any kind, the way we learn how to write maintainable code is by writing a lot of unmaintainable code and getting bitten by it. 

What my teammates have taught me over the last few months of working together is a mechanism for how we can use modules and versioning, two fundamental skills, to make our code more maintainable. So it makes sense to talk a little about why your infrastructure may be changing in the first place. And it usually comes down to changing requirements. But there's a couple of high level things that are shared across many organizations.

Perhaps you need to support a new deployment context. There's going to be a bare metal context we've never supported before. Maybe you have a multi-cloud situation or you have client-specific deployments that will occur and you need to make sure you can deploy to these new requirements that are unforeseen right now. And it's more complicated because you may have some of these particular requirements of your infrastructure code: automated tests, infrastructure code execution in a CI pipeline, or portable code.

So the first suggestion is that you split modules based on your build. So obey the build. Don't work against it. Take your networking code, which would be required for our machines to be able to speak to one another, and make that its own module. Then for the configuration for those machines, make that a module as well.

And finally, the services that will be deployed in there, those services need to be placed into their own module. These dependencies being modeled here will provide a common abstraction that you'll be able to use across all of your deployment contexts in the future.

So if you're using Terraform oftentimes the outputs of one of these "build modules" will be fed into the next module. In the networking module, where you have your VPC set up and subnets, the output of that information will flow into the compute layer. And after provisioning, perhaps you are provisioned Kubernetes or something else, then you will have all of the outputs there, ready to deploy services into it.

This is an example of how we can use this to achieve phased deployments. In your development environment, if you have some kind of error going on in your services module, then we can fix that error, make sure we address the failing tests before promoting into production. And similarly if we have a brand new deployment context, we can isolate and understand changes in these different "build modules" and talk about them using the same terminologies we've used before. 

Now, the second strategy that we've used is to create a separate child modules repository. And in this separate repository we can place things like the finer grained VPC configuration or maybe your authentication tooling. And when we place them all in this separate repository together, now we're able to reference them and consume them through all of our separate build modules that are in separate pipelines for our deployments. 

So, for example, in this development pipeline that we have, we may see that there's a failure, again, in our services module, and we're referencing certain child modules that are under development and we need to fix those before we can merge those back into the master. And we do fix those and we're at v1.24 and we want to push that out into production. We've now run tests in development. We've seen it running alongside other services, and we can just bump our references. And hopefully it's as simple as that, for the most part, in moving from development into production. 

Now if you have a brand new deployment context and you have new functionality that you haven't put into your system yet, then we'll bump again. We'll add the specialized modules and perhaps change the configuration more dramatically in the build modules but still just consume that new bumped version of the child modules.

To summarize, obey the build, don't work against it. Use that to guide your organization and keep it consistent. Decouple by placing your child modules on their own and version them. Make sure you can reference those versions and use that to specialize your new deployment contexts that you may not have been able to foresee as of yet. And then finally I would like to offer some acknowledgements and thanks, especially the team that I work with on a day-to-day basis. Especially Adarsh and Arielle. Thank you.

Interested in joining the conversation? Reach out to Michael Wytock to talk more on this topic!