Part 2 - Custom Terraform Providers

Introduction

Cards on the table, I’ve tried to have AI written this whole blog, but I failed. I didn’t like the style, it was full of errors regarding the different types of Terraform provider frameworks/SDKs and it followed a bit of a weird structure. Probably fixable by written a more extensive prompt, but I feel like I already spent pushing it. Off course there’s also ample somewhat scientific evidence that outsourcing creative work makes people less create, like here, here or here

So let’s go!

Why?

Most of the scenario is explained here, but to save you some time:

• We have a large terraform codebase that manages 100K resources, and are now confronted with a custom internal API whose underlying objects are closely integrated with other objects in the exising codebase.

What are they though?

Providers

So, now we’re on the same page, let’s start buliding this thing! But what is this actual thing? Let’s take a step back.

Think of Terraform providers like APIs for your Terraform code. They enable Terraform to communicate with the underlying infrastructure provider’s API, allowing you to create, manage, and configure resources in a declarative way. As such, they implement the actual logic behind the definition of your resources, data resources, etc and the resulting action towards the infrastructure provider’s APIs.

There are two types of Terraform providers:

1. Core providers: These are built into the Terraform distribution and provide access to popular cloud services like AWS, Azure, Google Cloud, etc.
2. Third-party providers: These are contributed by external organizations or individuals and can be installed separately using the terraform init command.

Some examples of Terraform core providers include:

• AWS Provider (for Amazon Web Services)
• Azure Provider (for Microsoft Azure)
• Google Cloud Provider (for Google Cloud Platform)
• Docker Provider (for Docker container management)

The Plugin Framework

HashiCorp offers two Go programming language Software Development Kits (SDKs) for building Terraform providers:

1. Terraform Plugin Framework: The most recent SDK that is easier to use and more extensible than SDKv2.
2. SDKv2: SDKv2 is the prior SDK that many existing providers use. It is maintained for Terraform versions 1.x and earlier, but we have stopped most feature development so we can focus on improving the framework.

There are still many providers out there that were built using the SDKs, especially the larger ones, but for newer ones Hashicorp advises to make use of the Terraform Plugin Framework. There are plenty of reasons for that, I won’t list them here. If you’re interested, there’s some documentation on it.

Why did they not call it the Provider Framework? Good question, without really a clear answer. Typically, extensions to larger codebases would be refered to as plugins, hence the name. Providers are more of a technical concept, which is there to allow/provide interaction with specific infrastructure platforms. So a provider typically is a plugin.

In general; A provider is a concept. A provider (whether HashiCorp’s or third-party) is always implemented as a plugin. A provider plugin, i.e. a plugin that implements a provider, is one kind of Terraform plugin.

When you run terraform init, it downloads all plugins needed by providers you use.

Next to that, there’s also the concept of provisioners. Again, it’s a bit of a grey area if you ask me and people tend to have slightly different notions of it too. To me, a provisioner executes scripts and commands on remote systems, rather than setting up those systems. But then again, the concept of a system is very broad and very abstract.

How?

There’s an amazing tutorial that explains in a whole lot more depth how this looks like. If you’re interested, you should definitely have a look. It’s also way easier then you might think, although some familiarity with Go might help you a lot.