Terraform, OpenTF, and the Shrinking Gap

introduction

Hashicorp recently made shock licensing changes to their stack to close their source code to ‘competitors’. There has been much public comment and Open Terraform has already been announced in response. I see a strong future for OpenTF and a move towards Terraform itself being less used directly and more as an intermediate layer or stage for higher level tools as the gap that it occupies shrinks, as detailed below.

What is impacted?

I am not a fan of repeating information that is publicly available so I don’t propose to rehash the public announcements or discussions. It isn’t just Terraform, or all of Terraform that is impacted however. Whilst I am sure all of the Hashicorp products have their adherents and some of them leverage each other, most of them have viable competition when working in a cloud environment.

Terraform Open source is the key tool impacted for which there isn’t much in the way of direct alternatives available on major cloud providers. Most of the alternatives to Terraform are either:

  • based around/leverage it, e.g. Spacelift; Atlantis; Terragrunt; etc.
  • Or specific to their own platforms, e.g. CloudFormation; ARM.

Pulumi seems to be the principal competitor right now that is cross-platform, not built around Terraform, and doesn’t require knowledge or set up of some other service, e.g. Kubernetes in the case of Crossplane. Paying customers for Terraform Enterprise or Terraform Cloud aren’t really impacted IMO.

Some of the rest of the Hashicorp stack is popular, but alternatives are available, at least in the cloud:

  • Vault (open source): Popular, although there are many cloud-based alternatives - see e.g. the available backends for the Kubernetes External Secrets Operator. Doesn’t help on-prem but I’m focussing on cloud here.

  • Packer: I am not sure at this point what the ‘enterprise’ sell is for Packer beyond the Open source offering. There are many alternatives available however so I can’t see it being a ‘lock-in’ for organisations.

  • Consul: Nice in its way and widely used, especially with other Hashicorp services but it’s basically a key-value store. There are many alternatives available.

I consider the following Hashicorp products to be of niche interest and so of little impact to the majority of organisations:

  • Boundary
  • Nomad
  • Waypoint
  • Vagrant

What is not affected?

HashiCorp APIs, SDKs, Terraform providers, and almost all other libraries will remain MPL 2.0.

The problem space

From a purely practical point of view the ‘big one’ is Terraform Core Open Source. I can’t see, e.g. AWS being happy with a more restrictive licence change on the AWS providers and some, such as the Cloud Control provider are created automatically. For almost everything else the providers are third-party and Hashicorp act mainly as a certification authority if they are involved at all.

In this section of this video interview from 1st April 2023 Dave McJannet, CEO at HashiCorp repeatedly uses the term ‘malicious’ in reference to Hashicorp’s approach to community building. We can debate the use of the term and how appropriate this sort of approach is to building a business but clearly we are in the land of hard-nosed decision making here. Certainly it would be naive to rule out the possibility of further licence changes in future based on some sort of imagined social contract. As has been referenced in a number of places, Hashicorp’s BSL is deliberately ambiguous in order to compel people to ask them directly for permission to do things that may not be 100% clear otherwise. This is a situation, at best, prone to inconsistency - not just between client organisations, but over time for a given client organisation. Maybe a client organisation isn’t a competitor (as judged by Hashicorp) now but we can’t be certain that that won’t change in future, either because of some change in the client’s own business direction, ownership, etc, some change in Hashicorp’s setup or simply on a whim.

The immediate and further future and OpenTF

Some people are already looking to move off of Terraform and some are locking Terraform at the final open source version with a view to seeing how things turn out- this is probably a fairly safe bet. I wouldn’t rush to migrate an existing large-scale deployment to an alternative right now but I wouldn’t want to double down either. Certainly any future switch will be easier from today’s point than years down the road.

Is OpenTF an idea with legs? Initially I took the position that Hashicorp were probably justified in stopping other people building businesses of off their work but as the conversation has developed I’ve come to revise that decision. As in this Register article, Hashicorp haven’t been suffering financially, any more than the other organisations that have closed their source code in a similar way. Elastic and MongoDB are cited although we could also consider Java, RedHat and Confluent Cloud amongst others. It might be more level-headed to consider those playbooks. From an organisation’s point of view, it’s clear why you might want to use OpenSearch rather than Elastic or OpenJDK instead of Oracle java. All of those are initiates that aren’t going to go away.

There is another thread however:

The future of Terraform itself

Terraform has become popular because (amongst other qualities) it has a low barrier to entry and it covers a very wide range of services. Despite all its strengths there are some significant clunkinesses however. Data structures in Terraform are painful compared to any half decent real programming language for example, and we can still joke about the missing twice flag even in 2023. I am not a fan of Terragrunt but it does solve the bootstrapping problem that Hashicorp have never seemed to want to solve themselves. We could go on. Fundamentally Terraform, like infrastructure itself, is always a part of the solution rather than a solution in itself.

It’s worth looking at how the landscape is changing however- CDK and CDK for Terraform are already here. I have recently written about playing with Wing in an article where I described using AI acceleration to write a Terraform module. As already mentioned, initiatives like AWS Cloud Control API allow Terraform providers to be created ‘automatically’.

From another angle I recently had a technical exercise at interview along the lines of ‘how would you build an Instagram’. I fed the brief to ChatGPT and then asked it to write terraform code to deploy the suggestion. Some minor tweaking and I was able to run a terraform plan, at which point I could use Pluralith to generate an architecture diagram.

Fundamentally Terraform is already at the point of becoming an intermediate tool or language- we can generate the top level Terraform code with our programming language of choice, an AI, or some other tool, and the providers are likely to remain open with clearly contracted interfaces. This to me looks like a shrinking gap - we don’t have to worry too much about writing HCL ourselves. The gap between the contract for the providers and the language specification is finite and made smaller as tooling develops.

This author’s take

I would expect OpenTF to be a real thing real soon and to achieve credibility within the first year. I would expect initiatives like Wing, OpenAI, CDK, CloudControl, etc. to leverage it to shrink the space it occupies as a top-level tool ever further. Terraform at this point is 9 years old. Serverless and managed initiatives aside, I don’t see it holding its crown in another nine, it simply won’t matter that much beyond intermediate level generated code and niche cases.

Get in touch