The official Terraform documentation describes all aspects of configuration in details. Read it carefully to understand the rest of this section.

This section describes key concepts which are used inside the book.

Resource

Resource is aws_vpc, aws_db_instance, etc. A resource belongs to a provider, accepts arguments, outputs attributes, and has a lifecycle. A resource can be created, retrieved, updated, and deleted.

Resource module

Resource module is a collection of connected resources which together perform the common action (for e.g., creates VPC, subnets, NAT gateway, etc). It depends on provider configuration, which can be defined in it, or in higher-level structures (e.g., in infrastructure module).

Infrastructure module

An infrastructure module is a collection of resource modules, which can be logically not connected, but in the current situation/project/setup serves the same purpose. It defines the configuration for providers, which is passed to the downstream resource modules and to resources. It is normally limited to work in one entity per logical separator (e.g., AWS Region, Google Project).

For example, module uses resource modules like and to manage the infrastructure required for running on .

Another example is module where multiple modules by are being used together to manage the infrastructure as well as using Docker resources to build, push, and deploy Docker images. All in one set.

Composition

Composition is a collection of infrastructure modules, which can span across several logically separated areas (e.g.., AWS Regions, several AWS accounts). Composition is used to describe the complete infrastructure required for the whole organization or project.

A composition consists of infrastructure modules, which consist of resources modules, which implement individual resources.

Data source

Data source performs a read-only operation and is dependant on provider configuration, it is used in a resource module and an infrastructure module.

Data source terraform_remote_state acts as a glue for higher-level modules and compositions.

The data source allows an external program to act as a data source, exposing arbitrary data for use elsewhere in the Terraform configuration. Here is an example from the where the filename is computed by calling an external Python script.

The data source makes an HTTP GET request to the given URL and exports information about the response which is often useful to get information from endpoints where a native Terraform provider does not exist.

Remote state

Store for each infrastructure module and composition in a remote backend, configured with ACLs, versioning, and logging. This single, authoritative source of truth keeps environments consistent and typically includes disaster-recovery features such as automated backups. Managing state locally can lead to collaboration issues and race conditions when multiple developers run Terraform at the same time, resulting in unpredictable outcomes.

Provider, provisioner, etc

Providers, provisioners, and a few other terms are described very well in the official documentation and there is no point to repeat it here. To my opinion, they have little to do with writing good Terraform modules.

Why so difficult?

While individual resources are like atoms in the infrastructure, resource modules are molecules (consisting of atoms). A module is the smallest versioned and shareable unit. It has an exact list of arguments, implement basic logic for such a unit to do the required function. e.g., module creates aws_security_group and aws_security_group_rule resources based on input. This resource module by itself can be used together with other modules to create the infrastructure module.

Access to data across molecules (resource modules and infrastructure modules) is performed using the modules' outputs and data sources.

Access between compositions is often performed using remote state data sources. There are .

When putting concepts described above in pseudo-relations it may look like this:

Questions related to Terraform code structure are by far the most frequent in the community. Everyone thought about the best code structure for the project at some point also.

How should I structure my Terraform configurations?

This is one of the questions where lots of solutions exist and it is very hard to give universal advice, so let's start with understanding what are we dealing with.

• What is the complexity of your project?

  • Number of related resources

  • Number of Terraform providers (see note below about "logical providers")

• How often does your infrastructure change?

  • From once a month/week/day

  • To continuously (every time when there is a new commit)

• Code change initiators? Do you let the CI server update the repository when a new artifact is built?

  • Only developers can push to the infrastructure repository

  • Everyone can propose a change to anything by opening a PR (including automated tasks running on the CI server)

• Which deployment platform or deployment service do you use?

  • AWS CodeDeploy, Kubernetes, or OpenShift require a slightly different approach

• How environments are grouped?

  • By environment, region, project

Getting started with the structuring of Terraform configurations

Putting all code in main.tf is a good idea when you are getting started or writing an example code. In all other cases you will be better having several files split logically like this:

• main.tf - call modules, locals, and data sources to create all resources

• variables.tf - contains declarations of variables used in main.tf

terraform.tfvars should not be used anywhere except .

How to think about Terraform configuration structure?

Common recommendations for structuring code

• It is easier and faster to work with a smaller number of resources

  • terraform plan and terraform apply both make cloud API calls to verify the status of resources

In this book, example projects are grouped by complexity - from small to very-large infrastructures. This separation is not strict, so check other structures also.

Orchestration of infrastructure modules and compositions

Having a small infrastructure means that there is a small number of dependencies and few resources. As the project grows the need to chain the execution of Terraform configurations, connecting different infrastructure modules, and passing values within a composition becomes obvious.

There are at least 5 distinct groups of orchestration solutions that developers use:

1. Terraform only. Very straightforward, developers have to know only Terraform to get the job done.

2. Terragrunt. Pure orchestration tool which can be used to orchestrate the entire infrastructure as well as handle dependencies. Terragrunt operates with infrastructure modules and compositions natively, so it reduces duplication of code.

3. In-house scripts. Often this happens as a starting point towards orchestration and before discovering Terragrunt.

With that in mind, this book reviews the first two of these project structures, Terraform only and Terragrunt.

See examples of code structures for or in the next chapter.

Terraform is powerful (if not the most powerful out there now) and one of the most used tools which allow management of infrastructure as code. It allows developers to do a lot of things and does not restrict them from doing things in ways that will be hard to support or integrate with.

Some information described in this book may not seem like the best practices. I know this, and to help readers to separate what are established best practices and what is just another opinionated way of doing things, I sometimes use hints to provide some context and icons to specify the level of maturity on each subsection related to best practices.

The book was started in sunny Madrid in 2018, available for free here at https://www.terraform-best-practices.com/.

A few years later it has been updated with more actual best practices available with Terraform 1.0. Eventually, this book should contain most of the indisputable best practices and recommendations for Terraform users.

Sponsors

Please if you want to become a sponsor.

Translations

Contact me if you want to help translate this book into other languages.

Contributions

I always want to get feedback and update this book as the community matures and new ideas are implemented and verified over time.

If you are interested in specific topics, please , or thumb up an issue you want to be covered. If you feel that you have content and you want to contribute, write a draft and submit a pull request (don't worry about writing good text at this point!).

Authors

This book is maintained by with the help of different contributors and translators.

License

This work is licensed under Apache 2 License. See LICENSE for full details.

The authors and contributors to this content cannot guarantee the validity of the information found here. Please make sure that you understand that the information provided here is being provided freely, and that no kind of agreement or contract is created between you and any persons associated with this content or project. The authors and contributors do not assume and hereby disclaim any liability to any party for any loss, damage, or disruption caused by errors or omissions in the information contained in, associated with, or linked from this content, whether such errors or omissions result from negligence, accident, or any other cause.

Copyright © 2018-2023 Anton Babenko.

Terraform code structures

What are the tools I should be aware of and consider using?

• - Orchestration tool

Source:

This example contains code as an example of structuring Terraform configurations for a small-size infrastructure, where no external dependencies are used.

Source:

This example contains code as an example of structuring Terraform configurations for a medium-size infrastructure which uses:

• 2 AWS accounts

• 2 separate environments (prod

Source:

This example contains code as an example of structuring Terraform configurations for a large-size infrastructure which uses:

• 2 AWS accounts

• 2 regions

There is also a workshop for people who want to practice some of the things described in this guide.

The content is here -

- List of people who work with Terraform very actively and can tell you a lot (if you ask them).

- A community of individuals who actively share their Terraform knowledge through content, events, and open collaboration.

- Curated list of resources on HashiCorp's Terraform.

Use locals to specify explicit dependencies between resources

Helpful way to give a hint to Terraform that some resources should be deleted before even when there is no direct dependency in Terraform configurations.

https://raw.githubusercontent.com/antonbabenko/terraform-best-practices/master/snippets/locals.tf

Terraform 0.12 - Required vs Optional arguments

1. Required argument index_document must be set, if var.website is not an empty map.

2. Optional argument error_document can be omitted.

Optional Object Attributes (Terraform 1.3+)

Use optional attributes in objects to provide default values for non-required fields:

Managing Secrets in Terraform

Secrets are sensitive data that can be anything from passwords and encryption keys to API tokens and service certificates. They are typically used to set up authentication and authorization for cloud resources. Safeguarding these sensitive resources is crucial because exposure could lead to security breaches. It’s highly recommended to avoid storing secrets in Terraform config and state, as anyone with access to version control can access them. Instead, consider using external data sources to fetch secrets from external sources at runtime. For instance, if you’re using AWS Secrets Manager, you can use the aws_secretsmanager_secret_version data source to access the secret value. The following example uses write-only arguments, which are supported in Terraform 1.11+, and keep the value out of Terraform state.

Variable Validation and Input Handling

Basic Variable Validation

Use validation blocks to ensure variables meet specific criteria:

Object and List Validation

Validate complex data structures to ensure they contain expected values:

General conventions

1. Use _ (underscore) instead of - (dash) everywhere (in resource names, data source names, variable names, outputs, etc).

2. Prefer to use lowercase letters and numbers (even though UTF-8 is supported).

Resource and data source arguments

1. Do not repeat resource type in resource name (not partially, nor completely):

1. Resource name should be named this if there is no more descriptive and general name available, or if the resource module creates a single resource of this type (eg, in there is a single resource of type aws_nat_gateway and multiple resources of typeaws_route_table, so aws_nat_gateway should be named this and aws_route_table should have more descriptive names - like private, public, database

Code examples of resource

Usage of count / for_each

Placement of tags

Conditions in count

Variables

1. Don't reinvent the wheel in resource modules: use name, description, and default value for variables as defined in the "Argument Reference" section for the resource you are working with.

2. Support for validation in variables is rather limited (e.g. can't access other variables or do lookups if using a version before Terraform 1.9). Plan accordingly because in many cases this feature is useless.

Outputs

Make outputs consistent and understandable outside of its scope (when a user is using a module it should be obvious what type and attribute of the value it returns).

1. The name of output should describe the property it contains and be less free-form than you would normally want.

2. Good structure for the name of output looks like {name}_{type}_{attribute} , where:

   1. {name}

Code examples of output

Return at most one ID of security group:

When having multiple resources of the same type, this should be omitted in the name of output:

Use plural name if the returning value is a list