Terraform Day 5: Enabling TF_LOG and Working with Sensitive Information

In this Blog, we explore how to enable logging in Terraform using environment variables, how to handle sensitive information such as passwords, and how to integrate AWS Secrets Manager for securely storing sensitive data. We also demonstrate deploying an RDS MySQL instance with Terraform.

Topics Covered

1. Enabling TF_LOG for Debugging

2. Working with Sensitive Information

3. Using AWS Secrets Manager with Terraform

4. Deploying RDS MySQL Instance

Enabling TF_LOG

Terraform provides the TF_LOG environment variable for controlling log verbosity. You can choose from different levels like TRACE, DEBUG, INFO, WARN, and ERROR.

Steps to Enable TF_LOG

1. Set TF_LOG for Detailed Trace Logs
To enable detailed trace logging:

PowerShell:

$env:TF_LOG = "TRACE"
terraform apply --auto-approve

Bash:

export TF_LOG="TRACE"
terraform apply --auto-approve

This level logs every action, providing a complete picture of the Terraform execution process.

---

2. Set TF_LOG for Error-Level Logging
For logging only errors:

PowerShell:

$env:TF_LOG = "ERROR"
terraform apply --auto-approve

Bash:

export TF_LOG="ERROR"
terraform apply --auto-approve

This is useful when you only want to capture critical issues without unnecessary verbosity.

---

3. Write Logs to a File
To save the logs for future reference or debugging, you can direct them to a file:

PowerShell:

$env:TF_LOG = "TRACE"
$env:TF_LOG_PATH = "./logs/terraform.log"
terraform apply --auto-approve

Bash:

export TF_LOG="TRACE"
export TF_LOG_PATH="./logs/terraform.log"
terraform apply --auto-approve

The logs will be written to terraform.log in the specified directory (./logs), ensuring you don't lose critical debugging information

.

---

Log Levels in TF_LOG

Terraform supports various log levels for different use cases:

• TRACE: Detailed logs for debugging.

• DEBUG: Logs for in-depth insights into operations.

• INFO: General logs to understand the flow.

• WARN: Logs for warnings during operations.

• ERROR: Logs for critical errors only.

Handling Sensitive Information

When working with sensitive data like usernames and passwords, it is important to avoid hardcoding them in the Terraform scripts. Instead, use variables marked as sensitive.

Example

In your variables.tf:

variable "username" {
  type      = string
  sensitive = true
}

variable "password" {
  type      = string
  sensitive = true
}

Storing Passwords Securely with AWS Secrets Manager

To securely store and retrieve sensitive information like passwords, you can use AWS Secrets Manager.

2. Generate a random password:

    resource "random_password" "db_password" {
      length           = 16
      special          = true
      override_special = "_!%^"
    }
   

3. Store the password in AWS Secrets Manager:

    resource "aws_secretsmanager_secret" "db_secret" {
      name = "test-db-password"
    }
   
    resource "aws_secretsmanager_secret_version" "db_secret_version" {
      secret_id     = aws_secretsmanager_secret.db_secret.id
      secret_string = random_password.db_password.result
    }
   

   

Deploying RDS MySQL Instance

Steps:

1. Retrieve the password when deploying RDS:

    data "aws_secretsmanager_secret" "db_secret" {
      name = "test-db-password"
    }
   
    data "aws_secretsmanager_secret_version" "db_secret_version" {
      secret_id = aws_secretsmanager_secret.db_secret.id
    }
   

2. Create a subnet group:

    resource "aws_db_subnet_group" "test_subnet_group" {
      name = "test-db-subnet-group"
      subnet_ids = [
        aws_subnet.subnet1-public.id,
        aws_subnet.subnet2-public.id,
        aws_subnet.subnet3-public.id,
      ]
      tags = {
        Name = "Test DB subnet group"
      }
    }
   

3. Deploy the RDS instance:

    resource "aws_db_instance" "test_db_instance" {
      identifier           = "testdb"
      allocated_storage    = 20
      storage_type         = "gp2"
      engine               = "mysql"
      engine_version       = "8.0.39"
      instance_class       = "db.t3.micro"
      username             = "adminuser"
      password             = data.aws_secretsmanager_secret_version.db_secret_version.secret_string
      publicly_accessible  = true
      db_subnet_group_name = aws_db_subnet_group.test_subnet_group.id
    }
   

Connecting to RDS via MySQL Workbench:

1. In AWS Console, go to RDS > Databases > testdb and copy the endpoint

   

   .

2. In MySQL Workbench, use:

   • Hostname: <copied endpoint>

   • Username: adminuser
     make sure port 3306 is open in security Group

     

     

   • Password: Fetch from AWS Secrets Manager.

     

     

     

Connected successfully

Destroy the Infrastructure

After testing, remember to clean up:

terraform destroy --auto-approve

Interview Tip: Handling Sensitive Information

When asked how to handle sensitive information in Terraform, you can explain that Terraform can integrate with AWS Secrets Manager to securely store and retrieve sensitive data. Sensitive variables should be defined in Terraform to avoid exposing sensitive information directly in the code.