Sravya Yakkati's Blog

Integrating HashiCorp Vault with Terraform

Sravya Yakkati's photo
Sravya Yakkati
·

5 min read

Table of contents

Introduction

This document provides a step-by-step guide to integrating HashiCorp Vault with an existing Terraform project to securely retrieve the EC2 private key for provisioning.

Prerequisites

  • AWS CLI installed and configured.

  • Terraform installed on your local machine.

  • HashiCorp Vault installed.

  • An AWS account with necessary permissions.

Vault Installation and Setup

Install Vault on the EC2 instance

Install gpg

sudo apt update && sudo apt install gpg

Download the signing key to a new keyring

wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg

Verify the key's fingerprint

gpg --no-default-keyring --keyring /usr/share/keyrings/hashicorp-archive-keyring.gpg --fingerprint

Add the HashiCorp repo to ubuntu

echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list

Finally, Install Vault

sudo apt install vault && sudo apt install vault

Start Vault

vault server -dev -dev-listen-address="0.0.0.0:8200"

  • When we run the vault server -dev we will get the above logs.

  • We need to use this root token and login to hashicorp vault UI so that it will give root access to the hasicorp vault UI

Authenticate with Vault

export VAULT_ADDR='http://<ec2-public-ip>:8200'

Login to Vault Console

Login with EC2 instance public ip

Use the above Root Token after typing the vault server command you will get one Root Token use that token for login into Hashicorp Vault Console.

  • Secret engines means the different type of secrets that we can create in the hashicorp vault.

  • Once Login, Go to Search Engines and select KV ( Key Value Pair )

  • Give path of the secret engine and enable engine.

  • Once secret generated click on create secret.

  • Give the name of the secret

  • Give key of the secret and give value ec2 instance private key

  • Save the secret.

Create and Apply Vault Policy

Create Vault Policy

vault policy write aws-private-key-policy - <<EOF
path "*" {
  capabilities = ["list", "read"]
}

path "kv/data/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}
EOF

export vault address with your ec2-instance public-ip

Create Vault AppRole

Enable the AppRole authentication method:

vault auth enable approle

Create an AppRole and attach the policy:

vault write auth/approle/role/aws-private-key-role \
    secret_id_ttl=10m \
    token_num_uses=10 \
    token_ttl=20m \
    token_max_ttl=30m \
    secret_id_num_uses=40 \
    token_policies=aws-private-key-policy

Retrieve the Role ID and Secret ID:

vault read auth/approle/role/ec2-role/role-id

vault write -f auth/approle/role/ec2-role/secret-id

Store these values securely as they will be used in Terraform for authentication.

Terraform Configuration

Providers

Update providers.tf to include the Vault provider:

terraform {
    required_providers {
      aws = {
        source = "hashicorp/aws",
        version = "~> 5.0"
      }
    }
}

provider "aws" {
  region = var.aws_region
}

provider "vault" {
  address = "http://<ec2-public-ip>:8200"
  skip_child_token = true
  auth_login {
    path = "auth/approle/login"

    parameters = {
      role_id   = "<role_id>"
      secret_id = "<secret-id>"
    }
  }
}

data "vault_kv_secret_v2" "pem_file" {
  mount = "kv"
  name  = "aws-pem-file"
}

  • Added data block also in the provider file to retrieve secrets from the hashicorp vault.

  • Give the role_id and secret_id which we saved while doing Valut configurations with commands

Update main.tf to read the private key secret from Vault

ec2_private_key = data.vault_kv_secret_v2.pem_file.data["public_key"]

# Define Locals for using dynamic variables with the variable terraform.workspace ( dev/prod )
locals {
  ec2_instance_name         = "${terraform.workspace}-${var.ec2_instance_name}"
  # s3_bucket_name            = "${terraform.workspace}-${var.s3_bucket_name}"
  # aws_dynamodb_table_name   = "${terraform.workspace}-${var.aws_dynamodb_table_name}"
  ec2_security_group_name   = "${terraform.workspace}-${var.ec2_security_group_name}"
}


# Defining module for EC2 module
module "ec2_instance_module" {
  source = "./modules/ec2-instance"
  ec2_ami_id = var.ec2_ami_id
  ec2_instance_name = local.ec2_instance_name
  ec2_instance_type = var.ec2_instance_type
  ec2_private_key = data.vault_kv_secret_v2.pem_file.data["public_key"]
  vpc_id = var.vpc_id
  ec2_security_group_name = local.ec2_security_group_name
}


# Defining module for S3 Bucket
module "s3_bucket_module" {
  source = "./modules/s3-bucket"
  s3_bucket_name = var.s3_bucket_name
}


# Deniging module for DynamoDB
module "dynamodb_table_terraform_lock_module" {
  source = "./modules/dynamo-db"
  aws_dynamodb_table_name = var.aws_dynamodb_table_name
}

EC2 Module

Update the EC2 module to use the private key retrieved from Vault

resource "aws_security_group" "web-app-security-group" {
  name = var.ec2_security_group_name
  vpc_id = var.vpc_id
}

resource "aws_vpc_security_group_ingress_rule" "allow_tls_ipv4_22" {
  security_group_id = aws_security_group.web-app-security-group.id
  cidr_ipv4         = "0.0.0.0/0"
  from_port         = 22
  ip_protocol       = "tcp"
  to_port           = 22
}

resource "aws_vpc_security_group_ingress_rule" "allow_tls_ipv4_80" {
  security_group_id = aws_security_group.web-app-security-group.id
  cidr_ipv4         = "0.0.0.0/0"
  from_port         = 80
  ip_protocol       = "tcp"
  to_port           = 80
}

resource "aws_vpc_security_group_ingress_rule" "allow_tls_ipv4_all" {
  security_group_id = aws_security_group.web-app-security-group.id
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "-1"
}

resource "aws_vpc_security_group_egress_rule" "allow_all_traffic_ipv4" {
  security_group_id = aws_security_group.web-app-security-group.id
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "-1" # semantically equivalent to all ports
}


resource "aws_instance" "aws_ec2_instance" {
  ami = var.ec2_ami_id
  instance_type = var.ec2_instance_type
  key_name = "first-instance-key-pair"
  vpc_security_group_ids = [ aws_security_group.web-app-security-group.id ]

  tags = {
    Name = var.ec2_instance_name
  }

  provisioner "remote-exec" {
    inline = [ 
        "sudo yum update -y",
        "sudo yum install -y httpd",
        "sudo systemctl start httpd",
        "sudo systemctl enable httpd",
        "echo '<h1>Hello, World!</h1>' | sudo tee /var/www/html/index.html"
     ]

    connection {
        user = "ec2-user"
        host = self.public_ip
        private_key = var.ec2_private_key
    }
  }
}

Deploying the Project

Step 1: Initialize Terraform

terraform init

Step 2: Select or Create a Workspace

terraform workspace new dev

terraform workspace select dev

Step 4: Plan the Configuration

terraform plan -var-file="./workspaces/dev.tfvars"

Step 4: Apply the Configuration

terraform apply -var-file="./workspaces/dev.tfvars"

When prompted to confirm the action, type yes and press Enter.

Complete Terraform Documentation

Project Structure

Here is the overview of the project structure

terraform-project/
├── modules/
│   ├── ec2_instance/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── outputs.tf
│   ├── s3_bucket/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── outputs.tf
│   └── dynamodb_table/
│       ├── main.tf
│       ├── variables.tf
│       └── outputs.tf
├── main.tf
├── providers.tf
├── backend.tf
├── variables.tf
├── outputs.tf
└── workspaces/
    ├── dev.tfvars
    └── prod.tfvars

Complete Project Documentation Link :https://hashnode.com/edit/clx7cisya000409mm8h8j28nr

Accessing the Application

Step 1: Get the Public IP of the EC2 Instance

After the deployment is complete, you can get the public IP address of the EC2 instance.

Step 2: Open the Web Application

Open your web browser and navigate to http://<public_ip>, replacing <public_ip> with the actual public IP address obtained in the previous step. You should see the "Hello, World!" message.

Conclusion

This document provided steps to integrate HashiCorp Vault with an Terraform project to securely retrieve the EC2 private key and use it for provisioning. Adjust the configurations as needed for your specific use case.

Terraformhashicorphashicorp-vaultterraform-cloud#Terraform #AWS #InfrastructureAsCode #Provisioning #Automation #CloudComputing