Step 3: Create the private EKS cluster

Step 3.1: Create a SSM bastion host to manage the EKS cluster

<aside> 📝

Resources needed for the bastion host:

IAM Role & Instance Profile
Security group
Key pairs
EC2 instance </aside>

Setup the bastion module, with vpc module’s outputs (VPC ID, subnet IDs):

/* modules/**vpc**/outputs.tf */
****
output "vpc_id" {
  value = aws_vpc.main_vpc.id
}

output "subnet_ids" {
  value = {
    eks1    = aws_subnet.eks1_subnet.id
    eks2    = aws_subnet.eks2_subnet.id
    rds1    = aws_subnet.rds1_subnet.id
    rds2    = aws_subnet.rds2_subnet.id
    alb1    = aws_subnet.alb1_subnet.id
    alb2    = aws_subnet.alb2_subnet.id
    bastion = aws_subnet.bastion_subnet.id
    natgw   = aws_subnet.natgw_subnet.id
  }
}

### CREATE A NEW MODULE: **bastion** ###
/* modules/**bastion**/variables.tf */

# Referencing from root
variable "project_name" {
  type = string
}
variable "vpc_id" {
  type = string
}
variable "subnet_ids" {
  type = map(string)
  validation {
    condition = (
      contains(keys(var.subnet_ids), "bastion")
    )
    error_message = "The subnet_ids must contain 'bastion'"
  }
}

/* main.tf */

module "bastion" {
  source       = "./modules/bastion"
  project_name = local.project_name
  vpc_id       = module.vpc.vpc_id
  subnet_ids   = module.vpc.subnet_ids
}

Create an IAM role for the SSM instance, with the following policies:

Policy for Session Manager:AmazonSSMManagedInstanceCore
Custom inline policy for kubectl (https://docs.aws.amazon.com/eks/latest/userguide/create-kubeconfig.html): eks:DescribeCluster

/* modules/**bastion**/main.tf */

# IAM Role, Policies, and the Instance Profile
resource "aws_iam_role" "bastion_eks_role" {
  name = "${var.project_name}-bastion-eks-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Sid    = ""
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      },
    ]
  })
}

resource "aws_iam_role_policy_attachment" "ssm_policy_bastion_eks" {
  role       = aws_iam_role.bastion_eks_role.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}

resource "aws_iam_role_policy" "eks_describe_cluster_policy" {
  name = "EKSDescribeCluster"
  role = aws_iam_role.bastion_eks_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "eks:DescribeCluster"
        ]
        Resource = "*"
      }
    ]
  })
}

resource "aws_iam_instance_profile" "bastion_eks_profile" {
  name = "${var.project_name}-bastion-eks-profile"
  role = aws_iam_role.bastion_eks_role.name
}

Create a security group for the bastion host:

Inbound rule: default none
Outbound rule: HTTPS (port 443) - Source: Everywhere IPv4 (0.0.0.0/0) → Port to use SSM

# Security Group
resource "aws_security_group" "bastion_eks_sg" {
  name        = "${var.project_name}-bastion-eks-sg"
  description = "SG of bastion host for EKS cluster"
  vpc_id      = var.vpc_id

  tags = {
    Name = "${var.project_name}-bastion-eks-sg"
  }
}

resource "aws_vpc_security_group_egress_rule" "bastion_eks_allow_https" {
  security_group_id = aws_security_group.bastion_eks_sg.id
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "tcp"
  from_port         = 443
  to_port           = 443
}

Create a key pair from our local machine’s console, then reference the public key:

# Replace "~" with "$HOME" on Windows, as PowerShell does set the "$HOME" variable automatically (or use any directory you desire)
ssh-keygen -t rsa -f ~/.ssh/eks-demo-bastion-eks

# Key pair
resource "aws_key_pair" "bastion_eks" {
  key_name   = "${var.project_name}-bastion-eks"
  public_key = file(pathexpand("~/.ssh/eks-demo-bastion-eks.pub"))
}

Create the EC2 access instance:

AMI & Architecture: Amazon Linux 2023 (should have SSM agent preinstalled) & 64-bit Arm (arm64)
Instance type: t4g.nano
Key pair: (created above)
Network settings:
- VPC & Subnet: VPC and the Bastion subnet on step 2
- Security group: the security group just created
EBS Storage: 10 GiB gp3
Advanced details:
- IAM Instance profile: the IAM role just created

# AWS AMI used for the bastions
data "aws_ami" "al2023-arm64" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["al2023-ami-2023.*-arm64"]
  }
}

# EC2 Instance
resource "aws_instance" "bastion_eks" {
  ami                    = data.aws_ami.al2023-arm64.id
  instance_type          = "t4g.nano"
  key_name               = aws_key_pair.bastion_eks.key_name
  subnet_id              = var.subnet_ids.bastion
  vpc_security_group_ids = [aws_security_group.bastion_eks_sg.id]
  iam_instance_profile   = aws_iam_instance_profile.bastion_eks_profile.name

  root_block_device {
    volume_size           = 10
    volume_type           = "gp3"
    encrypted             = true
    delete_on_termination = true
  }

  tags = {
    Name = "${var.project_name}-bastion-eks"
  }
}

We can output the bastion’s instance ID so we can SSH into it:

/* modules/**ecr**/**outputs.tf** */
****
output "bastion_eks_instance_id" {
  value = aws_instance.bastion_eks.id
}

/* outputs.tf (root) */

output "bastion_eks_instance_id" {
  value = module.bastion.bastion_eks_instance_id
}

Apply Terraform changes:

# Needed as we added a new module "bastion"
terraform init

terraform validate && terraform fmt
terraform plan -out tf.plan
terraform apply "tf.plan"

Before create a session from the local machine to the bastion host:

Install SSM plugin for the AWS CLI on local machine: https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html

Enable SSH connections over SSM, following the guide here: https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-getting-started-enable-ssh-connections.html

# Add this to SSH config file
## Linux / macOS: ~/.ssh/config
host i-* mi-*
    ProxyCommand sh -c "aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p'"

## Windows: C:\\Users\\<username>\\.ssh\\config
host i-* mi-*
    ProxyCommand C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe "aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters portNumber=%p"

Then using the EC2 access key, SSH over SSM to the bastion host using instance ID:
```
# path_to_access_key = ~/.ssh/eks-demo-bastion-eks
ssh -i <**path_to_access_key**> ec2-user@<**bastion_eks_instance_id**>
```
- If the SSM agent is not detected, try rebooting the bastion host