Step 6: Expose the apps with networking objects and services

Step 6.1: Setting up the AWS Load Balancer Controller (AWS ALB Ingress Controller)

<aside> ℹ️

Official documentation of AWS LCB’s installation guide: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.10/deploy/installation/

</aside>

Configure IAM role and k8s Service Account for the controller:
- Setup an IAM OpenID Connect provider for the cluster:
```
# From the bastion host
eksctl utils associate-iam-oidc-provider --cluster eks-demo-cluster --region us-east-1 --approve
```
- Copy the policy from https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.10.0/docs/install/iam_policy.json, then add it as an IAM policy AWSLoadBalancerControllerIAMPolicy through the console or AWS CLI (from the local machine).
- Copy the ARN of the policy, then create an IAM role and a k8s service account:
```
eksctl create iamserviceaccount \\
	--cluster=eks-demo-cluster \\
	--namespace=kube-system \\
	--name=aws-load-balancer-controller \\
	--attach-policy-arn=arn:aws:iam::<AWS_ACCOUNT_ID>:policy/AWSLoadBalancerControllerIAMPolicy \\
	--override-existing-serviceaccounts \\
	--region us-east-1 \\
	--approve
```
Install the cert-manager (resolving SSL/TLS certificates from Load Balancer):

<aside> ☝

Since the EKS cluster is private and has no access to the internet, you cannot simply apply the manifest file like this:

kubectl apply -f <https://github.com/cert-manager/cert-manager/releases/download/v1.12.3/cert-manager.yaml>

The images in the manifest file is not from your AWS ECR private repository: For example, quay.io/jetstack/cert-manager-cainjector:v1.12.3

This will apply to all the other helper manifests used in this project (including the next steps)

</aside>
- Download the manifest file: https://github.com/cert-manager/cert-manager/releases/download/v1.12.3/cert-manager.yaml
- Create 3 private ECR repositories called:
  - jetstack/cert-manager-webhook (in cert-manager.yaml)
  - jetstack/cert-manager-cainjector (in cert-manager.yaml)
  - jetstack/cert-manager-controller (in cert-manager.yaml)
- From the local machine pull the images from what the manifest file indicated, then push to your own repository:
```
docker pull --platform linux/arm64 quay.io/jetstack/cert-manager-webhook:v1.12.3

docker tag quay.io/jetstack/cert-manager-webhook:v1.12.3 <aws_id>.dkr.ecr.us-east-1.amazonaws.com/jetstack/cert-manager-webhook:v1.12.3

docker push <aws_id>.dkr.ecr.us-east-1.amazonaws.com/jetstack/cert-manager-webhook:v1.12.3

# Do the same for the other images
```
- Modify the cert-manager.yaml manifest file:
```
# cert-manager.yaml
# Change the images to the private ECR images
image: <aws_id>.dkr.ecr.us-east-1.amazonaws.com/jetstack/cert-manager-webhook:v1.12.3
image: <aws_id>.dkr.ecr.us-east-1.amazonaws.com/jetstack/cert-manager-cainjector:v1.12.3
image: <aws_id>.dkr.ecr.us-east-1.amazonaws.com/jetstack/cert-manager-controller:v1.12.3
```
- Then apply the cert-manager to the cluster:
```
# From bastion host
mkdir helpers

# From local machine, copy the file through SCP to the bastion host
scp -i <path_to_access_key> ./cert-manager.yaml ec2-user@<instance_id>:/home/ec2-user/helpers

# From bastion host
kubectl apply -f ./helpers/cert-manager.yaml
# Verify the certification manager server
kubectl get deployment -n cert-manager
```

Install the Load Balancer Controller:

Download the manifest files:
- https://github.com/kubernetes-sigs/aws-load-balancer-controller/releases/download/v2.10.0/v2_10_0_full.yaml
- https://github.com/kubernetes-sigs/aws-load-balancer-controller/releases/download/v2.10.0/v2_10_0_ingclass.yaml
Create a private ECR repository called:
- eks/aws-load-balancer-controller (in v2_10_0_full.yaml)

From the local machine pull the images from what the manifest file indicated, then push to your own repository:

docker pull --platform linux/arm64 public.ecr.aws/eks/aws-load-balancer-controller:v2.10.0

docker tag public.ecr.aws/eks/aws-load-balancer-controller:v2.10.0 <aws_id>.dkr.ecr.us-east-1.amazonaws.com/eks/aws-load-balancer-controller:v2.10.0

docker push <aws_id>.dkr.ecr.us-east-1.amazonaws.com/eks/aws-load-balancer-controller:v2.10.0

Modify the v2_10_0_full.yaml manifest file:

# v2_10_0_full.yaml
# Change the cluster name and the images to the private ECR images
containers:
- args:
	- --cluster-name=eks-demo-cluster
	...
	image: <aws_id>.dkr.ecr.us-east-1.amazonaws.com/eks/aws-load-balancer-controller:v2.10.0

# Remove the ServiceAccount from the manifest to avoid overwrite
# (already created using eksctl create iamserviceaccount)
---
apiVersion: v1
kind: ServiceAccount
metadata:
  # ...
  name: aws-load-balancer-controller
  namespace: kube-system
  # ...
---

Then apply the LBC to the cluster:

# From local machine, copy the file through SCP to the bastion host
scp -i <path_to_access_key> ./v2_10_0_full.yaml ec2-user@<instance_id>:/home/ec2-user/helpers/aws-lbc-full.yaml
scp -i <path_to_access_key> ./v2_10_0_ingclass.yaml ec2-user@<instance_id>:/home/ec2-user/helpers/aws-lbc-ingclass.yaml

# From bastion host
kubectl apply -f ./helpers/aws-lbc-full.yaml
## Wait a few seconds for first manifest file to load, then apply the ingress class manifest
kubectl apply -f ./helpers/aws-lbc-ingclass.yaml

# Verify the metrics server
kubectl get deployment -n kube-system aws-load-balancer-controller

Step 6.2: Create more VPC resources for the Application Load Balancer

Create a VPC Endpoint to connect the AWS ELB service to the private cluster:
- Service: com.amazonaws.us-east-1.elasticloadbalancing
- VPC & Subnet: EKS subnet 1 & EKS subnet 2
- Security group: ClusterSharedNodeSecurityGroup
Create an ACM certificate to redirect your HTTP (80) traffic from your cluster to the internet with HTTPS (443), using SSL redirect:
- Request a public certificate from AWS Certificate Manager:
  - FQDN: Your domain name - In this example, I use eks-demo.bachhv.com
  - Validation method: DNS validation
- After request, ACM will give a CNAME record requiring validation. Go to whatever your domain management site (e.g. Route 53, Cloudflare), and add the CNAME record.
- When the certificate is validated and issued, copy the certificate’s ARN for the next step (using it for the AWS LBC Ingress)

Step 6.3: Create network objects in the cluster and the Application Load Balancer

Creating the manifest to deploy the service for each app:

# svc-frontend.yaml
apiVersion: v1
kind: Service
metadata:
  name: coffeeshop-frontend-svc
spec:
  selector:
    app: coffeeshop-frontend
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080

Creating the manifest to deploy the ingress:

Pick a name for the ALB, and keep this in mind, we will need this to create a CloudFront distribution later. In my case, my ALB’s name is eks-demo-alb

# ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: coffeeshop-ingress
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/load-balancer-name: eks-demo-alb
    alb.ingress.kubernetes.io/subnets: <subnet_alb_1_id>,<subnet_alb_2_id>
    alb.ingress.kubernetes.io/certificate-arn: <certificate_arn>
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/ssl-redirect: '443'
spec:
  ingressClassName: alb
  rules:
    - host: eks-demo.bachhv.com # Domain that will eventually be used
	    http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: coffeeshop-frontend-svc
                port:
                  number: 8080
          - path: /api/customer
            pathType: Prefix
            backend:
              service:
                name: coffeeshop-customer-svc
                port:
                  number: 4000
          - path: /api/shopping
            pathType: Prefix
            backend:
              service:
                name: coffeeshop-shopping-svc
                port:
                  number: 4000

Applying the manifests. When the ingress is created, the Load Balancer Controller will also create an ALB on AWS based on the rules of the ingress manifest. It should take 5-10 minutes for the controller to create:

# From local machine, copy all services and ingress manifests
scp -i <path_to_access_key> ./svc-*.yaml ./ingress.yaml ec2-user@<instance_id>:/home/ec2-user/manifests

# From bastion host
kubectl apply -f './manifests/svc-*.yaml'
kubectl apply -f './manifests/ingress.yaml'