AWS Native CI/CD For K8s Environments

5 min readDec 8, 2021

The Centre Pompidou | Photo By Denys Nevozhai | Unsplash

The concept of building and running environments and applications to benefit from the distributed computing offered by cloud providers is the focal point in most tech businesses. However, this does not end with just creating virtual servers on the cloud and moving the workload there. The main goal to be reached is a scalable, reliable, and secure workload.

Today, we will see how we maintain the CI/CD (Continuous Integration and Continuous Deployment) processes of our workload running on Amazon EKS (Elastic Kubernetes Service) in a serverless manner on AWS.

We get the advantage of being cloud-native on AWS with keeping things within our VPC (Virtual Private Cloud) along with its security. This helps the workload stays in regulatory compliance and leads to improved security posture.
While keeping components together we also achieve speed in data transfer between them. This accelerates the agility of the process of the SDLC (Software Development Lifecycle).
We can also achieve scalability using serverless architectures.
Therewithal, scaling horizontally achieves increase aggregate workload high availability, and reliability.
This also provides cost optimization while the data stays inside the VPC in a serverless manner. It prohibits spending money on undifferentiated heavy lifting.

The time squandered while maintaining and retaining the CI/CD processes can be utilized on innovating and evolving.

Fig 2: Basic Architecture of Serverless & AWS Native CI/CD

In this pipeline, we will go through some of the key points;

CodeCommit: The codebase needs to be kept secure and scalable. Furthermore, you definitely need fine-grained access control; who can push which branch? who can create a merge request and who can merge it? Which branches need to be protected and who can create branches with which prefix? And codebase needs to be easily integrated with CI/CD components of your workload. CodeCommit is a managed, secure, and highly scalable service that hosts private git repositories within your VPC.

Doing everything like the usual SCM roles, authorizations, protected branches, merge request approval policy, and more through IAM Policies actually supports flexibility and innovation on your development team.
For a starting point, these IAM User Groups can be created;

Viewer
Developer
Maintainer
Approver
Specific-CodeCommit-Repositories (each for every logical group of repos)

Therefore, a fine-grained structure can be created by using these IAM Groups together.

Other than these, generic IAM Policies are also needed for common usage like protected branches;

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DenyBrachDeleting",
            "Effect": "Deny",
            "Action": "codecommit:DeleteBranch",
            "Resource": "arn:aws:codecommit:*:123456789012:*",
            "Condition": {
                "ForAnyValue:StringEquals": {
                    "codecommit:References": [
                        "refs/heads/master",
                        "refs/heads/prod",
                    ]
                }
            }
        }
    ]
}

CodeBuild: The main component of our setup is the AWS CodeBuild, which brings consistency to each build with an isolated environment. Also, it provides less infrastructure complexity with simple setup and maintenance.

Custom images for the building environment can be created regarding the needs. Also, managing releases on CodeBuild by installing the helm and kubectl tools. Passwords, secrets, or any parameter can be kept on AWS Secrets Manager or AWS Systems Manager Parameter Store; and can be used in buildspec.yaml (which includes all the commands, environment variables, passwords, etc.).

After the CodeBuild run;

reports can be generated with JunitXml or CucumberJson format
artifacts can be sent to the next steps of the CodePipeline flow or S3 Bucket
paths or files can be marked as a cached path that can be used on the next CodeBuild runs.

Simple build spec file for building an image and releasing to AWS EKS Cluster;
Assuming source repository includes related Dockerfile and Helm Chart.

version: 0.2env:
  variables:
    IMAGE_REPO_NAME: "calculator-app"
    IMAGE_TAG: "latest"phases:
  pre_build:
    commands:
      - $(aws ecr get-login --region $AWS_DEFAULT_REGION)
  build:
    commands:
      - docker build -f Dockerfile.build_env -t $IMAGE_REPO_NAME:latest .
      - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
      - docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
  post_build:
    commands:
      - aws eks update-kubeconfig --name $EKS_CLUSTER_NAME --region $AWS_DEFAULT_REGION
      - helm upgrade -i calculator-app calc-chart --set deploy.tag=$IMAGE_TAG

CodePipeline: It automates the build, test, and deploy stages of your CI/CD Pipeline every time there is a code change. It can trigger Lambda functions, Step functions, CloudFormation, ElasticBeanStalk. It costs one dollar for each active (ran at least once a month) pipeline. So you got them for peanuts! Despite the fact that CodePipeline is an Amazon Web Services utility, it is not limited to Amazon’s cloud. CodePipeline can send information about the execution, and this information can include aspects of your developer tool resources, including repository contents, build and deployment statuses, and pipeline executions.

Fig 3: CodePipeline Events That Can Trigger Notifications

ChatBot: The Chatbot service handles the integration of AWS services with Slack channels. It needs an SNS Topic to send CodePipeline events. It saves us the hassle of custom lambda functions just to process SNS events.
On the other hand, it can also run commands on AWS Services like Lambda.

Conclusion

Unlike other system architectures, we have built an AWS Native structure here. We took advantage of the features of managed services and achieved a scalable and durable CI/CD process.

In this way, you will be the one configuring the pipeline, not managing the underlying infrastructure.

AWS Native CI/CD For K8s Environments

Conclusion

References

Written by Engin Can Höke