CI/CD Pipelines

Duration: 130 min

Continuous Integration and Continuous Deployment (CI/CD) automate the process of building, testing, and deploying code. This module covers GitHub Actions, AWS CodePipeline, CodeBuild, and CodeDeploy—essential tools for modern DevOps workflows.

CI/CD Principles

CI/CD pipelines follow these principles:

Continuous Integration: Code changes are automatically built and tested
Continuous Deployment: Tested code is automatically deployed to production
Fast Feedback: Developers get immediate feedback on code quality
Automation: Manual steps are eliminated to reduce errors
Repeatability: Same process runs consistently every time

GitHub Actions

GitHub Actions is a CI/CD platform integrated with GitHub repositories:

name: Build and Deploy

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Set up Node.js
      uses: actions/setup-node@v3
      with:
        node-version: '18'
    
    - name: Install dependencies
      run: npm ci
    
    - name: Run linter
      run: npm run lint
    
    - name: Run tests
      run: npm test
    
    - name: Build application
      run: npm run build
    
    - name: Upload coverage
      uses: codecov/codecov-action@v3
      with:
        files: ./coverage/coverage-final.json

  deploy:
    needs: build
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Deploy to production
      run: |
        echo "Deploying to production..."
        # Deployment commands here

AWS CodePipeline

CodePipeline orchestrates CI/CD workflows on AWS:

# Create a CodePipeline using AWS CLI
aws codepipeline create-pipeline \
  --cli-input-json file://pipeline.json

# Example pipeline.json structure
cat > pipeline.json << 'EOF'
{
  "pipeline": {
    "name": "my-app-pipeline",
    "roleArn": "arn:aws:iam::123456789:role/CodePipelineRole",
    "stages": [
      {
        "name": "Source",
        "actions": [
          {
            "name": "SourceAction",
            "actionTypeId": {
              "category": "Source",
              "owner": "AWS",
              "provider": "CodeCommit",
              "version": "1"
            },
            "configuration": {
              "RepositoryName": "my-repo",
              "BranchName": "main"
            },
            "outputArtifacts": [
              {"name": "SourceOutput"}
            ]
          }
        ]
      },
      {
        "name": "Build",
        "actions": [
          {
            "name": "BuildAction",
            "actionTypeId": {
              "category": "Build",
              "owner": "AWS",
              "provider": "CodeBuild",
              "version": "1"
            },
            "configuration": {
              "ProjectName": "my-build-project"
            },
            "inputArtifacts": [
              {"name": "SourceOutput"}
            ],
            "outputArtifacts": [
              {"name": "BuildOutput"}
            ]
          }
        ]
      }
    ]
  }
}
EOF

AWS CodeBuild

CodeBuild compiles source code, runs tests, and produces deployable artifacts:

# buildspec.yml for CodeBuild
version: 0.2

phases:
  pre_build:
    commands:
      - echo "Logging in to Amazon ECR..."
      - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
      - REPOSITORY_URI=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/my-app
      - COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)
      - IMAGE_TAG=${COMMIT_HASH:=latest}

  build:
    commands:
      - echo "Building the Docker image on `date`"
      - docker build -t $REPOSITORY_URI:latest .
      - docker tag $REPOSITORY_URI:latest $REPOSITORY_URI:$IMAGE_TAG

  post_build:
    commands:
      - echo "Pushing the Docker images on `date`"
      - docker push $REPOSITORY_URI:latest
      - docker push $REPOSITORY_URI:$IMAGE_TAG
      - echo "Writing image definitions file..."
      - printf '[{"name":"my-app","imageUri":"%s"}]' $REPOSITORY_URI:$IMAGE_TAG > imagedefinitions.json

artifacts:
  files: imagedefinitions.json

AWS CodeDeploy

CodeDeploy automates application deployments to EC2, on-premises servers, or Lambda:

# appspec.yml for CodeDeploy
version: 0.0
Resources:
  - TargetService:
      Type: AWS::ElasticLoadBalancingV2::TargetGroup
      Properties:
        TargetGroupName: !Ref TargetGroupName
        HealthCheckPath: /health
        Matcher:
          HttpCode: 200

Hooks:
  - BeforeAllowTraffic: "pre-traffic-hook"
  - AfterAllowTraffic: "post-traffic-hook"

Permissions:
  - Object: "/"
    Pattern: "**"
    Owner: ec2-user
    Group: ec2-user
    Mode: 755
    Type:
      - directory
  - Object: "/"
    Pattern: "**"
    Owner: ec2-user
    Group: ec2-user
    Mode: 644
    Type:
      - file

GitHub Actions Workflow Examples

Docker Build and Push

name: Build and Push Docker Image

on:
  push:
    tags:
      - 'v*'

jobs:
  build:
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v2
    
    - name: Login to ECR
      uses: aws-actions/amazon-ecr-login@v1
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: us-east-1
    
    - name: Build and push
      uses: docker/build-push-action@v4
      with:
        context: .
        push: true
        tags: |
          ${{ secrets.ECR_REGISTRY }}/my-app:latest
          ${{ secrets.ECR_REGISTRY }}/my-app:${{ github.ref_name }}

Deploy to ECS

name: Deploy to ECS

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v2
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: us-east-1
    
    - name: Update ECS service
      run: |
        aws ecs update-service \
          --cluster production \
          --service my-app \
          --force-new-deployment

Pipeline Best Practices

Artifact Management

# Store build artifacts in S3
aws s3 cp build-output.zip s3://my-artifacts/builds/$(date +%Y%m%d-%H%M%S)/

# Retrieve artifacts for deployment
aws s3 cp s3://my-artifacts/builds/latest/build-output.zip .
unzip build-output.zip

Secrets Management

# Use GitHub Secrets for sensitive data
- name: Deploy with credentials
  env:
    DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
    API_KEY: ${{ secrets.API_KEY }}
  run: |
    ./deploy.sh

Notifications

# Slack notification on failure
- name: Notify Slack on failure
  if: failure()
  uses: slackapi/slack-github-action@v1
  with:
    webhook-url: ${{ secrets.SLACK_WEBHOOK }}
    payload: |
      {
        "text": "Pipeline failed for ${{ github.repository }}",
        "blocks": [
          {
            "type": "section",
            "text": {
              "type": "mrkdwn",
              "text": "Build failed: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
            }
          }
        ]
      }

Testing in CI/CD

# Comprehensive testing strategy
jobs:
  test:
    runs-on: ubuntu-latest
    
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Run unit tests
      run: npm run test:unit
    
    - name: Run integration tests
      run: npm run test:integration
      env:
        DATABASE_URL: postgresql://postgres:postgres@postgres:5432/test
    
    - name: Run e2e tests
      run: npm run test:e2e
    
    - name: Upload coverage
      uses: codecov/codecov-action@v3

❓ What is the main goal of Continuous Integration?

To deploy code to production automatically To automatically build and test code changes frequently To manage version control To monitor application performance

❓ What does GitHub Actions use to define workflows?

YAML files in the .github/workflows directory JSON configuration in the repository root Python scripts in the .github directory Bash scripts in the scripts directory

❓ What is the purpose of AWS CodeBuild?

To manage source code repositories To orchestrate CI/CD pipelines To compile code, run tests, and produce deployable artifacts To deploy applications to servers

❓ What file does AWS CodeDeploy use to define deployment steps?

buildspec.yml appspec.yml pipeline.json deploy.yaml

❓ How should sensitive data like API keys be handled in CI/CD pipelines?

Store in secrets management (GitHub Secrets, AWS Secrets Manager) Hardcode in configuration files Pass as command-line arguments Store in environment files in the repository