OptiZen, a healthcare analytics platform, found its manual processes time-consuming and an obstacle to innovation. To manage its operations with complex CI/CD pipelines across multiple AWS accounts, it adopted a DevOps approach. It streamlined its operations by automating infrastructure provisioning and deployment using GitHub Actions and Terraform.
This case study explores how OptiZen achieved a more efficient, secure, and scalable CI/CD process and highlights the technologies used.
Key Challenges: CI/CD Challenges in a Multi-Account, Multi-Stack Environment
OptiZen faced significant challenges in its development and deployment processes, including manual workflows, inconsistent build and test environments, and security risks from storing sensitive credentials. These included:
- Slow Deployment Processes: Manual processes hindered rapid deployment and increased the risk of human error.
- Security Risks: Storing sensitive credentials, such as IAM keys, on GitHub poses a security threat. Managing Diverse Build Environments: The company required a flexible CI/CD solution to support various build types (Docker, Java, GCC/C).
- Efficient Artifact Storage and Management: A reliable solution was needed to store and manage build artifacts across multiple AWS environments and accounts.
They aimed to automate and streamline their CI/CD pipeline to accelerate deployments, enhance security by protecting sensitive information, and support diverse build requirements for multiple application stacks.
Our Solution: A CI/CD Transformation with GitHub Actions and AWS
The team implemented a CI/CD pipeline to overcome the challenges using GitHub Actions, leveraging composite GitHub Actions to meet client requirements. Key components of the solution include:
1.GitHub Actions Task:
Developed <github-org-name>/org-actions-java-build@<version> , a composite GitHub Actions task that supports a wide range of build and deployment operations, including:
- Java Build: Compatible with both Gradle and Maven applications.
- C Build: Supports GCC for compiling C applications.
- Docker Operations: Enables Docker image building and pushing to repositories.
- Artifact Management: Facilitates the upload and management of build artifacts.
- Infrastructure Automation: Implements Terraform plan and apply for managing infrastructure as code.
- Security Scanning: Conducts static application security testing using Snyk.
Example of building applications using composite GitHub actions:
- name: Java Maven App build
uses: <github-org-name>/org-actions-java-build@<version>
with:
cleanMavenCache: ${{ toJson(inputs.cleanMavenCache) || 'false' }}
mavenTestEnabled: ${{ toJson(inputs.mavenTestEnabled) || 'true' }}
snykScanEnabled: ${{ toJson(inputs.snykScanEnabled) || 'false' }}
cleanDockerCache: ${{ toJson(inputs.cleanDockerCache) || 'false' }}
ecrRepository: ${{ env.ecrRepository }}
githubToken: ${{ secrets.YOUR_GITHUB_TOKEN }}
pushDockerImage: ${{ (github.event_name == 'push' && github.ref == 'refs/heads/main') || (github.event_name == 'workflow_dispatch' && toJson(inputs.buildDockerImage) == 'true') && 'true' || 'false' }}
2. AWS Services for Build Artifact Management:
- AWS Identity and Access Management (IAM): To provide secure access control to AWS resources.
- AWS ECR, S3, and CodeArtifact: Utilized to store and manage build artifacts across multiple environments and environment-specific AWS accounts.
3. Automated Testing and Deployment:
By implementing a CI/CD pipeline, we automated testing and deployment on AWS. To accommodate our multi-account AWS environment, we integrated GitHub’s OpenID Connect (OIDC) provider and customized the configuration for each environment.
4. Security Enhancements:
Utilizing OIDC, we managed resources, eliminating the need for long-term AWS access keys and ensuring compliance with security best practices.
Benefits
The implemented solution significantly enhanced efficiency, security, and reliability throughout the development and deployment. Key benefits included:
1. Seamless Build, Test, and Deploy Workflows:
Developers could build, test, and deploy applications directly within GitHub, reducing context-switching and streamlining processes. With the new system in place, OptiZen was able to foster enhanced collaboration and testing, as code reviews and testing could be conducted at any workflow stage, supporting advanced features like matrix builds and multi-container tests.
2. Integrated Ecosystem:
The adoption of the GitHub Actions CI/CD pipeline, integrated with Terraform and AWS services, enabled seamless integration of third-party tools and services into the GitHub repository, simplifying the overall build, test, and deployment processes.
3. Improved Monitoring and Error Management:
Provided robust tools for tracking errors, monitoring new application builds, and measuring performance to ensure quality and reliability.. Provided robust tools for tracking errors, monitoring new application builds, and measuring performance to ensure quality and reliability.
4. Stronger Security Practices:
Leveraged GitHub’s OIDC provider for secure AWS permissions management, eliminating the need for long-term access keys.. Leveraged GitHub’s OIDC provider for secure AWS permissions management, eliminating the need for long-term access keys.
5. Accelerated Deployment and Recovery:
- Achieved higher deployment frequency and reduced failure rates in new releases.
- Shortened lead time for fixes and improved mean time to recovery after incidents.
6. Automated CI/CD with GitHub Actions:
GitHub-hosted CI/CD workflows were automatically triggered by repository events, leveraging GitHub’s virtual machine infrastructure. The live pipeline could be easily monitored and managed through the GitHub Actions console, enabling swift updates and enhanced transparency.
Results
The adoption of the GitHub Actions CI/CD pipeline, integrated with Terraform and AWS services, delivered measurable improvements across multiple performance metrics:
- Increased Efficiency: Accelerated deployment times by 80% and reduced lead time for fixes by 50%.
- Enhanced Security: Minimized configuration drift and strengthened security posture by utilizing GitHub OIDC and AWS IAM.
- Improved Reliability: Achieved a high build success rate of over 95% and reduced MTTR to 45 minutes.
- Cost Optimization: Optimized container builds and storage, leading to a 35% reduction in operational costs.
- Enhanced Quality: Increased test coverage by 70% and ensured consistent infrastructure provisioning with 85% IaC coverage.
This CI/CD pipeline has become a cornerstone of OptiZen ‘s deployment strategy. It delivers rapid, secure, and cost-effective deployments while maintaining high-quality standards.
OptiZen implemented a robust CI/CD pipeline by leveraging GitHub Actions and AWS services, which resulted in faster deployment and improved reliability. The team achieved a more streamlined process by using GitHub OIDC to allow workflows to access resources securely in AWS.
