In modern software development, Continuous Integration and Continuous Deployment (CI/CD) have become essential practices for delivering high-quality applications efficiently. CI/CD pipelines automate the process of integrating code changes, testing, and deploying applications, reducing errors and accelerating the development cycle.
This tutorial will guide you through building a robust CI/CD pipeline using FastAPI and GitHub Actions. We will explore each stage of the pipeline, best practices, security considerations, and troubleshooting tips to ensure a seamless workflow.
Understanding CI/CD and Its Importance
Before diving into the implementation, let’s briefly understand what CI/CD is and why it’s crucial:
- Continuous Integration (CI): The practice of frequently integrating code changes into a shared repository, automatically building, and testing them to catch issues early.
- Continuous Deployment (CD): The process of automatically deploying tested code to production, ensuring quick and reliable software releases.
Benefits of CI/CD include:
- Faster software delivery
- Improved code quality and security
- Reduced manual intervention and deployment risks
- Enhanced collaboration among development teams
Overview of the CI/CD Pipeline Stages
A complete CI/CD pipeline consists of multiple stages that automate the software delivery process:
1. Development and Code Commit
- Follow a clear branching strategy (e.g., Gitflow) to manage feature development, bug fixes, and releases efficiently.
- Use meaningful commit messages and pull requests to document changes.
- Enforce coding standards with linting tools such as Flake8 for Python.
2. Build Stage
- Automate the build process to ensure consistency across different environments.
- Utilise containerisation tools like Docker to package applications with their dependencies.
- Implement dependency management to avoid conflicts.
3. Testing Stage
- Integrate automated testing into the pipeline:
- Unit Tests: Verify individual components using pytest.
- Integration Tests: Ensure different components work together correctly.
- Security Scans: Use tools like Bandit to check for vulnerabilities.
- Generate test reports for better debugging and visibility.
4. Deployment Stage
- Use GitHub Actions to automate deployments to various environments (staging, production).
- Implement deployment strategies such as blue-green deployments or canary releases to minimise downtime and risk.
- Set up rollback mechanisms in case of failed deployments.
Prerequisites
Before we begin, ensure you have:
- Python 3.11 installed
- Git installed
- A GitHub account
- Basic knowledge of Python and REST APIs
Project Overview
We’ll build a simple calculator API with these features:
- Basic arithmetic operations (add, subtract, multiply, divide)
- REST API endpoints
- Automated testing
- Docker containerisation
- CI/CD pipeline with GitHub Actions
Step 1: Setting Up Your Development Environment
First, let’s set up a proper Python development environment.
Installing Python 3.11# macOS (using Homebrew) brew install python@3.11
# Ubuntu/Debian sudo apt update sudo apt install software-properties-common sudo add-apt-repository ppa:deadsnakes/ppa sudo apt update sudo apt install python3.11 python3.11-venv
Creating the Project Structure
# Create project directory mkdir simple-cicd-demo cd simple-cicd-demo # Create virtual environment python3.11 -m venv venv # Activate virtual environment source venv/bin/activate # On Unix/macOS # or venv\Scripts\activate # On Windows # Create project structure mkdir -p src tests .github/workflows
Step 2: Setting Up the Project Files
1. Create requirements.txt
[code language="text"] fastapi==0.95.2 uvicorn==0.22.0 pytest==7.0.0 requests==2.26.0 pydantic==1.10.7
2. Install Dependencies
pip install -r requirements.txt
3. Create Calculator Logic (src/calculator.py)
class Calculator:
@staticmethod
def add(x: float, y: float) -> float:
return x + y
@staticmethod
def subtract(x: float, y: float) -> float:
return x - y
@staticmethod
def multiply(x: float, y: float) -> float:
return x * y
@staticmethod
def divide(x: float, y: float) -> float:
if y == 0:
raise ValueError("Cannot divide by zero")
return x / y
4. Create FastAPI Application (src/main.py)
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from .calculator import Calculator
app = FastAPI()
calculator = Calculator()
class CalculationRequest(BaseModel):
x: float
y: float
@app.post("/add")
def add(request: CalculationRequest):
return {"result": calculator.add(request.x, request.y)}
@app.post("/subtract")
def subtract(request: CalculationRequest):
return {"result": calculator.subtract(request.x, request.y)}
@app.post("/multiply")
def multiply(request: CalculationRequest):
return {"result": calculator.multiply(request.x, request.y)}
@app.post("/divide")
def divide(request: CalculationRequest):
try:
return {"result": calculator.divide(request.x, request.y)}
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
5. Create Tests (tests/test_calculator.py)
import pytest from src.calculator import Calculator def test_add(): assert Calculator.add(2, 3) == 5 assert Calculator.add(-1, 1) == 0 def test_subtract(): assert Calculator.subtract(5, 3) == 2 assert Calculator.subtract(1, 1) == 0 def test_multiply(): assert Calculator.multiply(2, 3) == 6 assert Calculator.multiply(-2, 3) == -6 def test_divide(): assert Calculator.divide(6, 2) == 3 assert Calculator.divide(5, 2) == 2.5 def test_divide_by_zero(): with pytest.raises(ValueError): Calculator.divide(5, 0)
Step 3: Setting Up CI/CD
1. Create Dockerfile
FROM python:3.11-slim WORKDIR /app COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt COPY src/ src/ CMD ["uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8000"]
2. Create GitHub Actions Workflow (.github/workflows/main.yml)
name: CI/CD Pipeline on: push: branches: [ main ] pull_request: branches: [ main ] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Set up Python uses: actions/setup-python@v2 with: python-version: '3.11' - name: Install dependencies run: | python -m pip install --upgrade pip pip install -r requirements.txt - name: Run tests run: | pytest tests/ build: needs: test runs-on: ubuntu-latest if: github.ref == 'refs/heads/main' steps: - uses: actions/checkout@v2 - name: Build Docker image run: docker build -t calculator-api .
Step 4: Running the Application
Local Development
# Make sure virtual environment is activated source venv/bin/activate # Run tests pytest tests/ # Start the application uvicorn src.main:app --reload
Testing the API
Once the application is running, you can test it using curl:
curl -X POST -H "Content-Type: application/json" -d'{"x": 10, "y": 5}' http://localhost:8000/add
Expected response:
[code language="json"]
{"result": 15}
Step 5: Setting Up GitHub Repository
1. Create a new repository on GitHub
2. Initialise and push your code:
git init git add . git commit -m "Initial commit" git branch -M main git remote add origin git push -u origin main
Understanding the CI/CD Pipeline
Our pipeline performs these steps automatically:
- Continuous Integration
- Runs on every push and pull request
- Executes all tests
- Ensures code quality
- Continuous Deployment
- Builds Docker image
- Ready for deployment to any cloud platform
Troubleshooting Common Issues
Import Errors
If you see import errors, ensure:
- Virtual environment is activated
- All dependencies are installed
- You’re running commands from the project root
Version Conflicts
If you encounter version conflicts:
- Delete the virtual environment
- Recreate it using the steps above
- Install dependencies fresh from requirements.txt
Next Steps
To extend this project, consider:
- Adding more test cases
- Implementing code coverage reporting
- Adding API documentation using Swagger
- Setting up automatic deployment to a cloud platform
- Adding branch protection rules in GitHub
Conclusion
You now have a working CI/CD pipeline with:
- Automated testing
- Docker containerisation
- GitHub Actions integration
- A solid foundation for further development
This setup provides a great starting point for learning CI/CD concepts and can be extended for more complex applications.