Airbyte Deployment

Overview

This example shows how to deploy Airbyte on Qovery using Terraform. Airbyte is deployed using Helm with a PostgreSQL database and proxy for authentication. This is a complete, production-ready example.

This is based on the qovery-airbyte repository - a real-world example of deploying a complex application stack with Qovery.

What You’ll Deploy

PostgreSQL Database: Managed database for Airbyte metadata
Airbyte (Helm): Complete Airbyte deployment from official Helm chart
Proxy Application: Authentication proxy for Airbyte web UI
Deployment Stages: Ordered deployment (Database → App → Proxy)

Prerequisites

Before deploying, gather these IDs from Qovery Console:

# Required variables
export TF_VAR_qovery_access_token="your-api-token"
export TF_VAR_qovery_organization_id="your-org-id"
export TF_VAR_qovery_project_id="your-project-id"
export TF_VAR_qovery_cluster_id="your-cluster-id"
export TF_VAR_qovery_github_token_id="your-github-token-id"

File Structure

qovery-airbyte/
├── main.tf
├── variables.tf
└── airbyte-values.yaml

variables.tf

variable "qovery_access_token" {
  description = "Qovery API token"
  type        = string
  sensitive   = true
}

variable "qovery_organization_id" {
  description = "Qovery Organization ID"
  type        = string
}

variable "qovery_project_id" {
  description = "Qovery Project ID"
  type        = string
}

variable "qovery_cluster_id" {
  description = "Qovery Cluster ID"
  type        = string
}

variable "qovery_github_token_id" {
  description = "GitHub token for Qovery"
  type        = string
}

variable "airbyte_helm_version" {
  description = "Airbyte Helm chart version"
  type        = string
  default     = "1.7.1"
}

variable "airbyte_service_name" {
  description = "Airbyte service name"
  type        = string
  default     = "Airbyte"
}

variable "qovery_airbyte_web_app_proxy_basic_auth" {
  description = "Basic Auth for Airbyte web app proxy (htpasswd format)"
  type        = string
  default     = ""
}

main.tf

terraform {
  required_providers {
    qovery = {
      source  = "qovery/qovery"
      version = "~> 0.48.2"
    }
  }
}

provider "qovery" {
  token = var.qovery_access_token
}

# Read Airbyte Helm values from file
data "local_file" "airbyte_values" {
  filename = "${path.module}/airbyte-values.yaml"
}

# Create environment for Airbyte
resource "qovery_environment" "airbyte" {
  project_id = var.qovery_project_id
  cluster_id = var.qovery_cluster_id
  name       = "airbyte"
  mode       = "PRODUCTION"

  deployment_stage_order = [
    "DATABASE",
    "APP",
    "PROXY"
  ]
}

# Create PostgreSQL database for Airbyte
resource "qovery_database" "airbyte_db" {
  environment_id = qovery_environment.airbyte.id
  name           = "airbyte-db"
  type           = "POSTGRESQL"
  version        = "17"
  mode           = "MANAGED"
  storage        = 20
  accessibility  = "PRIVATE"

  deployment_stage_id = qovery_environment.airbyte.deployment_stage_ids["DATABASE"]
}

# Add Airbyte Helm repository
resource "qovery_helm_repository" "airbyte" {
  organization_id = var.qovery_organization_id
  name            = "airbyte"
  kind            = "OCI_DOCKER_HUB"
  url             = "https://airbytehq.github.io/helm-charts"
}

# Deploy Airbyte using Helm
resource "qovery_helm" "airbyte" {
  environment_id = qovery_environment.airbyte.id
  name           = var.airbyte_service_name

  source = {
    helm_repository = {
      repository_id = qovery_helm_repository.airbyte.id
      chart_name    = "airbyte"
      chart_version = var.airbyte_helm_version
    }
  }

  # Allow Airbyte to create cluster-wide resources
  allow_cluster_wide_resources = true

  # Override default values
  values_override = {
    file = {
      raw = data.local_file.airbyte_values.content
    }
  }

  # Configure database connection using Qovery environment variables
  environment_variables = [
    {
      key   = "DATABASE_HOST"
      value = qovery_database.airbyte_db.internal_host
    },
    {
      key   = "DATABASE_PORT"
      value = tostring(qovery_database.airbyte_db.port)
    },
    {
      key   = "DATABASE_DB"
      value = "airbyte"
    },
    {
      key   = "DATABASE_USER"
      value = qovery_database.airbyte_db.login
    }
  ]

  secrets = [
    {
      key   = "DATABASE_PASSWORD"
      value = qovery_database.airbyte_db.password
    }
  ]

  deployment_stage_id = qovery_environment.airbyte.deployment_stage_ids["APP"]

  depends_on = [
    qovery_database.airbyte_db,
    qovery_helm_repository.airbyte
  ]
}

# Deploy proxy application for Airbyte web app
resource "qovery_application" "airbyte_webapp_proxy" {
  environment_id = qovery_environment.airbyte.id
  name           = "airbyte-webapp-proxy"

  git_repository = {
    url           = "https://github.com/Qovery/simple-example-with-htt-auth"
    branch        = "main"
    root_path     = "/"
    git_token_id  = var.qovery_github_token_id
  }

  build_mode      = "DOCKER"
  dockerfile_path = "Dockerfile"

  cpu    = 100
  memory = 128

  min_running_instances = 1
  max_running_instances = 1

  ports = [{
    internal_port       = 80
    external_port       = 443
    protocol            = "HTTP"
    publicly_accessible = true
    name                = "http"
  }]

  environment_variables = [
    {
      key   = "TARGET"
      value = "http://airbyte-airbyte-webapp-svc.${qovery_environment.airbyte.id}:80"
    },
    {
      key   = "HTPASSWD"
      value = var.qovery_airbyte_web_app_proxy_basic_auth
    }
  ]

  healthchecks = {
    readiness_probe = {
      type = {
        http = {
          path = "/"
          port = 80
        }
      }
      initial_delay_seconds = 30
      period_seconds        = 10
      timeout_seconds       = 5
      success_threshold     = 1
      failure_threshold     = 3
    }

    liveness_probe = {
      type = {
        http = {
          path = "/"
          port = 80
        }
      }
      initial_delay_seconds = 30
      period_seconds        = 10
      timeout_seconds       = 5
      success_threshold     = 1
      failure_threshold     = 3
    }
  }

  deployment_stage_id = qovery_environment.airbyte.deployment_stage_ids["PROXY"]

  depends_on = [qovery_helm.airbyte]
}

airbyte-values.yaml

Create a airbyte-values.yaml file to customize Airbyte configuration:

global:
  serviceAccountName: airbyte-admin
  deploymentMode: oss

  database:
    type: external
    host: ${DATABASE_HOST}
    port: ${DATABASE_PORT}
    database: ${DATABASE_DB}
    user: ${DATABASE_USER}
    password: ${DATABASE_PASSWORD}

webapp:
  enabled: true
  replicaCount: 1
  service:
    type: ClusterIP
    port: 80

server:
  enabled: true
  replicaCount: 1

worker:
  enabled: true
  replicaCount: 1

airbyte-bootloader:
  enabled: true

Deployment Steps

Clone or Create Files

# Option 1: Clone the repository
git clone https://github.com/evoxmusic/qovery-airbyte
cd qovery-airbyte

# Option 2: Create the files manually
# Create main.tf, variables.tf, and airbyte-values.yaml

Set Environment Variables

export TF_VAR_qovery_access_token="your-api-token"
export TF_VAR_qovery_organization_id="your-org-id"
export TF_VAR_qovery_project_id="your-project-id"
export TF_VAR_qovery_cluster_id="your-cluster-id"
export TF_VAR_qovery_github_token_id="your-github-token-id"

Optional: Set Basic Authentication

# Generate basic auth credentials
export TF_VAR_qovery_airbyte_web_app_proxy_basic_auth=$(htpasswd -nb admin yourpassword)

Initialize Terraform

terraform init

Plan and Apply

# Review changes
terraform plan

# Deploy
terraform apply

Access Airbyte

After ~5 minutes, access Airbyte:

# Option 1: Get URL from Qovery Console
# Navigate to the airbyte-webapp-proxy application

# Option 2: Use port-forward
qovery port-forward --application airbyte-webapp-proxy --port 8080:443

# Open http://localhost:8080

Cleanup

terraform destroy

Key Takeaways

This example demonstrates several advanced Terraform patterns:

Multi-Service Deployment

Deploying multiple interconnected services:

Managed PostgreSQL database
Helm chart application
Proxy application for authentication

Deployment Stages

Ensuring proper deployment order:

DATABASE: PostgreSQL deploys first
APP: Airbyte Helm chart deploys after database
PROXY: Authentication proxy deploys last

Dynamic Configuration

Using Qovery’s database connection details in Helm values:

Database host, port, credentials injected as environment variables
Variables referenced in airbyte-values.yaml

External Helm Charts

Integrating third-party Helm repositories:

Adding Airbyte’s official Helm repository
Deploying specific chart versions
Overriding values with custom configuration

Security

Optional basic authentication for web access
Private database accessibility
Secrets stored securely

Health Checks

Proper liveness and readiness probes:

Initial delay to allow services to start
Regular health checks
Automatic restarts on failure

Resource Management

Setting appropriate CPU/memory limits:

Minimal resources for proxy (100 CPU, 128 MB)
Managed database with defined storage
Cluster-wide resource permissions for Airbyte

Troubleshooting

Deployment Takes Too Long

Airbyte deployment typically takes 5-10 minutes:

Database must be ready first
Helm chart pulls multiple images
Airbyte bootloader initializes the database

Check deployment status:

qovery status --watch

Cannot Access Web UI

If you can’t access the Airbyte UI:

Verify all services are deployed
Check proxy application is running
Ensure basic auth credentials are correct
Try port-forwarding locally

Database Connection Issues

If Airbyte can’t connect to the database:

Verify database is in RUNNING state
Check environment variables are set correctly
Review airbyte-values.yaml configuration
Check Airbyte logs for connection errors

Next Steps

GitHub Repository

Complete source code and additional documentation

Airbyte Documentation

Learn more about Airbyte

Helm Configuration

Deploy more Helm charts

Advanced Patterns

Learn advanced Terraform techniques

Getting Started

Examples

Airbyte Deployment

Overview

What You’ll Deploy

Prerequisites

File Structure

variables.tf

main.tf

airbyte-values.yaml

Deployment Steps

Cleanup

Key Takeaways

Troubleshooting

Next Steps

GitHub Repository

Airbyte Documentation

Helm Configuration

Advanced Patterns

Getting Started

Examples

​Overview

​What You’ll Deploy

​Prerequisites

​File Structure

​variables.tf

​main.tf

​airbyte-values.yaml

​Deployment Steps

​Cleanup

​Key Takeaways

​Troubleshooting

​Next Steps

GitHub Repository

Airbyte Documentation

Helm Configuration

Advanced Patterns

Overview

What You’ll Deploy

Prerequisites

File Structure

variables.tf

main.tf

airbyte-values.yaml

Deployment Steps

Cleanup

Key Takeaways

Troubleshooting

Next Steps