> ## Documentation Index
> Fetch the complete documentation index at: https://www.qovery.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Google BigQuery Dataset

> Provision a Google BigQuery dataset with an optional dedicated service account and key, so an application on any cloud — AWS, Azure, Scaleway or GCP — can query it

Qovery creates it in your account with a third-party provider, using credentials you enter in the form, as a [Blueprint](https://www.qovery.com/docs/configuration/blueprints.md) service in one of your environments.

| | |
| - | - |
| Catalog name | `bigquery-dataset` |
| Category | Analytics |
| Provider | EXTERNAL |
| Engine | terraform |
| Major versions (latest release) | `default` (1.0.3) |
| Source | [EXTERNAL/bigquery/default](https://github.com/Qovery/service-catalog/tree/EXTERNAL/bigquery/default/1.0.3/EXTERNAL/bigquery/default) |

> [!TIP]
>
> **New to this?** You don't need to know Terraform or Kubernetes. Qovery sets up Google BigQuery Dataset in your account with the provider: you fill in a form, or paste the prompt below into your coding agent. The values it publishes, listed under Outputs, reach your other services as environment variables. Qovery tells you when a newer version of the Blueprint is available. Check the requirements in the Reference section before you start, and try it first in a development environment.

## Create it

> [tab] Console
>
> 1. Open the environment, then **New service** › **Blueprint**.
> 2. Find **Google BigQuery Dataset** in the catalog and pick the major version.
> 3. Fill in the variables. Required ones are marked; the others use their defaults.
> 4. Create the service, and deploy it now or later.

> [tab] CLI and API
>
> List the catalog to get the latest tag, then create the service from it. `qovery api` signs the requests with your CLI session.
>
> ```bash theme={null}
> ORG_ID="REPLACE_WITH_ORGANIZATION_ID"
> ENV_ID="REPLACE_WITH_ENVIRONMENT_ID"
>
> qovery api "organization/$ORG_ID/blueprint/catalog" | jq '.. | objects | select(.name? == "bigquery-dataset")'
>
> cat > blueprint.json <<'EOF'
> {
>   "name": "bigquery-dataset",
>   "tag": "EXTERNAL/bigquery/default/1.0.3",
>   "icon": "app://qovery-console/bigquery",
>   "variables": [
>     {
>       "name": "gcp_service_account_key",
>       "value": "REPLACE_WITH_GCP_SERVICE_ACCOUNT_KEY",
>       "is_secret": true
>     },
>     {
>       "name": "gcp_project_id",
>       "value": "REPLACE_WITH_GCP_PROJECT_ID"
>     },
>     {
>       "name": "dataset_id",
>       "value": "REPLACE_WITH_DATASET_ID"
>     }
>   ],
>   "spec_overrides": {
>     "engine_version": "1.9.7"
>   }
> }
> EOF
> BLUEPRINT_ID=$(qovery api "environment/$ENV_ID/blueprint" --method POST --input - < blueprint.json | jq -r .id)
> rm blueprint.json   # it can hold secret values
> qovery api "blueprint/$BLUEPRINT_ID/deploy" --method POST
> qovery api "blueprint/$BLUEPRINT_ID"   # status of the latest deployment
> ```
>
> Replace each `REPLACE_WITH_` value. Add optional variables to `variables` the same way.

> [tab] Coding agent
>
> Paste this into Claude Code, Cursor, Codex or another coding agent with the [Qovery agent skills](https://www.qovery.com/docs/getting-started/quickstart/agent-skills.md) installed (`curl -fsSL https://skill.qovery.com/install.sh | bash`).
>
> ```text Prompt wrap theme={null}
> Add the Qovery Blueprint Google BigQuery Dataset (bigquery-dataset) to one of my Qovery environments and deploy it. Ask me for gcp_service_account_key, gcp_project_id and dataset_id.
> Ask me before deploying to a production environment.
> Docs: https://www.qovery.com/docs/configuration/blueprints/bigquery-dataset.md
> ```

You don't add any file to your repository. Qovery creates the Blueprint from the Console or the API.

## Variables and outputs

Generated from `qbm.yml` at `EXTERNAL/bigquery/default/1.0.3`, the manifest the Console form is built from.

> [accordion] All 13 variables
>
> | Name | Type | Required | Default | Secret | Description |
> | - | - | - | - | - | - |
> | `gcp_service_account_key` | string | Yes | | Yes | JSON key Terraform runs as; needs BigQuery Admin. Also Service Account (Key) Admin with create\_service\_account, Project IAM Admin with grant\_job\_user. |
> | `gcp_project_id` | string | Yes | | | Google Cloud project the dataset is created in |
> | `dataset_id` | string | Yes | | | Up to 1024 letters, digits and underscores. No hyphens. |
> | `location` | string | | `US` | | Multi-region (US, EU) or region (europe-west1). Immutable: changing it recreates the dataset and destroys its tables. |
> | `friendly_name` | string | | | | Display name shown in the BigQuery console. Empty = show the dataset id. |
> | `dataset_description` | string | | | | Free text attached to the dataset. Empty = none. |
> | `default_table_expiration_ms` | number | | `0` | | Lifetime in ms of tables without their own expiration. 0 = never; otherwise at least 3600000 (1 hour). |
> | `default_partition_expiration_ms` | number | | `0` | | Lifetime in ms of partitions without their own expiration. 0 = never; otherwise at least 3600000 (1 hour). |
> | `delete_contents_on_destroy` | bool | | `false` | | Allow destroying the dataset while it still holds tables. Left false, a destroy fails rather than silently dropping data. |
> | `create_service_account` | bool | | `true` | | Create a dedicated service account with a key so an application running on any cloud can reach the dataset. Set false to manage access yourself. |
> | `service_account_id` | string | | | | Leave empty — derived as qovery-bq-\<dataset>. Set it only if that id collides with an existing service account. |
> | `service_account_role` | string | | `roles/bigquery.dataEditor` | | Dataset-level role granted to the created service account |
> | `grant_job_user` | bool | | `true` | | Also grant the created account roles/bigquery.jobUser. Without it, it can read and write tables but not run queries. |

Outputs become environment variables that other services in the environment can use. By default Qovery also creates an alias for each output, prefixed with the Blueprint's name; see [Aliases Qovery creates for outputs](https://www.qovery.com/docs/configuration/environment-variables.md#aliases-qovery-creates-for-outputs) for the naming rules and the cases where no alias is created.

> [accordion] All 8 outputs
>
> | Name | Secret | Description |
> | - | - | - |
> | `dataset_id` | | BigQuery dataset id |
> | `project_id` | | Project the dataset lives in |
> | `location` | | Dataset location |
> | `dataset_reference` | | project.dataset, the form BigQuery SQL and client libraries expect |
> | `self_link` | | Dataset REST resource URI |
> | `service_account_email` | | Email of the created service account (empty when create\_service\_account is false) |
> | `service_account_key_json` | Yes | Raw JSON key, for GOOGLE\_APPLICATION\_CREDENTIALS or a client library |
> | `service_account_key_base64` | Yes | Same key, base64-encoded: easier in a single-line environment variable |

> [!WARNING]
>
> Secret variables and outputs are encrypted and masked in Qovery, but Terraform can also keep their values, and values derived from them, in its state. With the Qovery-managed state backend (the default), Qovery stores the state. If you use your own backend, protect it and keep state files out of version control.

## Reference

From the Blueprint's [README](https://github.com/Qovery/service-catalog/blob/EXTERNAL/bigquery/default/1.0.3/EXTERNAL/bigquery/default/README.md).

Provisions a [BigQuery](https://cloud.google.com/bigquery) dataset (`google_bigquery_dataset`) and,
by default, a dedicated service account with a key (`google_service_account`,
`google_service_account_key`) plus the IAM bindings an application needs to query it, via the
official `hashicorp/google` provider (`~> 7.0`).

**Available on every cloud.** This is an `EXTERNAL` blueprint: it authenticates with a service
account key supplied as a variable, never with the cluster's own cloud identity, so it deploys the
same way from an AWS, Azure, Scaleway or GCP cluster. The generated key is what lets a workload
outside Google Cloud authenticate — there is no workload identity to fall back on there.

### Credentials

A Google **service account JSON key** in `gcp_service_account_key` (sensitive), supplied as a
variable (`credentials.default: env`). The account it belongs to needs:

| Role | Why |
| - | - |
| `roles/bigquery.admin` | create the dataset and set dataset IAM |
| `roles/iam.serviceAccountAdmin` | create the application service account (`create_service_account = true`) |
| `roles/iam.serviceAccountKeyAdmin` | issue its key (`create_service_account = true`) |
| `roles/resourcemanager.projectIamAdmin` | grant `roles/bigquery.jobUser` (`grant_job_user = true`, the default) |

Drop the last three by setting `create_service_account = false` and managing access yourself.

### Variables

#### Required

| Name | Type | Sensitive | Description |
| - | - | - | - |
| `gcp_service_account_key` | string | yes | JSON key of the Google service account Terraform authenticates as. Needs BigQuery Admin on the project, plus Service Account Admin and Service Account Key Admin when `create_service_account` is true, and Project IAM Admin when `grant_job_user` is true. |
| `gcp_project_id` | string | | Project the dataset is created in |
| `dataset_id` | string | | BigQuery dataset id, up to 1024 chars of letters, digits and underscores (BigQuery rejects hyphens) |

#### Optional

| Name | Type | Default | Description |
| - | - | - | - |
| `location` | string | `US` | Dataset `location`: a multi-region (US, EU) or a region (europe-west1, us-central1). Immutable — changing it recreates the dataset and destroys its tables. |
| `friendly_name` | string | | Display name. Empty = show the dataset id |
| `dataset_description` | string | | Free-text description attached to the dataset. Empty = no description. |
| `default_table_expiration_ms` | number | `0` | Lifetime in milliseconds of a table created without its own expiration. 0 = tables never expire. BigQuery rejects anything below 3600000 (1 hour). |
| `default_partition_expiration_ms` | number | `0` | Lifetime in milliseconds of a partition created without its own expiration. 0 = partitions never expire. BigQuery rejects anything below 3600000 (1 hour). |
| `delete_contents_on_destroy` | bool | `false` | Allow destroying a dataset that still holds tables |
| `create_service_account` | bool | `true` | Create the application service account and key |
| `service_account_id` | string | | Leave empty — derived from the dataset id as qovery-bq-\<dataset>. Set it only when that derived id collides with an existing service account in the project. |
| `service_account_role` | string | `roles/bigquery.dataEditor` | `dataViewer`, `dataEditor` or `dataOwner`, granted on the dataset |
| `grant_job_user` | bool | `true` | Also grant the created service account roles/bigquery.jobUser on the project. Without it the account can read and write table data but cannot run a query — job creation is a project-level right that no dataset role carries. |

### Outputs

| Name | Sensitive | Description |
| - | - | - |
| `dataset_id` | | Dataset id |
| `project_id` | | Project the dataset lives in |
| `location` | | Dataset location |
| `dataset_reference` | | `project.dataset`, the form SQL and client libraries expect |
| `self_link` | | Dataset REST resource URI |
| `service_account_email` | | Application service account email (empty when not created) |
| `service_account_key_json` | yes | Key as raw JSON — for `GOOGLE_APPLICATION_CREDENTIALS` or a client |
| `service_account_key_base64` | yes | Same key base64-encoded, for a single-line environment variable |

### Notes

* **`grant_job_user` is not optional in practice.** BigQuery splits data access from job creation:
  a dataset role lets the account read and write rows, but running a `SELECT` creates a job, which
  is a project-level right. Turning it off yields an account that fails every query with
  `Permission bigquery.jobs.create denied`.
* **The service account key must be a real key object.** The variable rejects any JSON that is not
  an object carrying `type: "service_account"`, `client_email` and `private_key`, so a wrong paste
  fails at variable resolution rather than as an opaque OAuth error minutes into the apply.
* **The service account key is a long-lived credential.** It is surfaced as a sensitive output and
  stored encrypted by Qovery. On a GCP cluster you can set `create_service_account = false` and use
  Workload Identity instead; on any other cloud the key is the only option.
* **`location` is immutable.** BigQuery cannot move a dataset, so a change here recreates it and
  destroys every table in it.
* **Destroy is guarded.** With `delete_contents_on_destroy = false` (the default) destroying the
  blueprint fails while the dataset still holds tables, rather than dropping them.
* Tables are deliberately out of scope: schemas belong with the application that owns them, and a
  blueprint variable cannot express one. Create them from your migration tooling against
  `dataset_reference`.


## Related topics

- [Blueprints](https://www.qovery.com/docs/configuration/blueprints.md)
- [Cluster Advanced Settings](https://www.qovery.com/docs/configuration/cluster-advanced-settings.md)
- [Google GKE Integration](https://www.qovery.com/docs/configuration/integrations/kubernetes/gke/overview.md)
- [Google Cloud Memorystore for Redis](https://www.qovery.com/docs/configuration/blueprints/gcp-memorystore-redis.md)
- [Qovery Observe](https://www.qovery.com/docs/configuration/integrations/observability/qovery-observe.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.