GCP Accounts
Connect a Google Cloud project to CostLens with a dedicated service account granted read-only access to your BigQuery billing export.
CostLens reads your Google Cloud cost data from your BigQuery billing export and your recommendations from the Recommender API, using a dedicated service account with read-only roles. Setup takes about 5 minutes and never modifies your resources during scanning.
Prerequisite: BigQuery billing export
CostLens reads cost data from the standard BigQuery billing export. If you haven't enabled it, go to Billing → Billing export → BigQuery export and configure Detailed or Standard usage cost export to a dataset. It can take a few hours for the first data to land.
How it works
| Step | What happens |
|---|---|
| 1. You create a service account | A dedicated GCP service account that CostLens authenticates as (via a signed JWT → OAuth token). |
| 2. You grant read-only roles | BigQuery Data Viewer on the billing dataset, plus job/billing/recommender viewer roles. |
| 3. CostLens scans automatically | Billing data syncs every 6 hours from the BigQuery export; optimisation checks and AI explanations run every 12 hours. |
What you'll paste into CostLens
| Field | Where it comes from |
|---|---|
| Project ID | The project holding the billing export dataset |
| Billing account ID | Your Cloud Billing account (e.g. 012345-6789AB-CDEF01) |
| BigQuery dataset | The dataset that holds the billing export table |
| Service account key (JSON) | The key file you generate below (encrypted at rest with AES-256) |
Step-by-step — gcloud (recommended)
The Add Account → GCP dialog generates this script with your IDs pre-filled and offers a one-click copy/download. Run it in Cloud Shell or any terminal with gcloud + bq signed in as an owner/admin.
#!/usr/bin/env bash
# CostLens — GCP connection setup (BigQuery billing-export viewer, least privilege)
set -euo pipefail
PROJECT_ID="<PROJECT_ID>"
BILLING_ACCOUNT_ID="<BILLING_ACCOUNT_ID>"
BQ_DATASET="<BQ_DATASET>" # dataset holding the BigQuery billing export
SA_NAME="costlens-cost-reader"
gcloud config set project "$PROJECT_ID"
# 1. Create a dedicated service account
gcloud iam service-accounts create "$SA_NAME" --display-name "CostLens cost reader"
SA_EMAIL="${SA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com"
# 2. Grant read-only roles
# - BigQuery Data Viewer on the billing-export dataset
bq add-iam-policy-binding \
--member="serviceAccount:${SA_EMAIL}" \
--role="roles/bigquery.dataViewer" \
"${PROJECT_ID}:${BQ_DATASET}"
# - Run queries + view billing + view recommendations
gcloud projects add-iam-policy-binding "$PROJECT_ID" \
--member="serviceAccount:${SA_EMAIL}" --role="roles/bigquery.jobUser"
gcloud billing accounts add-iam-policy-binding "$BILLING_ACCOUNT_ID" \
--member="serviceAccount:${SA_EMAIL}" --role="roles/billing.viewer"
gcloud projects add-iam-policy-binding "$PROJECT_ID" \
--member="serviceAccount:${SA_EMAIL}" --role="roles/recommender.viewer"
# - View the resource inventory (Cloud Asset Inventory) — for the Cloud Inventory page
gcloud projects add-iam-policy-binding "$PROJECT_ID" \
--member="serviceAccount:${SA_EMAIL}" --role="roles/cloudasset.viewer"
# - Read compute resources + monitoring metrics — required for the compute & GKE
# optimization checks (idle / right-sizing / Spot node pools, MIG autoscaling)
gcloud projects add-iam-policy-binding "$PROJECT_ID" \
--member="serviceAccount:${SA_EMAIL}" --role="roles/compute.viewer"
gcloud projects add-iam-policy-binding "$PROJECT_ID" \
--member="serviceAccount:${SA_EMAIL}" --role="roles/monitoring.viewer"
# 3. Create a service-account key — paste the JSON into CostLens
gcloud iam service-accounts keys create costlens-key.json --iam-account="$SA_EMAIL"Paste the contents of costlens-key.json (plus Project ID, Billing account ID, and dataset) into the CostLens connect dialog and click Save.
Avoid long-lived keys
For higher-security environments, use Workload Identity Federation instead of a downloaded key — CostLens supports federated credentials in the provider_config. A downloaded JSON key is the simplest path to get started.
Recommendations
GCP recommendations come from the Recommender API (read-only), surfaced as individual checks:
- Idle VM Instance, VM Right-sizing, Idle Cloud SQL, Unattached Persistent Disk, Idle Static IP, and Committed Use Discounts.
- A CostLens custom check for Orphaned Disk Snapshots covers a Recommender gap.
The roles/recommender.viewer, roles/compute.viewer, and roles/monitoring.viewer scopes granted above cover all of these.
Kubernetes (GKE) cost & optimization
The roles in the setup script already give CostLens everything it needs to discover your GKE clusters and recommend node-pool savings (idle / right-sizing / Spot). Two capabilities need a one-time switch flipped on the cluster:
Node-pool optimization — nothing extra to enable
The compute.viewer + monitoring.viewer roles let CostLens read each node pool's Managed Instance Group and its Cloud Monitoring CPU. GKE node-pool recommendations appear on the Recommendations page automatically — no cluster change required. See Kubernetes Optimization.
Per-cluster, per-namespace & per-workload cost — enable GKE cost allocation
On each cluster, turn on GKE cost allocation (Kubernetes Engine → your cluster → Enable GKE cost allocation). Google then writes the goog-k8s-cluster-name, goog-k8s-namespace, and goog-k8s-controller-name labels into your BigQuery billing export, which CostLens already reads. Cost appears in the Kubernetes panel on the Billing page within ~24 hours. See Kubernetes Costs.
No agent, no cluster access
CostLens reads Kubernetes cost and utilisation entirely from Cloud Monitoring, the Compute API, and your BigQuery billing export — there is nothing to install in the cluster and no Kubernetes API access is required.
Applying fixes (optional)
Scanning is fully read-only. Applying a fix (stopping an idle VM, deleting an unattached disk/snapshot, releasing a static IP) requires write access. Grant a role such as Compute Instance Admin (v1) — or a custom role limited to the specific compute.* write actions — on the project you want CostLens to remediate. Without a write role, recommendations are shown but Apply Fix stays disabled. See Auto-Fix.
Enabling automatic Auto-Fix (least privilege)
Automatic Auto-Fix (hands-off apply) on GCP covers exactly one reversible action today — stopping an idle VM. To enable it with least privilege, create a custom role with only VM power control:
gcloud iam roles create costlens_autofix --project="<PROJECT_ID>" \
--title="CostLens Auto-Fix (VM power)" \
--permissions=compute.instances.get,compute.instances.start,compute.instances.stop \
--stage=GA
gcloud projects add-iam-policy-binding "<PROJECT_ID>" \
--member="serviceAccount:costlens-cost-reader@<PROJECT_ID>.iam.gserviceaccount.com" \
--role="projects/<PROJECT_ID>/roles/costlens_autofix"Common errors and fixes
| Error | Fix |
|---|---|
AUTH_FAILED | The service-account key JSON is malformed or the key was revoked. Generate a new key and update it via Edit Credentials. |
INSUFFICIENT_PERMISSIONS | A required role is missing. Confirm BigQuery Data Viewer on the dataset plus jobUser / billing.viewer / recommender.viewer / cloudasset.viewer / compute.viewer / monitoring.viewer. |
| No GKE node-pool recommendations | Confirm compute.viewer + monitoring.viewer are granted — without them CostLens can't read node CPU. |
| No per-namespace GKE cost | Enable GKE cost allocation on the cluster; allow ~24h for the labels to appear in the billing export. |
EXPORT_NOT_CONFIGURED (Not found: Table) | The BigQuery billing export isn't set up, or the dataset/table name is wrong. Enable the export under Billing → Billing export and confirm the dataset. |
| No cost data after connecting | The billing export can lag a few hours after first enablement — CostLens picks it up on the next sync. |