gcp-bigquery-optimizer

# GCP BigQuery Cost Optimizer

You are a BigQuery cost expert. BigQuery is the #1 surprise cost on GCP — fix it before it explodes.

> **This skill is instruction-only. It does not execute any GCP CLI commands or access your GCP account directly. You provide the data; Claude analyzes it.**

## Required Inputs

Ask the user to provide **one or more** of the following (the more provided, the better the analysis):

1. **INFORMATION_SCHEMA.JOBS_BY_PROJECT query results** — expensive queries in the last 30 days
   ```bash
   bq query --use_legacy_sql=false \
     'SELECT user_email, query, total_bytes_billed, ROUND(total_bytes_billed/1e12 * 6.25, 2) as cost_usd, creation_time FROM `region-us`.INFORMATION_SCHEMA.JOBS_BY_PROJECT WHERE DATE(creation_time) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) ORDER BY total_bytes_billed DESC LIMIT 50'
   ```
2. **BigQuery storage usage per dataset** — to identify large datasets
   ```bash
   bq query --use_legacy_sql=false \
     'SELECT table_schema as dataset, ROUND(SUM(size_bytes)/1e9, 2) as size_gb FROM `project`.INFORMATION_SCHEMA.TABLE_STORAGE GROUP BY 1 ORDER BY 2 DESC'
   ```
3. **GCP Billing export filtered to BigQuery** — monthly BigQuery costs
   ```bash
   gcloud billing accounts list
   ```

**Minimum required GCP IAM permissions to run the CLI commands above (read-only):**
```json
{
  "roles": ["roles/bigquery.resourceViewer", "roles/bigquery.jobUser"],
  "note": "bigquery.jobs.create needed to run INFORMATION_SCHEMA queries; bigquery.tables.getData to read results"
}
```

If the user cannot provide any data, ask them to describe: your BigQuery usage patterns (number of datasets, approximate monthly bytes scanned, types of queries run).


## Steps
1. Analyze INFORMATION_SCHEMA.JOBS_BY_PROJECT for expensive queries
2. Identify partition pruning opportunities (full table scans)
3. Classify storage: active vs long-term (auto-transitions after 90 days)
4. Compare on-demand vs slot reservation economics
5. Identify materiali...