github/awesome-copilot
Audit Python and BigQuery data pipelines for cost, safe re-runs, and production readiness
About this skill
A text-based skill that guides the agent to review Python scripts that use BigQuery as an expert data engineer. It detects runaway cost risks, such as queries inside loops and a missing maximum_bytes_billed. It also checks for dry_run and execute modes, backfill design, query safety and scan size, write idempotency, and observability. The output is a structured report from section A to F with a PASS or FAIL verdict, plus a list of fixes ordered by severity that points to the affected functions and lines.
How do you use this skill?
1. Open the code repository or attach the Python script that performs BigQuery operations so the agent can read it.
2. Request the audit explicitly, for example: "Audit the pipelines/daily_scores.py script for BigQuery cost and re-run safety, and give me the list of fixes."
3. The agent will identify every call such as client.query, load_table_from_*, and external calls. It then estimates the worst-case number of jobs and checks for maximum_bytes_billed, the --mode flag, and date range limits.
4. Review sections A to F in the report: cost, dry run, loop design, query safety, safe writes, and monitoring. In each section, the agent suggests simple changes, such as using MERGE on a deterministic key or adding a run_id.
5. Use the final section to prioritize work: it includes a PASS or FAIL verdict for each section, the list of fixes by severity, and the top three cost risks with a rough estimate for each.
Requirements and limitations
- The skill requires the agent to have read access to the pipeline's Python code.
- It relies on Google's BigQuery client library, but the audit is performed by reading the code only. The skill does not run queries, does not connect to a Google Cloud account, and does not require credentials.
- Estimates in the report, such as job counts and cost, are approximate and based on static code analysis. They must be reviewed, and the suggested fixes tested, before applying them in a production environment.
- The skill is designed specifically for Python pipelines with BigQuery.
- Original license: MIT.
Installation by agent
Download and extract the ZIP. It contains the folder bigquery-pipeline-audit and the file SKILL.md with the original attachments and license.
Claude Code
- Create this path inside your project if it does not exist:
.claude/skills/ - Copy the entire extracted skill folder to this path:
.claude/skills/bigquery-pipeline-audit/SKILL.md - Open the project in your agent or refresh the skills list and look for bigquery-pipeline-audit.
- Request the task shown above and select the skill. Grant only necessary permissions.
GitHub Copilot
- Create this path inside your project if it does not exist:
.github/skills/ - Copy the entire extracted skill folder to this path:
.github/skills/bigquery-pipeline-audit/SKILL.md - Open the project in your agent or refresh the skills list and look for bigquery-pipeline-audit.
- Request the task shown above and select the skill. Grant only necessary permissions.
OpenAI Codex
- Create this path inside your project if it does not exist:
.agents/skills/ - Copy the entire extracted skill folder to this path:
.agents/skills/bigquery-pipeline-audit/SKILL.md - Open the project in your agent or refresh the skills list and look for bigquery-pipeline-audit.
- Request the task shown above and select the skill. Grant only necessary permissions.
Cursor
- Create this path inside your project if it does not exist:
.cursor/skills/ - Copy the entire extracted skill folder to this path:
.cursor/skills/bigquery-pipeline-audit/SKILL.md - Open the project in your agent or refresh the skills list and look for bigquery-pipeline-audit.
- Request the task shown above and select the skill. Grant only necessary permissions.
Devin / Windsurf
- Create this path inside your project if it does not exist:
.devin/skills/ - Copy the entire extracted skill folder to this path:
.devin/skills/bigquery-pipeline-audit/SKILL.md - Open the project in your agent or refresh the skills list and look for bigquery-pipeline-audit.
- Request the task shown above and select the skill. Grant only necessary permissions.
Gemini CLI
- Create this path inside your project if it does not exist:
.gemini/skills/ - Copy the entire extracted skill folder to this path:
.gemini/skills/bigquery-pipeline-audit/SKILL.md - Open the project in your agent or refresh the skills list and look for bigquery-pipeline-audit.
- Request the task shown above and select the skill. Grant only necessary permissions.