GitHub
parquet-coder
Columnar file patterns including partitioning, predicate pushdown, and schema evolution.
☆ 20View skill
devops data engineering AI Skills
Discover devops data engineering skills, compare source repositories and read the instructions for each workflow.
Categories: devops data engineering
devops data engineering Clear filters
Browse skills
GitHub
pydantic-validation
Record-level data validation using Pydantic models. Field validators, model validators, and batch validation patterns.
☆ 20View skill
GitHub
etl-patterns
Production ETL patterns orchestrator. Routes to core reliability patterns and incremental load strategies.
☆ 20View skill
GitHub
pandera-validation
DataFrame schema validation using pandera. Schema definitions, column checks, and decorator-based validation.
☆ 20View skill
GitHub
dbt-coder
dbt (data build tool) patterns for model organization, incremental strategies, and testing.
☆ 20View skill
GitHub
data-validation
Data validation patterns and pipeline helpers. Custom validation functions, schema evolution, and test assertions.
☆ 20View skill
GitHub
data-profiler
Generate comprehensive data profiles for DataFrames. Use for EDA, data discovery, and understanding dataset characteristics.
☆ 20View skill
GitHub
pandas-coder
DataFrame manipulation with chunked processing, memory optimization, and vectorized operations.
☆ 20View skill
GitHub
test-fixture-generator
Generate synthetic test data with edge cases for ETL pipeline testing.
☆ 20View skill
GitHub
etl-incremental-patterns
Incremental data loading patterns including backfill strategies, CDC, timestamp-based loads, and pipeline orchestration.
☆ 20View skill
GitHub
litestream-coder
This skill guides configuring Litestream for continuous SQLite backup in Rails 8+ apps. Use when setting up production backups for SQLite databases (Solid Queue, Solid Cache, Solid Cable).
☆ 20View skill
GitHub
great-expectations
Data validation using Great Expectations. Expectation suites, checkpoints, and data docs for pipeline monitoring.
☆ 20View skill
GitHub
etl-core-patterns
Core ETL reliability patterns including idempotency, checkpointing, error handling, chunking, retry logic, and logging.
☆ 20View skill
GitHub
polars-expertise
This skill should be used when the user asks about Polars DataFrame library (Apache Arrow) for Python or Rust. Triggers: "polars expressions", "lazy vs eager", "scan_parquet streaming", "convert pandas to polar...
☆ 20View skill
GitHub
incremental-fetch
Build resilient data ingestion pipelines from APIs. Use when creating scripts that fetch paginated data from external APIs (Twitter, exchanges, any REST API) and need to track progress, avoid duplicates, handle...
☆ 20View skill
GitHub
table-filters
Designs optimal filtering UX for data tables. Use when building a table that needs filters - analyzes the data columns and determines the best filter type for each. Outputs a unified filter field with inline he...
☆ 20View skill
GitHub
postgres-query-expert
A comprehensive guide for interacting with PostgreSQL 16 databases. Use this skill for constructing standard and advanced SQL queries, optimizing performance, debugging errors, managing schema objects, and intr...
☆ 20View skill
GitHub
reasoning-causal
Execute evidence-based decision-making through 6-stage causal flow. Use for known processes, operational execution, and decisions with clear cause-effect chains.
☆ 20View skill
GitHub
grey-haven-data-modeling
Design database schemas for Grey Haven multi-tenant SaaS - SQLModel models, Drizzle schema, multi-tenant isolation with tenant_id and RLS, timestamp fields, foreign keys, indexes, migrations, and relationships....
☆ 19View skill
GitHub
grey-haven-database-conventions
Apply Grey Haven database conventions - snake_case fields, multi-tenant with tenant_id and RLS, proper indexing, migrations for Drizzle (TypeScript) and SQLModel (Python). Use when designing schemas, writing da...
☆ 19View skill
GitHub
grey-haven-ontological-documentation
Create comprehensive ontological documentation for Grey Haven systems - extract domain concepts from TanStack Start and FastAPI codebases, model semantic relationships, generate visual representations of system...
☆ 19View skill
GitHub
grey-haven-data-validation
Comprehensive data validation using Pydantic v2 with data quality monitoring and schema alignment for PlanetScale PostgreSQL. Use when implementing API validation, database schema alignment, or data quality ass...
☆ 19View skill
GitHub
data-storytelling
Transform data into compelling narratives using visualization, context, and persuasive structure. Use when presenting analytics to stakeholders, creating data reports, or building executive presentations.
☆ 19View skill
GitHub
database-optimizer
Expert database optimizer specializing in modern performance tuning, query optimization, and scalable architectures. Masters advanced indexing, N+1 resolution, multi-tier caching, partitioning strategies, and c...
☆ 19View skill
The default order uses GitHub repository stars from our indexed data. Stars belong to the source repository and may be shared by multiple skills. They indicate repository interest, not skill installs or an independent quality score.