GitHub
stream-processing
Use when designing real-time data processing systems, choosing stream processing frameworks, or implementing event-driven architectures. Covers Kafka, Flink, and streaming patterns.
☆ 21View skill
tools data engineering AI Skills
Discover tools data engineering skills, compare source repositories and read the instructions for each workflow.
Categories: tools data engineering
tools data engineering Clear filters
Browse skills
GitHub
etl-elt-patterns
Use when designing data pipelines, choosing between ETL and ELT approaches, or implementing data transformation patterns. Covers modern data pipeline architecture.
☆ 21View skill
GitHub
data-engineer
Use when user needs scalable data pipeline development, ETL/ELT implementation, or data infrastructure design.
☆ 21View skill
GitHub
database-optimizer
Use when user needs database query optimization, performance tuning, index strategies, execution plan analysis, or scalability across PostgreSQL, MySQL, MongoDB, Redis, and other database systems.
☆ 21View skill
GitHub
csv-data-wrangler
Expert in high-performance CSV processing, parsing, and data cleaning using Python, DuckDB, and command-line tools. Use when working with CSV files, cleaning data, transforming datasets, or processing large tab...
☆ 21View skill
GitHub
sql-pro
Use when user needs SQL development, database design, query optimization, performance tuning, or database administration across PostgreSQL, MySQL, SQL Server, and Oracle platforms.
☆ 21View skill
GitHub
kafka-engineer
Expert in Apache Kafka, Event Streaming, and Real-time Data Pipelines. Specializes in Kafka Connect, KSQL, and Schema Registry.
☆ 21View skill
GitHub
data-analyst
Expert in business intelligence, SQL, data visualization, and translating data into actionable business insights.
☆ 21View skill
GitHub
agentic-jumpstart-database
Database patterns with Drizzle ORM and PostgreSQL including schema design, queries, migrations, transactions, and relationships. Use when working with database schemas, queries, migrations, indexes, joins, or w...
☆ 21View skill
GitHub
fastapi-backend
Build production-grade FastAPI backends with SQLModel, Pydantic, and JWT authentication. Use this skill when building REST APIs, integrating with Neon PostgreSQL, implementing Better Auth JWT verification, or c...
☆ 21View skill
GitHub
csv-wrangler
Handle messy CSVs with encoding detection, delimiter inference, and malformed row recovery.
☆ 20View skill
GitHub
parquet-coder
Columnar file patterns including partitioning, predicate pushdown, and schema evolution.
☆ 20View skill
GitHub
pydantic-validation
Record-level data validation using Pydantic models. Field validators, model validators, and batch validation patterns.
☆ 20View skill
GitHub
etl-patterns
Production ETL patterns orchestrator. Routes to core reliability patterns and incremental load strategies.
☆ 20View skill
GitHub
pandera-validation
DataFrame schema validation using pandera. Schema definitions, column checks, and decorator-based validation.
☆ 20View skill
GitHub
dbt-coder
dbt (data build tool) patterns for model organization, incremental strategies, and testing.
☆ 20View skill
GitHub
data-validation
Data validation patterns and pipeline helpers. Custom validation functions, schema evolution, and test assertions.
☆ 20View skill
GitHub
data-profiler
Generate comprehensive data profiles for DataFrames. Use for EDA, data discovery, and understanding dataset characteristics.
☆ 20View skill
GitHub
pandas-coder
DataFrame manipulation with chunked processing, memory optimization, and vectorized operations.
☆ 20View skill
GitHub
test-fixture-generator
Generate synthetic test data with edge cases for ETL pipeline testing.
☆ 20View skill
GitHub
etl-incremental-patterns
Incremental data loading patterns including backfill strategies, CDC, timestamp-based loads, and pipeline orchestration.
☆ 20View skill
GitHub
litestream-coder
This skill guides configuring Litestream for continuous SQLite backup in Rails 8+ apps. Use when setting up production backups for SQLite databases (Solid Queue, Solid Cache, Solid Cable).
☆ 20View skill
GitHub
great-expectations
Data validation using Great Expectations. Expectation suites, checkpoints, and data docs for pipeline monitoring.
☆ 20View skill
GitHub
etl-core-patterns
Core ETL reliability patterns including idempotency, checkpointing, error handling, chunking, retry logic, and logging.
☆ 20View skill
The default order uses GitHub repository stars from our indexed data. Stars belong to the source repository and may be shared by multiple skills. They indicate repository interest, not skill installs or an independent quality score.