GitHub
stream-processing
Use when designing real-time data processing systems, choosing stream processing frameworks, or implementing event-driven architectures. Covers Kafka, Flink, and streaming patterns.
☆ 21查看技能
tools data engineering AI 技能
发现 tools data engineering 相关 Skills，比较来源仓库和用途，查看适合你工作流的技能说明。
分类目录: tools data engineering
tools data engineering 清除筛选
全部技能
GitHub
etl-elt-patterns
Use when designing data pipelines, choosing between ETL and ELT approaches, or implementing data transformation patterns. Covers modern data pipeline architecture.
☆ 21查看技能
GitHub
data-engineer
Use when user needs scalable data pipeline development, ETL/ELT implementation, or data infrastructure design.
☆ 21查看技能
GitHub
database-optimizer
Use when user needs database query optimization, performance tuning, index strategies, execution plan analysis, or scalability across PostgreSQL, MySQL, MongoDB, Redis, and other database systems.
☆ 21查看技能
GitHub
csv-data-wrangler
Expert in high-performance CSV processing, parsing, and data cleaning using Python, DuckDB, and command-line tools. Use when working with CSV files, cleaning data, transforming datasets, or processing large tab...
☆ 21查看技能
GitHub
sql-pro
Use when user needs SQL development, database design, query optimization, performance tuning, or database administration across PostgreSQL, MySQL, SQL Server, and Oracle platforms.
☆ 21查看技能
GitHub
kafka-engineer
Expert in Apache Kafka, Event Streaming, and Real-time Data Pipelines. Specializes in Kafka Connect, KSQL, and Schema Registry.
☆ 21查看技能
GitHub
data-analyst
Expert in business intelligence, SQL, data visualization, and translating data into actionable business insights.
☆ 21查看技能
GitHub
agentic-jumpstart-database
Database patterns with Drizzle ORM and PostgreSQL including schema design, queries, migrations, transactions, and relationships. Use when working with database schemas, queries, migrations, indexes, joins, or w...
☆ 21查看技能
GitHub
fastapi-backend
Build production-grade FastAPI backends with SQLModel, Pydantic, and JWT authentication. Use this skill when building REST APIs, integrating with Neon PostgreSQL, implementing Better Auth JWT verification, or c...
☆ 21查看技能
GitHub
csv-wrangler
Handle messy CSVs with encoding detection, delimiter inference, and malformed row recovery.
☆ 20查看技能
GitHub
parquet-coder
Columnar file patterns including partitioning, predicate pushdown, and schema evolution.
☆ 20查看技能
GitHub
pydantic-validation
Record-level data validation using Pydantic models. Field validators, model validators, and batch validation patterns.
☆ 20查看技能
GitHub
etl-patterns
Production ETL patterns orchestrator. Routes to core reliability patterns and incremental load strategies.
☆ 20查看技能
GitHub
pandera-validation
DataFrame schema validation using pandera. Schema definitions, column checks, and decorator-based validation.
☆ 20查看技能
GitHub
dbt-coder
dbt (data build tool) patterns for model organization, incremental strategies, and testing.
☆ 20查看技能
GitHub
data-validation
Data validation patterns and pipeline helpers. Custom validation functions, schema evolution, and test assertions.
☆ 20查看技能
GitHub
data-profiler
Generate comprehensive data profiles for DataFrames. Use for EDA, data discovery, and understanding dataset characteristics.
☆ 20查看技能
GitHub
pandas-coder
DataFrame manipulation with chunked processing, memory optimization, and vectorized operations.
☆ 20查看技能
GitHub
test-fixture-generator
Generate synthetic test data with edge cases for ETL pipeline testing.
☆ 20查看技能
GitHub
etl-incremental-patterns
Incremental data loading patterns including backfill strategies, CDC, timestamp-based loads, and pipeline orchestration.
☆ 20查看技能
GitHub
litestream-coder
This skill guides configuring Litestream for continuous SQLite backup in Rails 8+ apps. Use when setting up production backups for SQLite databases (Solid Queue, Solid Cache, Solid Cable).
☆ 20查看技能
GitHub
great-expectations
Data validation using Great Expectations. Expectation suites, checkpoints, and data docs for pipeline monitoring.
☆ 20查看技能
GitHub
etl-core-patterns
Core ETL reliability patterns including idempotency, checkpointing, error handling, chunking, retry logic, and logging.
☆ 20查看技能
默认按已收录数据中的 GitHub 仓库 Stars 排序。Stars 属于来源仓库，同一仓库的多个技能可能共享相同数值；它反映仓库关注度，不代表单个技能的安装量或质量评分。