GitHub
airflow-dag-patterns
Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.
☆ 27,754查看技能
tools data engineering AI 技能
发现 tools data engineering 相关 Skills，比较来源仓库和用途，查看适合你工作流的技能说明。
分类目录: tools data engineering
tools data engineering 清除筛选
全部技能
GitHub
kpi-dashboard-design
Design effective KPI dashboards with metrics selection, visualization best practices, and real-time monitoring patterns. Use when building business dashboards, selecting metrics, or designing data visualization...
☆ 27,753查看技能
GitHub
sql-optimization-patterns
Master SQL query optimization, indexing strategies, and EXPLAIN analysis to dramatically improve database performance and eliminate slow queries. Use when debugging slow queries, designing database schemas, or...
☆ 27,753查看技能
GitHub
database-migration
Execute database migrations across ORMs and platforms with zero-downtime strategies, data transformation, and rollback procedures. Use when migrating databases, changing schemas, performing data transformations...
☆ 27,752查看技能
GitHub
data-storytelling
Transform data into compelling narratives using visualization, context, and persuasive structure. Use when presenting analytics to stakeholders, creating data reports, or building executive presentations.
☆ 27,752查看技能
GitHub
dbt-transformation-patterns
Master dbt (data build tool) for analytics engineering with model organization, testing, documentation, and incremental strategies. Use when building data transformations, creating data models, or implementing...
☆ 27,752查看技能
GitHub
projection-patterns
Build read models and projections from event streams. Use when implementing CQRS read sides, building materialized views, or optimizing query performance in event-sourced systems.
☆ 27,752查看技能
GitHub
ml-pipeline-workflow
Build end-to-end MLOps pipelines from data preparation through model training, validation, and production deployment. Use when creating ML pipelines, implementing MLOps practices, or automating model training a...
☆ 27,751查看技能
GitHub
spark-optimization
Optimize Apache Spark jobs with partitioning, caching, shuffle optimization, and memory tuning. Use when improving Spark performance, debugging slow jobs, or scaling data processing pipelines.
☆ 27,751查看技能
GitHub
data-quality-frameworks
Implement data quality validation with Great Expectations, dbt tests, and data contracts. Use when building data quality pipelines, implementing validation rules, or establishing data contracts.
☆ 27,751查看技能
GitHub
snowflake-semanticview
Create, alter, and validate Snowflake semantic views using Snowflake CLI (snow). Use when asked to build or troubleshoot semantic views/semantic layer definitions with CREATE/ALTER SEMANTIC VIEW, to validate se...
☆ 20,008查看技能
GitHub
n8n-workflow-patterns
Proven workflow architectural patterns from real n8n workflows. Use when building new workflows, designing workflow structure, choosing workflow patterns, planning workflow architecture, or asking about webhook...
☆ 18,947查看技能
GitHub
railway-database
Add official Railway database services (Postgres, Redis, MySQL, MongoDB). Use when user wants to add a database, says "add postgres", "add redis", "add database", "connect to database", or "wire up the database...
☆ 18,945查看技能
GitHub
zarr-python
Chunked N-D arrays for cloud storage. Compressed arrays, parallel I/O, S3/GCS integration, NumPy/Dask/Xarray compatible, for large-scale scientific computing pipelines.
☆ 18,942查看技能
GitHub
ray-data
Scalable data processing for ML workloads. Streaming execution across CPU/GPU, supports Parquet/CSV/JSON/images. Integrates with Ray Train, PyTorch, TensorFlow. Scales from single machine to 100s of nodes. Use...
☆ 18,942查看技能
GitHub
cocoindex
Comprehensive toolkit for developing with the CocoIndex library. Use when users need to create data transformation pipelines (flows), write custom functions, or operate flows via CLI or API. Covers building ETL...
☆ 18,940查看技能
GitHub
vaex
Use this skill for processing and analyzing large tabular datasets (billions of rows) that exceed available RAM. Vaex excels at out-of-core DataFrame operations, lazy evaluation, fast aggregations, efficient vi...
☆ 18,940查看技能
GitHub
senior-data-engineer
World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data mode...
☆ 18,939查看技能
GitHub
pinecone
Managed vector database for production AI applications. Fully managed, auto-scaling, with hybrid search (dense + sparse), metadata filtering, and namespaces. Low latency (
☆ 18,938查看技能
GitHub
polars
Fast DataFrame library (Apache Arrow). Select, filter, group_by, joins, lazy evaluation, CSV/Parquet I/O, expression API, for high-performance data analysis workflows.
☆ 18,938查看技能
GitHub
timescaledb
TimescaleDB - PostgreSQL extension for high-performance time-series and event data analytics, hypertables, continuous aggregates, compression, and real-time analytics
☆ 12,818查看技能
GitHub
query-writing
For writing and executing SQL queries - from simple single-table queries to complex multi-table JOINs and aggregations
☆ 8,916查看技能
GitHub
zarr-python
Chunked N-D arrays for cloud storage. Compressed arrays, parallel I/O, S3/GCS integration, NumPy/Dask/Xarray compatible, for large-scale scientific computing pipelines.
☆ 8,603查看技能
GitHub
vaex
Use this skill for processing and analyzing large tabular datasets (billions of rows) that exceed available RAM. Vaex excels at out-of-core DataFrame operations, lazy evaluation, fast aggregations, efficient vi...
☆ 8,603查看技能
默认按已收录数据中的 GitHub 仓库 Stars 排序。Stars 属于来源仓库，同一仓库的多个技能可能共享相同数值；它反映仓库关注度，不代表单个技能的安装量或质量评分。