GitHub
understanding-transformers
Use when working with transformers, understanding event validation/enrichment/redaction, or learning about transformer chaining. Covers interface, return values, and pipeline integration.
☆ 316View skill
tools data engineering AI Skills
Discover tools data engineering skills, compare source repositories and read the instructions for each workflow.
Categories: tools data engineering
tools data engineering Clear filters
Browse skills
GitHub
create-transformer
Use when creating a new walkerOS transformer. Example-driven workflow for validation, enrichment, or redaction transformers.
☆ 316View skill
GitHub
understanding-mapping
Use when transforming events at any point in the flow (source→collector or collector→destination), configuring data/map/loop/condition, or understanding value extraction. Covers all mapping strategies.
☆ 316View skill
GitHub
data-analyst
This skill should be used when analyzing CSV datasets, handling missing values through intelligent imputation, and creating interactive dashboards to visualize data trends. Use this skill for tasks involving da...
☆ 313View skill
GitHub
postgres-pro
Use when optimizing PostgreSQL queries, configuring replication, or implementing advanced database features. Invoke for EXPLAIN analysis, JSONB operations, extension usage, VACUUM tuning, performance monitoring...
☆ 312View skill
GitHub
pandas-pro
Use when working with pandas DataFrames, data cleaning, aggregation, merging, or time series analysis. Invoke for data manipulation, missing value handling, groupby operations, or performance optimization.
☆ 312View skill
GitHub
salesforce-developer
Use when developing Salesforce applications, Apex code, Lightning Web Components, SOQL queries, triggers, integrations, or CRM customizations. Invoke for governor limits, bulk processing, platform events, Sales...
☆ 312View skill
GitHub
ml-pipeline
Use when building ML pipelines, orchestrating training workflows, automating model lifecycle, implementing feature stores, or managing experiment tracking systems.
☆ 312View skill
GitHub
spark-engineer
Use when building Apache Spark applications, distributed data processing pipelines, or optimizing big data workloads. Invoke for DataFrame API, Spark SQL, RDD operations, performance tuning, streaming analytics...
☆ 312View skill
GitHub
database-optimizer
Use when investigating slow queries, analyzing execution plans, or optimizing database performance. Invoke for index design, query rewrites, configuration tuning, partitioning strategies, lock contention resolu...
☆ 312View skill
GitHub
convex-schema-validator
Defining and validating database schemas with proper typing, index configuration, optional fields, unions, and migration strategies for schema changes
☆ 297View skill
GitHub
auto-cdc
Apply Change Data Capture (CDC) with apply_changes API in Spark Declarative Pipelines. Use when user needs to process CDC feeds from databases, handle upserts/deletes, maintain slowly changing dimensions (SCD T...
☆ 293View skill
GitHub
data-analysis
High-performance data analysis using Polars - load, transform, aggregate, visualize and export tabular data. Use for CSV/JSON/Parquet processing, statistical analysis, time series, and creating charts.
☆ 293View skill
GitHub
validation-scripts
Data validation and pipeline testing utilities for ML training projects. Validates datasets, model checkpoints, training pipelines, and dependencies. Use when validating training data, checking model outputs, t...
☆ 283View skill
GitHub
dbt-transformation-patterns
Master dbt (data build tool) for analytics engineering with model organization, testing, documentation, and incremental strategies. Use when building data transformations, creating data models, or implementing...
☆ 279View skill
GitHub
sql-pro
Master modern SQL with cloud-native databases, OLTP/OLAP optimization, and advanced query techniques. Expert in performance tuning, data modeling, and hybrid analytical systems. Use PROACTIVELY for database opt...
☆ 279View skill
GitHub
database-migrations-sql-migrations
SQL database migrations with zero-downtime strategies for PostgreSQL, MySQL, SQL Server
☆ 279View skill
GitHub
database-architect
Expert database architect specializing in data layer design from scratch, technology selection, schema modeling, and scalable database architectures. Masters SQL/NoSQL/TimeSeries database selection, normalizati...
☆ 279View skill
GitHub
airflow-dag-patterns
Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.
☆ 279View skill
GitHub
kpi-dashboard-design
Design effective KPI dashboards with metrics selection, visualization best practices, and real-time monitoring patterns. Use when building business dashboards, selecting metrics, or designing data visualization...
☆ 279View skill
GitHub
projection-patterns
Build read models and projections from event streams. Use when implementing CQRS read sides, building materialized views, or optimizing query performance in event-sourced systems.
☆ 279View skill
GitHub
data-storytelling
Transform data into compelling narratives using visualization, context, and persuasive structure. Use when presenting analytics to stakeholders, creating data reports, or building executive presentations.
☆ 279View skill
GitHub
sql-optimization-patterns
Master SQL query optimization, indexing strategies, and EXPLAIN analysis to dramatically improve database performance and eliminate slow queries. Use when debugging slow queries, designing database schemas, or...
☆ 279View skill
GitHub
spark-optimization
Optimize Apache Spark jobs with partitioning, caching, shuffle optimization, and memory tuning. Use when improving Spark performance, debugging slow jobs, or scaling data processing pipelines.
☆ 279View skill
The default order uses GitHub repository stars from our indexed data. Stars belong to the source repository and may be shared by multiple skills. They indicate repository interest, not skill installs or an independent quality score.