python-data-engineering
Comprehensive Python data engineering patterns for AWS Data Lake, including PySpark, Pandas, Apache Airflow, AWS Glue, ETL pipelines, data quality, schema management, performance optimization, FastAPI services, streaming with Kafka/Kinesis, data validation with Great Expectations, testing strategies, error handling, logging, and production deployment on AWS EMR and Glue.
Installation
Follow the repository instructions for this skill and your agent. Review the files and dependencies before adding it to your workflow.
View source ↗GuidesSkill instructions
Features and purpose
Comprehensive Python data engineering patterns for AWS Data Lake, including PySpark, Pandas, Apache Airflow, AWS Glue, ETL pipelines, data quality, schema management, performance optimization, FastAPI services, streaming with Kafka/Kinesis, data validation with Great Expectations, testing strategies, error handling, logging, and production deployment on AWS EMR and Glue.
Quick start
Follow the repository instructions for this skill and your agent. Review the files and dependencies before adding it to your workflow.
Guides ↗Examples
Read the usage examples in the source repository.
View source ↗Source history
- Indexed on
- 2026-01-21
- Index updated
- 2026-01-26
- Branch
- main
Index and sync dates describe this directory. View the source repository for code history.
GitHub ↗chat Comments (0)
Sign in to join the discussion and leave a comment.
No comments yet. Be the first to share your thoughts!