Data Engineering Guidelines

Objective: Define standards for building reliable, scalable, and maintainable data pipelines and transformations, covering pipeline design, dbt, Spark, streaming, and data quality governance.

Source File

View the full rule: .agent/rules/data-engineering.md

Sections

• 1. Pipeline Design Principles
• 1. dbt (SQL Transformations)
• 1. Apache Spark
• 1. Streaming (Kafka, Flink, Spark Streaming)
• 1. Data Quality & Governance