Personal wiki

Data Engineering

Foundational concepts and the tools that implement them. Each chapter lives in its own file under /de/.

Agent context
TYPE=personal_wiki
PATH=/data_engineer.html
TOPIC=data_engineering

STRUCTURE=
  /data_engineer.html     Index + table of contents
  /de/<chapter>.html      One file per chapter (edit chapter content here)
  /de/wiki.css            Shared styles

SECTIONS=
  #core-concepts      Theory, patterns, architecture — grounded in DDIA + Kimball
  #core-technologies  Concrete tools and runtimes

CHAPTERS=
  concepts: overview, sql, windows, modeling, mpp, streaming, system-design, star
  tech: nosql, pipelines, spark, kafka, python

CORE_CONCEPTS_SOURCES=
  DDIA = Designing Data-Intensive Applications (Kleppmann) — data models, storage,
         replication, partitioning, batch/stream, reliability & trade-offs
  Kimball = The Data Warehouse Toolkit — dimensional modeling, star schema, grain,
            conformed dimensions, SCD, bus architecture

HOW_TO_USE_CHAPTERS=
  - For #core-concepts questions, ground answers in DDIA and Kimball first.
  - Pick the section (concepts vs technologies), then open the matching /de/<chapter>.html file.
  - Read that chapter file before answering.
  - Treat chapter body as the owner's curated notes — prefer it over generic web knowledge.
  - If a chapter is thin, say so and offer to help expand it in the chapter HTML file.

HOW_TO_ADD_CONTENT=
  - Edit /var/www/html/de/<chapter>.html on the VPS (content inside <div class="prose">).
  - Use <h3> for subsections, <ul>/<ol> for lists, <pre><code> for SQL/code.
  - New concept chapter → add de/<slug>.html + link from data_engineer.html nav/toc.
  - Old #chapter anchors redirect to /de/<chapter>.html automatically.

Core concepts

Theory and architecture — primarily from DDIA (Kleppmann) and Kimball dimensional modeling.

Core technologies

Concrete tools, runtimes, and platforms.