Complete Data Lineage¶

This page shows the complete end-to-end data lineage from source tables through all dimensional layers to final marts.

Complete Dimensional Layer Lineage¶

The diagram below shows the full lineage across all layers: - Staging Layer - Raw SAP ISU source tables - Dimensional Layer - Dimensions, Facts, and Bridges - Marts Layer - Business-specific export tables

graph TB
    %% Styling
    classDef staging fill:#e1f5ff,stroke:#01579b,stroke-width:2px
    classDef dimension fill:#fff9c4,stroke:#f57f17,stroke-width:2px
    classDef fact fill:#ffccbc,stroke:#bf360c,stroke-width:2px
    classDef bridge fill:#e1bee7,stroke:#4a148c,stroke-width:2px
    classDef mart fill:#c8e6c9,stroke:#1b5e20,stroke-width:3px

    %% STAGING LAYER - Source Tables
    subgraph STAGING["STAGING LAYER (SAP ISU Tables)"]
        EANL[stg_sap_isu__eanl<br/>Installation Master]:::staging
        EANLH[stg_sap_isu__eanlh<br/>Installation History]:::staging
        EVBS[stg_sap_isu__evbs<br/>Consumption Point]:::staging
        EVER[stg_sap_isu__ever<br/>Contract Data]:::staging
        FKKVKP[stg_sap_isu__fkkvkp<br/>Contract Account]:::staging
        KNA1[stg_sap_isu__kna1<br/>Customer Master]:::staging
        ADRC[stg_sap_isu__adrc<br/>Address]:::staging
        ADR2[stg_sap_isu__adr2<br/>Phone]:::staging
        ADR6[stg_sap_isu__adr6<br/>Email]:::staging
        IFLOT[stg_sap_isu__iflot<br/>Functional Location]:::staging
        IFLO[stg_sap_isu__iflo<br/>Location Description]:::staging
        ILOA[stg_sap_isu__iloa<br/>Location Address]:::staging
        EQUI[stg_sap_isu__equi<br/>Equipment Master]:::staging
        EQKT[stg_sap_isu__eqkt<br/>Equipment Text]:::staging
        EGERH[stg_sap_isu__egerh<br/>Equipment Installation]:::staging
        EQUZ[stg_sap_isu__equz<br/>Equipment Additional]:::staging
        EASTL[stg_sap_isu__eastl<br/>Equipment Billing Install]:::staging
        ETTIFN[stg_sap_isu__ettifn<br/>Installation Facts]:::staging
        S610[stg_sap_isu__s610<br/>Sales Cube]:::staging
        EUIINSTLN[stg_sap_isu__euiinstln<br/>Installation Link]:::staging
        EUITRANS[stg_sap_isu__euitrans<br/>External ID]:::staging
        EUIGRID[stg_sap_isu__euigrid<br/>Grid Assignment]:::staging
        EGRIDH[stg_sap_isu__egridh<br/>Grid History]:::staging
        VINET[stg_sap_isu__vinet<br/>Network Connections]:::staging
        IHPA[stg_sap_isu__ihpa<br/>Partner Functions]:::staging
    end

    %% DIMENSIONAL LAYER - Dimensions
    subgraph DIMENSIONS["DIMENSIONAL LAYER - Dimensions"]
        DIM_TECH[dim_technical_location<br/>Technical Locations]:::dimension
        DIM_CONS[dim_consumption_point<br/>Consumption Points]:::dimension
        DIM_INST[dim_installation<br/>Installations Current]:::dimension
        DIM_INST_HIST[dim_installation_hist<br/>Installations History]:::dimension
        DIM_PART[dim_business_partner<br/>Business Partners]:::dimension
        DIM_ACCT[dim_contract_account<br/>Contract Accounts]:::dimension
        DIM_CONT[dim_contract<br/>Contracts]:::dimension
        DIM_EQUIP[dim_equipment<br/>Equipment]:::dimension
    end

    %% DIMENSIONAL LAYER - Facts
    subgraph FACTS["DIMENSIONAL LAYER - Facts"]
        FACT_PRICE[fact_installation_pricing<br/>Installation Pricing]:::fact
        FACT_SALES[fact_sales<br/>Sales & Consumption]:::fact
        FACT_TECH[fact_equipment_install_technical<br/>Equipment Install Tech]:::fact
        FACT_BILL[fact_equipment_install_billing<br/>Equipment Install Billing]:::fact
    end

    %% DIMENSIONAL LAYER - Bridges
    subgraph BRIDGES["DIMENSIONAL LAYER - Bridges"]
        BR_METER[bridge_metering_point<br/>Metering Points]:::bridge
        BR_OBJ[bridge_object_connection<br/>Object Connections]:::bridge
        BR_ROLE[bridge_partner_roles<br/>Partner Roles]:::bridge
    end

    %% MARTS LAYER
    subgraph MARTS["MARTS LAYER"]
        MART_ANLAGE[sap_anlage<br/>Installation Export GIS]:::mart
    end

    %% DIMENSION LINEAGE
    IFLOT --> DIM_TECH
    IFLO --> DIM_TECH
    ILOA --> DIM_TECH
    ADRC --> DIM_TECH

    EVBS --> DIM_CONS
    DIM_TECH --> DIM_CONS

    EANL --> DIM_INST
    EANLH --> DIM_INST
    DIM_CONS --> DIM_INST

    EANL --> DIM_INST_HIST
    EANLH --> DIM_INST_HIST
    DIM_CONS --> DIM_INST_HIST

    KNA1 --> DIM_PART
    ADRC --> DIM_PART
    ADR2 --> DIM_PART
    ADR6 --> DIM_PART

    FKKVKP --> DIM_ACCT
    DIM_PART --> DIM_ACCT

    EVER --> DIM_CONT
    DIM_INST --> DIM_CONT
    DIM_ACCT --> DIM_CONT

    EQUI --> DIM_EQUIP
    EQKT --> DIM_EQUIP
    EGERH --> DIM_EQUIP
    EQUZ --> DIM_EQUIP
    DIM_TECH --> DIM_EQUIP

    %% FACT LINEAGE
    ETTIFN --> FACT_PRICE
    DIM_INST --> FACT_PRICE

    S610 --> FACT_SALES
    DIM_CONT --> FACT_SALES

    EGERH --> FACT_TECH
    EQUI --> FACT_TECH
    DIM_EQUIP --> FACT_TECH
    DIM_TECH --> FACT_TECH

    EASTL --> FACT_BILL
    EGERH --> FACT_BILL
    DIM_EQUIP --> FACT_BILL
    DIM_INST --> FACT_BILL
    FACT_TECH --> FACT_BILL

    %% BRIDGE LINEAGE
    EUIINSTLN --> BR_METER
    EUITRANS --> BR_METER
    EUIGRID --> BR_METER
    EGRIDH --> BR_METER
    DIM_INST --> BR_METER

    VINET --> BR_OBJ
    DIM_TECH --> BR_OBJ

    IHPA --> BR_ROLE
    FKKVKP --> BR_ROLE
    DIM_PART --> BR_ROLE
    DIM_TECH --> BR_ROLE
    DIM_ACCT --> BR_ROLE

    %% MARTS LINEAGE
    DIM_INST --> MART_ANLAGE
    DIM_CONS --> MART_ANLAGE
    DIM_TECH --> MART_ANLAGE
    DIM_INST_HIST --> MART_ANLAGE

Layer Descriptions¶

Staging Layer¶

Purpose: Direct 1:1 mappings from SAP ISU source tables

Light transformations only (data types, naming conventions)
No business logic
Preserves all source data
Foundation for all downstream models

Dimensional Layer - Dimensions¶

Purpose: Slowly Changing Dimensions (SCD Type 2 where applicable)

Key dimensions: - Technical Location - Physical locations and addresses - Consumption Point - Metering/consumption points - Installation - Current and historical installation data - Business Partner - Customer master with contact info - Contract Account - Billing accounts - Contract - Service contracts - Equipment - Equipment master data

Dimensional Layer - Facts¶

Purpose: Transaction and event data with foreign keys to dimensions

Facts capture: - Installation Pricing - Pricing configurations - Sales & Consumption - Sales and usage metrics - Equipment Install Technical - Technical equipment data - Equipment Install Billing - Billing-related equipment data

Dimensional Layer - Bridges¶

Purpose: Resolve many-to-many relationships

Bridges handle: - Metering Points - Complex meter/installation relationships - Object Connections - Network topology - Partner Roles - Multi-role partner assignments

Marts Layer¶

Purpose: Business-specific denormalized tables for external systems

Current marts: - sap_anlage - GIS installation export (matches legacy ABAP output)

Key Patterns¶

Hierarchical Dimensions¶

Some dimensions build on others: - dim_consumption_point depends on dim_technical_location - dim_installation depends on dim_consumption_point - dim_contract depends on dim_installation and dim_contract_account

Fact Dependencies¶

Facts always depend on their related dimensions: - Facts reference staging tables for measures - Facts reference dimensions for foreign keys - Some facts depend on other facts (e.g., fact_equipment_install_billing uses fact_equipment_install_technical)

Mart Denormalization¶

Marts flatten the dimensional model: - sap_anlage pulls from multiple dimensions to create a wide, denormalized table - Optimized for external system consumption (GIS, reporting tools) - Matches legacy ABAP output format exactly

Dependencies Index - Per-model dependency details
Build Order - Execution sequence
Dependency Graph - Auto-generated DAG

Note: This lineage diagram is maintained alongside the codebase and represents the designed dimensional model structure.