Chapter 7: Bringing It All Together

The Full System

With all six chapters complete, the system looks like this:

• Neo4j Aura. Holds the supply chain graph – 20 suppliers, 12 warehouses, 10 distribution centers and 30 retailers connected by shipping relationships, loaded once in Chapter 2 and updated in real time as shipments arrive. It answers graph questions: what’s the shortest path from a supplier to a retailer, which warehouse should we use when one goes offline, which nodes are the most connected hubs.

• Snowflake Postgres. Holds the live operational data – every shipment event consumed from Kafka, written in real time by the simulator. It answers OLTP questions: how many shipments are currently delayed, which shipments have been rerouted, when did W003 go offline and when did it come back.

• Snowflake. Holds the analytical history – shipment data synced from Snowflake Postgres into shipment_history and aggregated for trend analysis. It answers analytical questions: which suppliers have the highest delay rates, which warehouses are breaching SLA thresholds, how has delay volume changed over the last hour.

None of these systems knows about the others. Aura knows nothing about Snowflake Postgres shipment records. Snowflake Postgres knows nothing about the graph topology. Snowflake knows nothing about disruption routing. The intelligence sits in the application layer – the simulator, the Streamlit dashboards and the analytics notebook – which orchestrates queries across all three and combines the results.

Running the System

The order of operations matters. Work through the notebooks in sequence:

1. 02_neo4j_supply_chain.ipynb – Builds the supply chain graph in Aura.
2. 03_snowflake_postgres.ipynb – Creates the operational tables in Snowflake Postgres.
3. 04_snowflake_warehouse.ipynb – Creates the analytics table in Snowflake.
4. 05_simulator.ipynb – Runs the Kafka producer, consumer and disruption lifecycle.
5. 06_analytics.ipynb – Optional, for running five analytics queries interactively.
6. In a terminal: streamlit run app.py.
7. In a second terminal: streamlit run analytics_app.py --server.port 8502.

Let the simulator run for at least 20 minutes before opening the analytics dashboard. The delay trends, warehouse health and Sankey charts need enough data to tell a meaningful story.

What Each System Does That the Others Can’t

It’s worth noting why three systems are better than one for this use case.

Why not just Aura? Neo4j Aura is purpose-built for graph traversals – exactly what the disruption routing requires. Finding an alternative path that avoids an inactive warehouse mid-route is a natural graph query. In Cypher it’s a variable-length path with a WHERE NONE() filter. In SQL it would require a recursive CTE with exclusion logic that grows in complexity with each additional hop. The graph structure also makes it easy to answer questions about connectivity – which suppliers can reach which retailers, which warehouses are the most connected hubs – that would require expensive self-joins in a relational database.

Why not just Snowflake Postgres? Snowflake Postgres is purpose-built for operational data – high-frequency writes from the Kafka consumer, transactional consistency and low-latency reads for the live operations dashboard. It handles thousands of shipment inserts per session exactly as a production Postgres instance would. Standard psycopg2 connectivity, ON CONFLICT DO UPDATE for rerouted shipments and TIMESTAMPTZ for accurate event timing all work exactly as expected.

Why not just Snowflake? Snowflake is excellent for large-scale analytics and historical queries. Aggregating delay trends by minute, computing warehouse health scores across thousands of shipments and ranking suppliers by delay rate are all queries that benefit from Snowflake’s columnar storage and parallel execution. Running these queries against the operational Snowflake Postgres table would add load to the live system and slow down the simulator’s writes.

The three-system architecture isn’t complexity for its own sake. Each system does what it’s genuinely good at and the results are combined at the application layer.

Going Further

The system as built is a working demo. Here are some directions for making it more realistic.

Real supply chain topology. The supply chain graph is generated synthetically with a fixed random seed. A natural extension is to load a real supplier-warehouse-retailer network from a CSV or an ERP system export. The MERGE-based loading pattern in Chapter 2 is already idempotent, so replacing the synthetic data with real data requires only changing the source.

Multiple disruptions simultaneously. The simulator disrupts one warehouse at a time. In reality, multiple warehouses can go offline at once – due to weather events, regional power outages or carrier failures. Extending the disruption lifecycle to track a set of disrupted warehouses rather than a single one would make the routing more challenging and the analytics more interesting.

Kafka Streams processing. The simulator currently consumes events one at a time in a tight loop. A natural production extension is to add a Kafka Streams application between the producer and the consumer that enriches events in real time – adding routing decisions, delay predictions or SLA breach flags before they land in Snowflake Postgres. This decouples the enrichment logic from the consumer and scales independently.

Real delay prediction. Delay minutes are currently drawn from a uniform random distribution. A more realistic model would use a machine learning model trained on historical delay data to predict delay duration based on supplier, warehouse, route and time of day. The delay_minutes column is already in the schema – replacing the random draw with a model prediction requires only changing the event generation function.

Snowflake Dynamic Tables. The incremental sync from Snowflake Postgres to shipment_history is handled in application code. Snowflake’s Dynamic Tables feature can automate this – defining shipment_history as a Dynamic Table that refreshes automatically whenever the source data changes. This would remove the sync code from the analytics app entirely and keep the warehouse table current without any application involvement.

Streamlit Cloud deployment. Both Streamlit apps run locally in this demo. Deploying them to Streamlit Cloud would make the dashboards accessible from any browser without running a local server. The only change required is moving credentials from environment variables to Streamlit’s secrets management.