Part 4/12:

• Operational Limitations: No clear separation between storage and compute meant noisy queries could impact others, leading to high contention.

• Delayed Issue Detection: The inability to analyze data in near real-time meant problems like inventory shortages or system errors were detected too late.

• High Costs: Scaling infrastructure and managing concurrency was expensive.

These limitations prompted Zepto to rethink its entire data framework, seeking a solution that could support their speed and scale requirements.

Moving Toward a Modern Data Lakehouse

Zepto transitioned from the traditional warehouse to a more flexible, scalable architecture incorporating Delta Lake and data bricks. The new architecture leverages: