Data Lake

A data lake is a large, central store for raw data in whatever form it arrives: database exports, application logs, clickstream events, JSON files, images, and sensor readings. The data is kept as files, usually in inexpensive and highly scalable object storage. Unlike a data warehouse, a data lake does not require you to define a schema before you store something.

Files in a lake are organized into folders, often partitioned by date or source, and are commonly saved in columnar formats such as Parquet that make analytical queries efficient. A catalog records which datasets exist and what their columns look like, and query engines read the files directly with SQL, applying the structure at read time, an approach called schema-on-read. Open table formats such as Apache Iceberg and Delta Lake add transactions, schema changes, and time travel on top of plain files, which is the basis of the lakehouse approach that combines lake storage with warehouse-style tables.

If a data warehouse is a supermarket with labeled shelves of ready-to-use products, a data lake is a reservoir where water from many streams collects in its natural state and is filtered only when someone draws from it. Data lakes are used for machine learning training data, data science exploration, cheap long-term retention, and log archives, especially when nobody knows yet exactly which questions the data will answer.

The key confusion is with a data warehouse. A warehouse stores cleaned, structured, modeled data designed for business reporting and applies the schema when data is written, while a lake stores everything raw and is cheaper and more flexible but slower to query and harder to trust. Without a catalog, quality checks, and access control, a lake turns into a data swamp that nobody can navigate. Many organizations use both, with ETL or ELT pipelines refining lake data into warehouse tables.