Join the Delta Lake Community
Delta Lake is supported by more than 190 developers from over 70 organizations across multiple repositories.
Chat with fellow Delta Lake users and contributors, ask questions and share tips.
Scheduled and Latest Videos
Director of Platform Engineering, Scribd
Contribute
Help us build the simplest, most complete, battle-tested open-source storage framework ever!
Below are a few great ways to get started to contribute.
192 contributors | 43 organizations
Delta Lake connectors for Apache Spark™, with Scala/Java and Python APIs, and for Java-based engines like Hive, Flink, PrestoDB, etc.
50 contributors | 26 organizations
Delta Lake connector written natively in Rust, with Python bindings.
19 contributors | 2 organizations
An open protocol for simple and secure data sharing.
6 contributors | 2 organizations
Delta Lake connector for streaming data from Kafka into Delta tables.
7 contributors | 3 organizations
Delta Lake website and documentation built on Gatsby.js framework.
Refer to the Delta Lake contribution guide for the latest on communication, coding style, and how to sign your work.
Refer to the Delta Lake Protocol for the specification for the Delta Transaction Protocol, which brings ACID properties to large collections of data, stored as files, in a distributed file system or object store.
Want to help or to help out?
Looking for ways to help or to get help? Refer to the Getting Help Guide.
Governance
Delta Lake is an independent open-source project and not controlled by any single company. To emphasize this we joined the Delta Lake Project in 2019, which is a sub-project of the Linux Foundation Projects. Within the project, we make decisions based on these rules.
Delta Lake is supported by a wide set of developers from over 70 organizations across multiple repositories. Since 2019, more than 190 developers have contributed to Delta Lake! The Delta Lake community is growing by leaps and bounds with more than 6200 members in the Delta Users slack.
For more information, please refer to the founding technical charter.
Integrating a Data Warehouse and a Data Lake
FLORIAN VALEYE
STAFF DATA ENGINEER BACK MARKET
This conversation from the Open Source Summit in Spain provides valuable insights into the significance of Delta Lake, the role of Rust in data engineering, and the collaborative nature of open source communities.
The Delta Lake open source project integrates data lakes and data warehouses, a needed combination in this new age of scale-out data requiring reliability and controls.
A data lakehouse integrates the advanced data analytics and low-cost storage of a data lake with the performance and reliability of a data warehouse, said Florian Valeye, a data engineer with Back Market, in this episode of The New Stack Makers, recorded at the Open Source Summit in Bilbao, Spain, earlier this fall.
Delta Lake:
The Definitive Guide
Early Release (Raw & Unedited) on building modern data lakehouse architectures with Delta Lake
Download
Project Maintainers
Want to dive deeper into Delta Lake, please chat with any of our maintainers!
For more information on how to contribute, please refer to the Delta Lake contribution guide.
| Name | Organziation | Focal Areas |
|---|---|---|
| Michael Armbrust | Databricks | delta, connectors, delta-sharing |
| R. Tyler Croy | Scribd | delta, delta-rs, kafka-delta-ingest |
| Tathagata Das | Databricks | delta, connectors |
| QP Hou | Neuralink | delta, delta-rs |
| Will Jones | Voltron Data | delta, delta-rs |
| Venki Korukanti | Databricks | delta, connectors |
| Denny Lee | Databricks | delta, connectors, delta-sharing, delta-rs, website |
| Mykhailo Osypov | Scribd | delta-rs, kafka-delta-ingest |
| Robert Pack | BASF | delta, delta-rs |
| Allison Portis | Databricks | delta, connectors |
| Scott Sandre | Databricks | delta, connectors |
| Florian Valeye | Backmarket | delta, delta-rs |
| Thomas Vollmer | Microsoft | delta-rs |
| Christian Williams | Scribd | delta, delta-rs, kafka-delta-ingest |
| Ryan Zhu | Databricks | delta, connectors, delta-sharing |
| Gerhard Brueckl | Paiqo GmbH | delta, connectors, delta-sharing |
Organizations that have contributed to Delta Lake
Together we have made Delta Lake the most widely used lakehouse format in the world!
Project Governance
Delta Lake is an independent open-source project and not controlled by any single company. To emphasize this we joined the Delta Lake Project in 2019, which is a sub-project of the Linux Foundation Projects. Within the project, we make decisions based on these rules.