Data Engineering Weekly #228

Source: Substack

By Ananth PackkilduraiJuly 13, 2025

Free ELT Course from Dagster

Dagster is excited to announce the launch of ETL with Dagster, a comprehensive seven-lesson course. This free course guides you through practical ETL implementation and architectural considerations, from single-file ingestion to full-scale database replication.

Simply sign up at Dagster University to get started. Once enrolled, you can track your progress and learn at your own pace.

Sign up for the free course.

Thomas Kejser: Iceberg, The Right Idea – The Wrong Spec

In the article “Iceberg, the right idea, wrong spec,” the author presents several excellent points about the Iceberg spec, highlighting the operational complexities it entails. Storing metadata in this manner makes it significantly larger than necessary, leading to fragmented and bloated metadata, and the space management problem is a pressing issue.

The article is probably the beginning of the debate, along with Ducklake, about the next gen iteration of Lakehouse

https://database-doctor.com/posts/iceberg-is-wrong-1.html

https://database-doctor.com/posts/iceberg-is-wrong-2.html

Discuss

OnAir membership is required to make comments and add content.
Contact this post’s lead Curator/Moderator, Matthew Kovacev.

For more information, see our
DE Curation & Moderation Guidelines post. 

This is an open discussion on this news piece.

Home Forums Open Discussion

Viewing 1 post (of 1 total)
Viewing 1 post (of 1 total)
  • You must be logged in to reply to this topic.
Skip to toolbar