-
The lakehouse: Iceberg, Delta, and tables that time-travel
A plain object-storage data lake is wonderfully simple: put Parquet files in S3, Azure Data Lake Storage, Google Cloud Storage, HDFS, or another filesystem-like service, then point query engines at th
-
Orchestration beyond cron: DAGs, retries, and backfills with Airflow or Dagster
Cron is excellent at one job: starting a command at a particular time.
-
ELT and dbt: transforming data where it lives, with tests and lineage
ELT changes the center of gravity of a data pipeline.
-
Polars vs pandas: the DataFrame rewrite that changed the default
For years, the default choice for tabular Python was automatic: import pandas and start manipulating rows. That default has changed.
-
DuckDB: an analytics warehouse that fits in your laptop’s pocket
If you’ve ever wanted to analyze gigabytes of data without provisioning a database server or signing up for a cloud data warehouse, DuckDB is worth a serious look.
-
Parquet and Arrow: why columnar formats make your queries 100x faster
Large analytical datasets often start life as CSV files. They are easy to create, easy to open, and supported almost everywhere.