Poor mans datalake

layout: post title: “Poor mans datalake” subtitle: “DuckDb” date: 2023-02-01 author: “Niklas Hansson” URL: “/2023/02/01/Trition_with_post_and_pre_processing/”

This post is a deep dive playing with DuckDB doing a twist on Build a poor man’s data lake from scratch with DuckDB where we will do the following changes:

• Use Minio instead of S3
• DBT instead of dagster.

We will host it on Kubernetes and set it up so it all run locally.