History repeats itself. I remember when the web (1.0 at the time :-) was booming and people were struggling maintaining their website’s content. The options were either to have it all be part of the massive index.php with a phpmyadmin view to manage the content. Or have this elaborate content…

As a data engineer I often have to do stuff with kubernetes. Sometimes I’m lucky and get a client that uses some cloud hosted solution, but often they’ll have some mega kubernetes cluster running on something that is only accessible via some bastion host via a vpn connection. Which is secure and…

Day 1 of the journey on building my first Scala GraalVM ZIO application » Day 0 » Day 1 Continued from Day 0 Day 1: Getting builds under control Now that we have a basic project setup and somewhat building, we need to stabilise it a bit more, so we can spend all our future energy…

Day 0 of the journey on building my first Scala GraalVM ZIO application » Day 0 » Day 1 Caveat: Normally I write posts about how I tackled a problem and present the solution on a silver platter. In this series I’d like you to take you on a journey…

For each challenge there are many technology stacks that can provide the solution. I’m not claiming this approach is the holy grail of data processing, but this more the tale of my quest to combine these widely supported tools in a maintainable fashion. From the onset I’ve always tried to…

A scalable approach to fuzzy data matching — The challenge Recently a colleague at Datlinq asked me to help her with a data problem, that seemed very straightforward at a glance. She had purchased a small set of data from the chamber of commerce (Kamer van Koophandel: KvK) that contained roughly 50k small sized companies (5–20FTE), which can be hard…

Since my childhood I’ve always been a coder. I got started with some GW BASIC, but quickly moved to C and C++ during my high school years, though I never really considered it as a possible future occupation. It was more of a fun hobby and since my friends also…