-
Living with pyspark.pandas: interop, performance traps, and a pandas-to-Spark migration checklist
Lesson 11 mapped what the pandas API on Spark implements; lesson 12 proved on 103 million rows that the port can be a handful of changed lines. This closing lesson is about operating it: moving data b
-
One dataset, two engines: scaling California housing until pandas gives up, then rerunning it on pyspark.pandas
Lesson 11 mapped what the pandas API on Spark implements. This lesson measures. We take a dataset every DS course uses — scikit-learn’s California housing, 20,640 rows — and multiply it with jitter un
-
pandas API on Spark in depth: what pyspark.pandas covers, what it doesn’t, and how it executes
You already know the promise from earlier lessons in this track: when a dataset outgrows one machine, Spark distributes the work. The catch has always been the rewrite — a pandas notebook does not bec
-
Which model when: a practical decision playbook for tabular, text, images, and small data
Model selection gets easier when you stop asking, “What is the best machine-learning model?” and ask a more useful set of questions:
-
Orchestration beyond cron: DAGs, retries, and backfills with Airflow or Dagster
Cron is excellent at one job: starting a command at a particular time.
-
Topics Everyone Is Talking About No411
What Comes After FOSS? • Passphrase-less reboots using kexec under NixOS • AI;DR AI; Didnt Read • Stripe will reportedly acquire OpenRouter for 7B • Qwen 3.8 27B is excellent, but it defaults to overthinking things…
-
vLLM and PagedAttention: why LLM serving throughput jumped 10x
A language model can generate only one next token per sequence at a time. That sounds like an inherently serial workload, and at the level of one request it largely is.
-
SVMs and the kernel trick: separating the inseparable
Many machine-learning problems become easier once you stop thinking first about algorithms and start thinking about geometry.
-
Topics Everyone Is Talking About No410
Claude: System Prompts • Models Are Getting Dumber on Purpose • What happens when an LLM never sees material beyond fifth grade? • I thought I was building a C replacement. I was wrong • Every Fucking Website…
-
Topics Everyone Is Talking About No409
Zig Day Seattle, WA 3 • Why I remain a skeptic • Working with AI Feels More Like Leadership Than Coding • Abdominal fat predicts heart disease risk better than BMI • The First At-Home Test for Infected Ticks Could Improve Lyme Disease Diagnosis…