-
Spark ML feature pipelines: VectorAssembler, StringIndexer, OneHotEncoder, and the Pipeline API
Machine learning models rarely consume raw tables directly. Before training, data usually needs cleaning, categorical encoding, feature assembly, and a consistent transformation path for future predic
-
PySpark DataFrames, shuffles, and fast UDFs: pandas_udf + Arrow beat Python UDFs 10x
PySpark is often introduced as “Spark with Python,” but that description hides the most important performance detail: Python is not where Spark does most of its work.
-
Why PySpark for ML: when your data outgrows pandas and a single machine
If you build machine learning systems in Python, pandas is often the first tool you reach for. It is simple, expressive, and excellent for exploration. For many datasets, it is exactly the right choic
-
Logistic regression as a classifier you can actually explain
Machine learning often feels like a trade-off between accuracy and interpretability. While deep neural networks and gradient-boosted trees dominate many benchmarks, there are plenty of real-world prob
-
Missing data is a message: patterns, mechanisms, and honest imputation
Missing values are often treated as an inconvenience: fill them, drop them, move on.
-
Matrix multiplication is all you need: shapes, batching, and why GPUs love it
If you learn one operation in modern machine learning, make it matrix multiplication (often shortened to matmul ).
-
Backpropagation demystified: hand-compute the gradients, then verify with autograd
Backpropagation is the engine behind modern deep learning. Whether you’re training a tiny multilayer perceptron or a frontier-scale language model, every optimization step depends on efficiently compu
-
Ship the rule-based baseline first: why 20 lines of if/else gate every good ML project
When people start an ML project, the instinct is often to train a model immediately. Open a notebook, split the data, fit a classifier, and compare metrics.
-
Encoders shootout: one-hot, ordinal, target, and hashing — when each wins and when it leaks
Categorical features are everywhere in tabular machine learning:
-
Parquet and Arrow: why columnar formats make your queries 100x faster
Large analytical datasets often start life as CSV files. They are easy to create, easy to open, and supported almost everywhere.