-
Numerical stability: why log-sum-exp exists and how float32 silently lies
A machine learning model produces three logits: 1,000, 1,001, and 999. You apply softmax, expecting three probabilities that sum to one.
A machine learning model produces three logits: 1,000, 1,001, and 999. You apply softmax, expecting three probabilities that sum to one.