A neuron is just a weighted sum: build a perceptron in 20 lines of NumPy

LEARN · NEURAL NETWORKS FROM THE GROUND UP

Modern AI systems can contain billions of parameters, but the core idea behind a neural network starts with a much smaller building block.

A single artificial neuron is a simple computation:

Multiply each input by a learned weight.
Add the results together with a bias.
Apply an activation function.
Produce an output.

This is the foundation of the classic Perceptron, one of the earliest neural network models. In this lesson, you will build a Perceptron from scratch with NumPy and train it on a tiny dataset.

By the end, you will understand:

How a neuron makes a prediction.
How the Perceptron learning rule updates weights.
Why AND and OR are easy for a single neuron.
Why XOR exposed a fundamental limitation.
How this simple idea connects to modern neural networks.

The implementation is intentionally small. The goal is not to replace machine learning frameworks, but to understand the mechanism underneath them.

The neuron behind a Perceptron

A Perceptron computes a weighted sum of its inputs:

output = activation(x₁w₁ + x₂w₂ + … + xₙwₙ + bias)

Each part has a role:

Input: A feature from the dataset.
Weight: Controls how strongly an input influences the result.
Bias: Shifts the decision boundary.
Activation: Converts the score into an output.

For binary classification, a classic Perceptron uses a step activation:

def step(score):
    if score >= 0:
        return 1
    return 0

If the weighted sum reaches the threshold, the neuron outputs 1. Otherwise, it outputs 0.

Modern neural networks use more advanced activation functions, such as GELU or SiLU, and combine many layers of neurons. However, the core operation of combining inputs with learned weights remains central.

Teaching a neuron the AND function

A Perceptron can only learn patterns that can be separated by a straight decision boundary.

The logical AND function is one example of a linearly separable problem:

Input	Target
0, 0	0
0, 1	0
1, 0	0
1, 1	1

A single line can separate the positive example from the negative examples, so a Perceptron can learn this rule.

Building a Perceptron with NumPy

The entire model can fit into a small training loop:

import numpy as np

X = np.array([
    [0, 0],
    [0, 1],
    [1, 0],
    [1, 1],
], dtype=float)

y = np.array([0, 0, 0, 1])

weights = np.zeros(2)
bias = 0.0
learning_rate = 0.1

for epoch in range(20):
    for inputs, target in zip(X, y):
        score = np.dot(inputs, weights) + bias
        prediction = 1 if score >= 0 else 0

        error = target - prediction

        weights += learning_rate * error * inputs
        bias += learning_rate * error

print("Weights:", weights)
print("Bias:", bias)

for inputs in X:
    score = np.dot(inputs, weights) + bias
    prediction = 1 if score >= 0 else 0
    print(inputs.astype(int), "->", prediction)

A successful training run produces predictions like:

[0 0] -> 0
[0 1] -> 0
[1 0] -> 0
[1 1] -> 1

The model has learned the AND relationship.

What happens during training?

The training loop repeats three simple steps.

1. Calculate the weighted sum

The neuron combines the input values and current parameters:

score = np.dot(inputs, weights) + bias

For two inputs, this is equivalent to:

x₁w₁ + x₂w₂ + bias

The bias matters because it moves the decision boundary. Without it, the model would be forced to make every decision around the origin.

2. Convert the score into a prediction

The step activation creates a binary decision:

prediction = 1 if score >= 0 else 0

A classic Perceptron does not produce probabilities. It simply chooses a class.

3. Correct mistakes

The Perceptron learning rule changes the weights only when the prediction is wrong:

error = target - prediction

weights += learning_rate * error * inputs
bias += learning_rate * error

The update behavior is simple:

Correct prediction: error is 0, so parameters stay unchanged.
Incorrect prediction: weights move toward a better separating boundary.

Over many examples, the model adjusts itself.

Understanding the decision boundary

The weights describe the shape of the classifier.

Imagine the examples plotted on a graph:

Positive examples appear on one side.
Negative examples appear on the other side.
The Perceptron searches for a line separating them.

For example:

x₁(0.2) + x₂(0.2) – 0.3 >= 0

defines a possible decision rule.

Changing the weights changes the angle of the boundary.

Changing the bias moves the boundary.

This geometric idea extends into higher-dimensional spaces, where neural networks learn much more complex representations.

Why the Perceptron can converge

The Perceptron has an important theoretical property.

If the training data is linearly separable, the Perceptron learning algorithm is guaranteed to eventually find a separating boundary. This is known as the Perceptron Convergence Theorem.

The condition is important:

Linearly separable data: a single boundary can separate the classes.
Non-linearly separable data: no single boundary can solve the problem.

This explains both the power and limitation of the model.

Try OR with the same code

The OR function is also linearly separable.

Only the labels need to change:

y = np.array([0, 1, 1, 1])

The training process stays identical.

This is one of the most important lessons in machine learning: the algorithm is only part of the solution. The structure of the data determines what the model can learn.

Where one neuron fails: XOR

Now try the XOR function:

Input	Target
0, 0	0
0, 1	1
1, 0	1
1, 1	0

Change the labels:

y = np.array([0, 1, 1, 0])

A single Perceptron cannot solve XOR.

The reason is not a training bug. The problem is geometric. The positive examples and negative examples are arranged so that no single straight line can separate them.

A single neuron creates one linear decision boundary. Solving XOR requires combining multiple neurons into a network with hidden layers.

Cherry on the cake: the XOR story that changed AI history

In 1969, Marvin Minsky and Seymour Papert published Perceptrons, a detailed analysis of the mathematical limits of single-layer Perceptrons.

Their XOR result was correct: one Perceptron cannot represent XOR.

The surprising part was how the result affected the field. Many researchers interpreted these limitations as evidence that neural networks were not a promising direction, even though the analysis applied specifically to single-layer models.

Interest and funding for neural network research declined during the 1970s, contributing to a period often described as an AI winter.

The missing piece was multiple layers. Networks with hidden layers could represent functions that a single Perceptron could not, including XOR. Decades later, those same ideas became part of the foundation for modern deep learning.

The lesson remains useful: a limitation of one model architecture does not define the limits of an entire field.

How this connects to modern neural networks

Today’s neural networks are vastly larger than a Perceptron, but the basic building block is recognizable.

A neural network still:

Combines inputs with weights.
Adds biases.
Applies nonlinear transformations.
Adjusts parameters during training.

The difference is scale and architecture.

Modern systems use:

Many layers of neurons.
Large datasets.
Advanced optimization algorithms.
Specialized architectures such as transformers and convolutional networks.

A single Perceptron is not powerful enough for complex tasks, but it introduces the same fundamental ideas used in much larger models.

Experiments to try next

Once your Perceptron works, modify it and observe the results:

Count mistakes after each training epoch.
Stop training when an epoch has zero errors.
Start with random weights instead of zeros.
Create your own two-dimensional datasets.
Visualize the decision boundary.
Compare AND, OR, and XOR with the same code.

Small experiments create intuition that transfers directly to larger machine learning systems.

Key takeaways

A neuron begins with a weighted sum and an activation function.
A Perceptron is a simple binary classifier built from that idea.
NumPy is enough to implement one from scratch.
The Perceptron learning rule updates weights after mistakes.
AND and OR work because they are linearly separable.
XOR fails because one neuron cannot create enough complexity.
Modern neural networks expand this same foundation with many layers and parameters.

Continue your learning

Take this Perceptron implementation and upgrade it into a small neural network with a hidden layer. Your next goal is to make it solve XOR and understand how backpropagation allows networks to learn representations that a single neuron cannot.

Run the code, change the data, break the model, and experiment. Building intuition with small models is one of the fastest ways to understand large AI systems. Continue to the next lesson and build your first multi-layer neural network.