Adversarial examples: fooling image classifiers with invisible noise

Deep neural networks have transformed computer vision. They can classify objects, detect scenes, and power systems that operate at enormous scale. Yet the same models that achieve impressive benchmark accuracy can fail in ways that are unexpected from a human perspective.

A carefully designed adversarial example can cause a model to make an incorrect prediction while the input appears unchanged or nearly unchanged to a person. The attacker does not need to damage the entire image or introduce obvious visual corruption. Instead, they search for a small, optimized modification that exploits the mathematical structure of the model.

This creates an important security question:

If a computer vision model performs well on normal data, how confidently can we trust it when an attacker intentionally manipulates the input?

Adversarial machine learning studies this question by analyzing how models break, how attacks are constructed, and how defenses can improve reliability.

A typical adversarial example follows this pattern:

• A clean image is provided to a neural network.

• The model produces a prediction with high confidence.

• An attacker calculates a small perturbation.

• The modified image causes the model to predict incorrectly.

• The perturbation may be almost invisible to humans.

The key issue is that neural networks do not necessarily learn the same concepts humans use. A person may identify a dog through shape, texture, and context. A model may also rely on subtle pixel-level statistical patterns that are useful during training but fragile under intentional manipulation.

For production computer vision systems, this distinction matters. A model deployed in a factory, vehicle, security camera, or automated decision pipeline is not operating only against natural variation. It may face inputs designed specifically to exploit its weaknesses.

An image classifier maps an input image to a set of output scores. Internally, the network produces logits, which are converted into probabilities.

The predicted class is usually the class with the highest probability:

Prediction = class with the highest confidence

During training, the model learns parameters that minimize errors on its training data. However, deep neural networks often create decision boundaries in extremely high-dimensional spaces.

An image is not just a picture. From the model’s perspective, it is a point represented by thousands or millions of numerical values.

A classifier divides this space into regions:

• One region may contain images classified as cars.

• Another region may contain images classified as bicycles.

• Another region may contain images classified as animals.

An adversarial attack searches for a small movement in this space that crosses a decision boundary.

The attacker tries to satisfy three goals:

• Keep the perturbation small.

• Change the model output.

• Maximize the attack objective.

For an untargeted attack, the goal is simply to make the original prediction fail.

For a targeted attack, the attacker chooses a specific incorrect class and optimizes the image until the model predicts that target.

For example:

• Untargeted goal: “Make this stop sign classification wrong.”

• Targeted goal: “Make this stop sign classified specifically as a speed-limit sign.”

Targeted attacks are generally more difficult because the attacker must move the prediction toward a particular outcome rather than any incorrect outcome.

The difficulty of an adversarial attack depends heavily on what information the attacker has.

White-box attacks

A white-box attacker has access to internal model details:

• Architecture.

• Parameters.

• Training configuration.

• Gradients.

This is the easiest environment for attack development because the attacker can directly calculate how changing pixels affects the model output.

FGSM and PGD are common white-box attacks.

Black-box attacks

A black-box attacker cannot inspect the model internals.

Instead, they may interact with:

• A prediction API.

• A deployed application.

• A confidence score endpoint.

The attacker can estimate useful information by repeatedly querying the system.

Black-box attacks are especially relevant for commercial AI services because model weights are usually hidden.

Transfer attacks

One of the most surprising discoveries in adversarial research is transferability.

An adversarial image created against one model can sometimes fool another model.

This means an attacker may not need access to the exact production system. They can train or obtain a similar model, generate adversarial examples, and test whether those examples transfer.

The Fast Gradient Sign Method (FGSM) is one of the simplest and most influential adversarial attacks.

The idea is straightforward:

1. Run the input through the model.

2. Calculate the loss.

3. Compute the gradient of that loss with respect to the input image.

4. Modify pixels in the direction that increases the loss.

The simplified update is:

Adversarial image = original image + epsilon × sign(gradient)

Here:

• epsilon controls the perturbation size.

• sign(gradient) keeps only the direction of change.

• The attack attempts to maximize the model’s error.

FGSM is fast because it requires only one gradient calculation. However, its simplicity also limits its strength. More advanced attacks usually perform iterative optimization.

A common mistake when implementing adversarial attacks is ignoring preprocessing.

Pretrained torchvision models do not usually receive raw pixel values directly. Images are normalized using channel-specific mean and standard deviation values.

That means the model does not operate in the range 0 to 1 after preprocessing. An attack implementation must either:

• Perform the attack in normalized space and clamp using normalized bounds.

• Convert back to image space before applying pixel constraints.

The following example uses normalized-space attacks correctly.

Create an isolated environment:

python -m venv .venv

Activate it:

source .venv/bin/activate

Install tested package versions:

pip install torch==2.7.1 torchvision==0.22.1 pillow==11.1.0 matplotlib==3.10.3

The example assumes you have a normal everyday image such as an object, vehicle, animal, or household item stored as sample.jpg.

import torch
import torchvision.models as models
from PIL import Image
import matplotlib.pyplot as plt

device = "cuda" if torch.cuda.is_available() else "cpu"

weights = models.ResNet50_Weights.DEFAULT
model = models.resnet50(weights=weights)
model.eval()
model.to(device)

preprocess = weights.transforms()

image = Image.open("sample.jpg").convert("RGB")

clean_tensor = preprocess(image).unsqueeze(0).to(device)
clean_tensor.requires_grad = True

with torch.no_grad():
    clean_output = model(clean_tensor)

original_label = clean_output.argmax(dim=1)

output = model(clean_tensor)

loss = torch.nn.functional.cross_entropy(
    output,
    original_label
)

model.zero_grad()
loss.backward()

epsilon = 0.01

gradient = clean_tensor.grad.detach()

adversarial_tensor = (
    clean_tensor.detach()
    + epsilon * gradient.sign()
)

mean = torch.tensor(
    weights.transforms().mean,
    device=device
).reshape(1, 3, 1, 1)

std = torch.tensor(
    weights.transforms().std,
    device=device
).reshape(1, 3, 1, 1)

lower_bound = (torch.zeros_like(mean) - mean) / std
upper_bound = (torch.ones_like(mean) - mean) / std

adversarial_tensor = torch.max(
    torch.min(adversarial_tensor, upper_bound),
    lower_bound
)

with torch.no_grad():
    adversarial_output = model(adversarial_tensor)

adversarial_label = adversarial_output.argmax(dim=1)

print("Original class index:", original_label.item())
print("Adversarial class index:", adversarial_label.item())

display_image = (
    adversarial_tensor * std + mean
)

display_image = (
    display_image.squeeze(0)
    .permute(1, 2, 0)
    .cpu()
    .clamp(0, 1)
)

plt.imshow(display_image)
plt.axis("off")
plt.show()

This is a demonstration, not a complete robustness evaluation. Real security testing requires multiple images, different perturbation strengths, repeated trials, and careful reporting.

FGSM takes one optimization step. Projected Gradient Descent (PGD) takes many smaller steps.

A simplified PGD workflow is:

1. Start with the original image.

2. Add a small adversarial update.

3. Measure the new gradient.

4. Repeat.

5. Project the result back into the allowed perturbation region.

The projection step is essential because it keeps the adversarial example within a defined threat model.

PGD is frequently used as a robustness baseline, but it is not a universal measure of security. Results depend on:

• The attack norm.

• The perturbation budget.

• Number of iterations.

• Random restarts.

• Whether the attacker uses adaptive strategies.

• Whether the model contains defenses designed to confuse gradients.

A model that survives one PGD configuration may still fail against a different threat model.

Adversarial attacks usually define “small” using mathematical distance measures called norms.

Common choices include:

L-infinity attacks

L-infinity limits the largest individual pixel change.

It asks:

“How much can any single pixel change?”

This is popular for invisible noise attacks.

L2 attacks

L2 measures the overall Euclidean distance between images.

It asks:

“How much total change was introduced?”

L1 attacks

L1 measures total absolute change and can encourage sparse modifications.

The chosen norm changes the meaning of robustness. A model resistant to one type of perturbation may not be resistant to another.

The following function expects images that are already prepared exactly as the model expects. Because torchvision models use normalized inputs, the epsilon and alpha values here are interpreted in normalized space.

import torch


def pgd_attack(
    model,
    images,
    labels,
    epsilon=0.03,
    alpha=0.005,
    iterations=10
):
    original = images.detach()

    adversarial = images.detach().clone()

    for _ in range(iterations):
        adversarial.requires_grad = True

        outputs = model(adversarial)

        loss = torch.nn.functional.cross_entropy(
            outputs,
            labels
        )

        model.zero_grad()
        loss.backward()

        with torch.no_grad():
            step = alpha * adversarial.grad.sign()

            adversarial = adversarial + step

            delta = adversarial - original

            delta = torch.clamp(
                delta,
                -epsilon,
                epsilon
            )

            adversarial = original + delta

    return adversarial.detach()

A stronger evaluation would add:

• Random initialization inside the allowed perturbation region.

• Targeted attack support.

• Multiple restarts.

• Different norm constraints.

• Adaptive attack logic.

One of the most important lessons in adversarial security is that a failed attack does not always mean a robust model.

Some defenses accidentally hide gradients instead of improving security.

This is called gradient masking or gradient obfuscation.

A model may appear resistant because:

• Gradients become noisy.

• Optimization becomes unstable.

• The attack implementation is incompatible.

However, an attacker using a better method, black-box optimization, or an adaptive strategy may still succeed.

A proper evaluation should assume that attackers understand the defense and modify their approach accordingly.

Clean accuracy alone is not enough.

A robust evaluation includes:

Clean accuracy

How well does the model perform on normal inputs?

Attack success rate

How often does an attack force the intended failure?

Robust accuracy

How many examples remain correct after the attack?

Perturbation measurement

How large was the modification?

Useful reporting includes:

• Attack type.

• Norm constraint.

• Perturbation budget.

• Number of optimization steps.

• Success rate.

Security testing should describe the threat model clearly. A statement such as “the model is robust” is incomplete without specifying what attacks were considered.

No single defense eliminates every adversarial risk. Practical systems combine several techniques.

Adversarial training

Adversarial training includes attack-generated examples during training.

The model learns from:

• Normal examples.

• Deliberately manipulated examples.

A simplified workflow:

for images, labels in training_data:
    adversarial_images = create_attack(
        model,
        images,
        labels
    )

    combined_batch = concatenate(
        images,
        adversarial_images
    )

    train_model(
        model,
        combined_batch,
        labels
    )

The disadvantage is cost. Generating attacks during training increases computation significantly.

Input processing defenses

Some systems attempt to reduce attack impact through:

• Compression.

• Resizing.

• Random transformations.

• Noise reduction.

These approaches can help in specific environments but should not be treated as complete protection. Adaptive attackers can often optimize against preprocessing.

Certified robustness

Certified methods attempt to provide mathematical guarantees that certain perturbations cannot change the prediction.

These methods are valuable when guarantees are required, but they usually involve tradeoffs in scalability and model performance.

A particularly interesting real-world development came from researchers at Carnegie Mellon University, who published work on adversarial patches in 2017.

The paper “Adversarial Patch” by Tom B. Brown and collaborators demonstrated that a printed physical patch could fool image classifiers in real-world settings.

The researchers showed examples where a specially designed patch placed near an object could cause a classifier to misidentify it. Unlike invisible pixel perturbations, the patch was visible, but it exploited the way neural networks process visual patterns.

This changed the security conversation because it demonstrated that adversarial attacks were not limited to carefully modified digital files.

Physical attacks matter because many computer vision systems interact with the real world:

• Robots.

• Autonomous machines.

• Industrial inspection cameras.

• Security systems.

The lesson is significant: attackers may not need access to your image files. They may manipulate the environment itself.

Building reliable computer vision systems requires thinking beyond benchmark accuracy.

A responsible workflow includes:

• Define the attacker model before deployment.

• Test with adaptive attacks.

• Monitor unusual inputs.

• Avoid relying only on confidence scores.

• Keep human review for high-impact decisions.

• Continuously evaluate after deployment.

The important question is not only:

“Does the model work?”

It is also:

“How can this model fail under intentional pressure?”

Adversarial examples reveal a broader truth about artificial intelligence. Statistical performance does not automatically equal understanding, and accuracy does not automatically equal security.

The fastest way to understand adversarial security is to experiment. Build a small classifier, generate FGSM and PGD attacks, measure the failures, and evaluate defenses against realistic threat models.

Continue with the next lesson in this course, run the attack demonstrations on your own computer vision projects, and subscribe for more deep technical guides on building safer AI systems.