Clustering without labels: k-means, DBSCAN, and how to know if clusters are real

LEARN · CLASSICAL MACHINE LEARNING

Supervised machine learning starts with a comfortable assumption: examples come with answers.

You show a model houses with prices, emails with categories, or products with demand values, and the algorithm learns a relationship between inputs and outputs.

Clustering removes that safety net.

There is no target column saying:

“This customer belongs to segment A.”
“This machine is in failure mode B.”
“This group of users behaves differently from everyone else.”

Instead, a clustering algorithm searches for structure inside the data itself.

That sounds easier than prediction, but it creates the central challenge of unsupervised learning:

A clustering algorithm will always find something.

The important question is not:

“Did the algorithm create clusters?”

The important question is:

“Are these clusters meaningful, stable, and useful?”

This distinction separates professional clustering work from simply running an algorithm, coloring a scatter plot, and presenting the result as a discovery.

Two approaches appear frequently in practical machine learning projects:

k-means, a center-based method that groups observations around learned representatives.
DBSCAN, a density-based method that discovers connected regions and identifies isolated observations as noise.

They solve different problems. Understanding their failure modes is just as important as understanding their strengths.

What clustering is actually trying to discover

Imagine an online retailer analyzing customer behavior.

The company collects features such as:

Number of purchases in the last 90 days.
Average order value.
Number of product categories purchased.
Time between purchases.
Percentage of purchases made during promotions.

There is no correct label saying:

“This person is a premium customer.”
“This person is a bargain hunter.”
“This person is an occasional shopper.”

The clustering algorithm tries to discover groups where observations inside the same group are more similar to each other than observations in different groups.

The basic workflow is:

Represent each object as a point in a feature space.
Define similarity using a distance measurement.
Search for groups where internal similarity is high.
Validate whether those groups represent something useful.

The difficulty is that “similar” depends heavily on the data preparation.

A customer who spends €10,000 per year and a customer who buys 500 inexpensive items might represent completely different behaviors depending on whether the business values spending, frequency, or product diversity.

Clustering is therefore not only an algorithm decision. It is a complete process involving:

Feature engineering.
Scaling.
Algorithm selection.
Parameter tuning.
Validation.
Domain expertise.

k-means: fast, popular, and easy to misuse

The KMeans estimator in scikit-learn implements one of the most widely used clustering techniques.

The idea behind k-means is straightforward:

Choose the number of clusters, k.
Create k initial cluster centers.
Assign each point to the nearest center.
Move each center to the average position of its assigned points.
Repeat until the solution stabilizes.

The algorithm attempts to minimize the distance between points and their assigned centers.

This makes k-means extremely effective when the data naturally forms compact groups.

Examples where k-means often performs well:

Customer groups based on clearly separated purchasing behavior.
Sensor measurements with different operating states.
Geographic points that naturally form compact regions.

However, k-means has assumptions that are easy to forget.

It works best when clusters are:

Roughly spherical.
Similar in density.
Representable by a center point.
Not dominated by extreme outliers.

When those assumptions fail, k-means can confidently produce incorrect interpretations.

The problem is not that the algorithm is broken. The problem is that the question given to it does not match the structure of the data.

A practical k-means example

The following example creates synthetic data with three compact groups. This type of data matches the assumptions of k-means.

from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt

X, _ = make_blobs(
    n_samples=500,
    centers=3,
    cluster_std=1.2,
    random_state=42
)

model = KMeans(
    n_clusters=3,
    random_state=42,
    n_init="auto"
)

labels = model.fit_predict(X)

plt.scatter(
    X[:, 0],
    X[:, 1],
    c=labels,
    s=20
)

plt.title("k-means clusters")
plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.show()

The parameter n_init="auto" is part of the modern scikit-learn API behavior for KMeans and avoids relying on older implicit defaults.

The result looks convincing because the generated data was designed around the assumptions of the algorithm.

Real data is rarely so cooperative.

Where k-means breaks

Many real-world patterns do not look like separate blobs.

Consider:

Customers whose behavior follows a gradual spectrum rather than distinct groups.
Geographic activity following roads or rivers.
Network events forming irregular shapes.
User communities connected through relationships.

A center-based method struggles because every cluster must have a middle point.

A famous demonstration is the “moons” shape, where two groups form curved structures. k-means attempts to divide the space based on distance from centers rather than recognizing connected shapes.

from sklearn.datasets import make_moons
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt

X, _ = make_moons(
    n_samples=500,
    noise=0.08,
    random_state=42
)

model = KMeans(
    n_clusters=2,
    random_state=42,
    n_init="auto"
)

labels = model.fit_predict(X)

plt.scatter(
    X[:, 0],
    X[:, 1],
    c=labels,
    s=20
)

plt.title("k-means on non-spherical data")
plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.show()

The algorithm is following its design. It is searching for groups around centers.

The mistake is expecting a center-based method to discover shapes that are not center-based.

DBSCAN: discovering density instead of centers

DBSCAN takes a different approach.

Instead of asking:

“Which center is this point closest to?”

DBSCAN asks:

“Does this point belong to a dense neighborhood?”

It uses two main parameters:

eps: the maximum distance between neighboring points.
min_samples: the minimum number of nearby points required to form a dense area.

Points are classified as:

Core points: observations surrounded by enough neighbors.
Border points: observations near a dense region but not dense themselves.
Noise points: isolated observations.

This makes DBSCAN useful for:

Finding irregularly shaped groups.
Detecting unusual events.
Geographic clustering.
Separating signal from noise.

Running DBSCAN on curved data

DBSCAN can identify structures that k-means misses.

from sklearn.datasets import make_moons
from sklearn.cluster import DBSCAN
import matplotlib.pyplot as plt

X, _ = make_moons(
    n_samples=500,
    noise=0.08,
    random_state=42
)

model = DBSCAN(
    eps=0.15,
    min_samples=5
)

labels = model.fit_predict(X)

plt.scatter(
    X[:, 0],
    X[:, 1],
    c=labels,
    s=20
)

plt.title("DBSCAN on curved data")
plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.show()

DBSCAN is not automatically better.

Its quality depends heavily on choosing suitable parameters.

If eps is too small:

Many points become noise.
True clusters may be fragmented.

If eps is too large:

Different groups may merge.
Noise may become part of clusters.

DBSCAN also struggles when clusters have very different densities because one density threshold cannot perfectly describe every region.

How do you know if clusters are real?

This is the hardest part of clustering.

A colorful visualization is not proof that meaningful groups exist.

A model can create beautiful clusters from random patterns.

Professional clustering requires validation.

The elbow method

For k-means, a common approach is measuring within-cluster variation for different values of k.

The workflow:

Run k-means with several possible cluster counts.
Measure the remaining within-cluster distance.
Look for a point where additional clusters provide diminishing improvement.

This point is often called the elbow.

from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
import matplotlib.pyplot as plt

X, _ = make_blobs(
    n_samples=500,
    centers=4,
    random_state=42
)

scores = []

for k in range(1, 10):
    model = KMeans(
        n_clusters=k,
        random_state=42,
        n_init="auto"
    )
    model.fit(X)
    scores.append(model.inertia_)

plt.plot(range(1, 10), scores, marker="o")
plt.xlabel("Number of clusters")
plt.ylabel("Within-cluster distance")
plt.title("Elbow method")
plt.show()

The elbow method is useful, but it is not a mathematical truth detector.

Some datasets have:

No obvious elbow.
Multiple possible elbows.
A visually convincing but statistically weak structure.

Silhouette score

Another common evaluation method is the silhouette score.

It compares:

How close a point is to its own cluster.
How far it is from the nearest alternative cluster.

Scores close to 1 indicate well-separated groups.

Scores close to 0 indicate overlapping groups.

Negative values can suggest poor assignments.

from sklearn.metrics import silhouette_score
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs

X, _ = make_blobs(
    n_samples=500,
    centers=4,
    random_state=42
)

model = KMeans(
    n_clusters=4,
    random_state=42,
    n_init="auto"
)

labels = model.fit_predict(X)

score = silhouette_score(X, labels)

print(score)

However, a high silhouette score does not guarantee business value.

A company does not need mathematically attractive groups. It needs groups that improve decisions.

Why preprocessing matters

Distance-based algorithms are extremely sensitive to feature scale.

Imagine clustering server telemetry:

CPU usage ranges from 0 to 100.
Request count ranges from 0 to 1,000,000.

Without scaling, request count dominates the distance calculation.

The algorithm may effectively ignore CPU usage.

A common solution is standardization.

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

X_scaled = scaler.fit_transform(X)

The same principle applies to customer analytics, energy monitoring, retail data, and almost any dataset where features use different units.

Cherry on the cake: when machine learning artifacts became a security problem

Clustering models are still software artifacts, and software artifacts have security risks.

A memorable example is CVE-2020-13092, a scikit-learn issue involving unsafe deserialization of untrusted files passed through joblib.load(). The vulnerability involved the ability to execute commands when loading a malicious serialized object. The issue was also discussed as a responsibility boundary problem because unsafe deserialization is a known risk of pickle-based workflows.

The broader lesson is more important than the individual CVE:

A model file is not just data.

A serialized machine learning artifact can contain executable behavior.

Good practices include:

Never load model files from unknown sources.
Verify where artifacts came from.
Keep machine learning dependencies updated.
Treat model storage as part of your software supply chain.

Modern scikit-learn documentation also warns that pickle and joblib-based persistence formats should only be used with trusted artifacts because they can execute code during loading.

A practical clustering checklist

Before presenting clusters, ask:

Did I scale the features correctly?
Did I remove irrelevant columns?
Did I test multiple algorithms?
Are the clusters stable across random seeds?
Do domain experts recognize the patterns?
Do the clusters improve a real decision?
Would the groups still exist on new data?

If the answers are unclear, the clusters are an experiment, not a conclusion.

Final thoughts

k-means and DBSCAN represent two different philosophies.

k-means says:

“Groups are organized around centers.”

DBSCAN says:

“Groups are organized around density.”

Neither approach is universally superior.

The correct choice depends on:

Data shape.
Noise level.
Feature scaling.
Business context.
The decision you want to improve.

The biggest clustering skill is not knowing how to call an API.

It is knowing when the output deserves trust.

Start with visualization. Test assumptions. Measure quality. Talk to people who understand the domain. A colorful chart takes minutes to create. A reliable discovery requires much more care.

Take the examples from this guide, run them on your own non-sensitive dataset, compare KMeans and DBSCAN, and document why your final clusters deserve to exist.

Which follow-up would you prefer: a KMeans parameter guide or a DBSCAN troubleshooting guide?