Clustering & Unsupervised ML

Clustering is unsupervised ML: no labels, just data. The goal is to discover natural groupings — customer segments, anomaly clusters, document topics. Unlike classification, there's no "right answer"; the analyst's job is to pick a sensible algorithm, validate the clusters, and translate them into business segments stakeholders can act on.

What Clustering Actually Is

Given points in feature space, clustering algorithms group points so that:

• Points in the same cluster are close to each other.
• Points in different clusters are far apart.

"Close" depends on the distance metric, and "clusters" depend on the algorithm. There's no universal answer — different methods give different shapes.

The Three Algorithms That Cover 90% of Cases

Algorithm	Shape	Need K?	Handles noise?
K-Means	spherical	yes	no
Hierarchical (Agglomerative)	any (via linkage)	optional (cut tree)	no
DBSCAN	arbitrary	no (eps + min_samples)	yes ("noise" label)

Learn these three. Reach for GMM / HDBSCAN / Spectral once they aren't enough.

Beginner Mistakes to Skip

1. Forgetting to scale. Income (50000) overwhelms age (30). StandardScaler first, *always*. 2. Picking K by eye. Use the elbow method + silhouette score, not vibes. 3. Clustering raw categorical data. K-Means assumes Euclidean distance — one-hot encode or use K-Modes / Gower distance. 4. Trusting K-Means for non-spherical clusters. It draws sphere-ish boundaries; use DBSCAN for crescents/blobs. 5. Reading too much into clusters. They're a *hypothesis*, not truth. Validate with profile tables and business sense. 6. High-dimensional clustering without reducing first. Distances become meaningless past ~50 dims; reduce with PCA / UMAP.

Intermediate: Distance Metrics

Metric	When to use
Euclidean	continuous numeric data (default)
Manhattan	grid-like, sparse, robust to outliers
Cosine	text / TF-IDF / embeddings
Jaccard	binary set membership
Gower	mixed numeric + categorical

The metric is more important than the algorithm for getting useful clusters.

Intermediate: K-Means in Detail

Lloyd's algorithm:

1. Pick K random centroids. 2. Assign each point to the nearest centroid. 3. Move each centroid to the mean of its assigned points. 4. Repeat until assignments stop changing.

Use KMeans(n_clusters=K, n_init=10, random_state=42) — n_init runs the algorithm 10 times with different seeds and keeps the best (avoids bad local minima). Use K-Means++ initialisation (default) for faster convergence.

Intermediate: Choosing K — Elbow + Silhouette

from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
inertia, sil = [], []
for k in range(2, 11):
    km = KMeans(n_clusters=k, n_init=10, random_state=42).fit(X)
    inertia.append(km.inertia_)
    sil.append(silhouette_score(X, km.labels_))

• Elbow: plot inertia vs K, look for the bend. Subjective.
• Silhouette: numeric score in [-1, 1]. Pick K with the highest score. Objective.
• Combine both — silhouette breaks ties when the elbow is fuzzy.

Intermediate: Silhouette Coefficient

For each point i:

s(i) = (b(i) - a(i)) / max(a(i), b(i))

where a(i) = avg distance to own cluster, b(i) = avg distance to nearest other cluster.

• ~+1 — well-clustered.
• ~0 — on a boundary.
• < 0 — likely in the wrong cluster.

Mean silhouette is the single best one-number summary of clustering quality.

Intermediate: Hierarchical (Agglomerative) Clustering

Start with each point as its own cluster, repeatedly merge the closest pair until one remains. Output is a dendrogram — cut it at any height to choose K.

Linkage choices:

• single — nearest pair (chains, sensitive to noise)
• complete — farthest pair (compact)
• average — mean of all pairs
• ward — minimises within-cluster variance (best default)

Great for small datasets (<10k) and when you want to *see* the cluster structure.

Advanced: DBSCAN

Density-based: a cluster is a dense region of points; sparse regions are *noise*.

Two knobs:

• eps — neighbourhood radius. Tune with a k-distance plot: sort distances to k-th NN, look for the knee.
• min_samples — minimum points to form a dense region. Higher → denser, larger clusters.

Strengths: finds arbitrary shapes, no K, labels outliers as -1. Weaknesses: struggles with varying density (use HDBSCAN instead).

Advanced: Validating Clusters Internally vs Externally

• Internal (no ground truth): silhouette, Davies-Bouldin, Calinski-Harabasz.
• External (with labels): Adjusted Rand Index, Normalised Mutual Information.
• Stability: cluster on bootstrap samples — do points keep landing together?
• Business validity: build a cluster profile table (mean of each feature per cluster, count) and confirm clusters tell a coherent story.

Advanced: High-Dimensional & Mixed Data

• Dimensionality reduction first — PCA (linear), UMAP / t-SNE (non-linear). UMAP often *creates* the clusters DBSCAN then formalises.
• Mixed data — use K-Prototypes (numeric + categorical) or Gower distance with hierarchical.
• Text/embeddings — cosine distance + spherical K-Means or HDBSCAN on UMAP-reduced embeddings.

Advanced: Customer Segmentation Pipeline

Production-grade segmentation:

1. Engineer RFM features (Recency, Frequency, Monetary) plus behaviour signals. 2. Log-transform skewed monetary values; clip outliers. 3. StandardScaler → cluster (start K-Means K=4..6). 4. Build a profile table; name clusters ("VIP whales", "churning casuals"). 5. Persist labels back to the warehouse keyed by user_id. 6. Refresh quarterly; check stability — if 70% of users keep the same label, segments are robust.

Practice Path

1. Take a customer dataset, scale features, run K-Means for K=2..10, plot elbow + silhouette, pick K. 2. Profile the chosen clusters: mean of each feature per cluster + business-friendly names. 3. Re-run with DBSCAN; compare which points are flagged as noise vs which K-Means cluster they joined. 4. Reduce a high-dim embedding to 2D with UMAP and visualise the clusters.