Clustering with k-Means and Hierarchical Methods

A marketing manager at a Faisalabad exporter once told us "I have 400 buyers and I treat them all the same, because I don't know how else to treat them". No label to predict there, just a question: what kinds of buyers do we have? Clustering answers it by grouping rows so that rows inside a group look alike and rows in different groups don't. It drives marketing segments, store groupings, anomaly detection, even image compression. We'll cover k-means (the workhorse), hierarchical clustering (the one that draws a tree), and how to choose the number of clusters.

k-means: assign, move, repeat

1. Choose k and place k centroids at random data points.
2. Assign every row to its nearest centroid (Euclidean distance).
3. Move each centroid to the mean of the rows assigned to it.
4. Repeat 2 and 3 until assignments stop changing.

The algorithm minimises inertia, the total squared distance from each point to its centroid. Because the start is random, run it several times (n_init=10) and keep the lowest inertia. Forgetting this is the most common reason two students get different clusters from the same data.

k = 2 on six shops

Six shops, features already scaled, monthly orders (x) and average basket (y).

Shop	x	y
A	1.0	1.0
B	1.5	2.0
C	3.0	4.0
D	5.0	7.0
E	3.5	5.0
F	4.5	5.0

Initial centroids: c1 = A (1.0, 1.0), c2 = F (4.5, 5.0).

Shop	Distance to c1	Distance to c2	Assigned
A	0.00	sqrt(12.25 + 16) = 5.32	1
B	sqrt(0.25 + 1) = 1.12	sqrt(9 + 9) = 4.24	1
C	sqrt(4 + 9) = 3.61	sqrt(2.25 + 1) = 1.80	2
D	sqrt(16 + 36) = 7.21	sqrt(0.25 + 4) = 2.06	2
E	sqrt(6.25 + 16) = 4.72	sqrt(1 + 0) = 1.00	2
F	sqrt(12.25 + 16) = 5.32	0.00	2

Move: c1 = mean(A, B) = (1.25, 1.5). c2 = mean(C, D, E, F) = ((3 + 5 + 3.5 + 4.5) / 4, (4 + 7 + 5 + 5) / 4) = (4.0, 5.25).

Re-assign: C is now sqrt(3.06 + 6.25) = 3.05 from c1 and sqrt(1 + 1.56) = 1.60 from c2, still cluster 2. A and B stay in cluster 1; D, E, F stay in 2. Nothing moved, so the algorithm has converged after two rounds. {A, B} are the "small occasional shops" and {C, D, E, F} the "larger regular shops". Naming them is your job, not the algorithm's.

Choosing k

Method	How	Reading
Elbow	plot inertia for k = 1..10	pick the k where the curve stops dropping steeply
Silhouette	for each point, (b - a) / max(a, b) where a = mean distance to own cluster, b = to nearest other cluster	average near 1 is great, near 0 means overlapping clusters; pick the k with the highest average
Business	how many segments can marketing actually treat differently?	often 3 to 6, regardless of the maths

Hierarchical clustering

Agglomerative clustering starts with every row as its own cluster and keeps merging the two closest until one remains. The record of merges is the dendrogram, and cutting it at any height gives you that many clusters without re-running anything. "Closest" between clusters depends on the linkage: single (nearest members), complete (farthest members), average, or Ward (smallest increase in inertia, the usual default). It's slower than k-means (fine up to tens of thousands of rows) but needs no k in advance and copes with non-spherical shapes.

Preparing data for clustering

• Scale every feature; clustering is pure distance.
• Log-transform money and counts so a few huge customers do not form their own cluster.
• Drop IDs and leaky columns; the algorithm will happily cluster on customer number.
• Profile each cluster afterwards with means and counts to give it a name.

# run locally
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans, AgglomerativeClustering
from sklearn.metrics import silhouette_score

Xs = StandardScaler().fit_transform(X)
for k in range(2, 8):
    km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(Xs)
    print(k, round(km.inertia_, 1), round(silhouette_score(Xs, km.labels_), 3))

best = KMeans(n_clusters=3, n_init=10, random_state=0).fit(Xs)
X["cluster"] = best.labels_
print(X.groupby("cluster").mean().round(2))   # profile each segment

hc = AgglomerativeClustering(n_clusters=3, linkage="ward").fit(Xs)

Four buyer segments and what changed after: That Faisalabad textile exporter clustered its 400 international buyers on log(annual volume), order frequency, average payment delay and share of custom orders. k-means with k = 4 (chosen by silhouette and by what the sales team could act on) gave "large steady" (38 buyers, 61% of revenue), "small frequent" (150), "seasonal bulk" (90, orders bunched before the European winter) and "slow payers" (122, average delay 47 days). A dedicated account manager went to the first group, the fourth moved to advance-payment terms, and yarn purchases were planned around the third group's calendar. The cluster numbers meant nothing until the profiling table gave them names.

Before you move on

• k-means alternates between assigning points to the nearest centroid and moving centroids to the mean, until nothing changes.
• It minimises inertia. Run several random starts and keep the best.
• Choose k with the elbow, the silhouette score and business sense, and when they disagree, business sense wins.
• Hierarchical clustering builds a dendrogram you can cut at any k.
• Scale and log-transform first. Profile and name the clusters afterwards.

Three things to try

1. Rerun the worked example with initial centroids c1 = A and c2 = C. Do you reach the same final clusters?
2. A point has mean distance 0.8 to its own cluster and 2.4 to the nearest other cluster. Compute its silhouette value.
3. A Daraz seller wants to segment 50,000 customers by recency, frequency and monetary value (RFM). Describe the preprocessing steps and how you'd present the result to a marketing manager. The RFM features themselves are a three-line GROUP BY, which our SQL track covers.