Deep Learning: CNNs, RNNs and Transformers

Same neurons, same layers, same training loop. So why does one network read number plates and another translate Urdu? Wiring. How the layers connect decides what a network is good at, and three wiring patterns cover nearly everything in production today. Convolutional networks for images, recurrent networks for sequences, and transformers, which own text, code and audio and are taking over vision. Intuition only here, no heavy maths.

Convolutional neural networks (CNNs)

An image is a grid of pixel numbers. A fully connected network would treat pixel (1,1) and pixel (200,200) as strangers, ignoring the fact that neighbouring pixels form edges and shapes. A convolution slides a small filter (3×3, say) across the image and computes a weighted sum at each spot. Each filter learns to spot one thing, such as a vertical edge, a colour blob or a corner. Pooling then shrinks the map and keeps the strongest responses. Stack a few and the early layers find edges, the middle layers find parts (eyes, wheels, letters) and the late layers find whole objects.

CNNs sit behind CNIC and number-plate OCR, fabric defect detection, medical imaging and product photo search. They're also light enough to run on a phone, which matters in the many Pakistani deployments where the "server" is a Rs 40,000 handset.

Recurrent neural networks (RNNs)

Text, speech and sensor readings are sequences. An RNN reads one item at a time and carries a hidden "memory" vector forward, so the output at step 10 depends on steps 1 to 9. LSTM and GRU variants improved that memory, and RNNs ran translation and speech recognition until about 2018. Two weaknesses did for them. They process one step after another (slow on GPUs, which want to do everything at once), and they forget things far back in a long sequence.

Transformers

The 2017 Transformer threw out recurrence and replaced it with attention. Every token looks directly at every other token and decides how much each one matters to its own meaning. In "The parcel from Lahore was late; it was damaged", the token "it" attends strongly to "parcel" and barely to "Lahore". Since all tokens are processed together, transformers train in parallel on GPUs and cope with long context. That one change is why you're reading about LLMs at all.

Attention in three steps

1. Each token produces a query ("what am I looking for?"), a key ("what do I contain?") and a value ("what information do I pass on?").
2. The query of one token is compared with the keys of all tokens; the similarities are turned into weights that sum to 1.
3. The token's new representation is the weighted sum of all values.

Several attention "heads" run side by side, each picking up a different kind of relationship (grammar, who "it" refers to, topic). Position encodings add word order back in, because attention on its own doesn't care about order. One transformer layer is attention followed by a small feed-forward network, and an LLM stacks dozens of them.

Which wiring for which data?

Data	Classic choice	2025–26 reality
Images	CNN	CNNs still common on devices; vision transformers and multimodal LLMs for complex understanding
Text	RNN / LSTM	Transformers everywhere (all LLMs)
Speech	RNN	Transformer-based speech models, often inside multimodal models
Time series (sales, load)	Statistical models, gradient boosting, RNN	Still mostly classic ML; transformers used for very large, multi-series problems
Tabular business data	Gradient boosting	Still gradient boosting; deep learning rarely wins here

A convolution by hand

Here's a 3×3 vertical-edge filter run over a tiny image in plain Python. Wherever the image jumps from dark to bright going left to right, the response is large. Everywhere else it's zero.

image = [
    [0, 0, 0, 9, 9, 9],
    [0, 0, 0, 9, 9, 9],
    [0, 0, 0, 9, 9, 9],
    [0, 0, 0, 9, 9, 9],
]
kernel = [[-1, 0, 1],
          [-1, 0, 1],
          [-1, 0, 1]]

rows, cols = len(image), len(image[0])
out = []
for r in range(rows - 2):
    row = []
    for c in range(cols - 2):
        s = sum(image[r + i][c + j] * kernel[i][j] for i in range(3) for j in range(3))
        row.append(s)
    out.append(row)

for row in out:
    print(row)
# [0, 27, 27, 0] on every row: the edge is found at columns 1-2

A CNN learns the numbers inside that kernel instead of us typing them, and it learns hundreds of kernels per layer.

Two architectures on one factory floor

The Faisalabad exporter from lesson 4 mounted a line-scan camera over the loom. A CNN fine-tuned on a few thousand labelled fabric images flags holes, oil stains and broken threads as the cloth passes, and the QC team only looks at the flagged metres. Down the corridor, the export desk uses a transformer-based assistant to read buyer emails in English and Chinese, pull the order specs into a table and draft confirmations in Urdu. Nobody there calls either of these "AI". They call them the camera and the email thing, which is how you know a system has landed.

In one breath, then. CNNs slide filters over images to find patterns from edges up to objects. RNNs walk through sequences with a carried memory and have mostly been retired. Transformers use attention so every token can look at every other token at once, and they're the backbone of every LLM. For tabular business data, classic ML still usually wins, so don't let anyone sell you a transformer for a sales forecast.

Try this before the next lesson

1. Change the kernel in the example to detect horizontal edges and test it on a transposed image.
2. In the sentence "The bank refused the loan because it was risky", which word should "it" attend to? What about "because it was closed"?
3. For a Karachi hospital, name one CNN use, one transformer use and one problem where classic ML is the better choice.