A tiny function

For all their mystique, neural networks are built from an almost laughably simple unit: the artificial neuron. A single neuron takes some input numbers, multiplies each by a weight, adds them up, adds one more number called a bias, and passes the result through a non-linear activation function. That's it — a weighted sum followed by a simple squashing step. The neuron's "knowledge" lives entirely in its weights and bias, the numbers that get adjusted during learning.

Why the non-linearity matters

That activation function — the non-linear squash — is the secret ingredient. Without it, stacking neurons would just produce another weighted sum, no more powerful than a single layer; you could never model anything but straight-line relationships. The non-linearity is what lets networks bend, curve, and capture the messy, complex patterns of real data. It's the difference between a ruler and clay.

Why "deep"

One neuron is weak. The power comes from connecting many into layers and stacking those layers — which is exactly what "deep" means: many layers, one after another. Stacking creates a hierarchy of features:

• Early layers learn simple patterns — edges in an image, fragments of words.
• Middle layers combine those into parts — a wheel, a syllable.
• Later layers assemble parts into concepts — a car, a sentence's meaning.

Nobody programs this hierarchy; the network discovers it during training. That automatic feature-building is deep learning's superpower, and why it thrives on the unstructured data — images, audio, language — where hand-designing features is hopeless.

A neuron is just a weighted sum with a kink. Deep learning is what happens when you stack millions of them and let training decide what each one should notice.

Try this: Picture recognizing a face. What might the earliest layer detect (edges, light and dark)? The next (eyes, a nose)? The last (a specific person)? Sketching that ladder of increasingly complex features is exactly the hierarchy a deep network builds on its own.