The basic building block: the neuron
Every neural network is built from a simple unit. An artificial neuron takes several numbers as input, multiplies each by an adjustable weight, adds them together with a bias, and passes the total through an activation function — a simple nonlinear curve that decides how strongly the neuron “fires.” That is the whole mechanism. The intelligence does not live in any single neuron; it emerges from how millions of these tiny units are connected and tuned.
The analogy to brain cells is loose and historical. In practice a neuron is just weighted arithmetic followed by a squashing function, repeated at enormous scale.
The perceptron and its limits
The earliest neural network, the perceptron from the late 1950s, was a single layer of these units. It could learn to draw a straight dividing line between two classes — but no more. Famously, it could not learn the XOR pattern, where the right answer depends on a combination of inputs that no single line can separate. This limitation stalled neural network research for years, until people realised that stacking layers could overcome it.
Going deep: hidden layers and learning
A deep neural network places one or more hidden layers between input and output. Each layer transforms the data into a more abstract representation: in image recognition, early layers detect edges, middle layers detect shapes, and later layers detect whole objects. This hierarchy of features is what makes deep learning powerful.
The network learns these features automatically through backpropagation. After it makes a prediction, a loss function measures the error, and backpropagation traces that error backward to find how much each weight contributed. Gradient descent then adjusts every weight a little in the direction that lowers the error. Repeated across millions of examples, this process tunes the network into something useful — no human hand-coding the rules.
Specialised architectures
Not all networks are wired the same way. Convolutional neural networks (CNNs) slide small filters across images to detect local patterns efficiently, dominating computer vision for years. Recurrent neural networks (RNNs) process sequences one step at a time, carrying a memory forward, which made them natural for text and speech. Each architecture bakes in assumptions suited to its data type, which helps the network learn faster and generalise better.
The transformer revolution
RNNs struggled with long sequences: processing word by word is slow and tends to forget distant context. The transformer, introduced in 2017, fixed this with the attention mechanism. Instead of marching through a sequence step by step, attention lets every token directly look at — and weigh the importance of — every other token, all at once and in parallel.
This made transformers dramatically better at language and far easier to scale on modern hardware. Stack enough transformer layers, train them on vast text, and you get the large language models behind today’s AI assistants. The arc from a single perceptron to GPT is one long story of the same core idea — weighted units adjusted by error — combined with smarter wiring and far more scale.