Four neurons are enough to expose the wiring

Four neurons are enough to show that a layer is several opinions, not one big one.

A single neuron teaches the arithmetic cleanly: multiply the inputs, add the bias, run the activation, look at the result. That is useful, but it can also make the layer feel tidier than it really is. Four neurons is still small enough to inspect by hand, and large enough to show that the same input can be judged four different ways.

That is the point of this lesson. Not a useful model. Just enough wiring to see the shape of the thing.

four rows, four rules

In the interactive example, the input has three numbers.

Each neuron has three weights and one bias. Each row is its own little rule:

weighted sum = input_1 * weight_1
             + input_2 * weight_2
             + input_3 * weight_3
             + bias

Four rows means four neurons. They all look at the same input, but they do not ask the same question. One row may care a lot about the first input. Another may mostly react to the last two. Another may stay quiet unless the bias nudges it over the edge.

That is the useful mental model: a layer is a row of separate tests.

z is where the arithmetic still looks honest

The Z values are the raw weighted sums before the activation function touches anything.

I like looking at Z first because it shows the calculation in plain view. If a value surprises you there, the cause is in the input, the weights, or the bias. ReLU has not entered the story yet.

That matters more than it sounds like. Two neurons can both end up at zero after ReLU and still mean very different things. One raw value might be -1. Another might be -30. Both get clipped to zero, but they are not equally close to turning on.

That distance is useful later in training, and it is useful now because it keeps the layer from feeling like a yes/no machine.

relu turns one row into a gate

The A values show what survives after ReLU.

ReLU is simple:

if z > 0, keep z
if z <= 0, output 0

That simplicity is exactly why it works here. You can see which neurons stay active and which ones go silent for the current input. The weights do not change. The signal does.

If a row is negative before ReLU, it does not help the next layer for this input. If it is positive, it carries forward intact. That is the moment the layer starts to feel less like homework and more like a filtering step.

one input does not exercise every wire

The interactive module gets more interesting when you move one number at a time.

Change one weight in one row. Predict which Z value should move. Then run the code and check. If the matching input is large, the effect should be obvious. If the matching input is zero, the weight change should do nothing for this example.

That last case is worth noticing because it is easy to overread one example. A weight can exist and still not matter for the current input. That does not make it useless. It means the input did not touch that part of the layer.

One neuron can only tell you so much. Four neurons give you comparison.

the layer starts to look like a set of opinions

With four neurons, the differences inside the layer become visible.

One neuron may respond sharply to the first input. Another may mostly care about the second and third. Another may stay blocked by a negative bias. Another may stay active across a broad range of values.

The layer begins to look like four small opinions about the same input. None of them knows the whole problem. Each one contributes a partial signal. Later layers can combine those partial signals into something more useful.

That is the real structure to remember. Neural networks are not one large calculation. They are small calculations stacked into larger behavior.

what to keep from the example

The actual numbers are not the lesson. The wiring is.

• the same input feeds every neuron
• each row has its own weights and bias
• Z shows the raw score
• ReLU keeps positive values and drops negative ones
• silent rows stay silent for this input
• active rows keep moving forward

Once that shape feels normal, larger networks are less mysterious. A bigger layer is just more rows. A batch is more columns. Another layer is another pass through the same kind of arithmetic.

Follow the numbers. Predict before you run. The wrong guess is often the fastest way to see which wire you misunderstood.