Hidden layers are just intermediate guesses

The word hidden suggests a sealed box. In a small network, it is simply the layer between input and output.

Its values are not the final answer, but they are easy to inspect. The layer gives the network a chance to rewrite the input before the output layer makes a decision.

follow one input through the layer

The example starts with three numbers. The first layer turns them into four raw scores, one per neuron. ReLU turns those scores into four activations. The output layer reads those activations rather than reading the original input again.

That is the practical job of the hidden layer: create intermediate signals. One neuron can respond to one pattern while another responds to a different pattern. The next layer can combine those responses instead of rebuilding every pattern from the original numbers.

inspect both columns

The Z and A columns describe different points in the calculation.

Z is the weighted sum before the activation function. A is the value after ReLU. If Z is negative, A becomes zero. That looks like a simple gate, but the raw score is still useful because it tells you how far the neuron was from turning on.

Two hidden units can both output zero and still be in different states. One might be barely negative. Another might be deep in the negative range. Those are different internal conditions, even though the visible activation is the same.

Debugging only the post-activation values can hide useful information. The raw values show how close each neuron was to the boundary. The activations show which signals actually moved forward.

The output layer now combines those internal signals. If a hidden activation is zero, it contributes nothing for this input. If it is large, it can dominate the final score. The network is a chain of smaller calculations:

input -> hidden scores -> hidden activations -> output scores -> output activations

The first layer is where the representation changes.

two small experiments

Change one weight in the first layer. Watch the hidden Z values move, then see which activations survive ReLU. If the output changes, it changed because the output layer received a different set of hidden signals.

Now change a weight in the output layer. The hidden values should remain the same while the final answer moves. The first experiment changes the representation. The second changes how the same representation is read.

That distinction is easier to see in a four-neuron example than in a diagram full of arrows.

The layer is hidden only because its values are intermediate, not because they are inaccessible. You can print them, inspect them, and perturb them. Scale adds more neurons and more paths for a signal to travel, but it does not change that basic causal chain.

When the network gets larger, keep asking:

• what did this layer receive?
• what did it produce?
• which values went to zero?
• which values stayed alive?
• what changed when the earlier weight moved?

Those questions keep the calculation grounded. A hidden layer is one representation becoming another so the next layer has a more useful set of signals to read.