Neural Networks 101: Part 2

In Part 1, you learned what a neural network is made of. You saw neurons, weights, biases and layers working together to process information. That structure is the blueprint.

Now comes the important question.

How does this system actually learn?

A neural network does not wake up knowing how to recognize spam, faces or language. At the beginning, it is a guess. What turns those guesses into accurate predictions is a learning process built on feedback, error correction and repetition.

In this part, we will explore that learning loop. You will see how activation functions shape decisions, how data moves forward through the network and how backpropagation allows the system to fix its own mistakes. Once you understand this cycle, neural networks stop feeling mysterious and start feeling logical.

Weights and biases calculate the data. The activation function decides what to do with it. This component helps the network capture complex patterns.

Without this step, the entire network fails to learn nuance. All the layers collapse into a single linear step. The activation function allows the system to model curves and edges in the data instead of just straight lines.

Common Activation Types

You will usually see one of three standard options inside a network.

1. ReLU (Rectified Linear Unit): This is the speed demon. It is fast and widely used. The Rule: If the value is below zero, the output is zero. If the value is positive, the output equals the value.
2. Sigmoid: This acts like a probability meter. It squeezes every value into a neat range between 0 and 1.
3. Tanh: This is useful for centered outputs. It maps every value to a range between -1 and 1.

Forward Propagation is simply the data flowing from start to finish.

The Process in Three Steps

1. Input: You drop raw features into the first layer.
2. Process: The hidden layers multiply inputs by weights, add biases and apply activation functions.
3. Output: The final layer converts the numbers into a prediction.

The Result:

For sorting categories (like cats vs. dogs), a Softmax function turns numbers into probabilities. For scoring (like predicting house prices), a single neuron outputs a raw number.

Example: Spam Filter

To spot spam, you feed the network word counts and links. The layers crunch the numbers. The final output gives you a probability: 0.87 Spam. You pick the winner.

“Learning” in AI is actually just error correction. The goal is to adjust the weights and biases until the system’s prediction matches the correct answer.

The network runs a constant cycle of five steps to improve itself: The system runs a continuous five-step loop to correct itself:

1. Make a Guess. The network uses its random weights to make a prediction. At first, this prediction is usually wrong.
2. Measure the Error. The system checks its answer against the truth. It uses a loss function to calculate exactly how far off the prediction was from the correct label.
3. Assign Blame (Backpropagation). This is the diagnosis step. The system looks backward through the network to calculate how much each specific weight contributed to the error. It identifies which connections are responsible for the mistake.
4. The Fix (Gradient Descent). This is the treatment step. The system shifts the weights in the direction that reduces the error. It uses a learning rate to control the step size so it does not overcorrect and miss the target.
5. Repeat. The network runs this cycle over the data thousands of times. It continues until the loss stops improving and the pattern is captured.

Building a working neural network is a step-by-step process that ensures the model learns effectively and avoids bias.

Collect Data: You gather examples that are clearly labeled. Example: Emails labeled “spam” or “not spam.”

Split Data: You divide the data into three separate groups: the Training Set, the Validation Set, and the Test Set.

Normalize Features: You scale the numbers in the data so that the training process is stable and faster.

Train the Model: You run many passes over the Training Set to minimize the loss (error).

Validate: You check the model’s performance on the Validation Set. This step is crucial for catching overfitting, when the model memorizes the training data but fails on new data.

Test: You report the accuracy using the final Test Set. This provides an unbiased estimate of how the model will perform in the real world.

Real-Life Application:

When a support team wants quicker ticket triaging, they label past tickets by category. A small network learns from those labels and then routes new tickets to the right queue instantly, with the workflow above guiding the whole setup.

Neural networks do not memorize raw data. They learn internal features that allow them to make smart decisions.

These features are hierarchical. Simple features combine to create a complex understanding.

The Layered Vision

The learning process is broken down by the depth of the layers:

1. In Pictures: Early layers detect basic elements like edges and corners. Deeper layers then combine those simple elements to recognize shapes and entire objects.
2. In Text: The layers learn word patterns, sentiment cues and topic signals.
3. In Audio: The network learns frequencies, rhythms and phoneme patterns.

This hierarchical design is why deep networks (those with many layers) can master complex tasks like computer vision and language translation.

You now understand how a neural network learns. You have seen how data moves forward through the layers, how predictions are made and how mistakes are measured and corrected through backpropagation. This learning loop is what turns a static structure into a system that improves with experience.

Once training is complete, the network is no longer guessing. It has captured patterns from the data and can apply them to new situations with speed and consistency. At this stage, the core mechanics are in place, but one important question remains.

What kind of neural network should be used for different problems, and how do we control how these systems learn in real-world settings? Part 3 will answer that question. We will look at common neural network types, the tasks they are best suited for and the training controls that shape their behavior.

This final piece connects the learning process to the real AI systems you interact with every day.