Neural Networks Fundamentals

Duration: 4 min

Welcome to the Neural Networks Fundamentals module. In this module, we will explore the foundational concepts of neural networks, which are a subset of machine learning and a key component of artificial intelligence.

Concept 1: Neurons and Activation Functions

A neural network is composed of layers of neurons, where each neuron receives input, processes it, and then passes the output to the next layer. The processing is done through a weighted sum of inputs, followed by an activation function.

import numpy as np

def sigmoid(x):
    return 1 / (1 + np.exp(-x))

inputs = np.array([0, 1, -1])
weights = np.array([0.5, -0.5, 0.3])
bias = 0.1

output = sigmoid(np.dot(weights, inputs) + bias)
print(output)

Try it in Google Colab:

0.6590747663566236

Concept 2: Forward and Backward Propagation

Forward propagation is the process of passing an input through the network to obtain an output. Backward propagation, or backprop, is the method used to update the weights of the network by computing the gradient of the loss function.

💡 Tip: When implementing backpropagation, ensure that your weight updates are small enough to avoid overshooting the minimum of the loss function.