CNN Convolutional Neural Network Simulator

Interactively explore the core formulas of convolutional neural networks, the effect of the parameters and how the network structure changes. Adjust the parameters on the left and watch the visualization on the right update in real time.

CNN Core Formulas Explained

The following formulas are the mathematical foundation of convolutional neural networks. Each formula comes with detailed parameters and an intuitive explanation.

Convolutional Layer Output Size Formula
Computes the size of the feature map after a convolution. This formula describes how the output feature-map size is determined by the input size, the kernel size, the padding and the stride after the kernel slides over the input.
Parameters:
O โ€” output feature-map size (width or height)
W โ€” input feature-map size (width or height)
K โ€” convolution kernel size (usually odd, e.g. 3ร—3, 5ร—5)
P โ€” padding size, the number of zero-padding rings at the input border
S โ€” stride, the pixel distance the kernel moves each step
โŒŠยทโŒ‹ โ€” floor operation
2D Convolution Operation
The essence of convolution is a local weighted sum. For each position (i,j) of the output feature map, the kernel is multiplied element-wise with the corresponding region of the input feature map and summed, then the bias is added to obtain the output value at that position.
Parameters:
yi,j โ€” value of the output feature map at position (i,j)
x โ€” input feature map
w โ€” convolution kernel weight matrix
b โ€” bias term (scalar)
K โ€” convolution kernel size
S โ€” stride
m, n โ€” indices inside the kernel
Pooling Layer Output Size Formula
The pooling layer is used to down-sample the feature map and reduce its spatial dimensions. This formula is similar to the convolution output-size formula, but pooling usually does not use padding (P=0).
Parameters:
Opool โ€” output size after pooling
W โ€” input size before pooling
Kpool โ€” pooling window size (e.g. 2ร—2)
Spool โ€” pooling stride (usually equal to the pooling window size)
Max Pooling
Max pooling takes the maximum value within each pooling window as the output. It preserves the most salient feature of the local region and is invariant to small shifts, making it the most common pooling method in CNNs.
Parameters:
yi,j โ€” pooling output value
x โ€” input feature map
K โ€” pooling window size
S โ€” pooling stride
max(ยท) โ€” take the maximum within the window
Average Pooling
Average pooling computes the mean of all elements in the pooling window. Compared with max pooling, average pooling preserves the overall information more smoothly and is often used in the last few layers or for global pooling.
Parameters:
yi,j โ€” pooling output value
K โ€” pooling window size
Kยฒ โ€” total number of elements in the pooling window
other parameters are the same as max pooling
ReLU Activation Function
ReLU (Rectified Linear Unit) is the most commonly used activation function in CNNs. It sets all negative values to zero and keeps positive values unchanged, introducing non-linearity while staying computationally efficient.
Parameters:
x โ€” input value (output of a convolution or fully-connected layer)
f(x) โ€” output value after activation
Pros: simple computation, alleviates vanishing gradients, sparse activation
Cons: may cause 'dying neurons' (Dead ReLU)
Fully Connected Layer
The fully connected layer connects every neuron of the previous layer to every neuron of the current layer. In a CNN it usually sits at the end of the network and maps the features extracted by the convolutional layers to the final classification output.
Parameters:
zj โ€” weighted sum of the j-th neuron
wij โ€” connection weight
ai โ€” activation value of the i-th neuron in the previous layer
bj โ€” bias term
N โ€” number of neurons in the previous layer
Softmax Classification Function
Softmax converts the output of the fully connected layer into a probability distribution whose sum over all classes is 1. It is the most common output-layer activation for multi-class classification.
Parameters:
P(y=j|x) โ€” probability that input x belongs to class j
zj โ€” score (logit) of class j
C โ€” total number of classes
e โ€” natural constant (about 2.71828)
Cross-Entropy Loss
Cross-entropy loss measures the difference between the predicted probability distribution and the true label distribution. The closer the predicted probability is to the true label, the smaller the loss. It is the standard loss function for classification tasks.
Parameters:
L โ€” loss value
yi โ€” true label (one-hot encoding, correct class is 1, rest are 0)
ลทi โ€” predicted probability (Softmax output)
C โ€” total number of classes
Convolution Layer Parameter Count
Computes the total number of learnable parameters in a convolution layer. Each kernel contains Kยฒร—Cin weight parameters, plus a bias term for each output channel.
Parameters:
K โ€” convolution kernel size
Cin โ€” number of input channels
Cout โ€” number of output channels (number of kernels)
+Cout โ€” bias of each output channel
๐Ÿ‘๏ธ Receptive Field
The receptive field is the size of the region in the original input image that each pixel of the output feature map corresponds to. Deeper networks have larger receptive fields and can capture more global features.
Parameters:
RFl โ€” receptive field size of layer l
Kl โ€” kernel size of layer l
Si โ€” stride of layer i
โˆ โ€” product operation
Convolution Layer Compute (FLOPs)
FLOPs (floating-point operations) measure the computational complexity of a convolution layer. Each multiply-accumulate in the convolution counts as 2 floating-point operations (1 multiplication + 1 addition).
Parameters:
K โ€” convolution kernel size
Cin, Cout โ€” input / output channels
Hout, Wout โ€” height and width of the output feature map
ร—2 โ€” 2 operations per multiply-accumulate

Interactive Network Simulation

Adjust the parameters on the left and the charts on the right update in real time. Observe how the feature-map size, the parameter count and the compute change.

Network Parameter Configuration
Feature-map spatial size per layer
Parameter count distribution per layer
๐Ÿ” Convolution demo (input 5ร—5, kernel 3ร—3)

A Deep Dive into CNNs

Standard Introduction

A convolutional neural network (CNN) is a deep-learning model specialized for grid-structured data such as images, video and audio spectrograms. Through three core mechanisms โ€” local connectivity, weight sharing and down-sampling โ€” a CNN greatly reduces the number of parameters while still extracting the spatial hierarchical features of the data effectively. A typical CNN architecture is stacked from convolution layers, pooling layers, activation functions and fully connected layers. The convolution layer slides learnable kernels over the input to extract local features; the pooling layer down-samples the feature map to enhance translation invariance and reduce computation; finally the fully connected layer together with the Softmax function performs classification or regression. In 2012, AlexNet's breakthrough on the ImageNet competition marked the full rise of CNNs in computer vision.

Plain-Language Introduction

Imagine you are looking at a photo of a cat. Your brain automatically recognizes features such as the cat's ears, eyes and whiskers, and finally concludes 'this is a cat'. A CNN works in a similar way โ€” it is like a careful detective that first scans the image block by block with a small window (the convolution kernel) to find basic features such as edges and textures; then it gradually combines these features to recognize more complex shapes (such as eyes and a nose); and finally it combines all the information to make a judgment. The convolution kernel is like a set of learnable 'feature detectors' that automatically learn to recognize the most important patterns during training. The pooling layer acts like 'down-sampling', shrinking the image while keeping the key information, which makes the network more efficient.

What Problem Does It Solve

  • Parameter explosion: a traditional fully connected network needs a huge number of parameters to process an image (e.g. a 224ร—224ร—3 image connected to 1000 neurons needs about 150 million parameters); CNN reduces the parameter count by several orders of magnitude through weight sharing and local connectivity.
  • Loss of spatial structure: a fully connected network flattens the image into a 1D vector, destroying the spatial relationships between pixels. The convolution operation of a CNN naturally preserves the 2D spatial structure.
  • Need for translation invariance: the pooling layer makes the network insensitive to small shifts of the target, improving the model's robustness.
  • Hierarchical feature extraction: a CNN can automatically build high-level features (shapes, object parts) from low-level features (edges, colors) layer by layer, without hand-designed features.

Why It Matters

CNN is a cornerstone technology in computer vision, and its influence goes far beyond academia. Since AlexNet won the ImageNet competition by a huge margin in 2012, CNNs have driven leapfrog development in face recognition, autonomous driving, medical image diagnosis, remote-sensing image analysis and many other fields. The three core ideas of CNNs โ€” local receptive fields, weight sharing and hierarchical feature extraction โ€” have become fundamental principles of deep-learning architecture design. Even later Transformer architectures borrow the hierarchical feature-extraction idea from CNNs. Thanks to their efficiency and interpretability, CNNs remain the preferred choice for many real-world applications to this day.

Application Scenarios

  • Image classification recognize the object category in a picture (e.g. ImageNet competition, product recognition)
  • Object detection locate and recognize multiple objects in an image (e.g. YOLO, Faster R-CNN)
  • Face recognition identity verification, security surveillance, phone unlocking
  • Medical imaging CT/MRI image analysis, tumor detection, pathology slide diagnosis
  • Autonomous driving road-sign recognition, pedestrian detection, lane-line segmentation
  • Remote sensing satellite image classification, land-use monitoring, disaster assessment
  • Natural language processing text classification, sentiment analysis (TextCNN)
  • Video analysis action recognition, video classification, behavior detection

Limitations and Drawbacks

  • High compute-resource demand: deep CNNs need a lot of GPU resources for training and put high requirements on the hardware.
  • Need for large amounts of labeled data: a CNN usually needs tens of thousands to millions of labeled images to reach good performance, and acquiring the data is costly.
  • Lack of rotation invariance: a standard CNN is not sensitive to transformations such as rotation and scaling and needs data augmentation or special designs (e.g. spatial transformer networks).
  • Limited interpretability: although visualization techniques (such as Grad-CAM) have made progress, the internal decision process of a CNN is still not transparent enough.
  • Vulnerability to adversarial examples: tiny pixel perturbations can make a CNN produce completely wrong classification results.
  • Limited global-context modeling: a CNN has a limited receptive field and is inferior to a Transformer for tasks that need global reasoning (such as long-range dependencies).
  • Difficulty with small-object detection: after several pooling steps, the features of a small object may be lost entirely.

๐Ÿ’ก Core insight: Through local connectivity, weight sharing and hierarchical feature extraction, CNNs have moved image understanding from the era of hand-crafted feature engineering into the era of automatic learning. Understanding how CNNs work is the foundation for mastering modern deep learning. Although new architectures such as Transformers are emerging, the efficiency and maturity of CNNs keep them in an irreplaceable position for edge computing and real-time applications.