SOM Network Core Formulas

The Self-Organizing Map (SOM) was proposed by Teuvo Kohonen in 1982. The following formulas form its mathematical foundation.

1 Euclidean Distance

Meaning: Compute the distance between an input sample and each neuron's weight vector, used to measure the degree of similarity between the two. The smaller the distance, the better the match.
x — input sample vector, with dimension n
wj — weight vector of the j-th neuron
d(x, wj) — Euclidean distance between the two

2 Best Matching Unit (BMU) Selection

Meaning: Select the neuron with the smallest distance to the input sample from all neurons, called the Best Matching Unit (BMU). This neuron and its neighborhood will be activated and updated.
c — index of the best matching unit
arg min — take the parameter that minimizes the distance
J — the set of all neurons

3 Neighborhood Function

Meaning: Define which neurons around the BMU will be updated and the strength of the update. The commonly used Gaussian neighborhood function decays exponentially according to the grid distance.
hcj(t) — neighborhood function value
dcj — distance between neurons c and j in the grid
σ(t) — neighborhood radius, decaying over time

4 Weight Update Rule

Meaning: Adjust the weight vector according to the input sample and the neighborhood function, so that the BMU and its neighbors move toward the input sample. The learning rate α(t) controls the update step size.
wj(t) — current weight vector
wj(t+1) — updated weight vector
α(t) — learning rate, decaying over time
x(t) — current input sample

5 Learning Rate Decay

Meaning: The learning rate gradually decreases with training iterations, allowing the network to adjust quickly in the early stage of training and fine-tune in the later stage. τ₁ controls the decay speed.
α₀ — initial learning rate
t — current iteration number
τ₁ — learning rate decay time constant

6 Neighborhood Radius Decay

Meaning: The neighborhood radius gradually decreases with training iterations, enabling the network to transition from global organization to local refinement. τ₂ controls the decay speed.
σ₀ — initial neighborhood radius
t — current iteration number
τ₂ — neighborhood radius decay time constant

7 Topological Neighborhood Distance (Grid Distance)

Meaning: Compute the topological distance between two neurons on the SOM grid, usually using Euclidean distance (grid coordinates). This distance is used in the neighborhood function.
rc, rj — coordinates of the neurons in the grid
dcj — grid distance

8 Quantization Error

Meaning: A metric measuring the mapping quality of the SOM, defined as the average distance between all input samples and their corresponding BMUs. The smaller the error, the higher the mapping quality.
N — number of input samples
xi — the i-th input sample
wbmu(i) — the BMU weight corresponding to sample i

Parameter Controls

Iteration count 0
Current learning rate α —
Current neighborhood radius σ —
Current input sample —
BMU position —
Quantization error QE —

SOM Topology Mapping (Input Space)

Learning Rate and Neighborhood Radius Decay Curves


In-Depth Understanding of SOM Networks

A complete introduction and in-depth interpretation of Self-Organizing Maps

Standard Introduction

The Self-Organizing Map (SOM) is an unsupervised learning neural network proposed in 1982 by Teuvo Kohonen, a professor at the Helsinki University of Technology in Finland. Through competitive learning, SOM maps high-dimensional input data onto a low-dimensional (usually two-dimensional) discrete grid while preserving the topological relationships among the data. This mapping is topology-preserving, meaning that similar input data will be close together on the grid after mapping, while dissimilar data will be far apart.

A SOM network consists of an input layer and a competitive layer (output layer). The neurons of the competitive layer are arranged on a regular grid (usually a two-dimensional rectangular or hexagonal grid). Each neuron has a weight vector with the same dimension as the input. During training, for each input sample, the network finds the neuron whose weight vector best matches the input (the BMU), and then updates the weight vectors of that neuron and those within its neighborhood, moving them toward the input sample. Through iterative training, the weight vectors gradually cover the input space, and adjacent neurons learn similar patterns, thereby achieving topology preservation.

The core advantage of the SOM network lies in its ability to visualize high-dimensional data in a low-dimensional space while preserving the structure of the data. It is widely used in clustering analysis, data visualization, pattern recognition, feature extraction, and dimensionality reduction.

Plain-Language Introduction

Imagine you have a huge map and you want to place many different attractions (input data) on it so that similar attractions are close to each other and different ones are far apart. The SOM network is like an intelligent mapmaker: it starts by randomly placing some markers (neuron weights) on the map, then looks at the attractions one by one, finds the nearest marker, and moves that marker and the markers around it a little toward the attraction. After many such adjustments, the markers gradually spread out and cover the entire scenic area, and similar attractions naturally cluster together. In the end, the map becomes a visualization tool that reflects the data distribution and similarity.

The training process of SOM is like continuously "folding" and "stretching" an elastic grid to make it fit the shape of the input data. Adjacent neurons remain connected, so when a neuron moves, its neighbors are also pulled along, ultimately forming an ordered topological structure.

What Problem Does It Solve

  • High-dimensional data visualization: Map high-dimensional data that is difficult to understand intuitively onto a two-dimensional plane while preserving the similarity structure of the data.
  • Topology-preserving dimensionality reduction: Maintain the proximity relationships among samples in the original data while reducing dimensionality, helping to discover the intrinsic structure of the data.
  • Unsupervised clustering: Automatically discover categories or patterns in the data without pre-labeling or specifying the number of clusters.
  • Data exploration and understanding: Provide an intuitive way to observe data distribution, outliers, and potential groupings.
  • Feature extraction and encoding: The weight vectors of the SOM can serve as low-dimensional feature representations of the original data for subsequent tasks.

Why It Matters

SOM is one of the earliest models in the field of neural networks to be successfully applied to unsupervised learning. It introduced the concepts of competitive learning and topology preservation, and has had a profound impact on subsequent fields such as self-organizing systems, vector quantization, and data visualization. SOM provides a biologically plausible mechanism to explain the topological mapping phenomena that exist in the cerebral cortex (such as the mapping of the retina to the visual cortex). Its ideas have been widely applied in numerous fields such as Machine Learning, data mining, image processing, and speech recognition. Even in the era of deep learning, SOM is still used as an effective visualization tool and clustering method, and it has unique advantages in terms of interpretability.

Core insight: The key to SOM lies in the neighborhood update strategy: not only is the best-matching neuron updated, but its neighbors are also updated, which forces adjacent neurons to learn similar patterns, thereby preserving the topological continuity of the data in the mapping space.

Application Scenarios

  • Data visualization: Map high-dimensional datasets (such as gene expression, customer behavior, and text features) onto a two-dimensional plane to help discover patterns and clusters.
  • Image analysis: Feature clustering in image segmentation, texture classification, and object recognition.
  • Speech processing: Speech feature mapping, phoneme classification, and speaker recognition.
  • Financial analysis: Stock market pattern recognition, credit risk assessment, and customer segmentation.
  • Industrial engineering: Fault diagnosis, process monitoring, and quality control.
  • Text mining: Document clustering, topic discovery, and semantic mapping.
  • Bioinformatics: Gene expression data analysis and protein structure visualization.
  • Robotics: Sensor data fusion, environment mapping, and path planning.

Limitations and Drawbacks

  • Parameter sensitivity: The choice of learning rate, neighborhood radius, and their decay parameters greatly affects the results and requires careful tuning.
  • Lack of a probabilistic model: SOM does not provide a probabilistic interpretation, making it difficult to quantify uncertainty.
  • Uncertain convergence: Training may fall into a local optimum, and multiple runs may produce different mapping results.
  • Fixed grid topology: A predefined grid shape may not suit certain data distributions, leading to topological distortion.
  • Computational overhead: For large-scale datasets, training takes a long time, especially since distances between all neurons and the input must be computed.
  • Not suitable for high-dimensional sparse data: When the input dimension is very high and sparse, the distance metric may become ineffective.
  • Lack of supervised information: It is purely unsupervised and cannot be directly used for classification or regression tasks; subsequent steps (such as labeling neurons) are usually required.