Convolutional layers in neural networks are often referred to as "locally connected" because of the way they process input data. This terminology reflects the fact that convolutional layers establish local connections between neurons or units in the layer, which is in contrast to fully connected layers, where each neuron is connected to every neuron in the previous layer. Here's a more detailed explanation:
Local Receptive Fields: In a convolutional layer, each neuron is associated with a small, fixed-size region of the input data known as a "local receptive field" or "kernel." This local receptive field is typically a square or rectangular region of the input.
Shared Weights: Within a local receptive field, all neurons share the same set of learnable weights and biases. These shared weights are used to perform the convolution operation on the input data within the local receptive field.
Sliding Window: The convolutional layer processes the input data by sliding the local receptive field across the entire input volume. At each position of the sliding window, a convolution operation is performed between the local receptive field and the corresponding region of the input data.
Feature Map: The output of each convolution operation forms a single element in the "feature map" or "activation map." Each feature map represents the response of a specific filter or kernel to a particular feature or pattern in the input data.
Multiple Feature Maps: A convolutional layer typically consists of multiple filters or kernels, each of which produces a distinct feature map. These feature maps capture different patterns and features from the input data.
Sparse Connectivity: Due to the use of local receptive fields, convolutional layers exhibit "sparse connectivity." Neurons in a feature map only connect to a small subset of neurons in the previous layer, specifically those within their local receptive field. This is in contrast to fully connected layers, where every neuron connects to every neuron in the previous layer.
The advantages of this locally connected architecture in convolutional layers are as follows:
Translation Invariance: Local connectivity allows convolutional layers to be translation invariant, meaning they can detect the same features in different parts of the input image. This property is crucial for tasks like image recognition, where the position of an object in an image shouldn't affect the network's ability to recognize it.
Parameter Sharing: Sharing weights within a local receptive field reduces the number of learnable parameters in the model, making it more efficient and easier to train, especially when dealing with high-dimensional data like images.
Feature Hierarchy: Convolutional layers can be stacked to create a hierarchy of features, with lower layers capturing simple features like edges and textures and higher layers capturing more complex and abstract features.
Overall, the concept of "local connectivity" in convolutional layers is a fundamental design principle in convolutional neural networks (CNNs) and is crucial for their effectiveness in tasks like image analysis and computer vision.