Practical issues aside, such that the number of neurons in this hidden layer would grow enormously large, we do not need other network architectures. A simple feed-forward neural network could do the trick.<\/p>\n It is challenging to estimate how many network architectures have been developed.\u00a0<\/p>\n When you open the popular AI model platform\u00a0Hugging Face<\/a>\u00a0today, you will find more than one million pretrained models. Depending on the task, you will use different architectures, for example transformers for natural language processing and convolutional networks for image classification.<\/p>\n So, why do we\u00a0need<\/em>\u00a0so many Neural Network<\/a> architectures?<\/p>\n In this post, I want offer an answer to this question from a physics perspective. It is the structure in the data that inspires novel neural network architectures.<\/p>\n Physicists love symmetry. The fundamental laws of physics employ symmetries, such as the fact that the motion of a particle can be described by the same equations, regardless of where it finds itself in time and space.<\/p>\n Symmetry always implies invariance with respect to some transformation. These ice crystals are an example of translational invariance. The smaller structures look the same, regardless of where they appear in the larger context.<\/p>\n If you already know that a certain symmetry persists in your data, you can exploit this fact to simplify your neural network architecture.<\/p>\n Let\u2019s explain this with the example of image classification. The panel shows three scenes including a goldfish. It can show up in any location within the image, but the image should always be classified as\u00a0goldfish<\/em>.<\/p>\n A feed-forward neural network could certainly achieve this, given sufficient training data.\u00a0<\/p>\n This network architecture requires a flattened input image. Weights are then assigned between each input layer neuron (representing one pixel in the image) and each hidden layer neuron. Also, weights are assigned between each neuron in the hidden and the output layer.<\/p>\n Along with this architecture, the panel shows a \u201cflattened\u201d version of the three goldfish images from above. Do they still look alike to you?<\/p>\n By flattening the image, we have incurred two problems:<\/p>\n Convolutional networks, on the other hand, work with kernels. Kernel sizes typically range between 3 and 7 pixels, and the kernel parameters are learnable in training.<\/p>\n The kernel is applied like a raster to the image. A convolutional layer will have more than one kernel, allowing each kernel to focus on different aspects of the image.\u00a0<\/p>\n For example, one kernel might pick up on horizontal lines in the image, while another might pick up on convex curves.<\/p>\n Convolutional neural networks preserve the order of pixels and are great to learn localized structures. The convolutional layers can be nested to create deep layers. In conjunction with pooling layers, high-level features can be learned.<\/p>\n The resulting networks are considerably smaller than if you would use a fully-connected neural network. A convolutional layer only requires\u00a0kernel_size x kernel_size x n_kernel<\/em>\u00a0trainable parameters.\u00a0<\/p>\n You can save memory and computational budget, all by exploiting the fact that your object may be located\u00a0anywhere<\/em> within your image!<\/p>\n More advanced deep learning architectures that exploit symmetries are Graph Neural Networks and physics-informed neural networks.<\/p>\n Convolutional neural networks work great with images because they preserve the local information in your image. Instead of flattening all the pixels, rendering the image meaningless, kernels with learnable parameters pick up on local features.<\/p>\nSymmetry and invariance<\/h3>\n
![\"Regular](\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/1cyok-skFlIXHo3MxRMByEQ.jpg?ssl=1\")
Exploiting symmetries: convolutional neural\u00a0network<\/h3>\n
![\"Three](\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/1CjKp7lOtNsaXHxLy73ul9Q.jpg?ssl=1\")
![\"Schematic](\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/1gjdefs9wO-3Hrwp604OBaQ.jpg?ssl=1\")
\n
![\"\"](\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/1hcEZQnOOq2kR4oyB2DeheA.png?ssl=1\")
\nSummary<\/h3>\n
\nFurther reading<\/h3>\n