{"id":4581,"date":"2022-11-10T17:50:05","date_gmt":"2022-11-11T01:50:05","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=4581"},"modified":"2025-04-24T17:16:33","modified_gmt":"2025-04-24T17:16:33","slug":"deep-learning-techniques-you-should-know-in-2022","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/deep-learning-techniques-you-should-know-in-2022\/","title":{"rendered":"Deep Learning Techniques You Should Know in 2022"},"content":{"rendered":"\n\n\n\n\n
\n

Over the years, Deep Learning has really taken off. This is because we have access to a lot more data and more computational power, allowing us to produce more accurate outputs and build models more efficiently. The rise in Deep Learning has been applied in different sectors such as speech recognition, image recognition, online advertising, and more.<\/p>\n

Deep Learning has recently outperformed humans in solving particular tasks, for example, Image Recognition. The level of accuracy achieved by Deep Learning has made it become so popular, and everybody is figuring out ways to implement it into their business.<\/p>\n

If you would like to know more about Deep Learning before knowing about the different techniques, check out this article I wrote: Deep Learning: How it works.<\/p>\n

Deep Learning Techniques<\/h1>\n

There are various Deep Learning models which are used to solve complicated tasks.<\/p>\n

Multilayer Perceptrons (MLPs)<\/h2>\n

Multilayer Perceptrons is a feedforward artificial neural network, where a set of inputs are fed into the Neural Network to generate a set of outputs. MLPs are made up of input layers and an output layer that is fully connected.<\/p>\n

How does MLP work?<\/h2>\n
    \n
  1. The MLP Network feeds the data into the input layer.<\/li>\n
  2. The neuron layers are connected so that the signal passed through in one direction.<\/li>\n
  3. The MLP computes the input with existing weights between the input layer and the hidden layer.<\/li>\n
  4. Activation Functions are used to determine which nodes to fire, for example, sigmoid functions, and tanh.<\/li>\n
  5. Using the training dataset, the MLPs train the model to get a better understanding of the correlation and dependencies between the independent variable and the target variable.<\/li>\n<\/ol>\n
    \n
    \n
    \"\"<\/figure>
    Source: <\/picture>researchgate<\/a><\/div>\n<\/div>\n<\/figure>\n

    \n<\/p>

    Why do we need to know Multilayer Perceptrons?<\/strong><\/p>\n

      \n
    1. Adaptive learning<\/strong>: Multilayer Perceptrons have the ability to learn data effectively and perform well.<\/li>\n
    2. Very popular<\/strong>: Multilayer Perceptrons is a preferred technique for image identification, spam detection, and stock analysis.<\/li>\n
    3. Accuracy<\/strong>: Multilayer Perceptrons do not make assumptions in relation to the probability in comparison to other probability-based models.<\/li>\n
    4. Decision Making<\/strong>: Multilayer Perceptrons contain the required decision function through training.<\/li>\n
    5. Universal Approximation<\/strong>: Multilayer Perceptrons provide more than 2 layers allowing backpropagation has proven to find any mathematical function and can map attributes to outputs.<\/li>\n<\/ol>\n

      Convolutional Neural Network<\/h2>\n

      Convolutional Neural Network (CNN)<\/strong>, also known as a ConvNet is a feed-forward Neural Network. It is typically used in image recognition, by processing pixelated data to detect and classify objects in an image. The model has been built to solve complex tasks, preprocessing, and data compilation.<\/p>\n

      How do CNN\u2019s work?<\/h2>\n

      CNN\u2019s consists of multiple layers:<\/p>\n

      1. Convolution Layer<\/p>\n

      This layer extracts the features from the input image by transforming it. During the transformation process, the image is convolved with a kernel which is a small matrix that moves over the input data. It is also known as a convolution matrix or convolution mask.<\/p>\n

      2. Rectified Linear Unit (ReLU)<\/p>\n

      This is a non-linear activation function that is used to perform on multi-layer neural networks. It aims to achieve non-linear transformation of the data, with the hope that the transformed data will be linear or linearly separable.<\/p>\n

      3. Pooling Layer<\/p>\n

      This layer is used to reduce the dimensions of the feature maps. It does this by reducing the number of parameters the model learns and the computational power used in the network.<\/p>\n

      It takes a two-dimensional array from the pooled feature map and converts it into a single, long, continuous, linear vector via flattening.<\/p>\n

      4. Fully Connected Layer<\/p>\n

      A Fully Connected Layer is made up of a series of connected layers that link every neuron in one layer to every neuron in another layer.<\/p>\n

      It is formed when the flattened matrix from the pooling layer is fed as an input, which is used to classify and identify the images.<\/p>\n

      \n
      \n
      \"\"<\/figure>
      Source: <\/picture>MDPI<\/a><\/div>\n<\/div>\n<\/figure>\n

      \n<\/p>

      Why do we need to know Convolutional Neural Networks?<\/strong><\/p>\n

        \n
      1. Feature Learning<\/strong>: Convolutional Neural Networks can automatically detect the importance of features without any human supervision, by learning the different features.<\/li>\n
      2. Computationally effective<\/strong>: Convolutional Neural Networks use unique convolution, pooling, parameter sharing, and dimensionality reduction, making the models easy and quick to deploy.<\/li>\n
      3. Accuracy<\/strong>: Convolutional Neural Networks is a powerful and efficient model which has outperformed humans in solving particular tasks<\/li>\n<\/ol>\n

        Recurrent Neural Networks (RNNs)<\/h2>\n

        A Recurrent Neural Network is used to work for time series data or data that involve sequences. RNN uses the previous state\u2019s knowledge and uses it as an input value for the current prediction.<\/p>\n

        Therefore, RNN can memorize previous inputs using its internal memory. They are used for time-series analysis, handwriting recognition, Natural Language Processing, and more.<\/p>\n

        An example of RNN is Long Short Term Memory.<\/p>\n

        Long Short Term Memory Networks (LSTMs)<\/h2>\n

        Long Short Term Memory Networks <\/strong>is a type of Recurrent Neural Network which can learn and memorize long-term dependencies. Its default behavior and the aim of an LSTM is to remember past information for long periods.<\/p>\n

        How Do LSTMs Work?<\/h2>\n

        LSTM uses a series of \u2018gates\u2019 that control the processing of information, how the data comes in, how it is stored, and how it leaves the network.<\/p>\n

        LSTM has three gates:<\/p>\n

          \n
        1. Forget Gate \u2014 This is where the LSTM forgets (less weight) irrelevant parts of the previous state.<\/li>\n
        2. Input Gate \u2014 This is where new information is determined if it should be added to the cell state, the network\u2019s long-term memory. This is done by using the previous hidden state and new input data.<\/li>\n
        3. Output Gate \u2014 This is deciding the new hidden state. This is done by using the newly updated cell state, the previous hidden state, and the new input data.<\/li>\n<\/ol>\n
          \n
          \"\"<\/figure>
          Source: <\/picture>d2l<\/a><\/div>\n<\/figure>\n

          \n<\/p>

          Why do we need to know Long Short Term Memory Networks?<\/strong><\/p>\n

            \n
          1. Memory: Long Short Term Memory Networks’ ability to learn and memorize long-term dependencies is highly beneficial. This improves the overall performance of the model.<\/li>\n
          2. Sequencing: Long Short Term Memory Networks are very popular with Natural Language Processing due to sequencing. If you train the model on a piece of text the model has the ability to generate new sentences, mimicking the style of the text.<\/li>\n<\/ol>\n<\/div>\n\n\n\n
            <\/div>\n\n\n\n
            \n

            Join 16,000 of your colleagues at Deep Learning Weekly<\/a> for the latest products, acquisitions, technologies, deep-dives and more.<\/p><\/blockquote>\n<\/div>\n\n\n\n

            <\/div>\n\n\n\n
            \n

            Generative Adversarial Networks (GANs)<\/h2>\n

            Generative Adversarial Networks<\/strong> use two neural networks which compete with one another, hence the \u201cadversarial\u201d in the name.<\/p>\n

            The two neural networks used to build a GAN are called \u2018the generator\u2019 and \u2018the discriminator\u2019. The Generator learns to generate fake data whilst the Discriminator learns from that fake information. They are used to ensure accuracy in the model\u2019s predictions.<\/p>\n

            How Do GANs work?<\/h2>\n
              \n
            1. During the initial training phase, The Generator learns to generate fake data in the network.<\/li>\n
            2. The Discriminator learns to distinguish and learns the difference between the real sample data and the fake data generated by The Generator.<\/li>\n
            3. The GAN then sends these results to the Generator and the Discriminator to continuously keep updating the model.<\/li>\n<\/ol>\n
              \n
              \n
              \"\"<\/figure>
              Source: <\/picture>wiki.pathmind<\/a><\/div>\n<\/div>\n<\/figure>\n

              \n<\/p>

              Why do we need to know Generative Adversarial Networks?<\/strong><\/p>\n

                \n
              1. No Data Labelling: Generative Adversarial Networks are unsupervised therefore no labeled data is needed in order to train them. This heavily reduces costs.<\/li>\n
              2. Sharp Images: Generative Adversarial Networks currently produce the sharpest images in comparison to other techniques.<\/li>\n
              3. Backpropagation: Generative Adversarial Networks can be trained only using backpropagation.<\/li>\n<\/ol>\n

                Restricted Boltzmann Machines (RBMs)<\/h2>\n

                Restricted Boltzmann machine is a type of Recurrent Neural Network where the nodes make binary decisions with some bias. It was invented by Geoffrey Hinton and is used generally for dimensionality reduction, classification, regression, feature learning, and topic modeling.<\/p>\n

                RBMs uses two layers:<\/p>\n