{"id":6565,"date":"2023-06-29T11:28:44","date_gmt":"2023-06-29T19:28:44","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=6565"},"modified":"2025-04-24T17:15:16","modified_gmt":"2025-04-24T17:15:16","slug":"train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/","title":{"rendered":"Train Neural Networks Using a Genetic Algorithm in Python with PyGAD"},"content":{"rendered":"\n<div class=\"fh fi fj fk fl\">\n<div class=\"lt bg\">\n<figure class=\"lu lv lw lx ly lt bg paragraph-image\"><picture><img loading=\"lazy\" decoding=\"async\" class=\"bg lz ma c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:2500\/1*bKjXAtERq1m17q0IE2Eziw.jpeg\" alt=\"\" width=\"2400\" height=\"1667\"><\/picture><figcaption class=\"mb mc md me mf mg mh be b bf z dv\" data-selectable-paragraph=\"\">Photo by <a class=\"af mi\" href=\"https:\/\/unsplash.com\/@5tep5?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" target=\"_blank\" rel=\"noopener ugc nofollow\">Alexander Popov<\/a> on <a class=\"af mi\" href=\"https:\/\/unsplash.com\/s\/photos\/dna?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" target=\"_blank\" rel=\"noopener ugc nofollow\">Unsplash<\/a><\/figcaption><\/figure>\n<\/div>\n<div class=\"ab ca\">\n<div class=\"ch bg et eu ev ew\">\n<p id=\"be7b\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The genetic algorithm (GA) is a biologically-inspired optimization algorithm. It has in recent years gained importance, as it\u2019s simple while also solving complex problems like travel route optimization, training machine learning algorithms, working with single and multi-objective problems, game playing, and more.<\/p>\n<p id=\"dbeb\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Deep neural networks are inspired by the idea of how the biological brain works. It\u2019s a universal function approximator, which is capable of simulating any function, and is now used to solve the most complex problems in machine learning. What\u2019s more, they\u2019re able to work with all types of data (images, audio, video, and text).<\/p>\n<p id=\"0f2d\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Both genetic algorithms (GAs) and neural networks (NNs) are similar, as both are biologically-inspired techniques. This similarity motivates us to create a hybrid of both to see whether a GA can train NNs with high accuracy.<\/p>\n<p id=\"a5d6\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">This tutorial uses <a class=\"af mi\" href=\"https:\/\/pygad.readthedocs.io\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">PyGAD<\/a>, a Python library that supports building and training NNs using a GA. <a class=\"af mi\" href=\"https:\/\/pygad.readthedocs.io\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">PyGAD<\/a> offers both classification and regression NNs. Before starting, install <a class=\"af mi\" href=\"https:\/\/pygad.readthedocs.io\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">PyGAD<\/a> using <code class=\"cw ng nh ni nj b\">pip<\/code>.<\/p>\n<p id=\"2e0e\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">This is the outline:<\/p>\n<ul class=\"\">\n<li id=\"ac5b\" class=\"mj mk fo be b ml mm mn mo mp mq mr ms nk mu mv mw nl my mz na nm nc nd ne nf nn no np bj\" data-selectable-paragraph=\"\">Genetic Algorithm Overview<\/li>\n<li id=\"d3cf\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\">Build a Population of Neural Networks<\/li>\n<li id=\"000f\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\">Build the Fitness Function<\/li>\n<li id=\"a97e\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\">Build a Genetic Algorithm<\/li>\n<li id=\"626f\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\">Train Network using Genetic Algorithm<\/li>\n<li id=\"6aed\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\">Complete Code for Classification Neural Network<\/li>\n<li id=\"7c9e\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\">Regression Neural Networks<\/li>\n<li id=\"0ba3\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\">About the Author<\/li>\n<\/ul>\n<h1 id=\"2f17\" class=\"nv nw fo be nx ny nz oa ob oc od oe of og oh oi oj ok ol om on oo op oq or os bj\" data-selectable-paragraph=\"\">Genetic Algorithm Overview<\/h1>\n<p id=\"f6fc\" class=\"pw-post-body-paragraph mj mk fo be b ml ot mn mo mp ou mr ms mt ov mv mw mx ow mz na nb ox nd ne nf fh bj\" data-selectable-paragraph=\"\">This tutorial will not discuss the genetic algorithm in detail\u2014the reader should read about how it works. Just a brief overview is provided below, which serves the purposes of this tutorial. To dive deeper into how GAs work, check out this <a href=\"https:\/\/towardsdatascience.com\/introduction-to-optimization-with-genetic-algorithm-2f5001d9964b\">tutorial<\/a>.<\/p>\n<p id=\"b14e\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Assume the problem to be solved using a GA is represented according to the following equation, where <strong class=\"be ps\">Y<\/strong> is the output, <strong class=\"be ps\">X<\/strong> is the input, and <strong class=\"be ps\">a<\/strong> &amp; <strong class=\"be ps\">b<\/strong> are the parameters. Given that <strong class=\"be ps\">X=2<\/strong> and <strong class=\"be ps\">Y=4<\/strong>, how is a GA used to find the values of the parameters <strong class=\"be ps\">a<\/strong> and <strong class=\"be ps\">b<\/strong> to solve the problem?<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"1c12\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">Y = a * X + b, X=2 and Y=4<\/span><\/pre>\n<p id=\"e5df\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The next figure lists the steps in the pipeline of the genetic algorithm. The first step is to create an initial population of solutions. According to the above equation, the solution consists of the 2 parameters: <strong class=\"be ps\">a<\/strong> and <strong class=\"be ps\">b<\/strong>. By creating an initial population of solutions, we select different values for the 2 parameters <strong class=\"be ps\">a<\/strong> and <strong class=\"be ps\">b<\/strong>. Here are 5 random solutions:<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"1339\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">                a       b\nsolution 1:     -4      1\nsolution 2:     8       4\nsolution 3:     0.3     8\nsolution 4:     2       3\nsolution 5:     4       7<\/span><\/pre>\n<figure class=\"pt pu pv pw px lt me mf paragraph-image\">\n<div class=\"qh qi eb qj bg qk\" tabindex=\"0\" role=\"button\">\n<figure><img loading=\"lazy\" decoding=\"async\" class=\"bg lz ma c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*jBgF-TWLACBcnxJ6KmE2pg.jpeg\" alt=\"\" width=\"700\" height=\"903\"><\/figure><div class=\"me mf qg\"><picture><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/format:webp\/1*jBgF-TWLACBcnxJ6KmE2pg.jpeg 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/format:webp\/1*jBgF-TWLACBcnxJ6KmE2pg.jpeg 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/format:webp\/1*jBgF-TWLACBcnxJ6KmE2pg.jpeg 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/format:webp\/1*jBgF-TWLACBcnxJ6KmE2pg.jpeg 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/format:webp\/1*jBgF-TWLACBcnxJ6KmE2pg.jpeg 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/format:webp\/1*jBgF-TWLACBcnxJ6KmE2pg.jpeg 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/format:webp\/1*jBgF-TWLACBcnxJ6KmE2pg.jpeg 1400w\" type=\"image\/webp\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\"><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/1*jBgF-TWLACBcnxJ6KmE2pg.jpeg 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/1*jBgF-TWLACBcnxJ6KmE2pg.jpeg 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/1*jBgF-TWLACBcnxJ6KmE2pg.jpeg 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/1*jBgF-TWLACBcnxJ6KmE2pg.jpeg 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/1*jBgF-TWLACBcnxJ6KmE2pg.jpeg 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/1*jBgF-TWLACBcnxJ6KmE2pg.jpeg 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*jBgF-TWLACBcnxJ6KmE2pg.jpeg 1400w\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\" data-testid=\"og\"><\/picture><\/div>\n<\/div>\n<\/figure>\n<p id=\"eb5a\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">After the initial population is created, a fitness function is used to calculate a fitness value for each solution. The fitness function changes based on the problem being solved. Regarding the equation <code class=\"cw ng nh ni nj b\">Y = a * X + b<\/code>, a fitness function can calculate how close the prediction is to the target output <code class=\"cw ng nh ni nj b\">Y=4<\/code>using the following equation. The higher the fitness value, the better the solution.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"e36d\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">fitness = 1\/|predicted-4|<\/span><\/pre>\n<p id=\"d3f7\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Before the fitness is calculated, the predicted output is calculated according to the parameters of each solution. After that, the fitness value is calculated. Here are predicted outputs and fitness values of all solutions. The highest fitness value is 0.143, and the lowest is 0.05.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"b057\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">                a       b       Prediction      Fitness\nsolution 1:     -4      1       -7              0.09\nsolution 2:     8       4       20              0.05\nsolution 3:     0.3     8       8.6             0.116\nsolution 4:     2       3       7               0.143\nsolution 5:     4       7       15              0.06<\/span><\/pre>\n<p id=\"f534\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The solutions with the highest fitness values are selected as parents. Here is the order of the 5 solutions according to their fitness value, in descending order:<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"9781\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">Solution 4\nSolution 3\nSolution 1\nSolution 5\nSolution 2<\/span><\/pre>\n<p id=\"4a00\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">By selecting some of the top solutions as parents, the GA applies crossover and mutation operations between them to create new solutions.<\/p>\n<p id=\"04f4\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The previous discussion can be summarized into the following 2 points:<\/p>\n<ol class=\"\">\n<li id=\"2d66\" class=\"mj mk fo be b ml mm mn mo mp mq mr ms nk mu mv mw nl my mz na nm nc nd ne nf ql no np bj\" data-selectable-paragraph=\"\">Creating an initial population of solutions.<\/li>\n<li id=\"ea2a\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf ql no np bj\" data-selectable-paragraph=\"\">Creating a fitness function.<\/li>\n<\/ol>\n<p id=\"dc82\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The next section discusses how to build an initial population of neural networks using PyGAD.<\/p>\n<h1 id=\"9cab\" class=\"nv nw fo be nx ny nz oa ob oc od oe of og oh oi oj ok ol om on oo op oq or os bj\" data-selectable-paragraph=\"\">Build a Population of Neural Networks<\/h1>\n<p id=\"e839\" class=\"pw-post-body-paragraph mj mk fo be b ml ot mn mo mp ou mr ms mt ov mv mw mx ow mz na nb ox nd ne nf fh bj\" data-selectable-paragraph=\"\">Similar to the 2 parameters <strong class=\"be ps\">a<\/strong> and <strong class=\"be ps\">b<\/strong> in the previously-mentioned equation <code class=\"cw ng nh ni nj b\">Y = a * X + b<\/code>, a neural network has its own parameters (i.e. weights). For example, the network in the next figure has 8 parameters (<strong class=\"be ps\">W1<\/strong> to <strong class=\"be ps\">W8<\/strong>). Using these parameters, the network can make predictions.<\/p>\n<p id=\"f59d\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">One solution in the genetic algorithm has values for the 8 parameters. Usually, the population consists of several solutions. How, then, do we create a population of neural nets?<\/p>\n<figure class=\"pt pu pv pw px lt me mf paragraph-image\">\n<div class=\"qh qi eb qj bg qk\" tabindex=\"0\" role=\"button\">\n<figure><img loading=\"lazy\" decoding=\"async\" class=\"bg lz ma c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*G0_8TuEIUHN7O5mJuY-vdA.jpeg\" alt=\"\" width=\"700\" height=\"453\"><\/figure><div class=\"me mf qm\"><picture><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/format:webp\/1*G0_8TuEIUHN7O5mJuY-vdA.jpeg 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/format:webp\/1*G0_8TuEIUHN7O5mJuY-vdA.jpeg 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/format:webp\/1*G0_8TuEIUHN7O5mJuY-vdA.jpeg 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/format:webp\/1*G0_8TuEIUHN7O5mJuY-vdA.jpeg 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/format:webp\/1*G0_8TuEIUHN7O5mJuY-vdA.jpeg 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/format:webp\/1*G0_8TuEIUHN7O5mJuY-vdA.jpeg 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/format:webp\/1*G0_8TuEIUHN7O5mJuY-vdA.jpeg 1400w\" type=\"image\/webp\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\"><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/1*G0_8TuEIUHN7O5mJuY-vdA.jpeg 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/1*G0_8TuEIUHN7O5mJuY-vdA.jpeg 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/1*G0_8TuEIUHN7O5mJuY-vdA.jpeg 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/1*G0_8TuEIUHN7O5mJuY-vdA.jpeg 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/1*G0_8TuEIUHN7O5mJuY-vdA.jpeg 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/1*G0_8TuEIUHN7O5mJuY-vdA.jpeg 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*G0_8TuEIUHN7O5mJuY-vdA.jpeg 1400w\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\" data-testid=\"og\"><\/picture><\/div>\n<\/div>\n<\/figure>\n<p id=\"fc61\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The PyGAD library has a module named <code class=\"cw ng nh ni nj b\">gann<\/code> (Genetic Algorithm &#8211; Neural Network) that builds an initial population of neural networks using its class named <code class=\"cw ng nh ni nj b\">GANN<\/code>. To create a population of neural networks, just create an instance of this class.<\/p>\n<p id=\"bf34\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The constructor of the <code class=\"cw ng nh ni nj b\">GANN<\/code> class has the following parameters:<\/p>\n<ul class=\"\">\n<li id=\"d5a3\" class=\"mj mk fo be b ml mm mn mo mp mq mr ms nk mu mv mw nl my mz na nm nc nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">num_neurons_input<\/code>: Number of inputs to the network.<\/li>\n<li id=\"5434\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">num_neurons_output<\/code>: Number of outputs (i.e. number of neurons in the output layer).<\/li>\n<li id=\"51a1\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">num_neurons_hidden_layers<\/code>: Number of neurons in the hidden layer(s).<\/li>\n<li id=\"51e2\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">output_activation<\/code>: The activation function at the output layer.<\/li>\n<li id=\"fd2a\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">hidden_activations<\/code>: A list of activation functions for the hidden layers.<\/li>\n<li id=\"5f8e\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">num_solutions=None<\/code>: Number of solutions in the population. It must be at least 2.<\/li>\n<\/ul>\n<p id=\"8d4e\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The first example discussed centers on building a classification neural network for the XOR (Exclusive OR logic gate) problem. The inputs and outputs of this problem are prepared as NumPy arrays, as shown below:<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"a5f9\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">import numpy\n\u200b\ndata_inputs = numpy.array([[1, 1],\n                           [1, 0],\n                           [0, 1],\n                           [0, 0]])\n\u200b\ndata_outputs = numpy.array([0,\n                            1,\n                            1,\n                            0])<\/span><\/pre>\n<p id=\"4316\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The XOR problem has 4 samples, where each sample has 2 inputs and 1 output. As a result, the <code class=\"cw ng nh ni nj b\">num_neurons_input<\/code> parameter should be assigned a value of 2. The value of the <code class=\"cw ng nh ni nj b\">num_neurons_output<\/code> parameter should be 2 because there are 2 classes.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"979e\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">num_neurons_input = 2\nnum_neurons_output = 2<\/span><\/pre>\n<p id=\"a0a6\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The <code class=\"cw ng nh ni nj b\">num_neurons_hidden_layers<\/code> parameter accepts a list representing the number of neurons in the hidden layer. For example, <code class=\"cw ng nh ni nj b\">num_neurons_hidden_layers=[5, 2]<\/code> means there are 2 hidden layers, where the first layer has 5 neurons and the second layer has 2 neurons. For the XOR problem, a single hidden layer with 2 neurons is enough.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"6e4c\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">num_neurons_hidden_layers = [2]<\/span><\/pre>\n<p id=\"e96b\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The architecture of the network that simulates the XOR gate is shown below.<\/p>\n<figure class=\"pt pu pv pw px lt me mf paragraph-image\">\n<div class=\"qh qi eb qj bg qk\" tabindex=\"0\" role=\"button\">\n<figure><img loading=\"lazy\" decoding=\"async\" class=\"bg lz ma c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*Oe8vg4Ub-vekNNUGlb4mfQ.jpeg\" alt=\"\" width=\"700\" height=\"577\"><\/figure><div class=\"me mf qm\"><picture><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/format:webp\/1*Oe8vg4Ub-vekNNUGlb4mfQ.jpeg 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/format:webp\/1*Oe8vg4Ub-vekNNUGlb4mfQ.jpeg 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/format:webp\/1*Oe8vg4Ub-vekNNUGlb4mfQ.jpeg 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/format:webp\/1*Oe8vg4Ub-vekNNUGlb4mfQ.jpeg 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/format:webp\/1*Oe8vg4Ub-vekNNUGlb4mfQ.jpeg 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/format:webp\/1*Oe8vg4Ub-vekNNUGlb4mfQ.jpeg 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/format:webp\/1*Oe8vg4Ub-vekNNUGlb4mfQ.jpeg 1400w\" type=\"image\/webp\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\"><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/1*Oe8vg4Ub-vekNNUGlb4mfQ.jpeg 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/1*Oe8vg4Ub-vekNNUGlb4mfQ.jpeg 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/1*Oe8vg4Ub-vekNNUGlb4mfQ.jpeg 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/1*Oe8vg4Ub-vekNNUGlb4mfQ.jpeg 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/1*Oe8vg4Ub-vekNNUGlb4mfQ.jpeg 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/1*Oe8vg4Ub-vekNNUGlb4mfQ.jpeg 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*Oe8vg4Ub-vekNNUGlb4mfQ.jpeg 1400w\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\" data-testid=\"og\"><\/picture><\/div>\n<\/div>\n<\/figure>\n<p id=\"d88d\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The <code class=\"cw ng nh ni nj b\">gann<\/code> module supports 3 types of activation functions:<\/p>\n<ol class=\"\">\n<li id=\"2eed\" class=\"mj mk fo be b ml mm mn mo mp mq mr ms nk mu mv mw nl my mz na nm nc nd ne nf ql no np bj\" data-selectable-paragraph=\"\">Sigmoid<\/li>\n<li id=\"38a1\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf ql no np bj\" data-selectable-paragraph=\"\">Rectified linear unit (ReLU)<\/li>\n<li id=\"1c53\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf ql no np bj\" data-selectable-paragraph=\"\">Softmax: Used at the output layer to make predictions.<\/li>\n<\/ol>\n<p id=\"398e\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The activation function for the hidden layer\u2019s neurons is set to <code class=\"cw ng nh ni nj b\">relu<\/code>, according to the next line. The <code class=\"cw ng nh ni nj b\">hidden_activations<\/code> parameter accepts a list of the names of each hidden layer&#8217;s activation function.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"277c\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">hidden_activations=[\"relu\"]<\/span><\/pre>\n<p id=\"3e71\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">If there are multiple hidden layers, then the activation function of each layer is listed.<\/p>\n<p id=\"d8ed\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">For the output layer, softmax is used.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"24eb\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">output_activation = \"softmax\"<\/span><\/pre>\n<p id=\"86a4\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The last parameter to prepare its value is the <code class=\"cw ng nh ni nj b\">num_solutions<\/code> parameter, which accepts the number of solutions in the population. Let&#8217;s set it to 5. As a result, there are 5 different value combinations for the parameters in the neural network.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"13ef\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">num_solutions = 5<\/span><\/pre>\n<p id=\"c893\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">After preparing all the parameters, here is the Python code that creates an instance of the <code class=\"cw ng nh ni nj b\">GANN<\/code> class to build the initial population for networks solving the XOR problem.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"878a\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">import pygad.gann\n\u200b\nGANN_instance = pygad.gann.GANN(num_solutions=5,\n                                num_neurons_input=2,\n                                num_neurons_hidden_layers=[2],\n                                num_neurons_output=1,\n                                hidden_activations=[\"relu\"],\n                                output_activation=\"softmax\")<\/span><\/pre>\n<p id=\"e8c0\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">There is an attribute called <code class=\"cw ng nh ni nj b\">population_networks<\/code> inside the <code class=\"cw ng nh ni nj b\">GANN<\/code> class that returns references to the networks within the population. Here is how the networks&#8217; references are returned.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"163d\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">print(GANN_instance.population_networks)<\/span><\/pre>\n<p id=\"e6ea\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Result:<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"cd85\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">[\n &lt;pygad.nn.nn.DenseLayer at 0x2725e28b630&gt;,\n &lt;pygad.nn.nn.DenseLayer at 0x2725e28b9b0&gt;,\n &lt;pygad.nn.nn.DenseLayer at 0x2725e190ac8&gt;,\n &lt;pygad.nn.nn.DenseLayer at 0x2725e27eb00&gt;,\n &lt;pygad.nn.nn.DenseLayer at 0x2725e27e940&gt;\n]<\/span><\/pre>\n<p id=\"5bf2\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">There is also a function called <code class=\"cw ng nh ni nj b\">population_as_vectors()<\/code> in the <code class=\"cw ng nh ni nj b\">gann<\/code>module that returns a list of the solutions&#8217; parameters.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"887d\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">population_vectors = pygad.gann.population_as_vectors(population_networks=GANN_instance.population_networks)<\/span><span id=\"a2ca\" class=\"qc nw fo nj b ho qn qe l ie qf\" data-selectable-paragraph=\"\">print(population_vectors)<\/span><\/pre>\n<p id=\"d5e4\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Here are the parameters of the 5 solutions.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"ea34\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">[\n array([-0.01812899, -0.02474208, -0.07105532, -0.01654875, -0.04692194, -0.02432995]),\n array([-0.05717154,  0.04262176,  0.01157643,  0.09404094, -0.04565558, -0.01055631]),\n array([-0.03654666, -0.05663649, -0.01808185,  0.04675663,  0.01162651, 0.02960824]),\n array([ 0.03844111,  0.09247692, -0.0251123 , -0.07804498, -0.04800328, 0.09069625]),\n array([-0.07784676, -0.09123247,  0.03205176, -0.00280736,  0.03818657, -0.07136233])\n]<\/span><\/pre>\n<p id=\"5519\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Now, the population of networks is prepared. The next section discusses building the genetic algorithm.<\/p>\n<h1 id=\"371d\" class=\"nv nw fo be nx ny nz oa ob oc od oe of og oh oi oj ok ol om on oo op oq or os bj\" data-selectable-paragraph=\"\">Build the Genetic Algorithm<\/h1>\n<p id=\"5f4f\" class=\"pw-post-body-paragraph mj mk fo be b ml ot mn mo mp ou mr ms mt ov mv mw mx ow mz na nb ox nd ne nf fh bj\" data-selectable-paragraph=\"\">The PyGAD library has a module named <code class=\"cw ng nh ni nj b\">pygad<\/code> that builds instances of the genetic algorithm using its <code class=\"cw ng nh ni nj b\">GA<\/code> class. This class accepts more than 20 parameters, but the ones needed in this tutorial are listed below:<\/p>\n<ul class=\"\">\n<li id=\"ac6e\" class=\"mj mk fo be b ml mm mn mo mp mq mr ms nk mu mv mw nl my mz na nm nc nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">num_generations<\/code>: Number of generations.<\/li>\n<li id=\"fead\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">num_parents_mating<\/code>: Number of parents selected from the population.<\/li>\n<li id=\"b08e\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">initial_population<\/code>: The initial population.<\/li>\n<li id=\"0734\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">fitness_func<\/code>: The fitness function, by which each solution is assessed.<\/li>\n<li id=\"060f\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">mutation_percent_genes<\/code>: Percentage of genes to mutate.<\/li>\n<li id=\"6523\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">callback_generation<\/code>: A function called after each generation.<\/li>\n<\/ul>\n<p id=\"0431\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">num_generations<\/code> is assigned an integer representing the number of iterations\/generations within which the solutions are evolved. It\u2019s set to 50 in this example.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"7d34\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">num_generations = 50<\/span><\/pre>\n<p id=\"af60\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Given that there are 5 solutions within the population, the <code class=\"cw ng nh ni nj b\">num_parents_mating<\/code> could be set to 3.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"7662\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">num_parents_mating = 3<\/span><\/pre>\n<p id=\"1749\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The initial population was previously returned into the <code class=\"cw ng nh ni nj b\">population_vectors<\/code>variable, which is assigned to the <code class=\"cw ng nh ni nj b\">initial_population<\/code> parameter.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"a733\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">initial_population = population_vectors.copy()<\/span><\/pre>\n<p id=\"28b9\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The <code class=\"cw ng nh ni nj b\">fitness_func<\/code> parameter is critical and must be designed carefully. It accepts a user-defined function that returns the fitness value of a single solution. This function must accept 2 parameters:<\/p>\n<ol class=\"\">\n<li id=\"fba2\" class=\"mj mk fo be b ml mm mn mo mp mq mr ms nk mu mv mw nl my mz na nm nc nd ne nf ql no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">solution<\/code>: The solution to calculate its fitness value.<\/li>\n<li id=\"7dba\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf ql no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">sol_idx<\/code>: Index of the solution within the population.<\/li>\n<\/ol>\n<p id=\"268e\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">To measure the classification accuracy of a solution within the population, here\u2019s the implementation of the fitness function. It uses the <code class=\"cw ng nh ni nj b\">predict()<\/code>function from the <code class=\"cw ng nh ni nj b\">pygad.nn<\/code> module to make predictions based on the current solution&#8217;s parameters. This function accepts the following parameters:<\/p>\n<ol class=\"\">\n<li id=\"acac\" class=\"mj mk fo be b ml mm mn mo mp mq mr ms nk mu mv mw nl my mz na nm nc nd ne nf ql no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">last_layer<\/code>: A reference to the last layer in the neural network.<\/li>\n<li id=\"17af\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf ql no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">data_inputs<\/code>: The input samples to predict their outputs.<\/li>\n<li id=\"2009\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf ql no np bj\" data-selectable-paragraph=\"\"><code class=\"cw ng nh ni nj b\">problem_type<\/code>: Type of the problem, which can be either <code class=\"cw ng nh ni nj b\">\"classification\"<\/code> (default) or <code class=\"cw ng nh ni nj b\">\"regression\"<\/code>.<\/li>\n<\/ol>\n<p id=\"87a8\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Based on the network predictions returned by the <code class=\"cw ng nh ni nj b\">pygad.nn.predict()<\/code>function, the classification accuracy is calculated and saved into the <code class=\"cw ng nh ni nj b\">solution_fitness<\/code> variable.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"8d5e\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">def fitness_function(solution, sol_idx):\n    global GANN_instance, data_inputs, data_outputs<\/span><span id=\"b5c9\" class=\"qc nw fo nj b ho qn qe l ie qf\" data-selectable-paragraph=\"\">    predictions = pygad.nn.predict(last_layer=GANN_instance.population_networks[sol_idx],\n                                   data_inputs=data_inputs,\n                                   problem_type=\"classification\")\n    correct_predictions = numpy.where(predictions == data_outputs)[0].size\n    solution_fitness = (correct_predictions\/data_outputs.size)*100<\/span><span id=\"ad76\" class=\"qc nw fo nj b ho qn qe l ie qf\" data-selectable-paragraph=\"\">    return solution_fitness<\/span><\/pre>\n<p id=\"b751\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The above function is assigned to the <code class=\"cw ng nh ni nj b\">fitness_func<\/code> parameter.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"a332\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">fitness_func = fitness_function<\/span><\/pre>\n<p id=\"aa5c\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The next parameter is <code class=\"cw ng nh ni nj b\">mutation_percent_genes<\/code>, which accepts the percentage of genes to mutate. It\u2019s recommended to set a small percentage. This is why 5% is used.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"df7d\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">mutation_percent_genes = 5<\/span><\/pre>\n<p id=\"f27d\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Finally, the <code class=\"cw ng nh ni nj b\">callback_generation<\/code> parameter accepts a function that\u2019s called after each generation. For training a neural network, the purpose of this function is to update the network weights by those evolved using the GA. Here is the implementation of this function.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"5064\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">def callback_generation(ga_instance):\n    global GANN_instance\n\u200b\n    population_matrices = pygad.gann.population_as_matrices(population_networks=GANN_instance.population_networks, population_vectors=ga_instance.population)\n    GANN_instance.update_population_trained_weights(population_trained_weights=population_matrices)\n\u200b\n    print(\"Generation = {generation}\".format(generation=ga_instance.generations_completed))\n    print(\"Accuracy   = {fitness}\".format(fitness=ga_instance.best_solution()[1]))<\/span><\/pre>\n<p id=\"84e2\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The function above is assigned to the <code class=\"cw ng nh ni nj b\">callback_generation<\/code> parameter.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"5c36\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">callback_generation = callback_generation<\/span><\/pre>\n<p id=\"3e8a\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">After preparing all the parameters of the <code class=\"cw ng nh ni nj b\">GA<\/code> class, here is how we create an instance.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"8ae9\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">ga_instance = pygad.GA(num_generations=50,\n                       num_parents_mating=3,\n                       initial_population=population_vectors.copy(),\n                       fitness_func=fitness_func,\n                       mutation_percent_genes=5,\n                       callback_generation=callback_generation)<\/span><\/pre>\n<p id=\"fafb\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">At this time, the neural network and the genetic algorithm are both created. The next section discusses running the genetic algorithm to train the network.<\/p>\n<h1 id=\"2a92\" class=\"nv nw fo be nx ny nz oa ob oc od oe of og oh oi oj ok ol om on oo op oq or os bj\" data-selectable-paragraph=\"\">Train a Network using the Genetic Algorithm<\/h1>\n<p id=\"7ba0\" class=\"pw-post-body-paragraph mj mk fo be b ml ot mn mo mp ou mr ms mt ov mv mw mx ow mz na nb ox nd ne nf fh bj\" data-selectable-paragraph=\"\">To start evolving the GA, the <code class=\"cw ng nh ni nj b\">run()<\/code> method is called. This method applies the pipeline of the genetic algorithm by calculating the fitness values of the solutions, selecting the parents, mating the parents by applying the mutation and crossover operations, and producing a new population. This process lasts for the specified number of generations.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"369b\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">ga_instance.run()<\/span><\/pre>\n<p id=\"f0a3\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">After the <code class=\"cw ng nh ni nj b\">run()<\/code> method completes, the <code class=\"cw ng nh ni nj b\">plot_result()<\/code> method can be used to create a figure showing how the accuracy changes by generation.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"9546\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">ga_instance.plot_result()<\/span><\/pre>\n<p id=\"eab0\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">According to the next figure, the GA found a solution that has 100% classification accuracy after around 14 generations.<\/p>\n<figure class=\"pt pu pv pw px lt me mf paragraph-image\">\n<figure><img loading=\"lazy\" decoding=\"async\" class=\"bg lz ma c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:389\/1*IC9KyqLUf9K-DfF6FxAfQQ.png\" alt=\"\" width=\"389\" height=\"278\"><\/figure><div class=\"me mf qo\"><picture><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/format:webp\/1*IC9KyqLUf9K-DfF6FxAfQQ.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/format:webp\/1*IC9KyqLUf9K-DfF6FxAfQQ.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/format:webp\/1*IC9KyqLUf9K-DfF6FxAfQQ.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/format:webp\/1*IC9KyqLUf9K-DfF6FxAfQQ.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/format:webp\/1*IC9KyqLUf9K-DfF6FxAfQQ.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/format:webp\/1*IC9KyqLUf9K-DfF6FxAfQQ.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:778\/format:webp\/1*IC9KyqLUf9K-DfF6FxAfQQ.png 778w\" type=\"image\/webp\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 389px\"><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/1*IC9KyqLUf9K-DfF6FxAfQQ.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/1*IC9KyqLUf9K-DfF6FxAfQQ.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/1*IC9KyqLUf9K-DfF6FxAfQQ.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/1*IC9KyqLUf9K-DfF6FxAfQQ.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/1*IC9KyqLUf9K-DfF6FxAfQQ.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/1*IC9KyqLUf9K-DfF6FxAfQQ.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:778\/1*IC9KyqLUf9K-DfF6FxAfQQ.png 778w\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 389px\" data-testid=\"og\"><\/picture><\/div>\n<\/figure>\n<p id=\"5800\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Information about the best solution can be returned using the <code class=\"cw ng nh ni nj b\">best_solution()<\/code> method. It returns the solution itself, its fitness value, and its index within the population.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"2e41\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">solution, solution_fitness, solution_idx = ga_instance.best_solution()<\/span><span id=\"f51f\" class=\"qc nw fo nj b ho qn qe l ie qf\" data-selectable-paragraph=\"\">print(solution)\nprint(solution_fitness)\nprint(solution_idx)<\/span><\/pre>\n<p id=\"9824\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Result:<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"50ed\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">[-1.42839466  0.02073534  0.02709985 -0.1430065   0.46980429  0.01294253 -0.37210115 0.03971092]<\/span><span id=\"54da\" class=\"qc nw fo nj b ho qn qe l ie qf\" data-selectable-paragraph=\"\">100.0<\/span><span id=\"54c5\" class=\"qc nw fo nj b ho qn qe l ie qf\" data-selectable-paragraph=\"\">0<\/span><\/pre>\n<p id=\"7eca\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">The best solution can be used to make predictions according to the code below. Because the accuracy is 100%, then all the outputs are predicted successfully.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"1145\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">predictions = pygad.nn.predict(last_layer=GANN_instance.population_networks[solution_idx],\n                               data_inputs=data_inputs,\n                              problem_type=\"classification\")\nprint(\"Predictions of the trained network : {predictions}\".format(predictions=predictions))<\/span><\/pre>\n<p id=\"bffd\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Result:<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"a3b6\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">Predictions of the trained network : [0. 1. 1. 0.]<\/span><\/pre>\n<h1 id=\"7193\" class=\"nv nw fo be nx ny nz oa ob oc od oe of og oh oi oj ok ol om on oo op oq or os bj\" data-selectable-paragraph=\"\">Complete Code for Classification Neural Network<\/h1>\n<p id=\"a958\" class=\"pw-post-body-paragraph mj mk fo be b ml ot mn mo mp ou mr ms mt ov mv mw mx ow mz na nb ox nd ne nf fh bj\" data-selectable-paragraph=\"\">The complete code of building and training a neural network using the genetic algorithm is shown below.<\/p>\n<pre>import numpy\nimport pygad\nimport pygad.nn\nimport pygad.gann\n\u200b\ndef fitness_func(solution, sol_idx):\n    global GANN_instance, data_inputs, data_outputs\n\u200b\n    predictions = pygad.nn.predict(last_layer=GANN_instance.population_networks[sol_idx],\n                                   data_inputs=data_inputs)\n    correct_predictions = numpy.where(predictions == data_outputs)[0].size\n    solution_fitness = (correct_predictions\/data_outputs.size)*100\n\u200b\n    return solution_fitness\n\u200b\ndef callback_generation(ga_instance):\n    global GANN_instance\n\u200b\n    population_matrices = pygad.gann.population_as_matrices(population_networks=GANN_instance.population_networks,\n                                                            population_vectors=ga_instance.population)\n\u200b\n    GANN_instance.update_population_trained_weights(population_trained_weights=population_matrices)\n\u200b\n    print(\"Generation = {generation}\".format(generation=ga_instance.generations_completed))\n    print(\"Accuracy   = {fitness}\".format(fitness=ga_instance.best_solution()[1]))\n\u200b\ndata_inputs = numpy.array([[1, 1],\n                           [1, 0],\n                           [0, 1],\n                           [0, 0]])\n\u200b\ndata_outputs = numpy.array([0,\n                            1,\n                            1,\n                            0])\n\u200b\nGANN_instance = pygad.gann.GANN(num_solutions=5,\n                                num_neurons_input=2,\n                                num_neurons_hidden_layers=[2],\n                                num_neurons_output=2,\n                                hidden_activations=[\"relu\"],\n                                output_activation=\"softmax\")\n\u200b\npopulation_vectors = pygad.gann.population_as_vectors(population_networks=GANN_instance.population_networks)\n\u200b\nga_instance = pygad.GA(num_generations=50,\n                       num_parents_mating=3,\n                       initial_population=population_vectors.copy(),\n                       fitness_func=fitness_func,\n                       mutation_percent_genes=5,\n                       callback_generation=callback_generation)\n\u200b\nga_instance.run()\n\u200b\nga_instance.plot_result()\n\u200b\nsolution, solution_fitness, solution_idx = ga_instance.best_solution()\nprint(solution)\nprint(solution_fitness)\nprint(solution_idx)<\/pre>\n<h1 id=\"1090\" class=\"nv nw fo be nx ny nz oa ob oc od oe of og oh oi oj ok ol om on oo op oq or os bj\" data-selectable-paragraph=\"\">Regression Neural Networks<\/h1>\n<p id=\"42ac\" class=\"pw-post-body-paragraph mj mk fo be b ml ot mn mo mp ou mr ms mt ov mv mw mx ow mz na nb ox nd ne nf fh bj\" data-selectable-paragraph=\"\">In addition to building neural networks for classification, PyGAD also supports regression networks. Compared to the previous code, there are 2 changes:<\/p>\n<ol class=\"\">\n<li id=\"d204\" class=\"mj mk fo be b ml mm mn mo mp mq mr ms nk mu mv mw nl my mz na nm nc nd ne nf ql no np bj\" data-selectable-paragraph=\"\">Edit the <code class=\"cw ng nh ni nj b\">fitness_func()<\/code> to calculate the regression error rather than classification accuracy.<\/li>\n<li id=\"5819\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf ql no np bj\" data-selectable-paragraph=\"\">Set the <code class=\"cw ng nh ni nj b\">problem_type<\/code> parameter in the <code class=\"cw ng nh ni nj b\">pygad.nn.predict()<\/code>function to <code class=\"cw ng nh ni nj b\">\"regression\"<\/code> rather than <code class=\"cw ng nh ni nj b\">\"classification\"<\/code>.<\/li>\n<\/ol>\n<p id=\"5b14\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Here, some new training samples are used to train the NN:<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"a779\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">data_inputs = numpy.array([[2, 5, -3, 0.1],\n                           [8, 15, 20, 13]])\n\u200b\ndata_outputs = numpy.array([[0.1, 0.2],\n                            [1.8, 1.5]])<\/span><\/pre>\n<p id=\"9ee1\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">In this example, the mean absolute error is used inside the fitness function. Here is its new code. Note how the <code class=\"cw ng nh ni nj b\">problem_type<\/code> argument is set to <code class=\"cw ng nh ni nj b\">\"regression\"<\/code>.<\/p>\n<pre class=\"pt pu pv pw px py nj pz qa ax qb bj\"><span id=\"955f\" class=\"qc nw fo nj b ho qd qe l ie qf\" data-selectable-paragraph=\"\">def fitness_func(solution, sol_idx):\n    global GANN_instance, data_inputs, data_outputs\n\u200b\n    predictions = pygad.nn.predict(last_layer=GANN_instance.population_networks[sol_idx],\n                                   data_inputs=data_inputs,\n                                   problem_type=\"regression\")\n    solution_fitness = 1.0\/numpy.mean(numpy.abs(predictions - data_outputs))\n\u200b\n    return solution_fitness<\/span><\/pre>\n<p id=\"94d5\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">For the complete code of this example, check out <a class=\"af mi\" href=\"https:\/\/github.com\/ahmedfgad\/NeuralGenetic\/blob\/master\/example_regression.py\" target=\"_blank\" rel=\"noopener ugc nofollow\">this code<\/a>:<\/p>\n<h1 id=\"4b9f\" class=\"nv nw fo be nx ny nz oa ob oc od oe of og oh oi oj ok ol om on oo op oq or os bj\" data-selectable-paragraph=\"\">About the Author<\/h1>\n<p id=\"c920\" class=\"pw-post-body-paragraph mj mk fo be b ml ot mn mo mp ou mr ms mt ov mv mw mx ow mz na nb ox nd ne nf fh bj\" data-selectable-paragraph=\"\"><a class=\"af mi\" href=\"https:\/\/www.linkedin.com\/in\/fatima-ezzahra-jarmouni-341a6b167\" target=\"_blank\" rel=\"noopener ugc nofollow\">Fatima Ezzahra JARMOUNI<\/a> is a Moroccan junior data scientist who received an M.Sc. degree in data science from <a class=\"af mi\" href=\"http:\/\/ensias.um5.ac.ma\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">L\u2019\u00c9cole Nationale Sup\u00e9rieure d\u2019Informatique et d\u2019Analyse des Syst\u00e8mes (ENSIAS)<\/a>.<\/p>\n<p id=\"8b27\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Fatima Ezzahra is <strong class=\"be ps\">actively seeking<\/strong> for a job in data science and data analysis. If there is a chance, use any of these contacts:<\/p>\n<ul class=\"\">\n<li id=\"0d88\" class=\"mj mk fo be b ml mm mn mo mp mq mr ms nk mu mv mw nl my mz na nm nc nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><a class=\"af mi\" href=\"https:\/\/www.linkedin.com\/in\/fatima-ezzahra-jarmouni-341a6b167\" target=\"_blank\" rel=\"noopener ugc nofollow\">LinkedIn<\/a>: <a class=\"af mi\" href=\"https:\/\/www.linkedin.com\/in\/fatima-ezzahra-jarmouni-341a6b167\" target=\"_blank\" rel=\"noopener ugc nofollow\">https:\/\/www.linkedin.com\/in\/fatima-ezzahra-jarmouni-341a6b167<\/a><\/li>\n<li id=\"6c02\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><a class=\"af mi\" href=\"https:\/\/github.com\/JARMOUNI94\" target=\"_blank\" rel=\"noopener ugc nofollow\">GitHub<\/a>: <a class=\"af mi\" href=\"https:\/\/github.com\/JARMOUNI94\" target=\"_blank\" rel=\"noopener ugc nofollow\">https:\/\/github.com\/JARMOUNI94<\/a><\/li>\n<li id=\"a72d\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\">E-mail: fatimaezzahrajarmouni at gmail dot com<\/li>\n<li id=\"fc57\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><a class=\"af mi\" href=\"https:\/\/www.researchgate.net\/profile\/Fatima_Ezzahra_Jarmouni2\" target=\"_blank\" rel=\"noopener ugc nofollow\">ResearchGate<\/a>: <a class=\"af mi\" href=\"https:\/\/www.researchgate.net\/profile\/Fatima_Ezzahra_Jarmouni2\" target=\"_blank\" rel=\"noopener ugc nofollow\">https:\/\/www.researchgate.net\/profile\/Fatima_Ezzahra_Jarmouni2<\/a><\/li>\n<li id=\"b4f5\" class=\"mj mk fo be b ml nq mn mo mp nr mr ms nk ns mv mw nl nt mz na nm nu nd ne nf nn no np bj\" data-selectable-paragraph=\"\"><a class=\"af mi\" href=\"https:\/\/um5s.academia.edu\/FatimaEzzahraJARMOUNI\" target=\"_blank\" rel=\"noopener ugc nofollow\">Academia<\/a>: <a class=\"af mi\" href=\"https:\/\/um5s.academia.edu\/FatimaEzzahraJARMOUNI\" target=\"_blank\" rel=\"noopener ugc nofollow\">https:\/\/um5s.academia.edu\/FatimaEzzahraJARMOUNI<\/a><\/li>\n<\/ul>\n<h1 id=\"cb84\" class=\"nv nw fo be nx ny nz oa ob oc od oe of og oh oi oj ok ol om on oo op oq or os bj\" data-selectable-paragraph=\"\">Conclusion<\/h1>\n<p id=\"906f\" class=\"pw-post-body-paragraph mj mk fo be b ml ot mn mo mp ou mr ms mt ov mv mw mx ow mz na nb ox nd ne nf fh bj\" data-selectable-paragraph=\"\">This tutorial discussed how to build and train both classification and regression neural networks using the genetic algorithm using a Python library called <a class=\"af mi\" href=\"https:\/\/pygad.readthedocs.io\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">PyGAD<\/a>.<\/p>\n<p id=\"76c6\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">To summarize what we\u2019ve covered: The library has a module named <code class=\"cw ng nh ni nj b\">gann<\/code>that creates a population of neural networks. The <code class=\"cw ng nh ni nj b\">pygad<\/code> module implements the genetic algorithm and uses the parameters in the population of neural networks as the initial population. The genetic algorithm then makes slight changes over many generations. For each generation, PyGAD updates the weights and uses it for making predictions using the neural network. This is a time-consuming process, especially for complex problems.<\/p>\n<p id=\"d466\" class=\"pw-post-body-paragraph mj mk fo be b ml mm mn mo mp mq mr ms mt mu mv mw mx my mz na nb nc nd ne nf fh bj\" data-selectable-paragraph=\"\">Thus, efforts are needed to speed-up the implementation of PyGAD. Cython may be an option.<\/p>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Photo by Alexander Popov on Unsplash The genetic algorithm (GA) is a biologically-inspired optimization algorithm. It has in recent years gained importance, as it\u2019s simple while also solving complex problems like travel route optimization, training machine learning algorithms, working with single and multi-objective problems, game playing, and more. Deep neural networks are inspired by the [&hellip;]<\/p>\n","protected":false},"author":45,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[6],"tags":[],"coauthors":[153],"class_list":["post-6565","post","type-post","status-publish","format-standard","hentry","category-machine-learning"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.9 (Yoast SEO v25.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Train Neural Networks Using a Genetic Algorithm in Python with PyGAD - Comet<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Train Neural Networks Using a Genetic Algorithm in Python with PyGAD\" \/>\n<meta property=\"og:description\" content=\"Photo by Alexander Popov on Unsplash The genetic algorithm (GA) is a biologically-inspired optimization algorithm. It has in recent years gained importance, as it\u2019s simple while also solving complex problems like travel route optimization, training machine learning algorithms, working with single and multi-objective problems, game playing, and more. Deep neural networks are inspired by the [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2023-06-29T19:28:44+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:15:16+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/miro.medium.com\/v2\/resize:fit:2500\/1*bKjXAtERq1m17q0IE2Eziw.jpeg\" \/>\n<meta name=\"author\" content=\"Fatima Ezzahra Jarmouni\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Fatima Ezzahra Jarmouni\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"12 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Train Neural Networks Using a Genetic Algorithm in Python with PyGAD - Comet","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/","og_locale":"en_US","og_type":"article","og_title":"Train Neural Networks Using a Genetic Algorithm in Python with PyGAD","og_description":"Photo by Alexander Popov on Unsplash The genetic algorithm (GA) is a biologically-inspired optimization algorithm. It has in recent years gained importance, as it\u2019s simple while also solving complex problems like travel route optimization, training machine learning algorithms, working with single and multi-objective problems, game playing, and more. Deep neural networks are inspired by the [&hellip;]","og_url":"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2023-06-29T19:28:44+00:00","article_modified_time":"2025-04-24T17:15:16+00:00","og_image":[{"url":"https:\/\/miro.medium.com\/v2\/resize:fit:2500\/1*bKjXAtERq1m17q0IE2Eziw.jpeg","type":"","width":"","height":""}],"author":"Fatima Ezzahra Jarmouni","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Fatima Ezzahra Jarmouni","Est. reading time":"12 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/"},"author":{"name":"Fatima Ezzahra Jarmouni","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/6c396caf689b4b242b3c23b2b7bed9df"},"headline":"Train Neural Networks Using a Genetic Algorithm in Python with PyGAD","datePublished":"2023-06-29T19:28:44+00:00","dateModified":"2025-04-24T17:15:16+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/"},"wordCount":1922,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:2500\/1*bKjXAtERq1m17q0IE2Eziw.jpeg","articleSection":["Machine Learning"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/","url":"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/","name":"Train Neural Networks Using a Genetic Algorithm in Python with PyGAD - Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:2500\/1*bKjXAtERq1m17q0IE2Eziw.jpeg","datePublished":"2023-06-29T19:28:44+00:00","dateModified":"2025-04-24T17:15:16+00:00","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/#primaryimage","url":"https:\/\/miro.medium.com\/v2\/resize:fit:2500\/1*bKjXAtERq1m17q0IE2Eziw.jpeg","contentUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:2500\/1*bKjXAtERq1m17q0IE2Eziw.jpeg"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/train-neural-networks-using-a-genetic-algorithm-in-python-with-pygad\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Train Neural Networks Using a Genetic Algorithm in Python with PyGAD"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/6c396caf689b4b242b3c23b2b7bed9df","name":"Fatima Ezzahra Jarmouni","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/0b661b2463e09a76965e57fc0018812b","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/08\/1631710283075-96x96.jpg","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/08\/1631710283075-96x96.jpg","caption":"Fatima Ezzahra Jarmouni"},"url":"https:\/\/www.comet.com\/site\/blog\/author\/fatima-ezzahra-jarmouni\/"}]}},"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/6565","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/45"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=6565"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/6565\/revisions"}],"predecessor-version":[{"id":15607,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/6565\/revisions\/15607"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=6565"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=6565"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=6565"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=6565"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}