![\"\"](\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/06\/1NUil-wyDtAKmoD-HjWXBLw-1024x614.webp\")
<\/figure><\/p>\n
Artificial intelligence (AI) is a buzzword you see pretty much everywhere around you, even when you\u2019re not looking. It has completely dominated tech media, newsrooms, and is even credited with the success of many modern applications.<\/p>\n
But does it really work, or is it just hype? Truth is, it does. While there might be some hype around its capabilities, AI has been demonstrated both in research and industry to work really well for a variety of tasks and use cases.<\/p>\n
There exist many techniques to make computers learn intelligently, but neural networks are one of the most popular and effective methods, most notably in complex tasks like image recognition, language translation, audio transcription, and so on.<\/p>\n
In this two-part series, I\u2019ll walk you through building a neural network from scratch. While you won\u2019t be building one from scratch in a real-world setting, it is advisable to work through this process at least once in your lifetime as an AI engineer. This can really help you better understand how neural networks work.<\/p>\n
In this first article, you\u2019ll learn:<\/p>\n
\u2014What Artificial Intelligence is<\/p>\n
\u2014What Deep Learning is<\/p>\n
\u2014What a Neural Network Entails<\/p>\n
\u2014Why Neural Networks are Popular<\/p>\n
\u2014Build a Neural Network From Scratch<\/p>\n
- \n
- Why use Python for AI?<\/li>\n
- Understanding the Problem<\/li>\n
- The Layers of a Neural Network<\/li>\n
- The Weights and Biases<\/li>\n
- The Activation Function<\/li>\n
- The Loss Function<\/li>\n
- Going Forward: Forward Propagation<\/li>\n
- A Step Backward: Backpropagation<\/li>\n
- Optimization and Training of the Neural Network<\/li>\n
- Making Predictions<\/li>\n
- Putting It All Together<\/li>\n<\/ul>\n
And once you\u2019ve had a chance to work through this tutorial, head on over to part 2, where we actually train and test the network we build.<\/p>\n
\n\n\n\ud83d\udc49\ud83c\udffd Full code in Google Colab <\/span>here<\/a><\/div>\n<\/div>\n<\/div>\n<\/div>\nWhat Is AI<\/h1>\n
Artificial intelligence (AI) is an umbrella term used to describe the intelligence shown by machines (computers), including their ability to mimic humans in areas such as learning and problem-solving. This means with AI, you can automate how you think, reason, and make decisions. As such, you can teach a computer to do what humans do, without explicitly programming it.<\/p>\n
Despite this simple explanation above, this isn\u2019t as easy as it sounds. And while many scientists and researchers have been able to teach machines to act like humans in areas like computer vision and natural language processing, there\u2019s still serious work to be done before we can have efficient and fully functioning AI systems.<\/p>\n
I\u2019m guessing that\u2019s the reason you\u2019re here-to learn how neural networks and AI work in general, and how you can use them to automate your own processes, build customized user experiences, and more.<\/p>\n
AI is broad and has numerous subfields, of which machine learning is a part. Machine learning itself has numerous techniques, of which neural networks are one (albeit a very successful technique).<\/p>\n
\n ![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:680\/1*Rk7gQpY_4HY5v6cmO58HhQ.jpeg\")
<\/figure><\/picture><\/div> Difference between AI, ML, and DL (Image source<\/a>)<\/figcaption><\/figure>\n What Is Deep Learning<\/h1>\n
Deep learning\u2013a machine learning technique\u2013is an efficient way of learning that relies on big data, where features that can help a machine map an input to an output is automatically extracted from layers of \u201cneurons\u201d.<\/p>\n
Deep learning is the main technology behind:<\/p>\n
- \n
- Driverless cars<\/li>\n
- Large-scale recommendation engines like Spotify, YouTube, and Amazon<\/li>\n
- Language translation services like Google Translate<\/li>\n
- Chatbots like Siri and Google assistant.<\/li>\n<\/ul>\n
A neural network is a type of deep learning architecture, and it\u2019s our primary focus in this tutorial. Some specific architectures for deep neural networks include convolutional neural networks (CNN) for computer vision use cases, recurrent neural networks (RNN) for language and time series modeling, and others like generative adversarial networks (GANs) for generative computer vision use cases.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n\n\nThe future of machine learning is on the edge. Subscribe to the Fritz AI Newsletter to discover the possibilities and benefits of embedding ML models inside mobile apps.<\/p>\n
What Is a Neural Network<\/h1>\n
Neural networks are composed of simple building blocks called neurons. While many people try to draw correlations between a neural network neuron and biological neurons, I will simply state the obvious here: \u201cA neuron is a mathematical function that takes data as input, performs a transformation on them, and produces an output\u201d.<\/p>\n
This means that neurons can represent any mathematical function; however, in neural networks, we typically use non-linear functions.<\/p>\n
\n \n![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*NZc0TcMCzpgVZXvUdEkqvA.png\")
<\/figure><\/picture><\/div>\n<\/div>\n A single neuron in a network<\/figcaption>\n<\/figure>\n Looking at the neuron above, you can see that it\u2019s composed of two main parts: the summation and the activation function. A neuron takes data (x\u2081, x\u2082, x\u2083) as input, multiplies each with a specific weight (w\u2081, w\u2082, w\u2083), and then passes the result to a nonlinear function called the activation function to produce an output.<\/p>\n
A neural network combines multiple neurons by stacking them vertically\/horizontally to create a network of neurons-hence the name \u201cneural network\u201d. A simple one-neuron network is called a perceptron and is the simplest network ever.<\/p>\n
Another important concept I\u2019ll explain in later sections of this tutorial is how a neural network actually learns the weights it assigns to each input feature. In neural nets, the weights are everything. If you know the correct weight, you can easily output correct predictions.<\/p>\n
In summary, what machine learning and deep learning really boils down to is actually trying to find the right weights that generalize to any input.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n\n\nWhy Are Neural Networks Popular?<\/h1>\n
In the previous section, I introduced neural networks and briefly explained the building blocks. Now we\u2019ll explore why neural networks are popular today.<\/p>\n
Neural networks have been around for a really long time\u2014a few major problems with them, and reasons, why people didn\u2019t use them before now, was due to the fact that:<\/p>\n
- \n
- They were notoriously difficult to train, in the sense that it can be difficult to get the right weights that generalize to new inputs.<\/li>\n
- They need huge amounts of data.<\/li>\n
- Computing power was still low and expensive.<\/li>\n<\/ol>\n
When these barriers were overcome, neural nets became cool again, and numerous applications sprung up.<\/p>\n
Neural networks are also very popular now because of their effectiveness on a wide range of tasks. They can automatically extract features from unstructured data like texts, images, and sounds, and deep learning has greatly reduced the time spent to manually create features.<\/p>\n
To illustrate this, I\u2019ll tell you a short story about Google Translate. In the early days of Google Translate, thousands of engineers, language experts, and computer scientists had to work all day to manually extract and create features from texts.<\/p>\n
These manual features had to be fed into machine learning models. Even with this time consuming and expensive task, the performance of these systems was nothing close to human-like. But when Geoff Hilton’s team showed that a neural network could be trained using a technique called backpropagation, Google switched from manually engineering features to using deep neural nets, and this greatly improved performance.<\/p>\n
This anecdote shows that with enough data and compute power, neural networks can do better than other machine learning algorithms\u2014hence, their rising popularity.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n\n\nBuilding a Neural Network From Scratch<\/h1>\n
Now that you\u2019ve gotten a brief introduction to AI, deep learning, and neural networks, including some reasons why they work well, you\u2019re going to build your very own neural net from scratch. To do this, you\u2019ll use Python and its efficient scientific library Numpy.<\/p>\n
Why Python for AI?<\/h2>\n
Python is a high-level, interpreted, and general-purpose language that can be used for a wide variety of tasks. It\u2019s one of the easiest languages to learn, and that makes it the go-to for new programmers. Python is popular among AI engineers\u2014in fact, the majority of AI applications are built with Python and Python-related tools. There are many reasons for this, some of which include:<\/p>\n
- \n
- There is a large ecosystem of pre-built libraries for scientific computation. Libraries like NumPy, SciPy, and Pandas make doing scientific calculations easy and quick, as the majority of these libraries are well-optimized for common ML and DL tasks.<\/li>\n
- Python is platform-independent and can be run on almost all devices. This means Python is easily compatible across platforms and can be deployed almost anywhere.<\/li>\n
- Python has a helpful and supportive community built around it, and this community provides tons of guides and tutorials for working with the language. You can rest assured that most problems you encounter have already been solved.<\/li>\n<\/ol>\n
I love the comic below that shows a flying programmer *winks<\/em>. It depicts why Python is easy to learn, with numerous libraries that you can import and use for almost any task\u2014including antigravity \ud83d\ude02!<\/p>\n
\n ![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:518\/1*TmwjhHeBOr1TNde_UbVMFQ.png\")
<\/figure><\/picture><\/div>\n The flying programmer (Image source<\/a>)<\/figcaption>\n<\/figure>\n Before you start flying, it is essential you\u2019ve properly set up your machine learning environment. If not, you should visit this page<\/a> first before moving onto the next section.<\/p>\n
What Are You Trying to Solve?<\/h2>\n
Before you start writing code, let\u2019s talk about the problem you\u2019re going to solve, as a more complete understanding of the problem will help you form the solution. In this tutorial, you\u2019re are going to create a neural network that predicts if a person will have heart disease or not. You\u2019ll use a heart disease dataset from the UCL data repository. You can download it here<\/a>.<\/p>\n
\n \n![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*x_8Ktepu-zevdmkdM4ueng.png\")
<\/figure><\/picture><\/div>\n<\/div>\n UCL heart disease dataset page<\/figcaption>\n<\/figure>\n On the dataset page, click on Data Folder<\/strong> and download the heart.dat<\/strong> file. This comes in a .dat file format. Create a new directory where your Jupyter Notebook and Data will live. Then, copy the heart.dat file to the folder.<\/p>\n
Next, you can create a new notebook and add the following lines of code:<\/p>\n
#prepare data downloaded from UCL\n\nimport csv\nimport pandas as pd\n\n# add header names\nheaders = ['age', 'sex','chest_pain','resting_blood_pressure',\n 'serum_cholestoral', 'fasting_blood_sugar', 'resting_ecg_results',\n 'max_heart_rate_achieved', 'exercise_induced_angina', 'oldpeak',\"slope of the peak\",\n 'num_of_major_vessels','thal', 'heart_disease']\n\nheart_df = pd.read_csv('heart.dat', sep=' ', names=headers)<\/pre>\nIn the code block above, you first set the header, which is the column names for the data set. You can get these names from the dataset description file also in the data page. Notice the sep<\/em><\/strong> parameters passed to Pandas read function? this tells pandas that the data is separated by spaces and not the default commas.<\/p>\n
In the next code block, you\u2019ll print out the head of the data:<\/p>\n
heart_df.head()<\/span><\/pre>\n<\/div>\n<\/div>\n\n![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:1441\/1*HzzcN5QvmgRE8uaKCCQvUg.png\")
<\/picture>\nFirst ten rows of the dataset<\/figcaption>\n<\/figure>\n<\/div>\n \n\nFrom the head of the data, you can see the features present, and you can begin to imagine the kind of analysis you\u2019ll need to perform on the dataset. Next, print out the shape of the data:<\/p>\n
heart_df.shape<\/span><\/pre>\n(270, 14)<\/p>\n
There are 270 observations. This means that your neural network will have an input data of shape 270 x 13, excluding the target variable (
heart_disease<\/code>). The features present in the dataset are:<\/p>\n- \n
- age<\/li>\n
- sex<\/li>\n
- chest pain type (4 values)<\/li>\n
- resting blood pressure<\/li>\n
- serum cholesterol in mg\/dl<\/li>\n
- fasting blood sugar > 120 mg\/dl<\/li>\n
- resting electrocardiographic results (values 0,1,2)<\/li>\n
- maximum heart rate achieved<\/li>\n
- exercise-induced angina<\/li>\n
- oldpeak (ST depression induced by exercise relative to rest)<\/li>\n
- the slope of the peak exercise ST segment<\/li>\n
- number of major vessels (0\u20133) colored by fluoroscopy<\/li>\n
- thal (3 = normal; 6 = fixed defect; 7 = reversible defect)<\/li>\n
- heart_disease: absence (1) or presence (2) of heart disease<\/li>\n<\/ul>\n
Next, you can check for missing values and also the data types. A Neural Network expects all features to be numeric and not contain missing values.<\/p>\n
heart_df.isna().sum()<\/span><\/pre>\n\n ![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:326\/1*f3wihhgnvMMeMsxA_apntQ.png\")
<\/figure><\/picture><\/div>\n<\/figure>\n heart_df.dtypes<\/span><\/pre>\n\n ![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:373\/1*9_5UaQWQTmnsEX2AFHgtHA.png\")
<\/figure><\/picture><\/div>\n<\/figure>\n There are no missing values in the dataset, and all features are numeric. Next, you\u2019ll separate the target from the data, split into train and test set, and then standardize the data.<\/p>\n
import numpy as np\nimport warnings\nwarnings.filterwarnings(\"ignore\") #suppress warnings\nimport matplotlib.pyplot as plt\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import StandardScaler\n\n#convert imput to numpy arrays\nX = heart_df.drop(columns=['heart_disease'])\n\n#replace target class with 0 and 1\n#1 means \"have heart disease\" and 0 means \"do not have heart disease\"\nheart_df['heart_disease'] = heart_df['heart_disease'].replace(1, 0)\nheart_df['heart_disease'] = heart_df['heart_disease'].replace(2, 1)\n\ny_label = heart_df['heart_disease'].values.reshape(X.shape[0], 1)\n\n#split data into train and test set\nXtrain, Xtest, ytrain, ytest = train_test_split(X, y_label, test_size=0.2, random_state=2)\n\n#standardize the dataset\nsc = StandardScaler()\nsc.fit(Xtrain)\nXtrain = sc.transform(Xtrain)\nXtest = sc.transform(Xtest)\n\nprint(f\"Shape of train set is {Xtrain.shape}\")\nprint(f\"Shape of test set is {Xtest.shape}\")\nprint(f\"Shape of train label is {ytrain.shape}\")\nprint(f\"Shape of test labels is {ytest.shape}\")<\/pre>\n\n ![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:346\/1*qoDtKQH-GuJ9Et5qWovCeg.png\")
<\/figure><\/picture><\/div>\n<\/figure>\n In the code block above, first, you dropped the target from the training dataset, and the replace the classes with 0 and 1. Notice you reshaped the
y_label<\/code> to a 1-D array. This is important when you start performing dot products. Next, you used the handytrain_test_split<\/code> function from sklearn to split the data into train and test set, with the test set taking 20 percent of the data. Finally, you standardized the dataset using the StandardScaler module of sklearn.<\/p>\nThe Layers of a Neural Network<\/h2>\n
Now that you have downloaded and prepared the dataset, let\u2019s start building the neural network to make predictions. To do that, you first, need to understand the concept of layers<\/strong>.<\/p>\n
Remember when I said a neural network stacks multiple neurons together to build really large and complex mathematical functions? Well, the official name for it is a layer. The layer is a collection of nodes at different stages of computation in a neural network. Each node acts as a neuron and performs calculations on the data passed to it. Look at the illustration of a 3-layer neural network below:<\/p>\n
\n \n![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*Z3zHoX1nhK6Rsmd4yNPdsg.jpeg\")
<\/figure><\/picture><\/div>\n<\/div>\n A 3 layer neural network<\/figcaption>\n<\/figure>\n Yeah I know, you see four layers\u2014but in deep learning, you don\u2019t count the first layer. The first layer is called the input layer, and the number of nodes will depend on the number of features present in your dataset. In our case, it will be 13 nodes because we have 13 features.<\/p>\n
The final layer of the neural network is called the output layer, and the number depends on what you\u2019re trying to predict. For regression and binary classification tasks, you can use a single node; while for multi-class problems, you\u2019ll use multiple nodes, depending on the number of classes.<\/p>\n
![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:586\/1*tDZCZM0g5oJxyZ0JfqRJDQ.png\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*ZafDv3VUm60Eh10OeJu1vw.png\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/1*OUOB_YF41M-O4GgZH_F2rw.png\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*e8qhWEz_8PZBLmF37JHfSA.png\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*TkSDhSmzdiVWzaCfCGYgrA.png\")
<\/figure>