![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*0AKj8jEQSgE3TGcGWLFdvw.jpeg\")
<\/figure>Building successful data science projects is not straightforward and sometimes it can turn into a nightmare. There are many challenges from data ingestion to production, including feature engineering, modeling, testing, deployment, and infrastructure management. Until a few years ago, data scientists were trying to deal with all these challenges on their own, but they were having a hard time overcoming them. To address these challenges, new fields such as data engineering, feature engineering, and machine learning (ML) engineering have emerged. In this blog post, I\u2019ll walk you through how to become an ML engineer.<\/p>\n
Here are the topics I\u2019ll cover in this post:<\/p>\n
- \n
- What is ML engineering?<\/li>\n
- Data scientist vs ML engineer vs data engineer<\/li>\n
- What does an ML engineer do?<\/li>\n
- The machine project lifecycle<\/li>\n
- 7 Steps to become an ML engineer with courses and books<\/li>\n<\/ul>\n
Let\u2019s dive in!<\/p>\n
What is Machine Learning Engineering?<\/h1>\n
Machine learning is a modern technique for problem-solving and task automation. Machine learning is a subfield of AI that allows a machine to learn automatically and improve from experience without explicit instruction. Building a machine learning project is a complex process that requires a range of skills, from modeling to deployment and infrastructure management. ML engineering emerged to bridge the gap between data science and software engineering. Fortunately, you can easily tackle ML engineering challenges with recently developed libraries and platforms such as Scikit-Learn<\/em>, TensorFlow<\/em>, HuggingFace<\/em>, and Comet<\/em><\/a>.<\/p>\n
Data Scientist vs ML Engineer vs Data Engineer<\/h1>\n
There are three key roles in data science projects: data engineer, data scientist, and ML engineer. Data engineers<\/em><\/mark> create systems and pipelines that collect raw data, manage it, and turn it into information. The <\/mark>data scientist<\/em><\/mark>theoretically creates the model prototype. The <\/mark>ML engineer<\/em><\/mark> uses various tools to create the model and deploy them to production.<\/mark><\/p>\n
\n \n![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*VgXHfZhvmesRztUeEA_ISA.png\")
<\/figure><\/picture><\/div>\n<\/div>\n Data Roles (Image by Author)<\/figcaption>\n<\/figure>\n Let me explain these roles with an example. Let\u2019s say a company wants to perform a sentiment analysis project. Data engineers are responsible for properly exporting-loading-transforming (ETL) the data needed to build the model. If data is continuously generated by different sources, they\u2019ll build data pipelines that can transmit all this information to the right parts of the system at the right time without any delays or bottlenecks.<\/p>\n
Using this data, data scientists try to find the best model that predicts whether the data is positive, negative, or neutral. ML engineers will be responsible for building the model that fits the data and deploying that model in real life, as well as making sure it can perform.<\/p>\n
The Machine Learning Project Lifecycle<\/h1>\n
\n \n![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*ssqb0IYqhzsimKyuvGxg9w.png\")
<\/figure><\/picture><\/div>\n<\/div>\n Machine Learning Lifecycle (Image by Author)<\/figcaption>\n<\/figure>\n The ML lifecycle is an iterative and never-ending cycle between improving data, modeling, and deployment. This lifecycle consists of three main stages: data preparation, model building, and model deployment. Let\u2019s take a look at these stages.<\/p>\n
Data Preparation<\/h2>\n
Real-world datasets are usually not clean. These datasets are cleaned by data preprocessing. Garbage in, garbage out<\/em> is a common concept in computer science, but this concept can also be used for ML engineering; if you use a clean dataset to build the model, you can obtain a good model.<\/p>\n
\n \n![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*ZBenO51dDGl2A8ZP4Be8-g.png\")
<\/figure><\/picture><\/div>\n<\/div>\n Data Cleaning Processing (Image by Author)<\/figcaption>\n<\/figure>\n Model Building<\/h2>\n
ML engineers try to build the best model using clean data. When building a model, it is recommended to start with a simple model such as regression, and then try complex models such as neural networks. After you create the model, you need to evaluate the performance of the model with various statistical metrics such as accuracy, precision, recall, or F1.<\/p>\n
Model Deployment<\/h2>\n
After obtaining the best model, it\u2019s time to deploy, monitor, and maintain it. The purpose of the model deployment is to put the model into production. So the model in production can retrieve the data and return their predicts. ML engineers also are responsible for monitoring the model\u2019s performance and ensuring the model makes accurate predictions.<\/p>\n
7 Steps to become an ML Engineer<\/h1>\n
It is a challenge to become an ML engineer. After reviewing more than 500 machine learning engineer job postings, the 365 team discovered<\/a> the following skills for an ML engineer position:<\/p>\n
\n \n![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*CMAFCBPrnVKziH7xYyI41A.jpeg\")
<\/figure><\/picture><\/div>\n<\/div>\n General skills for ML engineers according to research by the 365 team<\/a><\/figcaption>\n<\/figure>\n As you can see, there are many skills to become an ML engineer. Let\u2019s take a closer look at the most important skills.<\/p>\n
1. Programming<\/h2>\n
To implement machine learning projects, it is necessary to know a programming language. The most used languages in the world of machine learning are Python<\/em> and R<\/em>. Python is used more in data science as it is a general-purpose and easy-to-learn language. With Python, you can do end-to-end machine projects from data cleaning to model deployment. In addition, many important machine learning frameworks such as Pytorch, Scikit-Learn, and PySpark are written in Python.<\/p>\n
Python Free Courses:<\/strong><\/p>\n
- \n
- Learn Python \u2014 Full Course for Beginners [Tutorial]<\/a> (YouTube)<\/li>\n
- Python Tutorial \u2014 Python Full Course for Beginners<\/a> (YouTube)<\/li>\n<\/ul>\n
Python Books:<\/strong><\/p>\n
- \n
- Python Crash Course<\/a><\/li>\n
- Learning Python<\/a><\/li>\n<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n\n<\/div>\n<\/div>\n<\/div>\n\n\n\n\n\n\n2. Machine Learning Algorithms<\/h2>\n
There is no magic algorithm that will solve all types of machine learning problems. You can try all the algorithms to build a good model, but it takes a lot of time. It\u2019s very important to be familiar with all the common machine learning algorithms so that you know where to use what algorithms. Here are some crucial algorithms that are often used by machine learning engineers:linear regression<\/a>, Naive Bayes<\/a>, KNN<\/a>, decision tree<\/a>, support vector machines<\/a>, random forest<\/a>, XGBoost<\/a>, K-means<\/a>, and PCA<\/a>.<\/p>\n
\n \n![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*Ev_oUHbmhKARMJbs7lhBuw.png\")
<\/figure><\/picture><\/div>\n<\/div>\n Machine Learning Algorithms (Image by Author)<\/figcaption>\n<\/figure>\n Machine Learning Courses:<\/strong><\/p>\n
- \n
- Machine Learning Specialization <\/a>(Coursera)<\/li>\n
- Supervised machine learning: regression and classification<\/a>(Coursera)<\/li>\n<\/ul>\n
Machine Learning Books:<\/strong><\/p>\n
- \n
- Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow by Geron Aurelien<\/a><\/li>\n
- Machine Learning with PyTorch and Scikit-Learn By Sebastian Raschka, Yuxi (Hayden) Liu, Vahid Mirjalili<\/a><\/li>\n
- Machine Learning Bookcamp by Alexey Grigorev<\/a><\/li>\n<\/ul>\n
3. Applied Mathematics<\/strong><\/h2>\n
Mathematics is a crucial skill in the arsenal of an ML engineer. Machine Learning involves a lot of applied mathematics concepts such as statistics, linear algebra, calculus, probability theory, and discrete maths. Mathematical formulas are applied while training the model coefficients. If you are familiar with these formulas, you can select the correct algorithm. Most machine learning algorithms are based on statistics, so they are very easy to understand if you have a strong foundation in mathematics and statistics.<\/p>\n
\n \n![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*_1DoSvNvrQYcl2682tFt3w.png\")
<\/figure><\/picture><\/div>\n<\/div>\n Math Topics for ML Engineering (Image by Author)<\/figcaption>\n<\/figure>\n Applied Mathematics Courses:<\/strong><\/p>\n
- \n
- Mathematics For Machine Learning: Linear Algebra<\/a> (Coursera)<\/li>\n
- Mathematics For Machine Learning: Multivariate Calculus<\/a> (Coursera)<\/li>\n
- Khan Academy: Statistics and Probability<\/a><\/li>\n<\/ul>\n
Applied Mathematics Books:<\/strong><\/p>\n
- \n
- Practical Statistics for Data Scientists by Peter Bruce, Andrew Bruce, Peter Gedeck<\/a><\/li>\n
- Essential Math for Data Science by Thomas Nield<\/a><\/li>\n
- Practical Linear Algebra for Data Science by Mike Cohen<\/a><\/li>\n<\/ul>\n
4. Deep Learning<\/h2>\n
Machine learning algorithms work well with medium and small datasets. However, when it comes to big data, these algorithms do not perform well. Deep learning techniques are used to analyze big data. Deep learning<\/a> is a subfield of machine learning and is an extension of artificial neural networks. Problems such as image classification, language-to-language translation, and driverless cars could be solved by deep learning techniques such as GPT-3 and BERT based on transformers.<\/p>\n
Deep learning works well with unstructured data and does not require feature engineering. On the other hand, deep learning models are a black box as it is not known how they work. Also, they require large amounts of data. Here are the deep learning algorithms that ML engineers should know: multilayer perceptron, convolutional neural networks, recurrent neural networks, long short-term memory networks, generative adversarial networks, and transformers.<\/p>\n
Deep Learning Courses:<\/strong><\/p>\n
- \n
- Deep Learning With Tensorflow 2.0, Keras and Python<\/a> (YouTube)<\/li>\n
- MIT 6.S191: Introduction to Deep Learning<\/a> (YouTube)<\/li>\n<\/ul>\n
Deep Learning Books:<\/strong><\/p>\n
- \n
- Deep Learning with Python by Fran\u00e7ois Chollet<\/a><\/li>\n
- AI and Machine Learning for Coders by Laurence Moroney<\/a><\/li>\n
- Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville<\/a><\/li>\n<\/ul>\n
5. Machine Learning Frameworks<\/h2>\n
You can build machine learning models from scratch, but there is no need to reinvent the wheel. Fortunately, great frameworks have been developed recently. These frameworks help you carry out machine learning projects more easily. For example, you can use Pandas for data preprocessing, Matplotlib and Seaborn for data visualization, Scikit-Learn to implement machine learning algorithms, Tensorflow and Pytorch for deep learning analysis, and Comet for model optimization.<\/p>\n
Machine Learning Framework Blog Posts:<\/strong><\/p>\n
- \n
- A Guide to the Best Machine Learning Frameworks of 2022<\/a><\/li>\n
- 10 Best Python Libraries for Data Science<\/a><\/li>\n<\/ul>\n
6. MLOps<\/h2>\n
\n \n![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*Ac1FFOFOdGeTxJoXCl7Mrw.png\")
<\/figure><\/picture><\/div>\n<\/div>\n MLOps (Image by Author)<\/figcaption>\n<\/figure>\n A machine learning project that is not deployed to a production environment is a dead project. Machine Learning Operations (MLOps) is a core function of ML engineering that aims to put machine learning models into production and then maintain and monitor them. In other words, MLOps is a bridge between model building and exporting the model to production. MLOps is a relatively new but rapidly growing field. It is the DevOps equivalent for machine learning. To perform MLOps steps, you can use various tools like MLflow, Kubeflow, MetaFlow, and DataRobot.<\/p>\n
MLOps Courses:<\/strong><\/p>\n
- \n
- Machine Learning Engineering for Production (MLOps)<\/a> (YouTube)<\/li>\n
- Introduction to Machine Learning in Production<\/a> (Coursera)<\/li>\n<\/ul>\n
MLOps Books:<\/strong><\/p>\n
- \n
- Machine Learning Engineering by Andriy Burkov<\/a><\/li>\n
- MLOps Engineering at Scale by Carl Osipov<\/a><\/li>\n<\/ul>\n
7. Cloud Computing<\/h2>\n
Machine learning projects require a lot of processing power, data storage, and many servers. Cloud computing helps you to train models on powerful machines with multiple GPUs, deploy those models, and run as many servers as you want. Cloud computing is currently a rising trend in data science. The most used cloud computing services for machine learning are Amazon SageMaker, Microsoft Azure Machine Learning, and GCP Vertex AI for ML engineering.<\/p>\n
Cloud Computing Courses:<\/strong><\/p>\n
- \n
- Introduction to Cloud Computing<\/a> (Coursera)<\/li>\n
- Cloud Computing Full Course In 11 Hours<\/a> (YouTube)<\/li>\n<\/ul>\n
Cloud Computing Books:<\/strong><\/p>\n
- \n
- Data Science on AWS by Chris Fregly, Antje Barth<\/a><\/li>\n
- Data Science on the Google Cloud Platform: Implementing by Valliappa Lakshmanan<\/a><\/li>\n<\/ul>\n
Additional Skills<\/h2>\n
There are many skills required to become an ML engineer. I mentioned the most important of them. After mastering these skills, you will be ready to work as an ML engineer. But if you learn the following skills, you\u2019ll stand out from the competition.<\/p>\n
- \n
- Data Visualization<\/a><\/li>\n
- SQL<\/a><\/li>\n
- NoSQL<\/a><\/li>\n
- PySpark<\/a><\/li>\n
- Hadoop<\/a><\/li>\n
- Docker<\/a><\/li>\n
- Kubernetes<\/a><\/li>\n
- CI-CD for Machine Learning<\/a><\/li>\n
- Git and GitHub<\/a><\/li>\n
- FastAPI<\/a><\/li>\n<\/ul>\n
Final Thoughts<\/h1>\n
Building a successful end-to-end machine learning project has many challenges. To deal with these challenges, an ML engineer needs to learn some skills and tools. In this blog post, I talked about a roadmap to become an ML engineer. ML engineering is a fast-growing, high-paying, and in-demand field that has emerged recently. If you are interested in both data science and software, ML engineering is for you.<\/p>\n
\n \n - SQL<\/a><\/li>\n
- Data Science on the Google Cloud Platform: Implementing by Valliappa Lakshmanan<\/a><\/li>\n<\/ul>\n
- Cloud Computing Full Course In 11 Hours<\/a> (YouTube)<\/li>\n<\/ul>\n
- MLOps Engineering at Scale by Carl Osipov<\/a><\/li>\n<\/ul>\n
- Introduction to Machine Learning in Production<\/a> (Coursera)<\/li>\n<\/ul>\n
- 10 Best Python Libraries for Data Science<\/a><\/li>\n<\/ul>\n
- AI and Machine Learning for Coders by Laurence Moroney<\/a><\/li>\n
- MIT 6.S191: Introduction to Deep Learning<\/a> (YouTube)<\/li>\n<\/ul>\n
- Essential Math for Data Science by Thomas Nield<\/a><\/li>\n
- Mathematics For Machine Learning: Multivariate Calculus<\/a> (Coursera)<\/li>\n
- Machine Learning with PyTorch and Scikit-Learn By Sebastian Raschka, Yuxi (Hayden) Liu, Vahid Mirjalili<\/a><\/li>\n
- Supervised machine learning: regression and classification<\/a>(Coursera)<\/li>\n<\/ul>\n
- Learning Python<\/a><\/li>\n<\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
- Python Tutorial \u2014 Python Full Course for Beginners<\/a> (YouTube)<\/li>\n<\/ul>\n
- Learn Python \u2014 Full Course for Beginners [Tutorial]<\/a> (YouTube)<\/li>\n