{"id":6615,"date":"2023-07-10T09:36:43","date_gmt":"2023-07-10T17:36:43","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=6615"},"modified":"2025-04-24T17:15:13","modified_gmt":"2025-04-24T17:15:13","slug":"working-with-audio-data-for-machine-learning-in-python","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/","title":{"rendered":"Working with Audio Data for Machine Learning in Python"},"content":{"rendered":"\n\n\n\n\n
\n
\n
\"\"<\/picture>
Photo by Thomas Le<\/a> on Unsplash<\/a><\/figcaption><\/figure>\n<\/div>\n
\n
\n

Most of the attention, when it comes to machine learning or deep learning models, is given to computer vision or natural language sub-domain problems.<\/p>\n

However, there\u2019s an ever-increasing need to process audio data, with emerging advancements in technologies like Google Home and Alexa that extract information from voice signals. As such, working with audio data has become a new trend and area of study.<\/p>\n

The possible applications extend to voice recognition, music classification, tagging, and generation, and are paving the way for audio use cases to become the new era of deep learning.<\/p>\n

Audio File Overview<\/h2>\n

Sound are pressure waves, and these waves can be represented by numbers over a time period. These air pressure differences communicates with the brain. Audio files are generally stored in .wav format and need to be digitized, using the concept of sampling.<\/p>\n

\n
\"\"<\/figure>
<\/picture><\/div>\n<\/figure>\n

The sampling frequency<\/strong> (or sample rate) is the number of samples (data points) per second in a ound. For example: if the sampling frequency is 44 khz, a recording with a duration of 60 seconds will contain 2,646,000 samples. In practice, sampling even higher than 10x helps measure the amplitude correctly in the time domain.<\/p><\/blockquote>\n

\n
\"\"<\/figure>
<\/picture><\/div>\n<\/figure>\n

Loading and Visualizing an audio file in Python<\/h2>\n

Librosa<\/mark> is a Python library that helps us work with audio data. For complete documentation, you can also refer to this link<\/a>.<\/p>\n

    \n
  1. Install the library : pip install librosa<\/code><\/li>\n
  2. Loading the file: The audio file is loaded into a NumPy array after being sampled at a particular sample rate (sr).<\/li>\n<\/ol>\n
    <\/figure>\n
    import librosa\n#path of the audio file\naudio_data = 'sampleaudio.wav'\n#This returns an audio time series as a numpy array with a default sampling rate(sr) of 22KHZ\nx = librosa.load(audio_data, sr=None)\n\n#We can change this behavior by resampling at sr=44.1KHz.\nx = librosa.load(audio_data, sr=44000)\n\n<\/pre>\n
    3. Playing Audio : Using,IPython.display.Audio<\/strong><\/code>, we can play the audio file in a Jupyter Notebook, using the command IPython.display.Audio(audio_data)<\/code><\/p>\n
    4. Waveform visualization : To visualize the sampled signal and plot it, we need two Python libraries\u2014Matplotlib and Librosa. The following code depicts the waveform visualization of the amplitude vs the time representation of the signal.<\/p>\n
    %matplotlib inline\nimport matplotlib.pyplot as plt\nimport librosa.display\n\nplt.figure(figsize=(14, 5))\n#plotting the sampled signal\nlibrosa.display.waveplot(x, sr=sr)<\/pre>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    5. Spectrogram : A spectrogram is a visual representation of the spectrum of frequencies of a signal as it varies with time. They are time-frequency portraits of signals. Using a spectrogram, we can see how energy levels (dB) vary over time.<\/p>\n
    #x: numpy array\nX = librosa.stft(x)\n#converting into energy levels(dB)\nXdb = librosa.amplitude_to_db(abs(X))\n\nplt.figure(figsize=(20, 5))\nlibrosa.display.specshow(Xdb, sr=sr, x_axis='time', y_axis='hz')\nplt.colorbar()<\/pre>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    6. Log-frequency axis: Features can be obtained from a spectrogram by converting the linear frequency axis, as shown above, into a logarithmic axis. The resulting representation is also called a log-frequency spectrogram. The code we need to write here is:<\/p>\n
    librosa.display.specshow(Xdb, sr=sr, x_axis=\u2019time\u2019, y_axis=\u2019log\u2019)<\/code><\/p>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    Creating an audio signal and saving it<\/h2>\n
    A digitized audio signal is a NumPy array with a specified frequency and sample rate. The analog wave format of the audio signal represents a function (i.e. sine, cosine etc). We need to save the composed audio signal generated from the NumPy array. This kind of audio creation could be used in applications that require voice-to-text translation in audio-enabled bots or search engines.<\/p>\n
    sr = 22050 # sample rate\nT = 5.0    # seconds\nt = np.linspace(0, T, int(T*sr), endpoint=False) # time variable\nx = 0.5*np.sin(2*np.pi*220*t)# pure sine wave at 220 Hz\n\n#playing generated audio\nipd.Audio(x, rate=sr) # load a NumPy array\n\nlibrosa.output.write_wav('generated.wav', x, sr) # writing wave file in .wav format<\/pre>\n
    So far, so good. Easy and fun to learn. But data pre-processing steps can be difficult and memory-consuming, as we\u2019ll often have to deal with audio signals that are longer than 1 second. Compared to the images or number of pixels in each training item in popular datasets such as MNIST or CIFAR, the number of data points in digital audio is much higher. This may lead to memory issues.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
    \n
    \n
    \n
    Pre-processing of audio signals<\/h1>\n
    Normalization<\/strong><\/h2>\n
    A technique used to adjust the volume of audio files to a standard set level; if this isn\u2019t done, the volume can differ greatly from word to word, and the file can end up unable to be processed clearly.<\/p>\n
    #min = minimum value for each row of the vector signal\n#max = maximum value for each row of the vector signal\ndef normalize(x, axis=0):\n    return sklearn.preprocessing.minmax_scale(x, axis=axis)\n\n#Plotting the Spectral Centroid along the waveform\nlibrosa.display.waveplot(x, sr=sr, alpha=0.4)\nplt.plot(t, normalize(spectral_centroids), color='r')<\/pre>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    Pre-emphasis<\/strong><\/h2>\n
    Pre-emphasis is done before starting with feature extraction. We do this by boosting only the signal\u2019s high-frequency components, while leaving the low-frequency components in their original states. This is done in order to compensate the high-frequency section, which is suppressed naturally when humans make sounds.<\/p>\n
    import matplotlib.pyplot as plt\ny, sr = librosa.load(audio_file.wav, offset=30, duration=10)\ny_filt = librosa.effects.preemphasis(y)\n# and plot the results for comparison\nS_orig = librosa.amplitude_to_db(np.abs(librosa.stft(y)), ref=np.max)\nS_preemph = librosa.amplitude_to_db(np.abs(librosa.stft(y_filt)), ref=np.max)\n\nlibrosa.display.specshow(S_orig, y_axis='log', x_axis='time')\nplt.title('Original signal')\nlibrosa.display.specshow(S_preemph, y_axis='log', x_axis='time')\nplt.title('Pre-emphasized signal')<\/pre>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    Feature extraction from audio signals<\/h1>\n
    Up until now, we\u2019ve gone through the basic overview of audio signals and how they can be visualized in Python. To take us one step closer to model building, let\u2019s look at the various ways to extract feature from this data.<\/p>\n
    Zero Crossing Rate<\/strong><\/h2>\n
    The number times over a given interval that the signal\u2019s amplitude crosses a value of zero. Essentially, it denotes the number of times the signal changes sign from positive to negative in the given time period. If the count of zero crossings is higher for a given signal, the signal is said to change rapidly, which implies that the signal contains the high-frequency information, and vice-versa.<\/p>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    #zero crossings to be found between a given time\nn0 = 9000\nn1 = 9100\nplt.figure(figsize=(20, 5))\nplt.plot(x[n0:n1])\nplt.grid()\n\nzero_crossings = librosa.zero_crossings(x[n0:n1], pad=False)\nzero_crossings.shape<\/pre>\n
    Spectral Rolloff<\/strong><\/h2>\n
    The rolloff frequency is defined as the frequency under which the cutoff of the total energy of the spectrum is contained, eg. 85%. It can be used to distinguish between harmonic and noisy sounds.<\/p>\n
    y, sr = librosa.load(librosa.util.example_audio_file())\n# Approximate maximum frequencies with roll_percent=0.85 (default)\nrolloff = librosa.feature.spectral_rolloff(y=y, sr=sr)\n\n# Approximate minimum frequencies with roll_percent=0.1\nrolloff = librosa.feature.spectral_rolloff(y=y, sr=sr, roll_percent=0.1)<\/pre>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    MFCC<\/strong><\/h2>\n
    One popular audio feature extraction method is the Mel-frequency cepstral coefficients (MFCC), which has 39 features. The feature count is small enough to force the model to learn the information of the audio. 12 parameters are related to the amplitude of frequencies. The extraction flow of MFCC features is depicted below:<\/p>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
      \n
    1. Framing and Windowing: The continuous speech signal is blocked into frames of N samples, with adjacent frames being separated by M. The result after this step is called spectrum.<\/li>\n
    2. Mel Frequency Wrapping: For each tone with a frequency f, a pitch is measured on the Mel scale. This scale uses a linear spacing for frequencies below 1000Hz and transforms frequencies above 1000Hz by using a logarithmic function.<\/li>\n
    3. Cepstrum: Converting of log-mel scale back to time. This provides a good representation of a signal\u2019s local spectral properties, with the result as MFCC features.<\/li>\n<\/ol>\n
      The MFCC features can be extracted using the Librosa Python library we installed earlier:<\/p>\n
      librosa.feature.mfcc(x, sr=sr)<\/code><\/p>\n
      Where x = time domain NumPy series and sr = sampling rate<\/p>\n
      \n
      \"\"<\/figure>
      <\/picture><\/div>\n<\/figure>\n
      Chroma Frequencies<\/strong><\/h2>\n
      The entire spectrum is projected onto 12 bins representing the 12 distinct semitones (or chroma) of the musical octave. The human perception of pitch is periodic in the sense that two pitches are perceived as similar if they differ by one or several octaves (where 1 octave=12 pitches).<\/p>\n
      x, sr = librosa.load('audio.wav')\nipd.Audio(x, rate=sr)\n\nhop_length = 512\n# returns normalized energy for each chroma bin at each frame.\nchromagram = librosa.feature.chroma_stft(x, sr=sr, hop_length=hop_length)\nplt.figure(figsize=(15, 5))\nlibrosa.display.specshow(chromagram, x_axis='time', y_axis='chroma', hop_length=hop_length, cmap='coolwarm')<\/pre>\n
      \n
      \"\"<\/figure>
      <\/picture><\/div>\n<\/figure>\n
      Conclusion<\/h1>\n
      In this article on how to work with audio signals in Python, we covered the following sub-topics:<\/p>\n
        \n
      • Loading and visualizing audio signals<\/li>\n
      • Techniques of pre-processing of audio data by pre-emphasis, normalization<\/li>\n
      • Feature extraction from audio files by Zero Crossing Rate, MFCC, and Chroma frequencies<\/li>\n<\/ul>\n
        Thanks for sticking till the end!<\/p>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
        Photo by Thomas Le on Unsplash Most of the attention, when it comes to machine learning or deep learning models, is given to computer vision or natural language sub-domain problems. However, there\u2019s an ever-increasing need to process audio data, with emerging advancements in technologies like Google Home and Alexa that extract information from voice signals. […]<\/p>\n","protected":false},"author":53,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[6],"tags":[],"coauthors":[155],"class_list":["post-6615","post","type-post","status-publish","format-standard","hentry","category-machine-learning"],"yoast_head":"\nWorking with Audio Data for Machine Learning in Python - Comet<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Working with Audio Data for Machine Learning in Python\" \/>\n<meta property=\"og:description\" content=\"Photo by Thomas Le on Unsplash Most of the attention, when it comes to machine learning or deep learning models, is given to computer vision or natural language sub-domain problems. However, there\u2019s an ever-increasing need to process audio data, with emerging advancements in technologies like Google Home and Alexa that extract information from voice signals. […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2023-07-10T17:36:43+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:15:13+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/miro.medium.com\/v2\/resize:fit:2500\/1*vB3QdvugPeA_4mw0PTWQeA.jpeg\" \/>\n<meta name=\"author\" content=\"Pragati Baheti\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Pragati Baheti\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"9 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Working with Audio Data for Machine Learning in Python - Comet","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/","og_locale":"en_US","og_type":"article","og_title":"Working with Audio Data for Machine Learning in Python","og_description":"Photo by Thomas Le on Unsplash Most of the attention, when it comes to machine learning or deep learning models, is given to computer vision or natural language sub-domain problems. However, there\u2019s an ever-increasing need to process audio data, with emerging advancements in technologies like Google Home and Alexa that extract information from voice signals. […]","og_url":"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2023-07-10T17:36:43+00:00","article_modified_time":"2025-04-24T17:15:13+00:00","og_image":[{"url":"https:\/\/miro.medium.com\/v2\/resize:fit:2500\/1*vB3QdvugPeA_4mw0PTWQeA.jpeg","type":"","width":"","height":""}],"author":"Pragati Baheti","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Pragati Baheti","Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/"},"author":{"name":"Pragati Baheti","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/54958874fd9a373469e70e19b6597439"},"headline":"Working with Audio Data for Machine Learning in Python","datePublished":"2023-07-10T17:36:43+00:00","dateModified":"2025-04-24T17:15:13+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/"},"wordCount":1040,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:2500\/1*vB3QdvugPeA_4mw0PTWQeA.jpeg","articleSection":["Machine Learning"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/","url":"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/","name":"Working with Audio Data for Machine Learning in Python - Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:2500\/1*vB3QdvugPeA_4mw0PTWQeA.jpeg","datePublished":"2023-07-10T17:36:43+00:00","dateModified":"2025-04-24T17:15:13+00:00","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/#primaryimage","url":"https:\/\/miro.medium.com\/v2\/resize:fit:2500\/1*vB3QdvugPeA_4mw0PTWQeA.jpeg","contentUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:2500\/1*vB3QdvugPeA_4mw0PTWQeA.jpeg"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/working-with-audio-data-for-machine-learning-in-python\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Working with Audio Data for Machine Learning in Python"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/54958874fd9a373469e70e19b6597439","name":"Pragati Baheti","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/851362323c20d10f17041155fc07cae2","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/08\/1535716570395-96x96.jpg","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/08\/1535716570395-96x96.jpg","caption":"Pragati Baheti"},"url":"https:\/\/www.comet.com\/site\/blog\/author\/pragatibaheti001gmail-com\/"}]}},"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/6615","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/53"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=6615"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/6615\/revisions"}],"predecessor-version":[{"id":15604,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/6615\/revisions\/15604"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=6615"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=6615"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=6615"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=6615"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}