MSC Sound Design 2023-2024 - Introduction to audio synthesis, Web Audio, generative AI for music

Introduction

In this course, we'll cover the basics of audio synthesis for music, in particular by studying how synthesizers work. We'll also look at Web Audio and Web Midi, two W3C standards that enable computer music applications to be designed and used in Web browsers. Finally, we'll look at recent advances in AIs for the automatic generation of music from text.

First Day: Introduction to music synthesis (using Web Audio Applications), Introduction to Web Audio

Second Day: Introduction to Machine Learning / Deep Learning

Course documents: see the powerpoint files on this mega link (2 gigabytes).

Videos:

• How does chatGTP works? (french with english subtitles)
  • INTERACTIVE DEMONSTRATION Word2Vec (words as vectors)
• How do AIs invent images (we'll address music in the next course) ? (french with english subtitles)

Also worth reading: "La revanche des neurones", an article by Dominique Cardon (in french), an excellent introduction to the 2012 AI learning revolution. The first four pages tell the story of the 18-year-old who programmed classic deep learning algorithms into a GPU and improved the object recognition record by an incredible amount, sparking a new craze for Machine Learning.

Third Day: Using generative AIs for music

• IA powered DAWs: amped studio, sounds.studio 
• https://www.suno.ai/ : generate songs through discord or copilot.
• IA music generation tool list (if there is a link in 3rd column, there is an online demo)
  • Musicgen better link.
  • Audiocraft plus 
• https://app.audiogen.co/ (music generation and sound design)
• List maintained automatically by chatGPT 
• Google piano transformer (generate piano tracks using transformers)
• https://soundraw.io/ (free trial available)
• VOICE CHANGER FOR COVERS : Back In Black (AC/DC) with Bon Scott IA generated voice swap and here aussi.
  • List of tools used (open source) : 
    https://github.com/SociallyIneptWeeb/AICoverGen with RCV for the IA models https://github.com/ardha27/AI-Song-Cover-RVC
      
  • DEMO BY THE TEACHER!
• Uberduck another text to music generator.
• A YouTube video channel with a complete course about IA generative music (beware, the course is for people with scientific/computer science background).
• Google MusicFX (and ImageFX): music and image generators (US only - you need a gmail account with US localization, see demo video)

To come (2024): multimodal text to XXX generators (video with voice, sound and music):

• See https://lumiere-video.github.io/#section_text_to_video 

For installing AI apps locally (Mac M1-M2-M3 or gamer PC recommended) : https://pinokio.computer/

Fourth Day: project! Compose music with the tools we've seen so far!