MSC Sound Design 2024-2025 - Introduction to audio synthesis, Web Audio, generative AI for music

In this course, we'll cover the basics of audio synthesis for music, in particular by studying how synthesizers work. We'll also look at Web Audio and Web Midi, two W3C standards that enable computer music applications to be designed and used in Web browsers. Finally, we'll look at recent advances in AIs for the automatic generation of music from text.

First Day: Introduction to music synthesis (using Web Audio Applications), Introduction to Web Audio

Learn synth basics

Learn synth basics with Ableton's Learning Synth Web tutorial. They also have an incredible Learning Music web site for playing with scales, modes, composition etc.
- For the curious ones, the author of previous tutorial also wrote an interactive course about signal processing.
Play with an online tutorial for the Oberheim OB-Xd synthesizer .
See what you can do with some online recreations of well-known synths
- The online Mini Moog
- The Pro54 (native instrument, ported to WebAudio by CMajor), use it with wam-studio, a web-based DAW Michel Buffa and his team are developing.
- Viktor-N1, an original synth recreated using same modules you can find on classic analog synths
- Or try simple ones like this synth101 one
With modern approaches (using digital/software) it is possible to create more complex oscillators for new waveforms in addition to traditional ones, try this demo.

Learn about Web Audio and Web Audio Modules (aka VSTs for the Web)

We'll use this summer School course I gave at the Sound and Music Computing Conference in 2022.
We'll finish by discovering how to make collaborative music on the Web using Sequencer Party.

Try WAXML for declarative music using an HTML like language

Check WAXML by Hans Lindrop

Try the audio orchestrator by BBC Research (for distributing music on smartphones)

BBC AUdio Orchestrator

Second Day: Introduction to Machine Learning / Deep Learning

Videos:

How does chatGTP works? (french with english subtitles)

INTERACTIVE DEMONSTRATION Word2Vec (words as vectors)

How do AIs invent images (we'll address music in the next course) ? (french with english subtitles)

Also worth reading: "La revanche des neurones", an article by Dominique Cardon (in french), an excellent introduction to the 2012 AI learning revolution. The first four pages tell the story of the 18-year-old who programmed classic deep learning algorithms into a GPU and improved the object recognition record by an incredible amount, sparking a new craze for Machine Learning.

Fourth Day: project! Compose music with the tools we've seen so far!

Assignment 1: use Web Audio tools (at least 5 different ones) for composing the Music for the Games on Web 2024 challenge (main anthem)

Assignments 2: use IAs for composing music, sound effects, visuals

Music for a video game developed for this challenge, inspired by the old "Track & Field" video game from the 80's, by Konami
- Main menu theme
- Level 1 theme (100m), level 2 theme (110m hurdle), level 3 theme etc.
- Game over music
- Sound effects (coin inserted, jump, steps, etc.)
- Cover of the virtual record (CD cover) created by an AI

Constraints:

Use at least 4 different NON COMMERCIAL IA applications presented in this course (audiocraft plus, musicgen, audiogen, mustango, etc. others from this list)
Provide a detailed README explaining your approach (ideas, goal, choosen tool, prompts, how you adapted the prompts etc.)
Provide all prompts + non definitive attemps in addition to the final results
Explain how you assembled / fine tuned the results using other classic tools (Logic, etc. -videos expected here)

Due date: 25 Mars 2024, Jordan Sintes will send me the final content (URL)

Online Exam 25 Mars 2024 à 11h: a quizz about IA (second day course), to check what you understood / know from the course. Date to come, I'll send the URL and you will have to pass it all a the same time, with a limited time (40mns). You will have to read the whole course I gave this day.