How our (first*) Shelter-in-Place Video Was Made
Written by George Yefchak, Music Director and producer of this video.
* Since this was written, SBP has produced several more shelter-in-place videos. The process is always similar, but we're learning all the time.
Brief Description of the Process
- First, I found a MIDI recording of the piece online from this website.
- I imported this file into the music notation program Sibelius. This program is mostly intended for writing music, but it also has some MIDI editing features. I adjusted the tempos to match what we had been rehearsing, turned on the click track (miraculously this can be generated automatically by Silbeius), and raised its volume to be heard easily over the music. I also extended the click track two beats before the piece started, so the players would have a prepartory "count-in."
- I exported the resulting audio from Sibelius and added it as a new playlist named "ClickTrack" to my iTunes account.
- I set up my video camera, said "Hey Siri, play my playlist ClickTrack," and recorded myself "conducting" the audio. (This is not a particularly interesting piece from a conducting point of view, so there wasn't much to look at here.)
- I imported this video into Adobe Premiere Pro, removed the camera-generated audio, and synced in my original audio file. I also added visual cues for "One Two Ready Go" in sync with the count-in beats.
- I posted this video on YouTube, and I sent an unlisted link out to the players along with instructions on how to record their part and transfer the file to me. I set up a shared Google Drive folder that players could use if needed.
- Each player recorded their own part, typically using their cell phone, while listening to the audio over headphones/earbuds from a second device, then sent their video to me.
- As each video arrived, I imported it into Adobe Premier Pro and attempted to synchronize it with the conducting video. I quickly discovered that this was going to be difficult. While people did a good job playing with the click track overall, it was still common for the music to get a little ahead or behind. In particular, timing of the eighth note pickups that begin the piece, and reoccur frequently, were remarkably different from person to person. After adjusting things here and there, I eventually decided that I should wait until all the videos were assembled before deciding how many notes needed to be shifted forwards or backwards in time.
- Once I got everything, I spent many, many hours adjusting the timing of countless notes. I was also able to replace a small number of wrong notes / glitches with other notes from the same player recorded elsewhere in the piece. And in three particularly noticeable spots, I was able to adjust the intonation. Note that this process could have gone on indefinitely. I decided my goal wasn't to make everything perfect, but rather to approximate how we had been sounding at our recent rehearsals. I think I got it about right for that, at least.
- The final step for the audio was adjusting the left-right mix of each channel in order to simulate a reasonable stereo sound.
- With the audio now finalized, I moved on to the video. I had twenty players, plus me as conductor, but one player had provided only audio and no video. So that meant I had 21 videos streams that needed to be shown (if I was going to put them all in screen at once). With a few extra weeks to learn Adobe After Effects, I could have done cool things like moving people around, zooming in and panning around, etc., but I wanted to finish quickly.
- So I set up a grid with three rows of five and one row of six. I drew this in Adobe Illustrator, using lines five pixels wide, and imported it into the project. This grid allowed me to position each video with a few pixels tolerance. I rescaled and cropped each video (they tended to come in different sizes and shapes), and aligned them all to my grid.
- My first attempt to output a movie file failed, because even my brand new Intel i9 PC, which I had configured specifically for video editing, was unable to simultaneously rescale twenty-one video streams. So...
- I rescaled and exported each video separately, creating an extra 8 GB of video files, then I merged those all into one.
- I recorded my little intro (this took several more tries than I'd like to admit), created the logo and roster frames, and merged everything into the final file you see online.
Media
