-Julian Schonbohm

For my final VR project, I worked together with a classmate on a 360 video, where we also incorporated the Sennheiser VR microphone we just recently got at Newhouse.

I want to structure this independent learning requirement in the way of a tutorial, so that students in future classes can benefit from it. The recording process is actually not that hard, but you should at least have a basic understanding for audio signal processing. It just makes it easier for you to understand. I will put some links to detailed information in here, so that you can get more into it, if you need to.

Recording

First of all, where to place the microphone? That was something I was wondering right at the beginning. In a regular movie production, the sound mixer is always holding the boom pole in a way, that it is not entering the frame. This is obviously not possible with a 360 video, since it captures everything. I placed the microphone right below the camera itself. Make sure that the tripod for the camera and the mic stand are as close as possible to each other. With this, you can prevent the microphone from appearing in the frame too much. The same applies to the digital recorder. Make sure that the recorder is placed within the 3 legs of the tripod, so that the recorder is noticeable as little as possible. You can hide this afterwards with a Nadir.

Since the microphone is a condenser, it needs phantom power. This is a specific power without a condenser microphone doesn’t work. This is due to microphones architecture. Phantom power can be provided by the recorder itself or by a battery. Pretty much every recorder has the feature to provide phantom power, which I would use. I’ve never used a battery for this purpose.

I don’t want to go into detail too much about the recorder, but just mention the most important things you have to set up in the menu before recording. There is a manual you can get here (http://cdn.sounddevices.com/download/guides/788t_en.pdf), that goes into detail of every single submenu.

The two most important things for recording in general are the bit- and sample rate. These two indicate how “precisely” the analogue signal from the microphone is converted into and reproduced as a digital signal inside the recorder. The standard for film is 24 bits for the bitrate and 48 kHz for the sample rate. You can also use 96 kHz instead, although it is unnecessary if you ask me, since your file gets much bigger and you will probably compress it again when you’re exporting the final video at the end. Whatever you choose, just make sure you are not going below the 48kHz and not below the 24 bits. I hope that the following pictures help to illustrate this.

The next important thing is the menu item “Record File Type”. The microphone produces a 4-channel .wav-file, since it has 4 capsules that capture sound from different locations around it. I really encourage you to select “Poly” for the Record File Type. When doing so, you will get an interleaved 4-channel file instead of 4 individual mono tracks for every take. I forgot this once and still had much more work to do after the recording, which is unnecessary.

The next thing you should keep in mind is that every individual input of the recorder has to be routed to a specific track inside the recorder. Otherwise it won’t know where to record and, you got it, won’t record anything at all. This can be achieved by the menu item “Track Routing”.

Another last thing that really is necessary, is to link the 4 gain potis of the recorder for the 4 channels you’re recording on. With this, you can ensure that you level every individual capsule signal from the microphone in the same way, which is necessary to get a proper working Ambisonic signal in the end. The localization of signals work with differences in level, which is why this is necessary.

I already mentioned that the microphone consists of 4 capsules. This is why it also has 4 XLR cables that need to go into 4 individual inputs in the recorder. It is absolutely necessary to use the right order of XLR outputs going into recorder inputs here. This is also provided by the menu, but I will mention it here anyway. The yellow XLR has to go into input 1, the red one into 2, the blue one into 3 and the green one into 4. This is important to keep the right channel order you later need for converting purposes of the Ambisonics signal.

Lastly, the aligning of the microphone itself is also important. There are 3 different ways to align the microphone. The first one is called upright which means that the microphone is front-up facing the sound main sound source. The second is called endfire, which means that the microphone is facing down but the capsule to the front. The last one is upside down. The capsule is facing the ground but the microphone still to the front. On the front side of the microphone is indicated that it is the front, so you can easily figure out if you’ve aligned it the right way.

I used the microphone upright, which I think is the most common way for 360 videos.

Converting into B-format

When you’re done with your recording you need to go over to the next step. The microphone delivers the 4-channel .wav-file in the so-called A-format. The A-format is the raw recording of the 4 individual capsules and is specific to the microphone itself. This is why you need to convert it to B-format which is a standardized audio format for Ambisonics.

Sennheiser provides a converter plug-in which allows to do this conversion. You can use a Digital Audio Workstation of your choice, although I encourage you to use Reaper. There is one easy reason for it. Reaper is free (at least for 60 days) and offers the opportunity to process 4-channel tracks, which not every DAW does. When you want to use Pro Tools instead, you will have to use the HD version.

In Reaper, you need to create a 4-channel track or just drag and drop the 4-channel files from the recorder to it. Afterwards, select the converter plugin in that specific track. In the converter Plug-In, make sure to use the AmbiX format instead of the FuMa format. Both of them are variations of the B-format, but with different channel orders. YouTube uses the standardized AmbiX format, which is why you need to use this if you want to have spatial audio in your YouTube video. You can select these at the right corner of the plugin. All you need to do from here on, is to render the Master Track. You may need to change the amount of channels of the master track to 4. The default is regular stereo, which is 2 channels. Also make sure, that you select the right sample- and bit-rate in the export menu, as well as the right number of channels, which is 4.

In case you did forget to use poly-mode for the recording and ended up with 4 individual mono tracks for every take, no worries, you can also “fix” this in Reaper.

Drag and drop all the 4 individual channels into Reaper. Make sure it will create a single individual mono track for every mono file in Reaper. Also, the order has to be the right one (the same order as for the recording).

The next step is to create a 4-channel track in Reaper. After doing so, you have to route every single mono track to the 4-channel track. So, mono track 1 to channel 1 on the 4-channel track, mono track 2 to channel 2 on the 4-channel track and so on. One important thing here is to deselect the automated routing to the master track from every single mono track, since you just want the converted 4-channel track to go to the master track.

From this point on, it is actually the same way as with the 4-channel .wav file. Select the converter plugin and export the Master track.

Editing in Premiere

Congratulations, from here on you’re actually done with the hardest part. Fortunately, Premiere offers presets for Ambisonic VR video processing and editing.

I highly recommend watching the following YouTube tutorial, since it helped me a lot.

I will just briefly go into the most important things you need to do in Premiere, since the video gets very much into detail.

Start by using the Preset sequences provided by Premiere for VR projects. Make sure to use the one with the Ambisonics tag. This will allow you to edit 4 channel tracks in Premiere Pro. You can find the Presets under the category “VR” when creating a new sequence.

Next, you will have to change your audio settings in Premiere and set the “Multichannel Mono Media” to “Adaptive”. You find the audio settings under Preferences in Premiere. You will also need to do this for every individual audio file you’re importing. For this, right click on your audio file and click “Modify”. There is also the option to set this to Adaptive. Besides that, select 4 channels and a maximum number of 1 clip. This will ensure that you have all the 4 channels of the file on one track.

Since you obviously can’t listen to 4 channels with your stereo headphones, there is a plugin provided by Premiere that allows you to listen to it with stereo headphones. It is called “Binauralizer” and you need to put it on the Master Channel. This will allow you to check how it would sound after exporting. You can also use this plugin to check if your audio sounds right in terms of location of the sound. If not, you can use the “Panner – Ambisonics” plugin to make any adjustments.

Most important, however, is to delete the plugin “Binauralizer” before you are exporting your video. It is just for preview reasons and needs to be deleted before the export. Otherwise your audio will not work properly.

When exporting the video, you need to make sure that “Video is VR” and “Audio is Ambisonics” is checked. Also for the Audio: It needs to be set to 4 channels and AAC format, which is what YouTube requires.

One last thing that drove me crazy a couple of times. After you did your upload on YouTube, be patient and give the video time to process. It will take some time until the Ambisonics are working, even if it is already uploaded. I sometimes ended up exporting the video several times, because I was thinking that the spatial audio didn’t work. After maybe half an hour, I figured that the video I’ve already uploaded is working all of a sudden. I’d suggest waiting for maybe half an hour to check if the spatial audio is actually working or not.

I hope that I could help you with that tutorial and wish you good luck and a lot of fun with your Ambisonics production!