The multimedia capabilities of windows phone are nice and exciting, but not all file formats are supported, especially the open-source lossless and lossy formats, such as flac and vorbis.

However, the Silverlight runtime comes with this class called a MediaStreamSource. Media stream sources provide the developer with direct access to the media pipeline, allowing decoding of formats which are not supported by Silverlight out of the box. Yes, at some point, you can support Flash on Silverlight if you want to, but that is a topic for another day.

We will need a flac decoder, written in pure C#. While this may seem trivial and obvious, finding one isn’t as easy as you may think. Most C# implementations of the flac decoder are using p/invoke, which will not work on windows phone due to obvious reasons.

So we are left with writing the decoder ourselves. Don’t panic, we won’t do that.

A good advice when facing daunting tasks in programming is to make sure nobody did it before you. In our case, some did do a flac decoder in pure C#, and made it open source.

http://flacbox.codeplex.com/

Our goal of making a flac media stream source seems achievable now. Writing the decoder from scratch in C# would take months.

Now, let’s get to work on the app.

In the interest of saving time, we shall use the sample already provided to us by the generous windows phone team.

http://msdn.microsoft.com/en-us/library/windowsphone/develop/hh202978%28v=vs.105%29.aspx

It will provide us with the basic skeleton for the background audio player.

Just make sure you replace the .mp3 files provided there with .flac files.

In case you don’t have flac files around, use this program to convert your mp3 files to flac files.

http://audacity.sourceforge.net/download/?lang=en

Now that we replaced the mp3 files with flac files (for the sake of simplicity, you can just convert the files given to you in the sample, so we won’t have to change too many things), we need to do some adjustments in the solution first.

In order to not have naming problems, we must replace every occurrence of the string .mp3 with .flac(assuming you converted the mp3 files in the sample to flac files and kept their names). This can be done very fast with the find and replace feature built into Visual Studio. Just make sure you select “entire solution” to make sure every occurrence is replaced.

The next step is to replace the URI value passed to the constructor and the Tag property of the audio tracks in the sample.

Before…

new AudioTrack(new Uri("Kalimba.mp3", UriKind.Relative),
 
                    "Kalimba",
 
                    "Mr. Scruff",
 
                    "Ninja Tuna",
 
                    null),

After…

    new AudioTrack(null,
                    "Kalimba",
                    "Mr. Scruff",
                    "Ninja Tuna",
                    Null,
                       "Kalimba.mp3",
EnabledPlayerControls.All),

Basically, we removed the URI (it is set to null), and the file path used to construct the Uri goes to the tag property.

Why null URI? Because the documentation says so, of course. If an audio track has a null URI, then the system knows it has to call the Background Streaming Agent to handle decoding.

http://msdn.microsoft.com/en-us/library/windowsphone/develop/hh394039%28v=vs.105%29.aspx

Next step is to add a Background Streaming Agent to our project. Make sure the main project (the one with the UI) has a reference to the newly created streaming agent, otherwise this will not work.

Now, the file created in our streaming agent has the following method already defined for us

protected override void OnBeginStreaming(AudioTrack track, AudioStreamer streamer);

And we are told to use the streamer.SetSource() method to set up the media stream source.

Now, let’s create the media stream source.

Import the flacbox project into your solution. Create a new project and add the files manually, and not the project file provided by flacbox, because it is created using a different runtime.

 

Now, add a file to the stream agent project, in which we will create the media stream source.

The MediaStreamSource class is an abstract class. Therefore, we can’t create an actual MediaStreamSource object unless we use inheritance.

public class FlacMediaStreamSource : MediaStreamSource, IDisposable

Use Visual Studio auto completing features to implement the abstract class and the IDisposable interface.

We will need a couple of extra fields to get the job done

/// <summary>
 
        /// The stream that we're playing back
 
        /// </summary>
 
        private Stream stream;
 
  
 
        /// <summary>
 
        /// The WavParser that can extract the data
 
        /// </summary>
 
        private WavParser wavParser;
 
  
 
        /// <summary>
 
        /// The stream description
 
        /// </summary>
 
        private MediaStreamDescription audioDesc;
 
  
 
        /// <summary>
 
        /// The current position in the stream.
 
        /// </summary>
 
        private long currentPosition;
 
  
 
        /// <summary>
 
        /// The start position of the data in the stream
 
        /// </summary>
 
        private long startPosition;
 
  
 
        /// <summary>
 
        /// The current timestamp
 
        /// </summary>
 
        private long currentTimeStamp;
 
  
 
        /// <summary>
 
        /// The sample attributes (not used so empty)
 
        /// </summary>
 
        private Dictionary<MediaSampleAttributeKeys, string> emptySampleDict = new Dictionary<MediaSampleAttributeKeys, string>();
 
  
 
        /// <summary>
 
        /// sample to feed to the audio pipeline
 
        /// </summary>
 
MediaStreamSample sample;

The stream field is self explanatory: we will need an actual stream of data to get samples from.

The wavParser and MediaStreamDescription will be used in the methods implemented by the abstract class.

currentTimeStamp is very important. If we mess this up, the media stream source will not work properly anymore.

protected override void CloseMedia()
        {
 
            // Close the stream, and dispose of other unmanaged resources
 
            stream.Close();
 
            stream.Dispose();
 
            this.startPosition = this.currentPosition = 0;
 
            this.wavParser = null;
 
            this.audioDesc = null;
 
            NotifyStreamComplete();
 
        }

This method is used to do clean-up. The NotifyStreamComplete () method is used to notify the background stream agent that the media stream source has completed its job.

We won’t bother with the

protected override void GetDiagnosticAsync(MediaStreamSourceDiagnosticKind diagnosticKind); //method. It serve no purpose for us.

The media stream source enables playback by feeding samples to the audio pipeline. This is pretty much how any audio-video decoder player actually works. While the media stream source gets a sample, the built-in audio chip plays the sample. For quality reasons, the sample should be acquired fast enough so the users won’t hear any gaps between the samples. If the sample takes too long to acquire, the system will shut down the media stream source along with the streaming and playback agents.

protected override void OpenMediaAsync()
        {
            // Create a parser
 
            this.wavParser = new WavParser(this.stream);
 
            // Parse the header
 
            this.wavParser.ParseWaveHeader();
 
            this.wavParser.WaveFormatEx.ValidateWaveFormat();
 
            this.startPosition = this.currentPosition = this.wavParser.DataPosition;
 
            // Init
 
            Dictionary<MediaStreamAttributeKeys, string> streamAttributes = new Dictionary<MediaStreamAttributeKeys, string>();
 
            Dictionary<MediaSourceAttributesKeys, string> sourceAttributes = new Dictionary<MediaSourceAttributesKeys, string>();
 
            List<MediaStreamDescription> availableStreams = new List<MediaStreamDescription>();
 
            // Stream Description
 
            streamAttributes[MediaStreamAttributeKeys.CodecPrivateData] = this.wavParser.WaveFormatEx.ToHexString();
 
            MediaStreamDescription msd = new MediaStreamDescription(MediaStreamType.Audio, streamAttributes);
 
            this.audioDesc = msd;
 
            availableStreams.Add(this.audioDesc);
 
            sourceAttributes[MediaSourceAttributesKeys.Duration] = this.wavParser.Duration.ToString();
 
            ReportOpenMediaCompleted(sourceAttributes, availableStreams);
 
        }


This method is pretty much the same for all media stream source. In case you wonder where it came from, Microsoft released a helpful example of media stream sources a few years back.

http://archive.msdn.microsoft.com/ManagedMediaHelpers

You will find the missing types there.

Now, the OpenMediaAsync method has a very important role, especially this line

streamAttributes[MediaStreamAttributeKeys.CodecPrivateData] = this.wavParser.WaveFormatEx.ToHexString();

 

It basically tells us the media pipe line that samples are “encoded” using the wav format.

Technically, our media stream source decodes chunks of flac encoded data into wav. Wav is the raw representation of sounds. The media stream source can decode the file chunks in pretty much any format supported by Silverlight, not just wav.

The codec private data can be found in the MediaStreamSource documentation

http://msdn.microsoft.com/en-us/library/system.windows.media.mediastreamsource%28VS.95%29.aspx

The core method of the media stream source is protected override void GetSampleAsync(MediaStreamType mediaStreamType);

 

This method provides the “chunks”, also known as samples, to the audio pipeline.

The ReportGetSampleCompleted() method sends the sample to the audio pipeline. If the sample contains no data, the media stream source is signaled to stop. Therefore, we have to check some conditions before we provide samples, to make sure the file hasn’t ended a long time ago and we just send a bunch of 0s to the pipeline.

We also need to make sure that if an exception is ever to occur, we will stop the media stream sample to prevent a complete shutdown of the audio streaming process.

There is another aspect we need to take into consideration: how big should the sample actually be?

The magic number is 4096 bytes. More or fewer bytes may also work, however, reading a lot of bytes from the stream may cause out of memory exceptions. Reading fewer bytes may cause disturbances in the audio playback or the song may not feel “right”.

So we read 4096 bytes from the stream, then feed it to the media stream sample constructor. And this is pretty much all it does.

Perhaps you are wondering why we are reading directly from the stream. This stream is a special kind of stream. The actual type used will not be the average IsolatedStorageFileStream, but the WaveOverFlacFileStream type, defined in the FlacBox project.

Its implementation of the .Read() method will provide us with WAV decoded FLAC data. The actual decoding occurs in this method.

protected override void GetSampleAsync(MediaStreamType mediaStreamType)
        {
            if (currentPosition < stream.Length)
 
            {
 
                byte[] k = new byte[4096];
 
                stream.Read(k, 0, 4096);
 
                using (MemoryStream ms = new MemoryStream(k))
 
                {
 
                    try
 
                    {
 
                        sample = new MediaStreamSample(
 
                        this.audioDesc,
 
                        ms,
 
                        0,
 
                        4096,
 
                        this.currentTimeStamp,
 
                        this.emptySampleDict);
 
                        // Move our timestamp and position forward
 
                        this.currentTimeStamp += this.wavParser.WaveFormatEx.AudioDurationFromBufferSize(4096);
 
                        this.currentPosition += k.Length;
 
                        ReportGetSampleCompleted(sample);
                    }
                    catch { ReportGetSampleCompleted(new MediaStreamSample(this.audioDesc, null, 0, 0, 0, this.emptySampleDict)); }
 
                }
            }
            else
            {
 
                ReportGetSampleCompleted(new MediaStreamSample(this.audioDesc, null, 0, 0, 0, this.emptySampleDict));
 
            }
 
        }


We must also notify the streaming agent when the job is done here, so we create a callback

public delegate void Foo();
 
        public event Foo FooComplete;
 
        protected virtual void NotifyStreamComplete()
        {
 
            if (FooComplete != null)
 
                FooComplete();
 
        } 

Now, we go back to the audio stream agent and the method

protected override void OnBeginStreaming(AudioTrack track, AudioStreamer streamer);

We are provided with the audio track that wanted streaming. And remember we set the URI to null and the tag was the file name.

So we will use the tag to get to the file, with an IsolatedStorageFileStream


var source = new IsolatedStorageFileStream(Track.Tag, System.IO.FileMode.Open, FileAccess.Read, FileShare.ReadWrite, IsolatedStorageFile.GetUserStoreForApplication());

Notice we won’t be using the IDisposable interface for the file stream this time. The stream gets disposed in the CloseMedia method of the media stream source.

Our media stream source constructor wanted a stream. But we won’t use this stream, at least not directly:


WaveOverFlacStream g = new WaveOverFlacStream(source, WaveOverFlacStreamMode.Decode);
 
FlacMediaStreamSource ms = new FlacMediaStreamSource(g);
 
ms.FooComplete += ms_FooComplete;
 
streamer.SetSource(ms);

void ms_FooComplete()
 
{
 
    NotifyComplete();
 
}


See Also

Another important place to find a huge amount of Windows Phone related articles is the TechNet Wiki itself. The best entry point is Windows Phone Resources on the TechNet Wiki.