All of the GStreamer code that previously used liboil has been converted to Orc, and will be part of the upcoming releases.  Plugins from -base that were converted are videotestsrc, videoscale, audioconvert, volume, and adder.  The change in speed is minor for now, since liboil was not used extensively.

The way that GStreamer uses Orc is intended to be very convenient for the end user:  by default when compiling from source, Orc will be used if an acceptable version is found, otherwise backup code (in C) will be used.  This backup code is automatically generated and checked in to git by a developer, so it always exists.  The configure options ‘–enable-orc’ and ‘–disable-orc’ affect this: the former causes configure to require Orc, and a configure failure is a good reminder to upgrade Orc.  The latter may be useful on architectures where Orc doesn’t generate code yet.  After a transitional phase, Orc will become required, since the more advanced uses of Orc cannot be duplicated by backup functions.

An Ogg/Theora video of Big Buck Bunny being played back on a Beagle Board via the C64x+ DSP coprocessor

The goal behind porting to the C64x+ is to run on OMAP3 SoC from TI, which has an ARM Cortex A8 core and also has a C64x+ DSP coprocessor. This SoC (System on Chip) is best known as being the base behind Nokia’s N series of mobiles (including the N900), the Motorola Droid, Palm Pre, and the Beagle Board. The DSP coprocessor is commonly used for audo and video processing, including video encoding and decoding, and TI makes codecs available for MPEG-4 video decoding, AAC decoding, etc.  Having Theora decoded on the DSP fits into Mozilla’s Fennec project, making Firefox with video useful on a mobile platform.

One of the engineering reasons behind having a separate processor for media handling is that it separates real-time tasks (media decoding) from non-real-time tasks, such as running web browser software. From the standpoint of software running on the ARM, the video decoder looks and acts just like a hardware video codec. The DSP on the OMAP3 is even more compelling for video decoding because attached to the DSP are several units that accelerate motion vector copying, VLC decoding, and loop deblocking. Unfortunately, these pieces are not publicly documented by TI, so the current Theora port (which is open source) is unable to use them. A future Entropy Wave project will likely add support for these acceleration units which would allow the performance of the Theora decoder to be similar to TI’s MPEG-4 codec, which can do 800×480 playback (possibly more?). As it looks now, the resulting code would necessarily be closed source until such a time when TI wishes to make the specifications public.

As it currently stands, the Theora decoder plays 640×360 24fps at slightly more than 100% speed on average. This isn’t quite good enough to call it “real time”, since some frames take longer than the allotted time to decode, but it’s pretty close and the results are good. Additional speed improvements in libtheora would require internal changes, which would be a project in itself. One clear area for improvement is that the DSP spends a substantial part of its time idle, because the host code is serialized with the DSP processing. Fixing this is likely to put the above case firmly into the “real time” category. Given that 640×360 is larger than the iPhone display resolution and almost as large as the N900 resolution, it’s clearly good enough, even if it is less than TI’s hardware accelerated MPEG-4.

On the Entropy Wave site is a page describing the demo, including where to download images and how to compile source code.

A big thanks to the people that laid the foundations for this work, especially Felipe Contreras.

Firefox does the conversion correctly, so it’s unlikely you’ll see the pattern. However, some video display drivers still get this wrong, so you might see the pattern when playing the video in a standalone program that uses XV. For those of you with working kit, I created a demonstration video that simulates a bad conversion:

Sometimes it’s possible to see the pattern very faintly due to rounding in even a correct conversion. This is unavoidable because the RGB->YCbCr->RGB round trip is lossy.