I recently added support for 10- and 16-bit encoding and decoding to Schrödinger, so I did a little release. Presenting Schrödinger-1.0.11. Also pushed changes to GStreamer to handle the new features. Although these changes have been in the works for some time, a little prompting from j-b caused me to finish this off, so this will probably appear in VLC soon, too.
This was the last piece needed to create a 10-bit master of Sintel, which I’ve been planning to do for some time.
Archive for the ‘gstreamer’ Category
New Schrödinger Release
Monday, January 23rd, 2012Live GStreamer Pipeline Graphing
Monday, December 12th, 2011Rather that fix bugs in my existing projects, I decided to have a “me” weekend the last few days. So I spent some time fixing up and adding new features to a very minor pet project, gst-gtk-widets. (It’s a few Gtk+ widgets for doing things related to GStreamer, like a video player widget. It also doesn’t do everything correctly, so look at the code with caution.) Specifically, I added the ability to change the pipeline from the UI in gst-launch-gtk, a GUI version of the venerable gst-launch tool. I also added some live pipeline graphing. Drumroll… and screenshot.
Screenshot of gst-launch-gtk with preliminary pipeline graphing
GStreamer SDI Capture Plugins
Thursday, March 24th, 2011I’m getting ready to push several commits to the gst-plugins-bad source repository that add plugins for capturing SDI and HD-SDI using cards from two different manufacturers: BlackMagic Design‘s DeckLink, and Linear Systems SDI Master capture card.
The Linear Systems cards are probably better known by their reseller, DVEO. Entropy Wave uses both of these cards in the E1000 Live Encoder appliance, we’ve found that aside from some motherboard incompatibilities in the DeckLink cards, they both work great in Linux. While we’re primarily interested in live capture at the moment, output has also been implemented.
We slightly prefer the Linear Systems cards – mainly because the drivers are open source, but also because the API allows lower level access to the hardware, including SDI clocking and raw VANC and HANC data. It also allows subframe latency, although not implemented in the GStreamer plugin, it will be nice to use in the future.
In comparison, the DeckLink driver and SDK are not open source (which means I can’t fix any bugs), although they conveniently provide open source headers and shim code for interfacing with the SDK. This allows the GStreamer plugin to be completely open source and legally distributable separately from the SDK, but will only work if the SDK libraries and driver are present. Optical fiber connections are only available in the DeckLink, and the DeckLink cards tend to be less expensive.
It will take a few weeks for these to be available as part of a GStreamer release, however, they are available in the Media SDK now.
(Reposted from my Entropy Wave blog.)
Orc NEON backend
Friday, June 25th, 2010The Orc NEON backend is now open-source and part of the 0.4.5 release. Thanks to Nokia for making this possible.
All of the GStreamer code that previously used liboil has been converted to Orc, and will be part of the upcoming releases. Plugins from -base that were converted are videotestsrc, videoscale, audioconvert, volume, and adder. The change in speed is minor for now, since liboil was not used extensively.
The way that GStreamer uses Orc is intended to be very convenient for the end user: by default when compiling from source, Orc will be used if an acceptable version is found, otherwise backup code (in C) will be used. This backup code is automatically generated and checked in to git by a developer, so it always exists. The configure options ‘–enable-orc’ and ‘–disable-orc’ affect this: the former causes configure to require Orc, and a configure failure is a good reminder to upgrade Orc. The latter may be useful on architectures where Orc doesn’t generate code yet. After a transitional phase, Orc will become required, since the more advanced uses of Orc cannot be duplicated by backup functions.
Theora on TI C64x+ DSP and OMAP3
Wednesday, November 11th, 2009For the last several months, Entropy Wave has been making Theora work on the TI C64x+ DSP as a project for Mozilla Corp.
An Ogg/Theora video of Big Buck Bunny being played back on a Beagle Board via the C64x+ DSP coprocessor
The goal behind porting to the C64x+ is to run on OMAP3 SoC from TI, which has an ARM Cortex A8 core and also has a C64x+ DSP coprocessor. This SoC (System on Chip) is best known as being the base behind Nokia’s N series of mobiles (including the N900), the Motorola Droid, Palm Pre, and the Beagle Board. The DSP coprocessor is commonly used for audo and video processing, including video encoding and decoding, and TI makes codecs available for MPEG-4 video decoding, AAC decoding, etc. Having Theora decoded on the DSP fits into Mozilla’s Fennec project, making Firefox with video useful on a mobile platform.
One of the engineering reasons behind having a separate processor for media handling is that it separates real-time tasks (media decoding) from non-real-time tasks, such as running web browser software. From the standpoint of software running on the ARM, the video decoder looks and acts just like a hardware video codec. The DSP on the OMAP3 is even more compelling for video decoding because attached to the DSP are several units that accelerate motion vector copying, VLC decoding, and loop deblocking. Unfortunately, these pieces are not publicly documented by TI, so the current Theora port (which is open source) is unable to use them. A future Entropy Wave project will likely add support for these acceleration units which would allow the performance of the Theora decoder to be similar to TI’s MPEG-4 codec, which can do 800×480 playback (possibly more?). As it looks now, the resulting code would necessarily be closed source until such a time when TI wishes to make the specifications public.
As it currently stands, the Theora decoder plays 640×360 24fps at slightly more than 100% speed on average. This isn’t quite good enough to call it “real time”, since some frames take longer than the allotted time to decode, but it’s pretty close and the results are good. Additional speed improvements in libtheora would require internal changes, which would be a project in itself. One clear area for improvement is that the DSP spends a substantial part of its time idle, because the host code is serialized with the DSP processing. Fixing this is likely to put the above case firmly into the “real time” category. Given that 640×360 is larger than the iPhone display resolution and almost as large as the N900 resolution, it’s clearly good enough, even if it is less than TI’s hardware accelerated MPEG-4.
On the Entropy Wave site is a page describing the demo, including where to download images and how to compile source code.
A big thanks to the people that laid the foundations for this work, especially Felipe Contreras.
YCbCr Gamut Checking
Wednesday, October 7th, 2009I recently added a pattern to GStreamer’s videotestsrc that can be used to check YCbCr to RGB conversion is being done correctly as part of video output. It is the result of a clever hack — some YCbCr values, when converted to RGB, are out of range, so as part of the conversion process, they are clamped to the nearest RGB value. The pattern generator creates a checkerboard pattern of a color (say, red) and a YCbCr value that upon correct conversion will result in the same color. Thus the pattern should be invisible. Usefully, these out-of-gamut YCbCr values are preserved by video codecs, so I can present to you a Theora video demonstrating this:
Firefox does the conversion correctly, so it’s unlikely you’ll see the pattern. However, some video display drivers still get this wrong, so you might see the pattern when playing the video in a standalone program that uses XV. For those of you with working kit, I created a demonstration video that simulates a bad conversion:
Sometimes it’s possible to see the pattern very faintly due to rounding in even a correct conversion. This is unavoidable because the RGB->YCbCr->RGB round trip is lossy.