Passing on the Left

I haven’t written about the Dirac project recently.  We’ve reorganized a bit recently: most information is on diracvideo.org now, “Dirac” is now used to describe the overall project, with various subprojects like Schrödinger, dirac-research (formerly the Dirac codebase, intermingled with the specification), and a bunch others.  I’m finding that a lot of my time recently is cat herding all the subprojects and creating a coherent whole.

One such subproject is Bart Wiegmans’ GSoC project to implement Dirac in Java and get it working in Cortado.  It’s moving along quite nicely, and has reached the feel-good milestone of creating a pretty picture.  Right now, it only handles a very limited set of bitstreams — intra pictures only, variable length coding, and a specific choice of wavelet filter.  I think Bart has fixed a bug related to chroma handling since the time I created this screenshot, which is why it’s in greyscale.

Another subproject, and another GSoC project, is Mattias Bolte’s OpenGL decoding backend for Schrödinger.  This is roughly similar to the existing CUDA backend, although hopefully will work on a wider range of hardware.  He’s uncovering a number of resource usage issues in Schrödinger, such as the decoder using up to half a GB of RAM for temporary storage in some cases.   Alas, my hardware does not handle GLSL, so there will be no screenshots here.  The scope of the current project will require the use of GLSL, since serious signal processing is beyond clever texture hacks.

On a related note, progress is being made in gst-plugins-gl, the GStreamer OpenGL plugins.  Julien Isorce has done some amazing infrastructure work expanding on my original gst-plugins-gl code, which was really just an experiment.  At some point, the schrodec GStreamer element will decode directly in hardware using OpenGL, and then deliver the pictures as textures to downstream GStreamer elements, which will render it directly to the display.  Fillipo Argolias has been writing some cool GStreamer video filters that use OpenGL, but on a different branch than Julien’s work.  (My current task is merging these branches.) These filters will be used for Cheese.

Maarten Lankhorst has been working on a DirectShow filter for encoding/decoding Dirac streams based on Schrödinger.  It currently only works with Dirac inside Ogg, and is nearing the point where it is useful for people to use.  We’ll be doing a testing release sometime soon.

I have been working on a QuickTime component in whatever time I’ve had remaining after trying to keep up with everyone else’s work.  It works only with the QuickTime container, but it encodes and decodes pretty consistently with the Apple tools.  It’s pretty much ready for a testing release as well; just need the time to get it out the door.

Update: A screenie of Julien’s glfiltercube example (on a Mac, FTW):

Orc is a new sub-project of Liboil that I’ve been working on for a few weeks.  Orc stands for Oil Runtime Compiler, which further expands to Optimized Inner Loop Runtime Compiler.  As its name implies, the Orc library compiles code into a runnable function, and does it using SIMD instructions when available.  The “code” is currently an intermediate form that is roughly a platform-agnostic assembly language that understands variables and simple arrays.  It’s an intermediate form in the sense that it’s currently only stored as a list of structs — there isn’t a parser yet.  Orc can generate MMX and Altivec code for a few simple functions that is as fast as the corresponding liboil function.  There are also software fallbacks for those not-so-mainstream architectures.

There have been many motivations for creating Orc, which I will go into at another time.

One of the primary goals of Orc is to create a simple compiler (actually, it’s a really fancy assembler that understands register allocation) that is user-expandable: an Orc user can create additional opcodes, variable types, and rules for translating those opcodes into machine code.  One of the potential applications for Orc is pixman, which would require adding types (ARGB pixels) and opcodes (compositing operations).

I have posted the slides from my talk at LugRadio Live 2008.

Contrary to what the LugRadio web site says, I will actually be talking about Dirac (and Schrödinger), not GStreamer, at LugRadio Live 2008.  If you’re interested in cutting edge developments, or open media, or simply watching open-source taking over another niche, come and listen to me rant^H^H^H^Htalk about the state of video compression, media on Linux, colorspace conversion, and kitties.  Or you could just come and watch Dirac in action.

Schroedinger-1.0.0 is now out. (download) The 1.0.0 means several things: One, early adopters can start using it for real work — it’s bitstream compliant and fairly well tested internally. Two, it’s API and ABI stable, so integrators can work at adding Dirac support to various frameworks and tools. Three, I’m going to be inundated with bug reports for the next few weeks.

Schro works best (right now) in combination with Ogg and GStreamer, and when compressing SD and HD720 video at constant bit rate. Integration with other projects is now a priority.

Encoded picture quality in CBR (constant bit rate) mode isn’t as good as I would have liked it for a release, but this is a good baseline to ensure progress. A week ago, picture quality was much better, but there was a bad bug that caused bit allocation for some pictures to be way too high, thus starving future pictures and causing the picture quality to drop to “unrecognizable”. The workaround for this release is to make the bit allocator less agressive, so average picture quality is lower, but worst-case is now tolerable.

It’s been a rather dismal last few years for democracy and freedom in the world, but this past week has yielded several symbolic but heartening events: the Protect America Act expired, Kosovo declared independence, and Pakistan democratically rejected its military government.

The Dirac specification is frozen. You may now party.

Update: Schro is now Dirac-2.0.0 compliant (I think).

I’ve been getting a lot of requests recently for people that have GStreamer experience and are available for full-time employment or contract work in San Francisco or Silicon Valley. The requests are varied, from junior- and senior- level programmers with a minor amount of GStreamer experience, but interested in writing GStreamer-based code, to l33t GStreamer hackers. If this describes you, and you are looking for employment now or in the next 6 months or so, please send me an email. Most of these opportunities require on-site work and US employment eligibility.

I spent about 4x as much time on this project as originally planned, but GStreamer now has a bunch of OpenGL-based elements and is capable of uploading and downloading video, as well as filtering. The GL sink (glimagesink) still needs to be updated to handle GL buffers, and there’s a host of other details that need to be filled in, but all the important stuff works. Requires -bad and -base from CVS, and is known to work on both NVidia and Intel drivers.

A simple GL filtering pipeline: videotestsrc ! glupload ! glfilter ! gldownload ! ffmpegcolorspace ! ximagesink.

At some point in the past few weeks, I deleted the mysql database behind my blog. Somewhere around here, there’s a backup for it, but I’m a bit lazy, and the interested reader can consult the Google cache.

In other news, I apparently had some creativity that needed to escape my head this week, so I started a bunch of mini-projects: Improvements to GStreamer’s OpenGL video sink element, generic code for handling raw video in GStreamer, support for pushing around XImages and/or OpenGL textures in GStreamer pipelines, a JIT compiler for liboil, the (first ever) live streaming of Dirac video (from one side of my office to the other), and experimental code for doing wavelet transforms in OpenGL with fragment programs.