Well, I've been exploring OpenCV on the Raspberry Pi. Using the Raspbian Wheezy build at the moment. I've found that my OpenCV program runs really slow decoding H264. But I found a non-OpenCV program that makes use of acceleration and does a great job of playback-- omxplayer. However, it's not as hackable as I'd like, I'd really like to see an OpenCV that takes advantage of the same improvements. As it turns out, the upcoming new release of Raspbian, "Jessie" is supposed to have the acceleration mods a bit more integrated. gstreamer is apparently able to take advantage as well, and there are some pieces in ffmpeg (though I don't know if it's sufficient to just link with the latest). It all works by getting the GPU to do some of the work, and looks like that might require the use of the dbus library/daemon for the associated IPC.
I've also encountered LibVA and VAAPI which are video acceleration libraries that are out there, though I don't know what stage they're in with regards to the Raspberry. Since I found out about those while looking for the author(s) of omxplayer, I think there may be some connection there so there's hope for that I would think. But I see no evidence on these forums that OpenCV is tying into them in any way as of yet...
I'm wondering what it's going to take to see (or make) an OpenCV build on Raspberry that takes advantage of video decoding acceleration-- it'd actually make OpenCV useful on Raspberry and I do like the features in OpenCV, so that would be my first preference.. If it looks like it'll be a long ways off I'll probably delve into gstreamer. Any thoughts?