I did a little bit [just a little] research on this, and I believe the main reason those things are slow is because Apple refuses to provide an API to allow 3rd parties to use the GPU for decoding video. Apple itself provides a codec for h264 that uses your graphics card, so that one performs well.
Eh? AFAIK, Accelerate.framework doesn't do anything with the GPU -- its "just" a set of highly optimized functions for math/DSP/image processing operations.
vImage is not GPU accelerated and provides no useful operations for decoding video. You may be thinking of OpenCL, which also provides no useful operations for decoding video, or VideoDecodeAcceleration, which is already supported by QuickTime and needs no further 3rd party support.
