I tried the mplayer a moment ago. -vo ps3 works really well. I preferred running it in the console because even the gnome terminal hack caused some flickering at the bottom of the video. The threaded h264 decoder was also supprisingly fast since my last attempts. I noticed that adding following additional parameters to it help even more: fast:skiploopfilter=all
I was able to watch intense 1280x720 24fps anime fansub so that it got slower than real-time only at some parts. Audio paused then, so the sync didnt go wrong. In total, 720p h.264 videos are now watchable.
With -vo null there doesnt seem to be any slowdowns except when stressing the frame buffer device with some other program at the same time, such as catting a long file to the console so that it scrolls. If there was some other method than the framebuffer to transfer the frames then there would probably be no cuts anymore, so the decoder at least is fast enough now. Maybe some upcoming RSX hacks will open the possibility.