There's A Gallium3D State Tracker For VDPAU

[phoronix]

Phoronix: There's A Gallium3D State Tracker For VDPAU

Committed to a branch of the Mesa repository over the weekend is an initial Gallium3D state tracker for providing VDPAU support. Yes, VDPAU as in NVIDIA's Video Decode and Presentation API for Unix that has become quite popular with Linux users and is supported by many media applications...

http://www.phoronix.com/vr.php?view=ODM3NQ

[ua=42]

Could someone explain exactly what a state tracker is exactly?

[Hibbelharry]

I'll try:

Basically every step of a process like Rendering a 3D Scene or Decoding a video is done in steps. While you go through these steps you always go through predefined states. A state tracker is something like a matrix of states with defined transitions from one state to the possible next ones. So a state tracker is a prerequisite for anything where a driver might hook in and manage to do the work to change from one state to another.

Greetz
Hibbel

[rohcQaH]

A state tracker is a fancy (yet mostly correct) word for an interface between the driver and the application. The interface doesn't do anything, it just translates a language understood by the application (openGL, openCL, ...) to a single language (TGSI) understood by the driver.
That way, drivers will only need to understand TGSI and don't have to worry about the specifics of each API they want to implement.

Of course, in the real world it's never that simple, but you get the general idea. The goal isn't a perfect abstraction, but to simplify things and remove redundancy in driver code.


In this case, the VDPAU state tracker is just a simple interface. Unfortunately, there is currently no g3d driver that exposes the video decoding hardware.

I don't know what the long-term goal for this state-tracker is. If it'll just wait for g3d drivers to eventually expose the video decoding hardware, it'll be useless for the next few years.
If it'll implement fallbacks (e.g. CPU or shader-based decoding), it'll be useful - but also a lot of work. Work that could have been done for ages without VDPAU, so I'd be surprised if something like that was planned.

[marek]

A state tracker translates commands of one API (e.g. OpenGL) to commands of another API (e.g. Gallium pipe driver interface). It's basically a wrapper.

[drag]

Do you know what a 'state' is (in terms of computer science)?

---------------

Say I have to develop a piece of security software that monitors the doors of a warehouse. Assume that doors can only be locked after they are closed.

So I have sensors that monitor whether the doors are opened or they are locked. So they have 2 bits of information for each door... or four 'states'.

1 closed and locked
2 closed and unlocked
3 open and unlocked
4 open and locked

So... out of the 4 states the first three are 'ok', but the third one is impossible without somebody breaking the lock.. if the lock is locked, but the door is open then the door has been broken into and my software should sound a alarm if that ever came up.

Of course some locks can be activated while the door is still open. So if I had to deal with that then I'd have to take into account if the door was locked and closed prior to being opened, or if it was just locked while being held open. Then I'd have to track other things like the time of day and so on and so forth.

Then I would have a series of actions the software would have to perform based on the various states. Like maybe it will automatically lock the doors after a certain time period, or if the door is unlocked after a certain time I would have the software check the ID of the person that unlocked it with a keycard or whatever. I would probably develop some sort of flow chart of when to activate alarms, when to contact the security personal and so on and so forth.

Pretty simple common sense stuff, right? So if I wrote a software to keep track of all that it would include what is known as a 'finite state machine'.

-------------------

So think about how Gallium is suppose to work:

* Multiple applications being used by multiple users
* Multiple Graphics APIs being used by applications
* Single driver (or small number of drivers)
* Single GPU (or small number of GPUs)

So since you have multiple APIs it needs a way to handle the 'fake' multitasking that happens in a modern OS were the kernel schedules tasks in a rapid manner to create the illusion of multitasking. (With multiple processors you can have maybe 2 or maybe 8 things happenning at the same time, but there will usually be dozens or even hundreds of applications running at the same time)


Here is how it looks (sorta)

Applications <---> Standardized APIs <---> State Tracker <---> Pipe Driver <---> Winsys <---> hardware


The 'State Tracker', 'Pipe Driver', and 'Winsys' are all part of Gallium driver model.

STATE TRACKER: API-specific, but mostly hardware independent.Meaning that you need a state tracker for each API you want to support, but the same state tracker can run on multiple Gallium drivers.

PIPE DRIVER: API Independent, Hardware specific. You'll need a new pipe driver for each major hardware family you have to support.

WINSYS: API Independent, Very hardware specific. Developers will try to contain as much as the hardware-specific code in Winsys as possible.



The way the relationships happen are like this:

1. Applications hook into standardized APIs (VDPAU, OpenGL, EXA) via application libraries.

Those Applications + libraries send API commands to the state tracker.

2. The state tracker keeps track of the command states as the system multitasks. It's main job is to translate the the standardized APIs to generic low-level graphics operations.

It sends those generic low-level graphics operations to the pipe driver

3. The pipe driver translates the generic low-level graphics operations to hardware-specific graphics operations and sends them along to the Winsys portion.

4. The Winsys portion takes the hardware-specific low-level graphics operations and translates that into binary code that gets executed on the actual GPU.


The idea behind this design is that you want to make hardware-specific code as small as possible to reduce the effort of making stable graphics drivers.

You also want to be able to support the ever-changing application APIs that can be executed on the GPU.

And you also want to be able to adapt the existing state tracker code and API code to new and radically different types of GPUs that will be coming out in the distant future.


edit:

Oh and this is just my personal understanding. Much of this information is probably misleading or inaccurate.

[droidhacker]

If it'll implement fallbacks (e.g. ... or shader-based decoding), it'll be useful - but also a lot of work. Work that could have been done for ages without VDPAU, so I'd be surprised if something like that was planned.

That is actually the ONLY sensible thing TO do. There is absolutely NO REASON to bring G3D into the picture if the only objective is to access the video decode hardware. That, by definition, should be done direct. Pumping it through G3D would be nothing more than additional complexity.

In fact, the objective here should be a COMBINATION of CPU and SHADER video decoding.

And for that matter, the video decoder part on GPUs actually *DOES* use shaders for (at least some of) its heavy lifting. The fact that it is typically HIDDEN from the user is irrelevant. Now to just move the hidden stuff out into the open in an open implementation.... and you'll see h.264 decode acceleration/assistance even on wimpy intel GPUs. Right, intel GPUs probably not strong enough to decode h.264. No, that doesn't mean that they can't add *something* to it. Every little bit of assistance the GPU can offer reduces the work that must be done by the CPU, and every little bit of work that the CPU *doesn't* have to do adds more systems into the pool of those capable of driving HD video decoding.

[bugmenot3]

A VA-API state tracker for comparison would be interesting

[Veerappan]

A VA-API state tracker for comparison would be interesting

Well, if it's done right, VA-API with a Gallium-based VDPAU backend should still work

[droidhacker]

This whole video decode API thing is starting to just get too messy.

TOO MANY APIs.

Note: A vaapi state tracker should be more-or-less the exact same as a vdpau state tracker. The back-end would be the same, just two different interfaces to what is fundamentally the same thing. There really isn't much to compare. I think the main thing to resist is to go through too many different layers of interfaces to get to the same objective. Though personally I root for vaapi, whichever API gets implemented that can be dealt with DIRECTLY by media players would be the ideal approach to USE.

I.e. player --> VDPAU/G3D
is better than
player --> VAAPI frontend to VDPAU --> VDPAU/G3D