There's A Direct3D 9.0 Gallium3D State Tracker

[crispy]

Just one question: do you plan to make some dummy implementation of d3d taken from wine first and then slowly implementing the real functions?

And people, don't blast me if this is a stupid question as ive only got very little knowledge of programming

[V!NCENT]

NOW STOP BLASTING YOUR UNFOUNDED CLAIMS AROUND.

Hey, this is a FLOSS website, mostly. Radical ideas, especially Microsoft tech, is blasted here.

Don't mind the naysayers. I, for one, am looking forward to your projects results

[zhasha]

Why is everyone convinced I'm trying to screw you all over? If you're seriously idiotic enough to think that executables store the memory address of all external functions statically, try compiling these 2 things:
Compile with: g++ lib.cpp -shared -o libfoo.so
lib.cpp

#include "lib.h"
#include <stdio.h>

void lib::foo(int i)
{
	printf("%d\n", i);
}

void lib::del()
{
	delete this;
}

lib *new_lib()
{
	return new lib;
}

lib.h

class lib
{
public:
	void foo(int i);
	void del();
};

lib *new_lib();

Compile with: g++ foo.cpp -o foo -lfoo -L.
foo.cpp

#include "lib.h"

int main()
{
    lib *l = new_lib();
    l->foo(10);
    l->foo(20);
    l->del();
    return 0;
}

then run LD_LIBRARY_PATH=. ./foo
it'll output:

10
20

now let's throw some shit around:
change lib.h to:

class lib
{
private:
    int j;

public:
    lib();
    ~lib();

	void foo(int i);
	virtual void add(int i);
	void del();
};

lib *new_lib();

and change lib.cpp to:

#include "lib.h"
#include <stdio.h>

lib::lib() : j(5)
{
    printf("constructor\n");
}

lib::~lib()
{
    printf("destructor\n");
}

void lib::foo(int i)
{
	printf("%d\n", j);
	add(i);
}

void lib::add(int i)
{
    j += i;
}

void lib::del()
{
	delete this;
}

lib *new_lib()
{
	return new lib;
}

compile the lib again.
run the same, untouched executable from before and you'll see:

constructor
5
15
destructor

oh, and just for arguments sake, try removing a function from the lib and see what the program outputs. I removed lib::del and got:

[zhasha@ztoshiba Desktop]$ LD_LIBRARY_PATH=. ./foo
constructor
5
15
./foo: symbol lookup error: ./foo: undefined symbol: _ZN3lib3delEv

EDIT: Now to put the final nail in the coffin for the myth you've started here. Try adding -Bsymbolic to the library compile line. What this does is resolve as many function pointers as close as it can on link time, or as the ld man page puts it:

       -Bsymbolic
           When creating a shared library, bind references to global symbols to the definition within the shared library, if any.  Normally,
           it is possible for a program linked against a shared library to override the definition within the shared library.  This option is
           only meaningful on ELF platforms which support shared libraries.

NOW STOP BLASTING YOUR UNFOUNDED CLAIMS AROUND.

[mdias]

I might have been a bit unclear here:
Presumably when you first call the function, the mangled name is resolved. This resolution is then stored so that further calls into the library don't need a lengthy procedure. The exact same thing needs to happen when using C libraries too, so in essence, the only difference is that C++ names are mangled to allow functions with the same names but different types.
You can completely screw up the vtable and still maintain binary compatibility. Provided of course you don't change any current function declarations.

In all honesty, I refuse to believe someone would make such a system. I know for a fact that in windows when you get a pointer to a function and call it, the asm code will call the procedure directly, and I suppose the exact same thing happens in linux.
So, assuming what I say is true (which I can't personally confirm right now, but google seems to agree with me), calling the address returned by dlsym() will call the procedure directly, so there's no way something else can interfere here to resolve names. dlsym() is the only one who takes a symbol name and returns an address, therefore if you change a class' vtable, you're going to get screwed, because the class' member names are never taken into consideration, and the data in the vtable will not be what your client's binary is expecting.

Actually, this (resolving virtual methods to addresses without exposing the implementation details of final classes) is why a vtable exists.

In other words: Name resolving only needs to happen once to get a pointer do Direct3DCreate9, after that it's all binary vtable data, no more name resolving. This is because your client's binary could never know what kind of object Direct3DCreate9 returns, all it gets is a pointer, and it hopes it's a pointer to a valid expected interface, so you better not change the interface on the library, or else things will break. Unless you recompile the client's binary using the new interface info.

[zhasha]

Wow, this is very weird behaviour. I mean, when does the lookup for the member names happen exacly? If it happens when you call the function, I see this as a very "eww" and slow trick. And I can't imagine this happening when you cast the returned pointer to the interface... or does it?

[edit] sorry for the kind of off-topic question here.

[edit2]: Or do you mean we have to recompile our binary if you change something?

[edit3]: I'm aware that the user's binary doesn't know about the real size of the final object, but my concern lies on you adding a member function somewhere in the middle of the existing ones, changing the offsets, and therefore becoming incompatible with the first user's *binary*.

I might have been a bit unclear here:
Presumably when you first call the function, the mangled name is resolved. This resolution is then stored so that further calls into the library don't need a lengthy procedure. The exact same thing needs to happen when using C libraries too, so in essence, the only difference is that C++ names are mangled to allow functions with the same names but different types.
You can completely screw up the vtable and still maintain binary compatibility. Provided of course you don't change any current function declarations.

[BlackStar]

Since the allocator functions are all inside the library, your binary never knows the size of any given object, nor does it need to. All it needs to concern itself about is a subset of the vtable filled with names (not offsets as I first thought).

I've tested this to be true on ELF binaries, and ELF binaries being the only thing I really care about for binary compat. we're pretty much good. Any wrapper like WINE will obviously use a wrapper still, but the difference will be that the wrapper will map every single call directly to a call into libd3d9.so

Does this work with e.g. Intel's compiler when libd3d9.so is built with GCC?

[mdias]

Thanks for that explanation. So as long as you don't want to use 'new' to create objects, you're good. And you don't need that in Nine, so it's all good.

On a completely unrelated note, the fact that you need factories to create objects from dynamically loaded classes in C++ is something I find ugly. I'm not convinced that this is unavoidable with a good linker implementation. Since the linker knows its own name-mangling scheme, there should not be a problem of mismatched name mangling. It should be possible to achieve source platform independence with a class-aware linker, just not binary platform independence.

Something like this:

MyClass *foo;
dlopen("myclass.so");
dl_class_sym(MyClass, "MyClass");
foo  = new MyClass();

It would work on all platforms that support class loading like this. The only thing that won't work is taking a compiled .so from one platform and load it on another. But that's not that important, unless you're Wine.

Well, I'd kill for that feature on C++, but I don't think it's possible, no... Only the library that implements MyClass would know how to build a new instance of that class. You can have a static member function that does just that though, but that's coming back to the factory thing really...

On wonderland you'd be able to do what you just said and a little bit more, like overriding member functions after instantiation and so on, but I don't even want to think about the complexity of that compiler-wise...

[Remco]

Thanks for that explanation. So as long as you don't want to use 'new' to create objects, you're good. And you don't need that in Nine, so it's all good.

On a completely unrelated note, the fact that you need factories to create objects from dynamically loaded classes in C++ is something I find ugly. I'm not convinced that this is unavoidable with a good linker implementation. Since the linker knows its own name-mangling scheme, there should not be a problem of mismatched name mangling. It should be possible to achieve source platform independence with a class-aware linker, just not binary platform independence.

Something like this:

MyClass *foo;
dlopen("myclass.so");
dl_class_sym(MyClass, "MyClass");
foo  = new MyClass();

It would work on all platforms that support class loading like this. The only thing that won't work is taking a compiled .so from one platform and load it on another. But that's not that important, unless you're Wine.

[mdias]

...
Your binary will still resolve the names of the functions you call just as it did before and all is good in wonderland. Since the allocator functions are all inside the library, your binary never knows the size of any given object, nor does it need to. All it needs to concern itself about is a subset of the vtable filled with names (not offsets as I first thought).
...

Wow, this is very weird behaviour. I mean, when does the lookup for the member names happen exacly? If it happens when you call the function, I see this as a very "eww" and slow trick. And I can't imagine this happening when you cast the returned pointer to the interface... or does it?

[edit] sorry for the kind of off-topic question here.

[edit2]: Or do you mean we have to recompile our binary if you change something?

[edit3]: I'm aware that the user's binary doesn't know about the real size of the final object, but my concern lies on you adding a member function somewhere in the middle of the existing ones, changing the offsets, and therefore becoming incompatible with the first user's *binary*.

[zhasha]

Since you're insisting on making a complete ass of yourself, let me clarify with examples from my own code.
The library has an allocator function (and automatic deallocator but peace be with that for now). The ONLY allocator function that's not a member of a class is Direct3DCreate9:

extern "C" IDirect3D9 *
Direct3DCreate9( UINT sdk_version )
{
    IDirect3D9 *d3d9 = NULL;

    /* Why? just because. */
    if (sdk_version != D3D_SDK_VERSION) { return NULL; }

    try {
        d3d9 = new IDirect3D9;
    } catch (std::bad_alloc e) {
        if (d3d9) { delete d3d9; }
        return NULL;
    }

    return d3d9;
}

It returns a pointer to an IDirect3D9 object. This object has member functions that you can call by simply using the -> operator as you would any static class. When you do that, ld.so will resolve the mangled function name in the library (automatically) and call that particular entry.
In code, to the end user, it looks like this:

#include <d3d9.h>
...
LPDIRECT3D9 d3d9;
d3d9 = Direct3DCreate9(D3D_SDK_VERSION);

To achieve this, simply do what you do with every other C/C++ library. In gcc you would simply specify -ld3d9 and it will do all the work for you. None of that retarded bullshit.
Now let's say I update one of the classes you're using by adding a couple of functions to it for my own amusement. What happens? Absolutely nothing happens. Your binary will still resolve the names of the functions you call just as it did before and all is good in wonderland. Since the allocator functions are all inside the library, your binary never knows the size of any given object, nor does it need to. All it needs to concern itself about is a subset of the vtable filled with names (not offsets as I first thought).

I've tested this to be true on ELF binaries, and ELF binaries being the only thing I really care about for binary compat. we're pretty much good. Any wrapper like WINE will obviously use a wrapper still, but the difference will be that the wrapper will map every single call directly to a call into libd3d9.so

EDIT:
If you want to see first hand how close this looks to actual D3D on windows, look in ut.cpp. Be wary though, that the XD3D9 API will change very soon to map more closely to D3D9. It was very much a rush job.