Announcement

Collapse
No announcement yet.

LoadLibrary: Support For Loading Windows DLLs On Linux

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #51
    Originally posted by Weasel View Post
    The issue is that loading a library via LD_PRELOAD can change symbols being looked up, because of ELF's global symbols. It's not a loader quirk, it's ELF being able to store global symbols.
    Your argument sounds like the people who believe in the War on Drugs.

    Removing global symbols wouldn't magically eliminate LD_PRELOAD. It would just have been designed differently... and probably something like LD_OVERRIDE="/lib/x86_64-linux-gnu/libc.so.6=/foo/libc_override.so;..."

    At the same time, I imagine that there'd be some kind of SYSTEM namespace which the loader treats as a global namespace to allow shared dynamic lookup of symbols like malloc. rather than embracing Windows's mess of "every compilation unit has to be assumed to have its own allocator".

    Whether it's economics or programming, attempting to stamp out supply without killing demand just creates workarounds like black markets... and the creators of ELF felt there was a need for a global namespace. If you don't fix the humans, the humans will do what the humans will do.

    As I've said when talking about DRM and P2P file-sharing, there's no such thing as a technical solution for a social problem.
    Last edited by ssokolow; 15 March 2020, 01:05 PM.

    Comment


    • #52
      Originally posted by Weasel View Post
      Symbols resolution should not depend on library load order, period.
      That is not in the ELF specification. ELF specification dynamic symbol resolution should be based on DT_NEEDED list in each file to be following ELF specification.
      http://notes.secretsauce.net/notes/2...eded-tags.html

      At times having incorrect or missing DT_NEEDED explodes back in your face because you run into different elf loaders with different levels of correct/incorrect behaviours.

      Originally posted by Weasel View Post
      How else could the loader resolve them? Give an error because they same global symbol has been found twice? That's just as much of a problem. (failure to load)
      The correct behaviour it straight forwards. In correctly functional to ELF specification this has only happened 1 very limited way. A file making up an application has including 2 .so files by DT_NEEDED that contain a symbol named exactly the same name.

      So example and this is the corner case in the ELF standard.
      Problem_child.so or problem_child dynamic elf application
      has
      DT_NEEDED libcx.so
      DT_NEEDED libcb.so
      Yes a list order as in the order symbol resolve should go.

      These examples are using malloc that happens to resolve from both yes both are a libc. The last on list there libcb there is the symbol used but the loader should have found it in libcx then found it again in libcb so raising warning of error. This is before ELF format had symbol versioning added. If symbol versioning information says that the symbol is libcx version not libcb.so version then libcx.so version would be passed though to application.

      This is why in elf standard Standard filtering and Auxiliary filtering you have a SONAME because without a SONAME you are meant to be ignored in resolve and you don't pick up the versioning information so you may or may not replace the symbol using LD_PRELOAD. Basically LD_PRELOAD is not a factor here because by ELF standard it should not work.

      Please note this is not how it goes bell up with the GNU loader all of the time.
      This here is the GNU loader disaster that is not to specification:
      mainprogram
      Has DT_NEEDED library1, library2 main program has to resolve global symbol malloc.
      - library1 provides to man program malloc as it provides that as a global export.
      - library2 DT_NEEDED library3 to provide it dependants it has malloc and GNU loader ignores this DT_NEEDED saying and gives library2 the malloc loaded for mainprogram from library1 because it doing a global resolve instead of a segregated dynamic linking resolve that would have resulted in library2 seeing library3 malloc.

      See the problem here is a global dynamic linking resolve instead of a segregated dynamic linking resolve this difference is the biggest cause of screw up by a large margin.

      Notice something segregated dynamic linking as per ELF specification LD_PRELOAD basically stops working. LD_PRELOAD normal has no soname. No soname no resolve by DT_NEEDED so everything loaded by LD_PRELOAD of old disappears because it does not have Standard filtering or Auxiliary filtering stating what SONAME is meant to be overlapping with.

      Weasel basically if everyone of your points go back and references LD_PRELOAD you are wrong.

      Originally posted by Weasel View Post
      The issue is that loading a library via LD_PRELOAD can change symbols being looked up, because of ELF's global symbols. It's not a loader quirk, it's ELF being able to store global symbols.
      That only works because the loader you are looking at is not conforming to ELF specifications requirement to to process DT_NEEDED on dynamic files. No SONAME value no DT_NEEDED mapping so by ELF standards a LD_PRELOAD should have zero visibility and replaced nothing and should need to be recoded as "Standard filtering or Auxiliary filtering" that are in the ELF specifications. Working LD_PRELOAD is sign of ELF standard not conformance loader with most likely worse issues on ELF platforms.

      So the issue you are talking about is a breach of ELF standard.

      Global symbols by ELF standard for dynamic loaded are not application wide. The problem with the ELF standard it self is a very small corner case for dynamically loaded. Now non conformance ELF loaders like the GNU libc loader have a lot larger issue area.

      You never bothered checking if LD_PRELOAD should work by the ELF standard yes the answer is that it should not work at all. Problem with LD_PRELOAD was known in the coff/a.out days and Standard if ELF was designed to provide a fixed solution to LD_PRELOAD being "Standard filtering or Auxiliary filtering".

      Weasel can you make your arguement against ELF without LD_PRELOAD remember LD_PRELOAD functional is not to ELF standard at all. So every time you use LD_PRELOAD as part of you arguement against ELF you are basically straight up wrong you are talking about ELF with a broken loader like it or not. You said I could not use Hacked PE loaders against PE format and that is what you do every time you use LD_PRELOAD when talking about ELF format.

      Comment


      • #53
        Personally I like PE, it's a really sound format, it provides a lot of features when dealing with binaries and is extremely versatile. Regardless of any specific situation, the self-serving anti-industry standard thought pattern is destructive to Linux, it is a toxin that serves only vested interests that benefit from Linux being handicapped and isolated.

        Then again I'm no longer in the free software community, they always have some ideological justification for not serving the users, that somehow conveniently benefits established players. I use Linux every day, it's my main OS, I don't even use anything except Linux... but there is no future here, the users are more dis-empowered than they were before, and Google and Apple work to help it stay that way. Open source is not all there is to freedom, open platforms are a necessary foundation that cannot be superseded by merely releasing source code.
        Last edited by techzilla; 18 March 2020, 01:45 PM.

        Comment


        • #54
          Originally posted by techzilla View Post
          Personally I like PE, it's a really sound format, it provides a lot of features when dealing with binaries and is extremely versatile.
          This is what happens when you have not read the specifications. ELF format supports more CPU types than PE does so some cases you don't have the choice PE on some platforms. Some platforms PE is simply is not usable.

          ELF is one format for static and dynamic linking. You see some people call PE the PE/Coff this is because PE is you executable format and object files are Coff for static linking so two formats instead of one.

          There is a performance that weasel never wants to talk about with versioned symbols instead says you can create multi versions named differently and that completely misses what versioned is mostly used for..

          https://lwn.net/Articles/691932/

          Symbol versioning is used for multi things including having more than 1 function of the same function optimised for different cpu types..

          ELF supports the loader dynamic calling function that returns a function pointer this is the ifunc feature . This is code that runs only once per run when the loader program whats X symbol and X symbol has a ifunc it runs the ifunc that returns to the loader what function should for this system on this platform from this library so transparent on the fly optimisation.

          https://gcc.gnu.org/onlinedocs/gcc-4...ttributes.html yes ifunc is listed here.

          There is no way to properly do ifunc in PE format because the format does not support it at all. This does cause a performance problem with PE because it does not support as much on the fly optimisation as the ELF format does.

          The symbol problem weasel complains about with ELF comes about from basically the same thing that stuff has been optimised for speed and size.

          PE format has it major flaws and its not as versatile as you would want to make out. Being highly versatile and highly optimised is what causes ELF solutions to have many ways those are implementing can make it nasty. Basically the higher versatile equals atleast a percentage of pure nasty implementations existing. ELF is extremely versatile.

          Something to remember the first ELF format is from 1983. ELF format is an old timer here it does have it issues but when you look at development of PE you find something horrible. But the younger PE format was based off older COFF yes a format that predates ELF that did not have the newer features in ELF. The features offered by PE format are mostly older and poorer than what ELF format offers yet exact match to what COFF offered.

          I don't know what thinking there was no license cost to use ELF format when Microsoft developers made PE why they did not base on ELF and take in the improvements to COFF that ELF had. Unix COFF/a.out did not have ifunc and symbol versioning and so PE does not have this stuff either.
          Last edited by oiaohm; 19 March 2020, 12:32 AM. Reason: I was annoyed and missed a few words in typing.

          Comment


          • #55
            Originally posted by oiaohm View Post
            ELF is one format for static and dynamic linking. You see some people call PE the PE/Coff this is because PE is you executable format and object files are Coff for static linking so two formats instead of one.
            You realize that's exactly one of the reasons PE is better, right? In fact, global symbols exist primarily because of object files.

            The fact ELF can also act as object file format (static library) is the reason for half of its idiocy.

            Comment


            • #56
              Originally posted by Weasel View Post
              You realize that's exactly one of the reasons PE is better, right? In fact, global symbols exist primarily because of object files.

              The fact ELF can also act as object file format (static library) is the reason for half of its idiocy.
              No its that you are a idiot who is making stuff up without reading the ELF specification. Because elf format symbols are not global until the loader or linker does it.

              https://www.gabriel.urdhr.fr/2015/09...le-format/#plt

              The Procedure Linkage Table(PLT) is part of the ELF design. That each elf object dynamic has it own PLT. The dynamic PLT is not merged out of existence by the loader. This is why lib-capsule is so simple in fact you just need to fill in the PLT of each elf file making up a application with the correct PLT information for that file this is exactly how ELF is designed to work. PLT is not a global thing so the symbols used by a .so in a application are not in fact required by elf format to be global resolved symbols.

              ELF is design from the ground up in dynamic for for segregated dynamic linking. Heck its design to be suitable for segregated static linking as what was historically done by sun ldd relinker.

              Weasel how many more times will you say ELF format has global symbols when really by elf format in static or dynamic linking they don't in fact exist as anything other than import/exports in the elf specification. Don't exist as in world global in the elf specification does not mean global resolved symbols.

              You are meant to be able to have multi symbols the same name inside ELF and have each section get the right symbol based on link order that if the linker/loader is implemented how the ELF format says.

              The only difference between PE and dynamic ELF is that dynamic ELF has a single table per file instead of PE multi tables per file and those implementing loaders have save space by not requiring putting the soname enteries in that table and not processing soname enteries in that table even that there is a allocated option in the ELF specification.

              The difference you are complaining about is broken ELF implementations like it or not. There is not a fault in the ELF specification causing it.

              Comment


              • #57
                Originally posted by oiaohm View Post
                No its that you are a idiot who is making stuff up without reading the ELF specification. Because elf format symbols are not global until the loader or linker does it.

                https://www.gabriel.urdhr.fr/2015/09...le-format/#plt

                The Procedure Linkage Table(PLT) is part of the ELF design. That each elf object dynamic has it own PLT. The dynamic PLT is not merged out of existence by the loader. This is why lib-capsule is so simple in fact you just need to fill in the PLT of each elf file making up a application with the correct PLT information for that file this is exactly how ELF is designed to work. PLT is not a global thing so the symbols used by a .so in a application are not in fact required by elf format to be global resolved symbols.

                ELF is design from the ground up in dynamic for for segregated dynamic linking. Heck its design to be suitable for segregated static linking as what was historically done by sun ldd relinker.
                Are you retarded or something? The PLT example you linked shows call abort, whether it's in the PLT or not. Do you know what that fucking means? That it's a global symbol.

                If it weren't, it would need to have the library name stored along with it somewhere. Such as "[email protected]". Object files have global symbols by design, since when you call a function in C/C++, you use just its name. That's global you moron.

                That's why .def files exist for the linker on Windows, because that happens during linking (i.e. after object files are already compiled, and those have global symbols) to map the global symbols into whatever you want. This works, because after linking, you get a different format (PE) that separates symbols by the libraries/modules they are from.

                Once again you link shit without understanding anything.

                Comment


                • #58
                  Originally posted by Weasel View Post
                  Are you retarded or something? The PLT example you linked shows call abort, whether it's in the PLT or not. Do you know what that fucking means? That it's a global symbol.
                  NO you are retarded because that is not what is means. PLT does not store symbol names it stores function pointers.

                  Originally posted by Weasel;n1166767If it weren't, it would need to have the library name stored along with it somewhere. Such as "[email protected]". Object files have global symbols by design, since when you call a function in C/C++, you use just its name. [B
                  That's global you moron.[/B]
                  Valid question to me would be the somewhere. The correct somewhere be.

                  "[email protected]" this here. Reality is libname and abort are stored in different tables inside PE or ELF format. That @ does not exist in the PE or ELF file output. This is more information to the linker to so puts the information in the right table. This is just something to make it simple to the end user programmer not something that is part of the on disc format..

                  Originally posted by Weasel View Post
                  That's why .def files exist for the linker on Windows, because that happens during linking (i.e. after object files are already compiled, and those have global symbols) to map the global symbols into whatever you want. This works, because after linking, you get a different format (PE) that separates symbols by the libraries/modules they are from.
                  But you don't need def files for the linker. Mingw for example is able to use the straight up .dll to get the information to fill in the import table under windows. This should have you questioning.

                  ELF was not design to have you link to a static library to perform a dynamic call. Notice that .def by Microsoft is built into a .a that the linking then links so you need to embedded dll name inside.

                  If you look at the .def format you see

                  Code:
                  LIBRARY name_of_dll
                  
                  EXPORTS
                  func1
                  func2
                  func3
                  Do we see something like this in the ELF standard yes we do. In your elf dynamic Symbol Table. I will walk you thought the part ELF implementations optimised that lead mess.

                  STT_FILE entry. This is design to take the name/soname of the object that the linker resolved the symbol from at build time of the dynamic executable. Guess what GNU binutils does not populate this symbol.

                  In dynamic symbol table you should been seeing.
                  Code:
                  STT_FILE  some_so_name
                  STT_FUNC func1
                  STT_FUNC func2
                  STT_FUNC func3
                  STT_FILE some_other_so_name
                  STT_FUNC func1
                  STT_FUNC func2
                  STT_FUNC func3

                  Now lets start optimising this. Searching the dynamic table top to bottom to find all .so file required is slow was of processing time if library is missing. So now ELF standard adds the DT_NEEDED entries that list the in a short list the need .so files. Of course as you have stated C you cannot have same function name twice ADA language you can.

                  Here comes the path to the first bad choice. STT_FILE entries from the dynamic symbol export table. This saved some space now we can save some more space if we delete the STT_FILE entries(yes the first bad choice) from the dynamic symbol import table and used DT_NEEDED resolve order instead and push the processing of what function owns to what .so into the loader. This is only minor-ally bad. DT_NEEDED does not exist in the first draft of the ELF standard. Up until this optimisation there was basically no difference between PE and ELF other than the fact ELF was using single tables and PE was using multi tables. Yes to the optimisation that nukes STT_FILE entries that start of problem. Someone implementing PE could basically say stuff it we are having one table name GLOBAL as well and create the same problem.

                  Heck a loader like the GNU glibc one that ignores STT_FILE entries in the dynamic symbol table is broken at this point but this break is not that bad.

                  At this point ELF symbols are still not process wide global as they are still meant to be processed on a per loaded file base.

                  The next optimisation is the worst killer. "We can speed up the dynamic loader by using global resolve instead of segregated resolve idea." Now finally with this optimisation do we now have the symbols gone uncontrolled global. This is not part of the ELF standard and is effectively a broken loader. Of course this is going to be good program loaded plugins not to need DT_NEEDED either so instead lets just using the already loaded symbols for resolve and only resolve by name nothing more. This optimisation is not written as part of the ELF specification either.

                  http://notes.secretsauce.net/notes/2...eded-tags.html
                  Yes the reason why above works with python yet not with a normal loader is this worst killer that leads to sections of executable ELF having their symbols resolved from libraries that are not in their DT_NEEDED entries.

                  So two complete parts of the standard are ignored/not implemented in the dynamic loader to cause the problem one part is not being done by the linker making the ELF files.

                  Yes the linker is not putting in STT_FILE entries into the dynamic symbol table the effect of this should be insanely minor as long as the DT_NEEDED table is being used by the dynamic loader correctly.

                  Weasel its 2 levels of failure of implementation so the ELF standard has to be broken twice by those implementing to cause the problem you are talking about. Basically like it or not the ELF standard is not defective. ELF implemented correctly your problem does not exist. Those implementing ELF have done defective implementations. So if the same groups take PE and implement they can screw it up the same way because they have track record of ignoring standards when it helps performance.

                  Comment


                  • #59
                    Originally posted by oiaohm View Post
                    NO you are retarded because that is not what is means. PLT does not store symbol names it stores function pointers.
                    Nobody is talking about the function pointers in memory, we're talking about the symbols. In object files, symbols are NOT stored with the library they are from, because when compiling a file, you don't even specify which library it's imported from.

                    All C/C++ do when compiling is they keep externally (undefined) symbols and refer to them by the symbol name. Like "call abort", that's how it ends up compiled: it's an undefined symbol as far as C/C++ are concerned. The linker then fills these undefined symbols with references to specific symbols when it finds the imports. All the library arguments like -l are for the LINKER only, which is AFTER object files are already compiled.

                    The linker. How can C/C++ know which symbols are from which libraries when they don't even know WHICH libraries you link to? That's information that's sent later after C/C++ already done their job.

                    So the object file format stores no reference to any libraries, because it hasn't even been linked yet, idiot. Object files store only global undefined symbols. And that's exactly why ELF supports global symbols, because it can also act as object file, and why it's degenerated.

                    Originally posted by oiaohm View Post
                    Valid question to me would be the somewhere. The correct somewhere be.

                    "[email protected]" this here. Reality is libname and abort are stored in different tables inside PE or ELF format. That @ does not exist in the PE or ELF file output. This is more information to the linker to so puts the information in the right table. This is just something to make it simple to the end user programmer not something that is part of the on disc format..
                    Nope, PE stores it in an array referred to from the library import itself. You have a list of libraries to import, and then each entry in this list has a reference to an array holding all the symbols to import from that library.

                    Originally posted by oiaohm View Post
                    But you don't need def files for the linker. Mingw for example is able to use the straight up .dll to get the information to fill in the import table under windows. This should have you questioning.
                    It does it by "guessing" where it's from using the name or automatically doing name decoration (for stdcall for example). This is just a guess. It can use .def files though, and they are required in some cases.

                    So shut the hell up already.
                    Last edited by Weasel; 21 March 2020, 09:04 AM.

                    Comment


                    • #60
                      Originally posted by Weasel View Post
                      So the object file format stores no reference to any libraries, because it hasn't even been linked yet, idiot. Object files store only global undefined symbols. And that's exactly why ELF supports global symbols, because it can also act as object file, and why it's degenerated.

                      Code:
                      STT_FILE some_so_name
                      STT_FUNC func1 STT_FUNC func2
                      STT_FUNC func3
                      STT_FILE some_other_so_name
                      STT_FUNC func1
                      STT_FUNC func2
                      STT_FUNC func3
                      This is where you are completely wrong. As the ELF format was in fact designed in object mode to have a STT_FILE entry as well. The linking not done yet does not mean you don't want to have space to record a asm directive saying get X function from X library. Yes this was include in the ELF specification but its not being used.

                      ELF is a superior object format if it features are used. So your presume that resolve to library has to be done in the linking stage is wrong ELF allow it to be done in object construction by the assembler. This is added DT_NEEDED or STT_FUNC entries in object or other special things like symbol versions.

                      [QUOTE=Weasel;n1166935]Nope, PE stores it in an array referred to from the library import itself. You have a list of libraries to import, and then each entry in this list has a reference to an array holding all the symbols to import from that library.[/CODE]

                      This is a multi array store and you don't need a multi array todo it. ELF was designed todo it in a single array.

                      Please note you are saying to have to list the libraries with PE there is really no such thing if you have a non conforming PE loader that decide to accept a single array. ELF non conforming loader accepts dyanmic symbols without STT_FILE entries in the symbols table. There are two ways to skin this cat and two ways to screw it up. The fact one of these screws has been done does not magically make another format superior.

                      http://bestofbr33d.blogspot.com/2012...xecutable.html

                      The fun little point you keep on missing a ELF built executable by ELF design is meant to be a object that you can feedback into the linker and link again. Yes if that worked case of program linked against old version of symbol or newer version of symbol with glibc you should be able to force it with a relink but GNU binutils basically does not support that..

                      Possible contents of a object and the Possible contents of a executable with ELF is the same stuff. You are thinking there is a divide between what executable has and what object has. This is the PE/Coff world problem of having two formats instead of one.

                      Code:
                      __asm__(".symver realpath,[email protected]_2.2.5");
                      Weasel how do you think the above line works with ELF. When you saw this stuff should have been telling you something is more advanced in ELF than PE. With you PE system symver stuff would have to be done at the link stage because the coff format has no where to store it because none of the dynamic link tables don't exist at the object stage. ELF format each object can put stuff in the .symver section that is one of the dynamic link tables then have the linker merge those sections and add extra as required.

                      Now same applies the the ELF dynamic symbol table where that can exist in the ELF object file as a optional part.

                      PE/COFF of windows is missing ifunc as well because same reason because you cannot be populating the dynamic tables in the object file plus there is not a slot in the PE dynamic tables to store ifunc.

                      The split output and object fault of PE causes it own set of problems that you are totally ignoring Weasel.

                      Really you just want to say ELF is broken so you don't have to accept its a more advanced format that is being miss used.

                      Comment

                      Working...
                      X