Announcement

Collapse
No announcement yet.

Fedora 29 Might Make Change To Eliminate Unnecessary Linking

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #21
    Originally posted by coder View Post
    Don't bash what you clearly don't understand.

    Shared library symbol references are only global if they're not resolved. If you link to a library providing those symbols, then they're now bound to that library. If every library links in its immediate dependencies, like everyone here seems to agree they should, then there's no problem.
    They're not bound to that library as evidenced by the fact that you can LD_PRELOAD a completely different library before and it steals those symbols. Obviously, once a symbol is resolved, it won't resolve again, that doesn't mean it's "bound" to a library since this can be changed on every load of the library via LD_PRELOAD. Bound means that it is bound AT COMPILATION TIME.

    Originally posted by coder View Post
    Now, try using "1 million versions of a library" in a single executable and pass around data structures between different users of the different versions of those libraries and have it not blow up in your face, please. Can't do it? Gee, then maybe linking in different versions of the same shared library is a bad idea! ...not that you can't do it, on properly versioned shared libraries, but people generally don't because it's so perilous.
    Why do people come up with such stupid arguments? Who the fuck uses multiple versions of a library from ONE code? Come on, stop grasping at straws.

    Multiple versions can exist in the same addressing space if, say, a dependency that your program needs uses a different version of a library that your program directly accesses. But those data structs will never be messed up: your code calls version X of library and passes structs to it, while the other (deeper) dependency calls version Y of library with its own structs, and there's literally no conflict.

    Originally posted by pal666 View Post
    The stupidity of phoronix commenters never ceases to amaze me.
    it is the other way around: by default library is linked in even when no symbols are needed
    Took the words right out of my mouth with the first phrase there. Not what the guy I quoted said. Maybe keep context next time unless you want to look like an idiot.

    He was outlining why removing such "dead links" to libraries can lead to breakage, but this only happens due to the stupidity in Linux userland.

    Originally posted by pal666 View Post
    what a wall of shit
    you described a world where symbol interposition does not exist, i.e. no custom malloc replacements or no ld_preload
    Good because LD_PRELOAD is a hack that shouldn't exist in the first place.

    Originally posted by pal666 View Post
    imbecile, consider your library takes const std :: string & parameter
    who the fuck is going to call its constructor, which does allocation ?
    Hey moron, did you know if your library takes non-basic built-in types, you're a retarded programmer? Standard library is just a collection of templated classes, so they're far from "basic types".

    Go back to school. I'm not interested in "what ifs" from code developed by rookies.

    Here, educate yourself: https://stackoverflow.com/a/22797419

    Also see: https://stackoverflow.com/a/33783918
    Originally posted by Ben Voigt
    This is something that Linux programmers weren't accustomed to dealing with, because g++'s libstdc++ was a de-facto standard and virtually all programs used it, thus satisfying the ODR. clang's libc++ broke that assumption, and then C++11 came along with mandatory changes to nearly all Standard library types.

    Just don't share Standard library types between modules. It's undefined behavior.
    I'm guessing you don't even know what "undefined behavior" is.


    ________________

    Originally posted by pal666 View Post
    this is pile of shit to workaround windows deficiency and it will not work anyway.
    consider your handle from liba is for some large object which has field of some other type governed by libb.
    Types should be local to a library. This is called encapsulation, and it's one of the first things any C or C++ programmer learns about.

    What the fuck kind of spagetti bullshit do you describe? In C++, a class should implement all associated interfaces within itself, or free functions within the same namespace (for ADL lookup). Two libraries must NEVER share namespace.

    If liba can't handle everything it can with its type then it's time to redesign your shitty liba.

    Originally posted by pal666 View Post
    pages of different files are not identical
    Thanks for the info Captain Obvious. We're talking about sharing the same library in memory. Same library that, you know, gets mapped as COW.

    Originally posted by pal666 View Post
    it has nothing to do with fork. it is an example of windows not supporting position independent code.
    LMFAO what has Windows to do with the CPU? Are you saying Windows can't use the CPU's rip register in 64-bit mode?

    Don't waste people's time with your bullshit.

    https://en.wikipedia.org/wiki/Positi...e#Windows_DLLs
    Originally posted by Wikipedia
    In Windows Vista and later versions of Windows, the relocation of DLLs and executables is done by the kernel memory manager, which shares the relocated binaries across multiple processes. Images are always relocated from their preferred base addresses, achieving address space layout randomization (ASLR).[4]
    This forum is so full of morons who never coded a thing in their life and still talk.
    Last edited by Weasel; 04 July 2018, 07:42 AM.

    Comment


    • #22
      Originally posted by coder View Post
      I think you've got the causality wrong. You're accepting Windows' constraint as given and then looking to justify it.

      I've been down the road of custom allocators, a few times, and never found them justified. Especially for API-public types. But that's not even what we're talking about. We're talking about the exact opposite - a separate allocator for each library, by default, even though 99% of the time it's just the same underlying implementation. Were I to match your hyperbole, I could use some choice words to describe that architecture.
      No, I'm saying that the allocator is local to a library and should be local, otherwise your library is designed poorly. Whoever does it should get fired.

      An allocator is not just malloc. It includes free.

      Think of C++: an allocator is a class. The class implements both new and delete operators. You don't fucking let the caller of the library to implement and call delete on YOUR class, WTF?? That's such a straw man argument from people who have nothing against Windows' superior DLL design.

      Same thing in C. malloc/free are a PAIR and should always be used as SUCH. A malloc from library X must be considered distinct from library Y, and thus its free should also be distinct. Also from app Z.


      Other post awaiting moderation because I destroyed some rookies with too many links I guess...
      Last edited by Weasel; 04 July 2018, 07:43 AM.

      Comment


      • #23
        Wow, so many smart people here!
        Can someone explain to me how Macho-o compares to elf and PE?

        Comment


        • #24
          Originally posted by grigi View Post
          Actually it (-Wl,--as-needed) has been default on Gentoo for years. No need to set it in your make.conf
          It is possible I added it to my make.conf before Gentoo defaulted to using -Wl,--as-needed.

          The file /usr/portage/profiles/default/linux suggests that it has been enabled in Gentoo in year 2010 or sooner.

          Searching the personal history on my machine reveals that I added it to my make.conf on 19 april 2009.

          Comment


          • #25
            Originally posted by Weasel View Post
            No, I'm saying that the allocator is local to a library and should be local, otherwise your library is designed poorly. Whoever does it should get fired.

            An allocator is not just malloc. It includes free.
            That's pretty much the issue. Lots of library functions allocate memory the caller is responsible for freeing, and many don't bother to wrap free(). You're using the argument of an extreme corner case where it might be justifiable to use a custom allocator to say that all memory allocation must always be wrapped, which has a real cost in runtime and API weight and forcing layers of wrapping and abstraction or copying on simple API types like strings or raw data buffers that might be passed across several library boundaries.

            If you weight down your code with abstractions to handle every corner case that might apply to any library - not just yours - you run a greater risk of introducing or hiding bugs. And libraries that are cumbersome to use generally aren't.

            Originally posted by Weasel View Post
            Think of C++: an allocator is a class. The class implements both new and delete operators. You don't fucking let the caller of the library to implement and call delete on YOUR class, WTF?? That's such a straw man argument from people who have nothing against Windows' superior DLL design.
            In my experience, custom new & delete operators are mercifully quite rare.

            Comment


            • #26
              Originally posted by coder View Post
              That's pretty much the issue. Lots of library functions allocate memory the caller is responsible for freeing, and many don't bother to wrap free().
              Such as? (give me examples of Windows libraries btw, since you used this example against Windows -- I'm not surprised if Linux libs do have this "problem" since, well, the quality of most libraries is kind of subpar)

              Of course, in Windows it's perfectly fine for a library to ask you to use one of VirtualFree, LocalFree or GlobalFree, because those are system calls and always the same no matter the runtime or dependencies (i.e. things won't break). So those don't count.

              Originally posted by coder View Post
              You're using the argument of an extreme corner case where it might be justifiable to use a custom allocator to say that all memory allocation must always be wrapped, which has a real cost in runtime and API weight and forcing layers of wrapping and abstraction or copying on simple API types like strings or raw data buffers that might be passed across several library boundaries.

              If you weight down your code with abstractions to handle every corner case that might apply to any library - not just yours - you run a greater risk of introducing or hiding bugs. And libraries that are cumbersome to use generally aren't.
              Seriously? Every library worth its salt has its own free or destroy or destructor, it's just common sense practice. This is not a corner case, this is the norm. Most of the time you don't even just "wrap free", you have to do more work to begin with when destroying the library's object.

              For example, zlib, flac, etc.

              If you think destroying an object via an extra function call is some sort of "serious overhead" then you shouldn't be using a dynamic library in the first place, static link it instead with LTO. That's like 0.01% of the performance hit.

              Comment


              • #27
                Originally posted by moonshadow565 View Post
                Wow, so many smart people here!
                Can someone explain to me how Macho-o compares to elf and PE?
                Given the nonsense I had to reply to, I think it's safe to say I'm the only programmer in this discussion (no, you don't have to take my word for it, I supplied links to prove I know what I'm talking about if you are *really* interested).

                And unfortunately for you, I couldn't care less about the Mac, so I have no idea about anything regarding it. Unless someone else chimes in, that is. Something tells me it's not any better than elf tho.

                Comment


                • #28
                  Originally posted by Weasel View Post
                  Dude, what?

                  There is literally zero context switches between libraries because they're in the same address space (unless you mean different threads but that's a completely different thing). That's the whole point of libraries compared to processes in the first place. I have seriously NO idea where you got that from but it's so wrong it's not even funny.
                  Sorry to burst that bubble dude but this is really happening, I've programmed Windows applications for decades and have seen many young devs fall for this.

                  If your application is linked with runtime A and you then use a DLL that is linked with runtime B and your process calls a function in that DLL then Windows will switch the current runtime context for your thread to B and when the function returns it will switch back to A again. However the major bug here is that if code in the DLL calls back to your application then the runtime context is still B and not A.

                  You can test this extremely easy yourself with libraries such as cURL or Expat. E.g with Expat you define callbacks to handle when an XML-tag starts and ends:
                  Code:
                  struct XMLUSERDATA {
                      char *test;
                  };
                  
                  static void xml_end (void *udata, const XML_Char *name)
                  {
                  }
                  
                  static void xml_start (void *udata, const XML_Char *name, const XML_Char **atts)
                  {
                      struct XMLUSERDATA *userdata = udata;
                  
                      userdata->test = strdup (name); /* we allocate a string in our own code here */
                  }
                  
                  int main (int argc, char **argv)
                  {
                      struct XMLUSERDATA userdata;
                      memset (&userdata, 0, sizeof userdata);
                  
                      XML_Parser xml = XML_ParserCreate (NULL);
                  
                      XML_SetUserData (xml, &userdata);
                      XML_SetElementHandler (xml, xml_start, xml_end);
                  
                      XML_Parse (xml, argv[1], strlen (argv[1]), 1);
                      XML_ParserFree (xml);
                  
                      free (userdata.test); /* this will crash because the previous strdup() was done with the C runtime context of libexpat and not our application */
                  }
                  If this application is linked with a different C runtime than libexpat is (which e.g can happen if one of them is debug-compiled) then this code crashes hard even though both the call to strdup() and free() is done only in our own code (all form of error checking have been ignored due to visibility).
                  Last edited by F.Ultra; 04 July 2018, 02:25 PM.

                  Comment


                  • #29
                    Originally posted by moonshadow565 View Post
                    Wow, so many smart people here!
                    Can someone explain to me how Macho-o compares to elf and PE?
                    Depends on which context you are thinking about regarding differences. If we are talking about the very topic that is discussed in this thread then the executable format does not make any difference (i.e unnecessary linking is done at the linking stage and not due to the format of the file). The major difference between Mach-O and ELF is that Mach-O can store multiple binaries for different architectures in the same file (which ELF can do with an extension).

                    Comment


                    • #30
                      Originally posted by Weasel View Post
                      Such as? (give me examples of Windows libraries btw, since you used this example against Windows -- I'm not surprised if Linux libs do have this "problem" since, well, the quality of most libraries is kind of subpar)

                      Of course, in Windows it's perfectly fine for a library to ask you to use one of VirtualFree, LocalFree or GlobalFree, because those are system calls and always the same no matter the runtime or dependencies (i.e. things won't break). So those don't count.

                      Seriously? Every library worth its salt has its own free or destroy or destructor, it's just common sense practice. This is not a corner case, this is the norm. Most of the time you don't even just "wrap free", you have to do more work to begin with when destroying the library's object.

                      For example, zlib, flac, etc.

                      If you think destroying an object via an extra function call is some sort of "serious overhead" then you shouldn't be using a dynamic library in the first place, static link it instead with LTO. That's like 0.01% of the performance hit.
                      Yes libraries that originate on Windows or have a large usebase on Windows have their own free() functions, not because such a function make a library better but because they are necessary on Windows while they are completely useless on every other system (unless of course if we talk about objects and opaque handlers since their destruction can be more complex than a simple call to free()).

                      Comment

                      Working...
                      X