Announcement

Collapse
No announcement yet.

Ubuntu 21.04 To Turn On LTO Optimizations For Its Packages

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • ms178
    replied
    Originally posted by dirlewanger88

    Why would it work for one but not the other...?
    Support was added only for Clang, as I am not a programmer, you'd have to ask someone else for the details. From my understanding GCC and Clang LTO are different enough from each other that some more work is needed for GCC. Its not that Clang and GCC are interchangeable for everything. The Kernel just began very recently to compile with Clang at all.

    Leave a comment:


  • Zan Lynx
    replied
    Originally posted by dirlewanger88

    You have no idea what you're talking about.
    I suppose not. I've only been writing C code since 1992.

    Leave a comment:


  • flashmozzg
    replied
    Originally posted by Weasel View Post
    You completely misunderstood the point. The code doesn't segfault at runtime. The compiler segfaults during linking (since the compiler is involved during linking in LTO).

    It's a compiler bug. Yes, compilers crash too. Who would've thought?
    Sorry, quoted wrong comment. I was replying to the one above yours.

    Leave a comment:


  • Weasel
    replied
    Originally posted by flashmozzg View Post
    What is always the case is that accessing memory outside of the object the pointer points too is UB. So whether it'll lead to a SIGSEV/crash or will read some wrong value and fail later or will work correctly just by chance doesn't matter. The compiler will rightfully assume that it won't ever happen and so remove the unnecessary checks.
    You completely misunderstood the point. The code doesn't segfault at runtime. The compiler segfaults during linking (since the compiler is involved during linking in LTO).

    It's a compiler bug. Yes, compilers crash too. Who would've thought?

    Leave a comment:


  • flashmozzg
    replied
    Originally posted by PerformanceExpert View Post

    No this is not true for array access since reading beyond the bounds will almost always succeed. So the compiler cannot rely on a crash like with the NULL pointer case, and thus it can't optimise the check. In most cases a compiler does not know about array bounds as pointers don't have a bound.
    What is always the case is that accessing memory outside of the object the pointer points too is UB. So whether it'll lead to a SIGSEV/crash or will read some wrong value and fail later or will work correctly just by chance doesn't matter. The compiler will rightfully assume that it won't ever happen and so remove the unnecessary checks.
    Last edited by flashmozzg; 31 January 2021, 10:19 AM.

    Leave a comment:


  • Weasel
    replied
    Originally posted by PerformanceExpert View Post
    Do you have a link for the bugreport? I can't find actxprxy or DXVK in any GCC bug reports. Does it happen in all optimization levels?
    I didn't open a bug for Wine because I gave up, and it also requires a workaround patch to even compile in the first place.

    FYI it's not just actxprxy that's the problem. The amount of modules that segfault is huge, and yes I tested without any optimizations at all and it still segfaults (but not with -fno-lto).

    I ran a script overnight that tried to compile each module in turn, and enabling LTO for each separate translation file in turn, until it failed to compile, then marked those translation units as "bad" (i.e. causing segfaults). Here's the massive list of such translation units that had to be compiled with -fno-lto to not segfault:

    Code:
    dlls/actxprxy/actxprxy_urlhist_p
    dlls/advpack/install
    dlls/appwiz.cpl/appwiz
    dlls/avifil32/avifil32_p
    dlls/avifile.dll16/main
    dlls/bcrypt/sha512
    dlls/browseui/aclmru
    dlls/browseui/progressdlg
    dlls/cabinet/fdi
    dlls/comctl32/syslink
    dlls/commdlg.dll16/printdlg
    dlls/crtdll/math
    dlls/crtdll/string
    dlls/cryptnet/cryptnet_main
    dlls/d2d1/geometry
    dlls/d3d8/vertexdeclaration
    dlls/d3d9/vertexdeclaration
    dlls/d3drm/texture
    dlls/d3dxof/mszip
    dlls/dbgeng/dbgeng
    dlls/ddeml.dll16/ddeml
    dlls/dhcpcsvc/dhcpcsvc
    dlls/dispex/disp_ex_p
    dlls/dplayx/name_server
    dlls/dpnet/server
    dlls/dsdmo/main
    dlls/dsound/mixer
    dlls/dxva2/main
    dlls/evr/presenter
    dlls/fontsub/main
    dlls/gameux/gameexplorer
    dlls/gameux/gamestatistics
    dlls/gdi32/font
    dlls/gdi32/metafile
    dlls/glu32/quad
    dlls/hhctrl.ocx/search
    dlls/hid/hidp
    dlls/hidclass.sys/descriptor
    dlls/iccvid/iccvid
    dlls/ieframe/navigate
    dlls/imagehlp/integrity
    dlls/imm32/imm
    dlls/inetcomm/protocol
    dlls/inetcpl.cpl/general
    dlls/inetcpl.cpl/security
    dlls/infosoft/wordbreaker
    dlls/inseng/inseng_main
    dlls/itss/chm_lib
    dlls/joy.cpl/main
    dlls/jscript/number
    dlls/kernel32/process
    dlls/kernel32/sync
    dlls/kernel32/volume
    dlls/kernelbase/main
    dlls/kernelbase/memory
    dlls/kernelbase/volume
    dlls/krnl386.exe16/ioports
    dlls/mapi32/sendmail
    dlls/mciavi32/mciavi
    dlls/mciavi32/mmoutput
    dlls/mciqtz32/mciqtz
    dlls/mf/session
    dlls/mfmediaengine/main
    dlls/mfplat/mediatype
    dlls/mmsystem.dll16/mmsystem
    dlls/msacm32/internal
    dlls/msctfp/msctfp_p
    dlls/msdaps/row_server_p
    dlls/mshtml/htmlevent
    dlls/mshtml/nsio
    dlls/msi/streams
    dlls/mspatcha/lzxd_dec
    dlls/msrle32/msrle32
    dlls/mstask/task
    dlls/msvidc32/msvideo1
    dlls/ntdll/env
    dlls/ntdll/time
    dlls/ntdll/wcstring
    dlls/ntdsapi/ntdsapi
    dlls/ntoskrnl.exe/ntoskrnl
    dlls/ntprint/ntprint
    dlls/odbc32/proxyodbc
    dlls/odbccp32/odbccp32
    dlls/ole2.dll16/memlockbytes
    dlls/ole2disp.dll16/ole2disp
    dlls/ole32/datacache
    dlls/ole32/storage32
    dlls/oleaut32/vartype
    dlls/packager/packager_main
    dlls/prntvpt/main
    dlls/qcap/avimux
    dlls/qdvd/navigator
    dlls/qmgr/file
    dlls/qmgrprxy/qmgrprxy_p
    dlls/quartz/dsoundrender
    dlls/riched20/writer
    dlls/rpcrt4/rpc_transport
    dlls/rsaenh/mpi
    dlls/sapi/token
    dlls/setupapi/parser
    dlls/setupx.dll16/infparse
    dlls/shdocvw/shdocvw_main
    dlls/shlwapi/ordinal
    dlls/shlwapi/string
    dlls/shlwapi/wsprintf
    dlls/snmpapi/main
    dlls/spoolss/router
    dlls/sspicli/main
    dlls/toolhelp.dll16/toolhelp
    dlls/twain_32/dsm_ctrl
    dlls/urlmon/uri
    dlls/user32/wsprintf
    dlls/userenv/userenv_main
    dlls/uxtheme/draw
    dlls/vcomp/main
    dlls/vnbt.vxd/vnbt
    dlls/w32skrnl/w32skernel
    dlls/webservices/writer
    dlls/wer/main
    dlls/windowscodecs/palette
    dlls/winhttp/request
    dlls/wininet/http
    dlls/wininet/urlcache
    dlls/winmm/mci
    dlls/winmm/waveform
    dlls/winsock.dll16/socket
    dlls/wintab32/context
    dlls/wmp/player
    dlls/wmvcore/reader
    dlls/wsdapi/xml
    dlls/wsock32/socket
    dlls/wtsapi32/wtsapi32
    programs/expand/expand
    programs/oleview/oleview
    programs/ping/ping_main
    programs/rundll.exe16/rundll
    programs/wineboot/wineboot
    programs/winhelp.exe16/winhelp
    Note that COMPILING the above files works perfectly fine with LTO. It's during linking that it segfaults, unless the above files are compiled with -fno-lto (the others can be with -flto and it works).

    Actually there's more files btw, all d3dcompiler_* and msvc* dlls, but I gave up and disabled LTO completely on those.

    Leave a comment:


  • Zan Lynx
    replied
    Originally posted by PerformanceExpert View Post

    No this is not true for array access since reading beyond the bounds will almost always succeed. So the compiler cannot rely on a crash like with the NULL pointer case, and thus it can't optimise the check. In most cases a compiler does not know about array bounds as pointers don't have a bound.
    I had to fix a bug in my own code a while ago caused by exactly this, because I always build my C++ code with maximum O3 level optimizations, LTO and PGO when possible.

    Yes GCC will break your code if you do this. Here is the patch I had to use to fix it:

    Code:
    for(
        unsigned j=0;
    -   table[table_i].values[j] && j<category_map_count && to_i<category_array_len;
    +   j<category_map_count && table[table_i].values[j] && to_i<category_array_len;
       ++j
    ) {
    The check for j less than category_map_count must happen first. Otherwise GCC removes it because j was already used to read the table[table_i].values array.
    Last edited by Zan Lynx; 30 January 2021, 03:24 PM.

    Leave a comment:


  • PerformanceExpert
    replied
    Originally posted by Weasel View Post
    Well depends what "before" means, it won't even compile since 4.0 or 5.0 due to compilation errors (functions that are only called from assembly are not marked as "used" so LTO discards them).

    But recently, even if you fix this with a patch, GCC segfaults on some modules, like actxprxy. Wonderful. Oh, and it's a long-standing issue, since I've tried all GCC versions since version 7 (the latest stable in each), they all segfault.
    Do you have a link for the bugreport? I can't find actxprxy or DXVK in any GCC bug reports. Does it happen in all optimization levels?

    Leave a comment:


  • Weasel
    replied
    Originally posted by Vistaus View Post
    I've used Wine with LTO before, so if it doesn't work for you, then please report a bug.
    Well depends what "before" means, it won't even compile since 4.0 or 5.0 due to compilation errors (functions that are only called from assembly are not marked as "used" so LTO discards them).

    But recently, even if you fix this with a patch, GCC segfaults on some modules, like actxprxy. Wonderful. Oh, and it's a long-standing issue, since I've tried all GCC versions since version 7 (the latest stable in each), they all segfault.

    Leave a comment:


  • Weasel
    replied
    Originally posted by Zan Lynx View Post
    As with almost every compiler optimization feature, the programs that break with LTO are the ones that were already broken.

    In the past programmers hid their C undefined behavior by putting into a separate module. Then the compiler had to use the common shared ABI to call across the modules.

    With LTO that is no longer true. Watch out, if the code assumes stack or register layout, or violates strict aliasing with int and float casts, or tries to check pointers for NULL... LTO is coming for you!
    I half agree, but it's not always the case. To quote the DXVK dev, "LTO is notoriously broken" and I'm not talking about miscompiled binaries only. I'm talking about segfaults when linking (well, when LTO is applied). You can't even compile because GCC segfaults.

    I know, "report the bug". Tons of reports already. Some fixed, some not. Still segfaults 3 years later, obviously more than just one bug here.

    Leave a comment:

Working...
X