Originally posted by Zan Lynx
View Post
Announcement
Collapse
No announcement yet.
Ubuntu 21.04 To Turn On LTO Optimizations For Its Packages
Collapse
X
-
Originally posted by atomsymbol
What do you mean?
If you write C code that uses a pointer, the compiler assumes that the pointer must have been a good value. Otherwise your program would have crashed. Therefore code that checks it for NULL after using the pointer can be removed by the optimizer. It could not have been NULL because it would have crashed.
It turns out though that this is a bad assumption in a kernel. The kernel is free to read and write from a NULL pointer without crashing, and removing the check for NULL created a security hole. The kernel code was still wrong though.
This can appear in slightly different ways too. Instead of NULL it could be array bound checks. If your C code accesses the array and THEN checks if it was inside the array size the compiler can remove the array size check. Because your code already used that array index. So it must have been OK; no need to check.
- Likes 3
Comment
-
Originally posted by Zan Lynx View PostThis can appear in slightly different ways too. Instead of NULL it could be array bound checks. If your C code accesses the array and THEN checks if it was inside the array size the compiler can remove the array size check. Because your code already used that array index. So it must have been OK; no need to check.
Comment
-
Originally posted by Zan Lynx View PostAs with almost every compiler optimization feature, the programs that break with LTO are the ones that were already broken.
In the past programmers hid their C undefined behavior by putting into a separate module. Then the compiler had to use the common shared ABI to call across the modules.
With LTO that is no longer true. Watch out, if the code assumes stack or register layout, or violates strict aliasing with int and float casts, or tries to check pointers for NULL... LTO is coming for you!
I know, "report the bug". Tons of reports already. Some fixed, some not. Still segfaults 3 years later, obviously more than just one bug here.
Comment
-
Originally posted by Vistaus View PostI've used Wine with LTO before, so if it doesn't work for you, then please report a bug.
But recently, even if you fix this with a patch, GCC segfaults on some modules, like actxprxy. Wonderful. Oh, and it's a long-standing issue, since I've tried all GCC versions since version 7 (the latest stable in each), they all segfault.
Comment
-
Originally posted by Weasel View PostWell depends what "before" means, it won't even compile since 4.0 or 5.0 due to compilation errors (functions that are only called from assembly are not marked as "used" so LTO discards them).
But recently, even if you fix this with a patch, GCC segfaults on some modules, like actxprxy. Wonderful. Oh, and it's a long-standing issue, since I've tried all GCC versions since version 7 (the latest stable in each), they all segfault.
- Likes 1
Comment
-
Originally posted by PerformanceExpert View Post
No this is not true for array access since reading beyond the bounds will almost always succeed. So the compiler cannot rely on a crash like with the NULL pointer case, and thus it can't optimise the check. In most cases a compiler does not know about array bounds as pointers don't have a bound.
Yes GCC will break your code if you do this. Here is the patch I had to use to fix it:
Code:for( unsigned j=0; - table[table_i].values[j] && j<category_map_count && to_i<category_array_len; + j<category_map_count && table[table_i].values[j] && to_i<category_array_len; ++j ) {
Last edited by Zan Lynx; 30 January 2021, 03:24 PM.
- Likes 1
Comment
-
Originally posted by PerformanceExpert View PostDo you have a link for the bugreport? I can't find actxprxy or DXVK in any GCC bug reports. Does it happen in all optimization levels?
FYI it's not just actxprxy that's the problem. The amount of modules that segfault is huge, and yes I tested without any optimizations at all and it still segfaults (but not with -fno-lto).
I ran a script overnight that tried to compile each module in turn, and enabling LTO for each separate translation file in turn, until it failed to compile, then marked those translation units as "bad" (i.e. causing segfaults). Here's the massive list of such translation units that had to be compiled with -fno-lto to not segfault:
Code:dlls/actxprxy/actxprxy_urlhist_p dlls/advpack/install dlls/appwiz.cpl/appwiz dlls/avifil32/avifil32_p dlls/avifile.dll16/main dlls/bcrypt/sha512 dlls/browseui/aclmru dlls/browseui/progressdlg dlls/cabinet/fdi dlls/comctl32/syslink dlls/commdlg.dll16/printdlg dlls/crtdll/math dlls/crtdll/string dlls/cryptnet/cryptnet_main dlls/d2d1/geometry dlls/d3d8/vertexdeclaration dlls/d3d9/vertexdeclaration dlls/d3drm/texture dlls/d3dxof/mszip dlls/dbgeng/dbgeng dlls/ddeml.dll16/ddeml dlls/dhcpcsvc/dhcpcsvc dlls/dispex/disp_ex_p dlls/dplayx/name_server dlls/dpnet/server dlls/dsdmo/main dlls/dsound/mixer dlls/dxva2/main dlls/evr/presenter dlls/fontsub/main dlls/gameux/gameexplorer dlls/gameux/gamestatistics dlls/gdi32/font dlls/gdi32/metafile dlls/glu32/quad dlls/hhctrl.ocx/search dlls/hid/hidp dlls/hidclass.sys/descriptor dlls/iccvid/iccvid dlls/ieframe/navigate dlls/imagehlp/integrity dlls/imm32/imm dlls/inetcomm/protocol dlls/inetcpl.cpl/general dlls/inetcpl.cpl/security dlls/infosoft/wordbreaker dlls/inseng/inseng_main dlls/itss/chm_lib dlls/joy.cpl/main dlls/jscript/number dlls/kernel32/process dlls/kernel32/sync dlls/kernel32/volume dlls/kernelbase/main dlls/kernelbase/memory dlls/kernelbase/volume dlls/krnl386.exe16/ioports dlls/mapi32/sendmail dlls/mciavi32/mciavi dlls/mciavi32/mmoutput dlls/mciqtz32/mciqtz dlls/mf/session dlls/mfmediaengine/main dlls/mfplat/mediatype dlls/mmsystem.dll16/mmsystem dlls/msacm32/internal dlls/msctfp/msctfp_p dlls/msdaps/row_server_p dlls/mshtml/htmlevent dlls/mshtml/nsio dlls/msi/streams dlls/mspatcha/lzxd_dec dlls/msrle32/msrle32 dlls/mstask/task dlls/msvidc32/msvideo1 dlls/ntdll/env dlls/ntdll/time dlls/ntdll/wcstring dlls/ntdsapi/ntdsapi dlls/ntoskrnl.exe/ntoskrnl dlls/ntprint/ntprint dlls/odbc32/proxyodbc dlls/odbccp32/odbccp32 dlls/ole2.dll16/memlockbytes dlls/ole2disp.dll16/ole2disp dlls/ole32/datacache dlls/ole32/storage32 dlls/oleaut32/vartype dlls/packager/packager_main dlls/prntvpt/main dlls/qcap/avimux dlls/qdvd/navigator dlls/qmgr/file dlls/qmgrprxy/qmgrprxy_p dlls/quartz/dsoundrender dlls/riched20/writer dlls/rpcrt4/rpc_transport dlls/rsaenh/mpi dlls/sapi/token dlls/setupapi/parser dlls/setupx.dll16/infparse dlls/shdocvw/shdocvw_main dlls/shlwapi/ordinal dlls/shlwapi/string dlls/shlwapi/wsprintf dlls/snmpapi/main dlls/spoolss/router dlls/sspicli/main dlls/toolhelp.dll16/toolhelp dlls/twain_32/dsm_ctrl dlls/urlmon/uri dlls/user32/wsprintf dlls/userenv/userenv_main dlls/uxtheme/draw dlls/vcomp/main dlls/vnbt.vxd/vnbt dlls/w32skrnl/w32skernel dlls/webservices/writer dlls/wer/main dlls/windowscodecs/palette dlls/winhttp/request dlls/wininet/http dlls/wininet/urlcache dlls/winmm/mci dlls/winmm/waveform dlls/winsock.dll16/socket dlls/wintab32/context dlls/wmp/player dlls/wmvcore/reader dlls/wsdapi/xml dlls/wsock32/socket dlls/wtsapi32/wtsapi32 programs/expand/expand programs/oleview/oleview programs/ping/ping_main programs/rundll.exe16/rundll programs/wineboot/wineboot programs/winhelp.exe16/winhelp
Actually there's more files btw, all d3dcompiler_* and msvc* dlls, but I gave up and disabled LTO completely on those.
Comment
-
Originally posted by PerformanceExpert View Post
No this is not true for array access since reading beyond the bounds will almost always succeed. So the compiler cannot rely on a crash like with the NULL pointer case, and thus it can't optimise the check. In most cases a compiler does not know about array bounds as pointers don't have a bound.Last edited by flashmozzg; 31 January 2021, 10:19 AM.
Comment
-
Originally posted by flashmozzg View PostWhat is always the case is that accessing memory outside of the object the pointer points too is UB. So whether it'll lead to a SIGSEV/crash or will read some wrong value and fail later or will work correctly just by chance doesn't matter. The compiler will rightfully assume that it won't ever happen and so remove the unnecessary checks.
It's a compiler bug. Yes, compilers crash too. Who would've thought?
Comment
Comment