Backports for Julia 1.11.8 #59521
Draft
Backports for Julia 1.11.8 #59521
+210
−99
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ion for null ptr comparisons") to Julia 1.11.x (#59446) Backports #59259 to 1.11 (cherry picked from commit 920df7a). For more details, see #59445 (comment). Targets `backports-release-1.11` (#59336). --------- Co-authored-by: Gabriel Baraldi <baraldigabriel@gmail.com>
…ls (#59470) > Inspired by a bug I hit on MinGW recently with weak symbol resolution. I'm not sure why we started using these in 70000ac, since I believe we should be able to lookup these symbols just fine in our `jl_exe_handle` as long as it was linked / compiled with `-rdynamic` > > Migrating away from weak symbols seems a good idea, given their uneven implementation across platforms. (cherry picked from commit 4a6ada6) @gbaraldi asked if this could be backported separately from #59227. Co-authored-by: Cody Tapscott <84105208+topolarity@users.noreply.github.com> Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
This was originally added as a workaround for the behavior of sh to try to become the terminal process group leader, but no longer relevant since #25006 removed that flag again: ``` julia> run(detach(`bash -i -c "sleep 0"`)) bash: cannot set terminal process group (-1): Inappropriate ioctl for device bash: no job control in this shell Process(`bash -i -c 'sleep 0'`, ProcessExited(0)) ``` Long explanation: Julia recieves SIGTTOU when a process like "sh -i -c" exits (at least for bash and zsh, but not dash, ksh, or csh), since "sh -i" sometimes takes over the controlling terminal, causing julia to stop once the bash process exits. This can be quite annoying, but julia goes through some pains (in libuv) to ensure it doesn't steal the controlling terminal from the parent, and apparently bash is not so careful about it. However, since PR #25006, julia hasn't needed this workaround for this issue, so we can now follow the intended behavior when the child is in the same session group and steals the controlling terminal from the parent. Even if such behavior seems odd, this seems to be the intended behavior per posix design implementation that it gets SIGTTOU when julia later tries to change mode (from cooked -> raw after printing the prompt): http://curiousthing.org/sigttin-sigttou-deep-dive-linux. For some background on why this is a useful signal and behavior and should not be just blocked, see nodejs/node#35536. According to glibc, there's a half page of code for how to correctly implement a REPL mode change call: https://www.gnu.org/software/libc/manual/html_node/Initializing-the-Shell.html ``` $ ./julia -q shell> bash -i -c "sleep 1" [1]+ Stopped ./julia julia> run(`zsh -i -c "sleep 0"`) Process(`zsh -i -c 'sleep 0'`, ProcessExited(0)) julia> [1]+ Stopped ./julia ``` (cherry picked from commit 0cbe466)
This was in DAECompiler.jl code found by @serenity4. He also mentioned that writing up how one might go and fix a bug like this so i'll give a quick writeup (this was a very simple bug so it might not be too interesting) The original crash which looked something like > %19 = alloca [10 x i64], align 8 %155 = insertelement <4 x ptr> poison, ptr %19, i32 0 Unexpected instruction > [898844] signal 6 (-6): Aborted in expression starting at /home/gbaraldi/DAECompiler.jl/test/reflection.jl:28 pthread_kill at /lib/x86_64-linux-gnu/libc.so.6 (unknown line) gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line) abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line) RecursivelyVisit<llvm::IntrinsicInst, LateLowerGCFrame::PlaceRootsAndUpdateCalls(llvm::ArrayRef<int>, int, State&, std::map<llvm::Value*, std::pair<int, int> >)::<lambda(llvm::AllocaInst*&)>::<lambda(llvm::Use&)> > at /home/gbaraldi/julia4/src/llvm-late-gc-lowering.cpp:803 operator() at /home/gbaraldi/julia4/src/llvm-late-gc-lowering.cpp:2560 [inlined] PlaceRootsAndUpdateCalls at /home/gbaraldi/julia4/src/llvm-late-gc-lowering.cpp:2576 runOnFunction at /home/gbaraldi/julia4/src/llvm-late-gc-lowering.cpp:2638 run at /home/gbaraldi/julia4/src/llvm-late-gc-lowering.cpp:2675 run at /home/gbaraldi/julia4/usr/include/llvm/IR/PassManagerInternal.h:91 which means it was crashing inside of late-gc-lowering, so the first thing I did was ran julia and the same test with LLVM_ASSERTIONS=1 and FORCE_ASSERTIONS=1 to see if LLVM complained about a malformed module, and both were fine. Next step was trying to get the failing code out for inspection. Easiest way is to do `export JULIA_LLVM_ARGS="--print-before=LateLowerGCFrame --print-module-scope"` and pipe the output to a file. The file is huge, but since it's a crash in LLVM we know that the last thing is what we want, and that gave me the IR I wanted. To verify that this is failing I did `make -C src install-analysis-deps` to install the LLVM machinery (opt...). That gets put in the `tools` directory of a julia build. Then I checked if this crashed outside of julia by doing `./opt -load-pass-plugin=../lib/libjulia-codegen.dylib --passes=LateLowerGCFrame -S test.ll -o tmp3.ll `. This is run from inside the tools dir so your paths might vary (the -S is so LLVM doesn't generate bitcode) and my code did crash, however it was over 500 lines of IR which makes it harder to debug and to write a test. Next step then is to minimize the crash by doing [`llvm-reduce`](https://llvm.org/docs/CommandGuide/llvm-reduce.html) over it (it's basically creduce but optimized for LLVM IR) which gave me a 2 line reproducer (in this case apparently just having the insertelement was enough for the pass to fail). One thing to be wary is that llvm-reduce will usually make very weird code, so it might be useful to modify the code slightly so it doesn't look odd (it will have unreachable basic-blocks and such). After the cleanup fixing the bug here wasn't interesting but this doesn't apply generally. And also always transform your reduced IR into a test to put in llvmpasses. (cherry picked from commit 906d348)
[The "parsing" section of the "eval" dev docs page](https://docs.julialang.org/en/v1/devdocs/eval/#dev-parsing) was not updated when JuliaSyntax.jl was adopted in 1.10. This updates the text to reflect that it is the default parser and briefly mentions how to switch back. (cherry picked from commit ec27274)
The fields of `StringIndexError` are abstractly typed, so there's no reason to specialize on a concrete type. The change seems like it could prevent some invalidation on loading user code. --------- Co-authored-by: Shuhei Kadowaki <40514306+aviatesk@users.noreply.github.com> (cherry picked from commit bc33a3e)
…59329) We already save some memory here by deleting the `jl_native_code_desc_t` after we're done serializing the combined module, but some data in the module's `LLVM::Context` lives on until the end of the scope in `jl_dump_native_impl`. Once we're done with the module, move the lock and `ThreadSafeContext` so the reference count drops to zero. A crude measurement shows that when compiling the Base sysimage, about 3 GiB is in use. Deleting the `jl_native_code_desc_t` (as before) saves about 600 MiB, and cleaning up the context saves an additional ~500 MiB. (cherry picked from commit ceeb661)
(cherry picked from commit 6c02a21)
The current method :jl_type_to_llvm takes a pointer-to-bool parameter isboxed, which if non-null, is set to true if the Julia Type requires boxing. However, one may want to know the julia type without boxing and there is no mechanism for doing so. Now there is `julia_struct_to_llvm`. (cherry picked from commit 85f1b8c)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backported PRs:
using Foo: Foo, ...in package code (was: "using considered harmful") #42080Ptrvalues static-show w/ type-information #58584similarmethod forType{<:CodeUnits}#57826Need manual backport:
@time_imports@trace_compile@trace_dispatch#58011exp_impltopow#58062onefor theSizedArraytest helper #58209_aligned_msizeto prevent conflict with mingw64 defintion #58238Coremethods from newly-inferred list #58510IteratorSizemethod forGenerator#58110hygienic-scopes in inner macro expansions #58965Contains multiple commits, manual intervention needed:
structto hard #56755--project=@scriptwhen outside script directory #56351jl_type_hashis concrete evaluated) #58401Non-merged PRs with backport label:
CoreLogging: prevent someAnnotated*-related instability #59467--trace-compile#58535transcode: prevent Windows sysimage invalidation #58038displaysizeto theIOContextused by the REPL #55499isfile_casesensitivefixes on Windows #55220@inboundsandBase.@propagate_inbounds#50157