cpython: add support for experimental jit compilation #348786
Draft
+43
−23
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
the
6.topic: python
Member
Author
|
Currently stuck on this error. Also happens with |
mweinelt
force-pushed
the
python313-jit
branch
from
5b801a5 to
025a6cf
Compare
natsukium
reviewed
Oct 15, 2024
mweinelt
force-pushed
the
python313-jit
branch
from
025a6cf to
bb71c07
Compare
Member
Same error here on JIT compilation just works on Edit: This derivation is buildable on # Python 3.13.1 with JIT compiler.
with import (builtins.fetchTarball "https://github.com/NixOS/nixpkgs/tarball/890e5f24567d6d940b14241c8e05cb28129e3c59") { };
let
llvmPackages = llvmPackages_18;
in
python313.overrideAttrs (
final: prev: {
nativeBuildInputs = [
llvmPackages.libllvm # this must be prepended
llvmPackages.clang
] ++ prev.nativeBuildInputs;
env = prev.env // {
PYTHON_FOR_REGEN = lib.getExe pkgsBuildBuild.python3Minimal;
};
configureFlags = prev.configureFlags ++ [ "--enable-experimental-jit=yes-off" ];
}
) |
rennsax
reviewed
Dec 24, 2024
| @@ -161,6 +167,9 @@ let | |||
| pythonOnBuildForHost | |||
| ] ++ optionals (stdenv.cc.isClang && (!stdenv.hostPlatform.useAndroidPrebuilt or false) && (enableLTO || enableOptimizations)) [ | |||
| stdenv.cc.cc.libllvm.out | |||
Member
There was a problem hiding this comment.
To adapt for darwin system, this should be modified because it overrides llvm at the last of nativeBuildInputs. Python 3.13.1 tries to find llvm-readobj-18 and libllvm.out defaults to 16, which is not acceptable.
A possible solution is adding a condition && (!jitSupport), but this is a little ugly.
Member
Author
There was a problem hiding this comment.
I don't think we want to support this if it pins a specific llvm version. We have 19 on master for both darwin and linux in the meantime.
Member
|
I've posted an issue to CPython: python/cpython#130673 |
Member
|
This PR probably fixes our problem and it almost reaches the final stage. |
Member
Author
Picked and confirmed it builds on x86_64-linux and aarch64-darwin for Unfortunately I had no luck confirming that the JIT actually works. |
Member
CPython 3.14 will pin to LLVM 19. Since LLVM is only used for building the JIT compiler (in fact, a header file |
wegank
added
the
2.status: merge conflict
Member
|
This patch is backported (python/cpython#131749). So |
Member
Author
|
And in 3.13.3, which is on master. Unfortunately I'm currently a bit swamped. |
ofborg
bot
removed
the
2.status: merge conflict
Member
Author
|
Rebased and updated to use llvm/clang 18. Please give it a whirl, if you can. |
wegank
added
the
2.status: merge conflict
13 tasks
mweinelt
force-pushed
the
python313-jit
branch
from
5b4f6f3 to
ca75b89
Compare
Member
Author
|
Rebased. |
nixpkgs-ci
bot
added
10.rebuild-linux: 1-10
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Things done
WIP JIT support.
nix.conf? (See Nix manual)sandbox = relaxedsandbox = truenix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)Add a 👍 reaction to pull requests you find important.