Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ permissions:
contents: read

jobs:
update_default_gems:
update_default_gems_list:
name: Update default gems list

permissions:
Expand Down
65 changes: 65 additions & 0 deletions .github/workflows/sync_default_gems.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: Sync default gems
on:
workflow_dispatch:
inputs:
gem:
required: true
description: 'Name of the gem to be synchronized'
type: string
before:
required: true
description: 'Gem commit SHA before sync'
type: string
after:
required: true
description: 'Gem commit SHA after sync'
type: string

jobs:
sync_default_gems:
name: Sync default gems

permissions:
contents: write # for Git to git push

runs-on: ubuntu-latest

if: ${{ github.repository == 'ruby/ruby' }}

steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
name: Check out ruby/ruby
with:
fetch-depth: 999999 # Fetch all history to follow past renames. Not using 0 to avoid fetching tags/branches.
token: ${{ github.repository == 'ruby/ruby' && secrets.MATZBOT_AUTO_UPDATE_TOKEN || secrets.GITHUB_TOKEN }}

- name: Increase rename limit
run: git config merge.renameLimit 999999

- name: Run tool/sync_default_gems.rb
id: sync
run: |
ruby_before=$(git rev-parse HEAD)
set -x
ruby tool/sync_default_gems.rb "${gem_name}" "${gem_before}..${gem_after}"
if [[ "$(git rev-parse HEAD)" != "$ruby_before" ]]; then
echo update=true >> $GITHUB_OUTPUT
fi
env:
gem_name: ${{ github.event.inputs.gem }}
gem_before: ${{ github.event.inputs.before }}
gem_after: ${{ github.event.inputs.after }}
EMAIL: svn-admin@ruby-lang.org
GIT_AUTHOR_NAME: git
GIT_COMMITTER_NAME: git

- name: Push
run: |
git pull --ff-only origin ${GITHUB_REF#refs/heads/}
git push origin ${GITHUB_REF#refs/heads/}
if: ${{ steps.sync.outputs.update }}

- uses: ./.github/actions/slack
with:
SLACK_WEBHOOK_URL: ${{ secrets.SIMPLER_ALERTS_URL }} # ruby-lang slack: ruby/simpler-alerts-bot
if: ${{ failure() }}
2 changes: 1 addition & 1 deletion NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,7 @@ The following default gems are updated.

* RubyGems 3.8.0.dev
* bundler 2.8.0.dev
* erb 5.0.2
* erb 5.0.3
* etc 1.4.6
* fcntl 1.3.0
* io-console 0.8.1
Expand Down
160 changes: 83 additions & 77 deletions doc/zjit.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,113 +3,102 @@
## Build Instructions

To build ZJIT on macOS:
```

```bash
./autogen.sh
./configure --enable-zjit=dev --prefix=$HOME/.rubies/ruby-zjit --disable-install-doc --with-opt-dir="$(brew --prefix openssl):$(brew --prefix readline):$(brew --prefix libyaml)"
make -j miniruby
```

## Useful dev commands
./configure \
--enable-zjit=dev \
--prefix="$HOME"/.rubies/ruby-zjit \
--disable-install-doc \
--with-opt-dir="$(brew --prefix openssl):$(brew --prefix readline):$(brew --prefix libyaml)"

To view YARV output for code snippets:
```
./miniruby --dump=insns -e0
```

To run code snippets with ZJIT:
```
./miniruby --zjit -e0
make -j miniruby
```

You can also try https://www.rubyexplorer.xyz/ to view Ruby YARV disasm output with syntax highlighting
in a way that can be easily shared with other team members.

## Documentation

You can generate and open the source level documentation in your browser using `cargo doc --open --document-private-items`.
You can generate and open the source level documentation in your browser using:

```bash
cargo doc --document-private-items -p zjit --open
```

## Testing

Make sure you have a `--enable-zjit=dev` build, and install the following tools:
- `brew install cargo-nextest` - Required for running tests
- `cargo install cargo-insta` - Required for updating snapshots
Note that tests link against CRuby, so directly calling `cargo test`, or `cargo nextest` should not build. All tests are instead accessed through `make`.

### make zjit-check
### Setup

This command runs all ZJIT tests: `make zjit-test` and `test/ruby/test_zjit.rb`.
First, ensure you have `cargo` installed. If you do not already have it, you can use [rustup.rs](https://rustup.rs/).

```
make zjit-check
Make sure to add `--enable-zjit=dev` when you run `configure`, then install the following tools:

```bash
cargo install cargo-nextest
cargo install cargo-insta
```

### make zjit-test
`cargo-insta` is used for updating snapshots. `cargo-nextest` runs each test in its own process, which is valuable since CRuby only supports booting once per process, and most APIs are not thread safe.

This command runs Rust unit tests using `insta` for snapshot testing.
### Running unit tests

```
For testing functionality within ZJIT, use:

```bash
make zjit-test
```

You can also run a single test case by specifying the function name:

```
```bash
make zjit-test ZJIT_TESTS=test_putobject
```

#### Snapshot Testing

ZJIT uses [insta](https://insta.rs/) for snapshot testing. When tests fail due to snapshot mismatches, pending snapshots are created. The test command will notify you if there are pending snapshots:
ZJIT uses [insta](https://insta.rs/) for snapshot testing within unit tests. When tests fail due to snapshot mismatches, pending snapshots are created. The test command will notify you if there are pending snapshots:

```
Pending snapshots found. Accept with: make zjit-test-update
```

To update/accept all the snapshot changes:

```
```bash
make zjit-test-update
```

You can also review snapshot changes interactively one by one:

```
```bash
cd zjit && cargo insta review
```

Test changes will be reviewed alongside code changes.

<details>

<summary>Setting up zjit-test</summary>

ZJIT uses `cargo-nextest` for Rust unit tests instead of `cargo test`.
`cargo-nextest` runs each test in its own process, which is valuable since
CRuby only supports booting once per process, and most APIs are not thread
safe. Use `brew install cargo-nextest` to install it on macOS, otherwise, refer
to <https://nexte.st/docs/installation/pre-built-binaries/> for installation
instructions.

Since it uses Cargo, you'll also need a `configure --enable-zjit=dev ...` build
for `make zjit-test`. Since the tests need to link against CRuby, directly
calling `cargo test`, or `cargo nextest` likely won't build. Make sure to
use `make`.

</details>

### test/ruby/test\_zjit.rb
### Running integration tests

This command runs Ruby execution tests.

```
```bash
make test-all TESTS="test/ruby/test_zjit.rb"
```

You can also run a single test case by matching the method name:

```
```bash
make test-all TESTS="test/ruby/test_zjit.rb -n TestZJIT#test_putobject"
```

### Running all tests

Runs both `make zjit-test` and `test/ruby/test_zjit.rb`:

```bash
make zjit-check
```

## Statistics Collection

ZJIT provides detailed statistics about JIT compilation and execution behavior.
Expand Down Expand Up @@ -161,38 +150,55 @@ Through [Stackprof](https://github.com/tmm1/stackprof), detailed information abo
./miniruby --zjit-trace-exits script.rb
```

A file called `zjit_exit_locations.dump` will be created in the same directory as `script.rb`. Viewing the side exited methods can be done with Stackprof:
A file called `zjit_exit_locations{timestamp}.dump` will be created in the same directory as `script.rb`. Viewing the side exited methods can be done with Stackprof:

```bash
stackprof path/to/zjit_exit_locations.dump
stackprof path/to/zjit_exit_locations{timestamp}.dump
```

## Useful dev commands

To view YARV output for code snippets:

```bash
./miniruby --dump=insns -e0
```

To run code snippets with ZJIT:

```bash
./miniruby --zjit -e0
```

You can also try https://www.rubyexplorer.xyz/ to view Ruby YARV disasm output with syntax highlighting
in a way that can be easily shared with other team members.

## ZJIT Glossary

This glossary contains terms that are helpful for understanding ZJIT.

Please note that some terms may appear in CRuby internals too but with different meanings.

| Term | Definition |
| --- | -----------|
| HIR | High-level Intermediate Representation. High-level (Ruby semantics) graph representation in static single-assignment (SSA) form |
| LIR | Low-level Intermediate Representation. Low-level IR used in the backend for assembly generation |
| SSA | Static Single Assignment. A form where each variable is assigned exactly once |
| `opnd` | Operand. An operand to an IR instruction (can be register, memory, immediate, etc.) |
| `dst` | Destination. The output operand of an instruction where the result is stored |
| VReg | Virtual Register. A virtual register that gets lowered to physical register or memory |
| `insn_id` | Instruction ID. An index of an instruction in a function |
| `block_id` | The index of a basic block, which effectively acts like a pointer |
| `branch` | Control flow edge between basic blocks in the compiled code |
| `cb` | Code Block. Memory region for generated machine code |
| `entry` | The starting address of compiled code for an ISEQ |
| Patch Point | Location in generated code that can be modified later in case assumptions get invalidated |
| Frame State | Captured state of the Ruby stack frame at a specific point for deoptimization |
| Guard | A run-time check that ensures assumptions are still valid |
| `invariant` | An assumption that JIT code relies on, requiring invalidation if broken |
| Deopt | Deoptimization. Process of falling back from JIT code to interpreter |
| Side Exit | Exit from JIT code back to interpreter |
| Type Lattice | Hierarchy of types used for type inference and optimization |
| Constant Folding | Optimization that evaluates constant expressions at compile time |
| RSP | x86-64 stack pointer register used for native stack operations |
| Register Spilling | Process of moving register values to memory when running out of physical registers |
| Term | Definition |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| HIR | High-level Intermediate Representation. High-level (Ruby semantics) graph representation in static single-assignment (SSA) form |
| LIR | Low-level Intermediate Representation. Low-level IR used in the backend for assembly generation |
| SSA | Static Single Assignment. A form where each variable is assigned exactly once |
| `opnd` | Operand. An operand to an IR instruction (can be register, memory, immediate, etc.) |
| `dst` | Destination. The output operand of an instruction where the result is stored |
| VReg | Virtual Register. A virtual register that gets lowered to physical register or memory |
| `insn_id` | Instruction ID. An index of an instruction in a function |
| `block_id` | The index of a basic block, which effectively acts like a pointer |
| `branch` | Control flow edge between basic blocks in the compiled code |
| `cb` | Code Block. Memory region for generated machine code |
| `entry` | The starting address of compiled code for an ISEQ |
| Patch Point | Location in generated code that can be modified later in case assumptions get invalidated |
| Frame State | Captured state of the Ruby stack frame at a specific point for deoptimization |
| Guard | A run-time check that ensures assumptions are still valid |
| `invariant` | An assumption that JIT code relies on, requiring invalidation if broken |
| Deopt | Deoptimization. Process of falling back from JIT code to interpreter |
| Side Exit | Exit from JIT code back to interpreter |
| Type Lattice | Hierarchy of types used for type inference and optimization |
| Constant Folding | Optimization that evaluates constant expressions at compile time |
| RSP | x86-64 stack pointer register used for native stack operations |
| Register Spilling | Process of moving register values to memory when running out of physical registers |
2 changes: 1 addition & 1 deletion lib/erb/version.rb
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# frozen_string_literal: true
class ERB
# The string \ERB version.
VERSION = '5.0.2'
VERSION = '5.0.3'
end
3 changes: 2 additions & 1 deletion prism/prism.c
Original file line number Diff line number Diff line change
Expand Up @@ -22524,9 +22524,10 @@ parse_program(pm_parser_t *parser) {
statements = wrap_statements(parser, statements);
} else {
flush_block_exits(parser, previous_block_exits);
pm_node_list_free(&current_block_exits);
}

pm_node_list_free(&current_block_exits);

// If this is an empty file, then we're still going to parse all of the
// statements in order to gather up all of the comments and such. Here we'll
// correct the location information.
Expand Down
5 changes: 5 additions & 0 deletions test/prism/fixtures/string_concatination_frozen_false.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# frozen_string_literal: false

'foo' 'bar'

'foo' 'bar' "baz#{bat}"
5 changes: 5 additions & 0 deletions test/prism/fixtures/string_concatination_frozen_true.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# frozen_string_literal: true

'foo' 'bar'

'foo' 'bar' "baz#{bat}"
1 change: 0 additions & 1 deletion test/ruby/test_thread.rb
Original file line number Diff line number Diff line change
Expand Up @@ -1592,7 +1592,6 @@ def frame_for_deadlock_test_2

# [Bug #21342]
def test_unlock_locked_mutex_with_collected_fiber
bug21127 = '[ruby-core:120930] [Bug #21127]'
assert_separately([], "#{<<~"begin;"}\n#{<<~'end;'}")
begin;
5.times do
Expand Down