Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ The following bundled gems are added.

The following bundled gems are updated.

* minitest 5.26.1
* minitest 5.26.2
* power_assert 3.0.1
* rake 13.3.1
* test-unit 3.7.1
Expand Down
163 changes: 163 additions & 0 deletions doc/contributing/vm_stack_and_frames.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
# Ruby VM Stack and Frame Layout

This document explains the Ruby VM stack architecture, including how the value
stack (SP) and control frames (CFP) share a single contiguous memory region,
and how individual frames are structured.

## VM Stack Architecture

The Ruby VM uses a single contiguous stack (`ec->vm_stack`) with two different
regions growing toward each other. Understanding this requires distinguishing
the overall architecture (how CFPs and values share one stack) from individual
frame internals (how values are organized for one single frame).

```text
High addresses (ec->vm_stack + ec->vm_stack_size)
[CFP region starts here] ← RUBY_VM_END_CONTROL_FRAME(ec)
[CFP - 1] New frame pushed here (grows downward)
[CFP - 2] Another frame
...

(Unused space - stack overflow when they meet)

... Value stack grows UP toward higher addresses
[SP + n] Values pushed here
[ec->cfp->sp] Current executing frame's stack pointer
Low addresses (ec->vm_stack)
```

The "unused space" represents free space available for new frames and values. When this gap closes (CFP meets SP), stack overflow occurs.

### Stack Growth Directions

**Control Frames (CFP):**

- Start at `ec->vm_stack + ec->vm_stack_size` (high addresses)
- Grow **downward** toward lower addresses as frames are pushed
- Each new frame is allocated at `cfp - 1` (lower address)
- The `rb_control_frame_t` structure itself moves downward

**Value Stack (SP):**

- Starts at `ec->vm_stack` (low addresses)
- Grows **upward** toward higher addresses as values are pushed
- Each frame's `cfp->sp` points to the top of its value stack

### Stack Overflow

When recursive calls push too many frames, CFP grows downward until it collides
with SP growing upward. The VM detects this with `CHECK_VM_STACK_OVERFLOW0`,
which computes `const rb_control_frame_struct *bound = (void *)&sp[margin];`
and raises if `cfp <= &bound[1]`.

## Understanding Individual Frame Value Stacks

Each frame has its own portion of the overall VM stack, called its "VM value stack"
or simply "value stack". This space is pre-allocated when the frame is created,
with size determined by:

- `local_size` - space for local variables
- `stack_max` - maximum depth for temporary values during execution

The frame's value stack grows upward from its base (where self/arguments/locals
live) toward `cfp->sp` (the current top of temporary values).

## Visualizing How Frames Fit in the VM Stack

The left side shows the overall VM stack with CFP metadata separated from frame
values. The right side zooms into one frame's value region, revealing its internal
structure.

```text
Overall VM Stack (ec->vm_stack): Zooming into Frame 2's value stack:

High addr (vm_stack + vm_stack_size) High addr (cfp->sp)
↓ ┌
[CFP 1 metadata] │ [Temporaries]
[CFP 2 metadata] ─────────┐ │ [Env: Flags/Block/CME] ← cfp->ep
[CFP 3 metadata] │ │ [Locals]
──────────────── │ ┌─┤ [Arguments]
(unused space) │ │ │ [self]
──────────────── │ │ └
[Frame 3 values] │ │ Low addr (frame base)
[Frame 2 values] <────────┴───────┘
[Frame 1 values]
Low addr (vm_stack)
```

## Examining a Single Frame's Value Stack

Now let's walk through a concrete Ruby program to see how a single frame's
value stack is structured internally:

```ruby
def foo(x, y)
z = x.casecmp(y)
end

foo(:one, :two)
```

First, after arguments are evaluated and right before the `send` to `foo`:

```text
┌────────────┐
putself │ :two │
putobject :one 0x2 ├────────────┤
putobject :two │ :one │
► send <:foo, argc:2> 0x1 ├────────────┤
leave │ self │
0x0 └────────────┘
```

The `put*` instructions have pushed 3 items onto the stack. It's now time to
add a new control frame for `foo`. The following is the shape of the stack
after one instruction in `foo`:

```text
cfp->sp=0x8 at this point.
0x8 ┌────────────┐◄──Stack space for temporaries
│ :one │ live above the environment.
0x7 ├────────────┤
getlocal x@0 │ < flags > │ foo's rb_control_frame_t
► getlocal y@1 0x6 ├────────────┤◄──has cfp->ep=0x6
send <:casecmp, argc:1> │ <no block> │
dup 0x5 ├────────────┤ The flags, block, and CME triple
setlocal z@2 │ <CME: foo> │ (VM_ENV_DATA_SIZE) form an
leave 0x4 ├────────────┤ environment. They can be used to
│ z (nil) │ figure out what local variables
0x3 ├────────────┤ are below them.
│ :two │
0x2 ├────────────┤ Notice how the arguments, now
│ :one │ locals, never moved. This layout
0x1 ├────────────┤ allows for argument transfer
│ self │ without copying.
0x0 └────────────┘
```

Given that locals have lower address than `cfp->ep`, it makes sense then that
`getlocal` in `insns.def` has `val = *(vm_get_ep(GET_EP(), level) - idx);`.
When accessing variables in the immediate scope, where `level=0`, it's
essentially `val = cfp->ep[-idx];`.

Note that this EP-relative index has a different basis than the index that comes
after "@" in disassembly listings. The "@" index is relative to the 0th local
(`x` in this case).

### Q&A

Q: It seems that the receiver is always at an offset relative to EP,
like locals. Couldn't we use EP to access it instead of using `cfp->self`?

A: Not all calls put the `self` in the callee on the stack. Two
examples are `Proc#call`, where the receiver is the Proc object, but `self`
inside the callee is `Proc#receiver`, and `yield`, where the receiver isn't
pushed onto the stack before the arguments.

Q: Why have `cfp->ep` when it seems that everything is below `cfp->sp`?

A: In the example, `cfp->ep` points to the stack, but it can also point to the
GC heap. Blocks can capture and evacuate their environment to the heap.
77 changes: 0 additions & 77 deletions doc/yarv_frame_layout.md

This file was deleted.

2 changes: 2 additions & 0 deletions doc/zjit.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,8 @@ Ruby execution involves three distinct stacks and understanding them will help y

The Ruby VM uses a single contiguous memory region (`ec->vm_stack`) containing two sub-stacks that grow toward each other. When they meet, stack overflow occurs.

See [doc/contributing/vm_stack_and_frames.md](contributing/vm_stack_and_frames.md) for detailed architecture and frame layout.

**Control Frame Stack:**

- **Stores**: Frame metadata (`rb_control_frame_t` structures)
Expand Down
12 changes: 6 additions & 6 deletions ext/json/fbuffer/fbuffer.h
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ typedef struct FBufferStruct {
unsigned long initial_length;
unsigned long len;
unsigned long capa;
#ifdef JSON_DEBUG
#if JSON_DEBUG
unsigned long requested;
#endif
char *ptr;
Expand Down Expand Up @@ -45,14 +45,14 @@ static void fbuffer_stack_init(FBuffer *fb, unsigned long initial_length, char *
fb->ptr = stack_buffer;
fb->capa = stack_buffer_size;
}
#ifdef JSON_DEBUG
#if JSON_DEBUG
fb->requested = 0;
#endif
}

static inline void fbuffer_consumed(FBuffer *fb, unsigned long consumed)
{
#ifdef JSON_DEBUG
#if JSON_DEBUG
if (consumed > fb->requested) {
rb_bug("fbuffer: Out of bound write");
}
Expand Down Expand Up @@ -122,7 +122,7 @@ static void fbuffer_do_inc_capa(FBuffer *fb, unsigned long requested)

static inline void fbuffer_inc_capa(FBuffer *fb, unsigned long requested)
{
#ifdef JSON_DEBUG
#if JSON_DEBUG
fb->requested = requested;
#endif

Expand All @@ -148,7 +148,7 @@ static inline void fbuffer_append(FBuffer *fb, const char *newstr, unsigned long
/* Appends a character into a buffer. The buffer needs to have sufficient capacity, via fbuffer_inc_capa(...). */
static inline void fbuffer_append_reserved_char(FBuffer *fb, char chr)
{
#ifdef JSON_DEBUG
#if JSON_DEBUG
if (fb->requested < 1) {
rb_bug("fbuffer: unreserved write");
}
Expand All @@ -174,7 +174,7 @@ static void fbuffer_append_str_repeat(FBuffer *fb, VALUE str, size_t repeat)

fbuffer_inc_capa(fb, repeat * len);
while (repeat) {
#ifdef JSON_DEBUG
#if JSON_DEBUG
fb->requested = len;
#endif
fbuffer_append_reserved(fb, newstr, len);
Expand Down
2 changes: 1 addition & 1 deletion ext/json/generator/extconf.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
else
append_cflags("-std=c99")
$defs << "-DJSON_GENERATOR"
$defs << "-DJSON_DEBUG" if ENV["JSON_DEBUG"]
$defs << "-DJSON_DEBUG" if ENV.fetch("JSON_DEBUG", "0") != "0"

if enable_config('generator-use-simd', default=!ENV["JSON_DISABLE_SIMD"])
load __dir__ + "/../simd/conf.rb"
Expand Down
2 changes: 1 addition & 1 deletion ext/json/parser/extconf.rb
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# frozen_string_literal: true
require 'mkmf'

$defs << "-DJSON_DEBUG" if ENV["JSON_DEBUG"]
$defs << "-DJSON_DEBUG" if ENV.fetch("JSON_DEBUG", "0") != "0"
have_func("rb_enc_interned_str", "ruby/encoding.h") # RUBY_VERSION >= 3.0
have_func("rb_str_to_interned_str", "ruby.h") # RUBY_VERSION >= 3.0
have_func("rb_hash_new_capa", "ruby.h") # RUBY_VERSION >= 3.2
Expand Down
4 changes: 2 additions & 2 deletions ext/json/vendor/fpconv.c
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
#include <string.h>
#include <stdint.h>

#ifdef JSON_DEBUG
#if JSON_DEBUG
#include <assert.h>
#endif

Expand Down Expand Up @@ -472,7 +472,7 @@ static int fpconv_dtoa(double d, char dest[28])
int ndigits = grisu2(d, digits, &K);

str_len += emit_digits(digits, ndigits, dest + str_len, K, neg);
#ifdef JSON_DEBUG
#if JSON_DEBUG
assert(str_len <= 32);
#endif

Expand Down
2 changes: 1 addition & 1 deletion gems/bundled_gems
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
# - revision: revision in repository-url to test
# if `revision` is not given, "v"+`version` or `version` will be used.

minitest 5.26.1 https://github.com/minitest/minitest
minitest 5.26.2 https://github.com/minitest/minitest
power_assert 3.0.1 https://github.com/ruby/power_assert
rake 13.3.1 https://github.com/ruby/rake
test-unit 3.7.1 https://github.com/test-unit/test-unit
Expand Down
23 changes: 20 additions & 3 deletions lib/bundler/cli/common.rb
Original file line number Diff line number Diff line change
Expand Up @@ -94,11 +94,14 @@ def self.ask_for_spec_from(specs)
end

def self.gem_not_found_message(missing_gem_name, alternatives)
require_relative "../similarity_detector"
message = "Could not find gem '#{missing_gem_name}'."
alternate_names = alternatives.map {|a| a.respond_to?(:name) ? a.name : a }
suggestions = SimilarityDetector.new(alternate_names).similar_word_list(missing_gem_name)
message += "\nDid you mean #{suggestions}?" if suggestions
if alternate_names.include?(missing_gem_name.downcase)
message += "\nDid you mean '#{missing_gem_name.downcase}'?"
elsif defined?(DidYouMean::SpellChecker)
suggestions = DidYouMean::SpellChecker.new(dictionary: alternate_names).correct(missing_gem_name)
message += "\nDid you mean #{word_list(suggestions)}?" unless suggestions.empty?
end
message
end

Expand Down Expand Up @@ -134,5 +137,19 @@ def self.clean_after_install?
clean &&= !Bundler.use_system_gems?
clean
end

def self.word_list(words)
if words.empty?
return ""
end

words = words.map {|word| "'#{word}'" }

if words.length == 1
return words[0]
end

[words[0..-2].join(", "), words[-1]].join(" or ")
end
end
end
Loading