FFI support

[headius]

This PR will build out support for loading the RBS library via FFI, avoiding the C extension on implementations that don't support the CRuby extension API.

[headius]

I believe these two classes are the only two defined by the C extension, correct? I am taking the approach of trying to mimic the C ext's API but implement what it does via FFI (not yet in this PR).

[headius]

I've made an attempt to start binding the FFI backend. I had to come up with my own Makefile and figured out generally how to use the API, but I'm still getting a crash when running anything through rbs_parse_signature, erroring while updating the constant pool. Seems like I'm doing something wrong, but I don't see it.

Help wanted!

-- C level backtrace information -------------------------------------------
/Users/headius/.rubies/ruby-4.0.0/bin/ruby(rb_vm_bugreport+0xbc8) [0x10431c140]
/Users/headius/.rubies/ruby-4.0.0/bin/ruby(rb_bug_for_fatal_signal+0x10c) [0x10413f380]
/Users/headius/.rubies/ruby-4.0.0/bin/ruby(sigsegv+0x90) [0x1042713a8]
/usr/lib/system/libsystem_platform.dylib(_sigtramp+0x38) [0x19d9296a4]
/Users/headius/work/rbs/build/librbs.dylib(rbs_constant_pool_insert+0x128) [0x10a67ee14]
/Users/headius/work/rbs/build/librbs.dylib(parse_simple+0xce8) [0x10a6795a8]
/Users/headius/work/rbs/build/librbs.dylib(parse_optional+0x34) [0x10a678814]
/Users/headius/work/rbs/build/librbs.dylib(parse_intersection+0x44) [0x10a67584c]
/Users/headius/work/rbs/build/librbs.dylib(rbs_parse_type+0x58) [0x10a67571c]
/Users/headius/work/rbs/build/librbs.dylib(parse_type_decl+0x138) [0x10a67aeb8]
/Users/headius/work/rbs/build/librbs.dylib(parse_module_members+0x4c4) [0x10a67d780]
/Users/headius/work/rbs/build/librbs.dylib(parse_class_decl+0x38c) [0x10a67bef0]
/Users/headius/work/rbs/build/librbs.dylib(parse_module_members+0xa6c) [0x10a67dd28]
/Users/headius/work/rbs/build/librbs.dylib(parse_module_decl+0x464) [0x10a67b920]
/Users/headius/work/rbs/build/librbs.dylib(rbs_parse_signature+0x998) [0x10a6774b0]
/Users/headius/.gem/ruby/4.0.0/gems/ffi-1.17.3-arm64-darwin/lib/4.0/ffi_c.bundle(ffi_call_SYSV+0x50) [0x10a458050]

[ParadoxV5]

erroring while updating the constant pool.

The thing in #2826?

[headius]

@ParadoxV5 I looked through that and I'm not sure if it fixes anything for me. I suspect it's something I'm not setting up or initializing properly. It fails with either CRuby or JRuby.

It can be tested pretty easily by running make shared to get a dynamic library build, commenting the code shown below, and running the code below that...

diff --git a/lib/rbs.rb b/lib/rbs.rb
index d2b04d36..4c78316f 100644
--- a/lib/rbs.rb
+++ b/lib/rbs.rb
@@ -70,8 +70,8 @@ require "rbs/type_alias_regularity"
 require "rbs/collection"
 
 begin
-  require "rbs_extension"
-rescue LoadError
+#   require "rbs_extension"
+# rescue LoadError
   require "rbs/ffi/parser"
   require "rbs/ffi/location"
 end

$ LD_LIBRARY_PATH="$LD_LIBRARY_PATH:build" ruby -I lib -r rbs -e 'result = RBS::Parser.parse_signature(File.read("sig/method_types.rbs"))'

I was trying to follow the implementation of the C extension as implement my FFI-based _parse_signature the same way, but I must be missing something.

[soutaro]

@headius I want to confirm if the code initializes the global constant pool.

rbs/ext/rbs_extension/main.c

Line 486 in 088d66a

rbs_constant_pool_init(RBS_GLOBAL_CONSTANT_POOL, num_uniquely_interned_strings);

(And as @ParadoxV5 says, I'm working to delete the global constant pool to make the parser thread safe.)

[headius]

@soutaro My reading of the code showed that rbs_parser_new calls that function for me on the constant pool it carries. Is there another constant pool I need to initialize?

Thread safety is an excellent goal! Happy to help if I can, perhaps by testing it on JRuby once I get past this issue?

[headius]

@CufeHaco I appreciate your interest but it seems that perhaps you or Claude misunderstood what I am trying to do here. I don't think voltage monitoring has anything to do with memory seg faults which is what I'm running into in the current code. I gave permission for @soutaro to hide your comments so they don't confuse anyone else finding this PR for the first time.

[headius]

@soutaro I think that might have been it! The following patch appears to fix the crash. I'm unclear why there's a constant pool on both the parser and in a C global, but I'm glad you're working to eliminate it!

diff --git a/include/rbs/util/rbs_constant_pool.h b/include/rbs/util/rbs_constant_pool.h
index 30d569f6..81ff085d 100644
--- a/include/rbs/util/rbs_constant_pool.h
+++ b/include/rbs/util/rbs_constant_pool.h
@@ -82,6 +82,7 @@ typedef struct {
 } rbs_constant_pool_t;
 
 // A global constant pool for storing permenant keywords, such as the names of location children in `parser.c`.
+RBS_EXPORTED_FUNCTION
 extern rbs_constant_pool_t *RBS_GLOBAL_CONSTANT_POOL;
 
 /**
@@ -91,6 +92,7 @@ extern rbs_constant_pool_t *RBS_GLOBAL_CONSTANT_POOL;
  * @param capacity The initial capacity of the pool.
  * @return Whether the initialization succeeded.
  */
+RBS_EXPORTED_FUNCTION
 bool rbs_constant_pool_init(rbs_constant_pool_t *pool, uint32_t capacity);
 
 /**
diff --git a/lib/rbs/ffi/parser.rb b/lib/rbs/ffi/parser.rb
index c8095472..7188af42 100644
--- a/lib/rbs/ffi/parser.rb
+++ b/lib/rbs/ffi/parser.rb
@@ -180,8 +180,12 @@ module RBS
     attach_function :rbs_parser_free, [:pointer], :void
     attach_function :rbs_parse_type, [:pointer, :pointer, :bool, :bool, :bool], :bool
     attach_function :rbs_parse_signature, [:pointer, :pointer], :bool
+    attach_function :rbs_constant_pool_init, [:pointer, :uint32], :bool
+    attach_variable :RBS_GLOBAL_CONSTANT_POOL, :pointer
   end
 
+  Native.rbs_constant_pool_init(Native.RBS_GLOBAL_CONSTANT_POOL, 7)
+
   class Parser
 
     def self.new_parser(str, start_pos, end_pos)

[headius]

Things are moving along and I've added struct definitions for most of the core types.

This may or may not be the best way to use this library from non-C-extension languages but it's a quick way to get it functional. Regardless of how the library's native types and functions are accessed, there's still quite a bit of C extension code to be reproduced in Ruby. The current logic works to parse signatures and get a list of nodes from the RBS parser.

This is running on JRuby using just FFI and the dynamic library version of RBS.

$ LD_LIBRARY_PATH="$LD_LIBRARY_PATH:build" ruby -I lib -r rbs -e 'result = RBS::Parser.parse_signature(File.read("sig/method_types.rbs")); pp result[2]'
[#<RBS::Native::NodeListNode node=#<RBS::Native::Node type=10 location=#<RBS::Native::Location rg=#<RBS::Native::Range start=#<RBS::Native::Position byte_pos=0 char_pos=0 line=1 column=0> end=#<RBS::Native::Position byte_pos=1692 char_pos=1692 line=58 column=3>> children=#<RBS::Native::LocChildren len=6 cap=6 required_p=7 entries=#<FFI::StructLayout::ArrayProxy:0x51b01960>>>> next=nil>]

[CufeHaco]

That wasn't claude. Thats all me. I know its something not widely known. This is straight out of my Electrical background dealing with Ladder Logic @headius . I just had claude write a layout. My dad was TMDE Test Measurement, and Diagnostic Equipment in the Army and National Guard. He taught me hardware.

In ladder logic (industrial control programming), you have constant memory pools that represent physical I/O states. When you have a segfault pattern like this, it reminds me of scanning issues where the controller tries to read from address space that hasn't been initialized yet, similar to how a PLC might fault when scanning rungs before the I/O table is fully mapped.

The voltage monitoring analogy I was thinking about: in electrical systems, you monitor voltage to verify proper state before switching loads. Here, the constant pool needs proper initialization/state verification before FFI can reference those memory addresses safely. The segfault pattern suggests we're accessing the pool before it's in a known good state.

Not saying the fix is voltage monitoring. I'm saying the diagnostic pattern from electrical troubleshooting applies:

verify your reference state exists before you try to use it. That's what the constant pool patch appears to do.

The actual hands on part, is we need to use multi meters to read the voltage differences between relays and PLCs. This case, the cpu and mmu. We need to know voltage differences to address the rungs correctly. Ladder logic is as close as you can get to hardware/software abstraction. This is software debugging via P=IE.