Skip to content

Conversation

@kou
Copy link
Member

@kou kou commented Jan 18, 2026

What was the end-user or developer problem that led to this PR?

I can't reproduce this again but I got the following error when I install many gems at once:

Fetching pg-1.6.3.gem
#<Thread:0x00007ff77fe16210 /var/tmp/local/lib/ruby/4.1.0+1/rubygems/request_set.rb:168 run> terminated with exception (report_on_exception is true):
/var/tmp/local/lib/ruby/4.1.0+1/rubygems.rb:841:in 'IO#write': "\x8B" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)
	from /var/tmp/local/lib/ruby/4.1.0+1/rubygems.rb:841:in 'block in Gem.write_binary'
	from /var/tmp/local/lib/ruby/4.1.0+1/rubygems/util/atomic_file_writer.rb:44:in 'block in Gem::AtomicFileWriter.open'
	from /var/tmp/local/lib/ruby/4.1.0+1/rubygems/util/atomic_file_writer.rb:32:in 'IO.open'
	from /var/tmp/local/lib/ruby/4.1.0+1/rubygems/util/atomic_file_writer.rb:32:in 'Gem::AtomicFileWriter.open'
	from /var/tmp/local/lib/ruby/4.1.0+1/rubygems.rb:840:in 'Gem.write_binary'
	from /var/tmp/local/lib/ruby/4.1.0+1/rubygems/remote_fetcher.rb:304:in 'Gem::RemoteFetcher#cache_update_path'
	from /var/tmp/local/lib/ruby/4.1.0+1/rubygems/remote_fetcher.rb:151:in 'Gem::RemoteFetcher#download'
	from /var/tmp/local/lib/ruby/4.1.0+1/rubygems/source.rb:195:in 'Gem::Source#download'
	from /var/tmp/local/lib/ruby/4.1.0+1/rubygems/resolver/specification.rb:113:in 'Gem::Resolver::Specification#download'
	from /var/tmp/local/lib/ruby/4.1.0+1/rubygems/request_set.rb:174:in 'block (2 levels) in Gem::RequestSet#install'
ERROR:  While executing gem ... (Encoding::UndefinedConversionError)
    "\x8B" from ASCII-8BIT to UTF-8
	/var/tmp/local/lib/ruby/4.1.0+1/rubygems.rb:841:in 'IO#write'
	/var/tmp/local/lib/ruby/4.1.0+1/rubygems.rb:841:in 'block in Gem.write_binary'
	/var/tmp/local/lib/ruby/4.1.0+1/rubygems/util/atomic_file_writer.rb:44:in 'block in Gem::AtomicFileWriter.open'
	/var/tmp/local/lib/ruby/4.1.0+1/rubygems/util/atomic_file_writer.rb:32:in 'IO.open'
	/var/tmp/local/lib/ruby/4.1.0+1/rubygems/util/atomic_file_writer.rb:32:in 'Gem::AtomicFileWriter.open'
	/var/tmp/local/lib/ruby/4.1.0+1/rubygems.rb:840:in 'Gem.write_binary'
	/var/tmp/local/lib/ruby/4.1.0+1/rubygems/remote_fetcher.rb:304:in 'Gem::RemoteFetcher#cache_update_path'
	/var/tmp/local/lib/ruby/4.1.0+1/rubygems/remote_fetcher.rb:151:in 'Gem::RemoteFetcher#download'
	/var/tmp/local/lib/ruby/4.1.0+1/rubygems/source.rb:195:in 'Gem::Source#download'
	/var/tmp/local/lib/ruby/4.1.0+1/rubygems/resolver/specification.rb:113:in 'Gem::Resolver::Specification#download'
	/var/tmp/local/lib/ruby/4.1.0+1/rubygems/request_set.rb:174:in 'block (2 levels) in Gem::RequestSet#install'

I think that it's not related to pg gem. We can reproduce similar error by the following script:

require "fileutils"
path = "/tmp/abc.txt"
FileUtils.rm_f(path)
flags = File::RDWR|File::CREAT|File::EXCL|File::BINARY
Encoding.default_internal = Encoding::UTF_8 # This is the important part. This change file.external_encoding to Encoding::UTF_8
File.open(path, flags) do |file|
  file.write("\x8b".b)
end
/var/tmp/xxx.rb:7:in 'IO#write': "\x8B" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)
	from /var/tmp/xxx.rb:7:in 'block in <main>'
	from /var/tmp/xxx.rb:6:in 'IO.open'
	from /var/tmp/xxx.rb:6:in '<main>'

We can solve this by adding file.binmode:

require "fileutils"
path = "/tmp/abc.txt"
FileUtils.rm_f(path)
flags = File::RDWR|File::CREAT|File::EXCL|File::BINARY
Encoding.default_internal = Encoding::UTF_8
File.open(path, flags) do |file|
  file.binmode
  file.write("\x8b".b)
end

What is your fix for the problem, implemented in this PR?

Add missing .binmode that is included in the original change in GH-9202. I commented that temp_file.binmode is redundant. But I was wrong. We need temp_file.binmode even when we specify File::BINARY. Sorry.

See also: https://docs.ruby-lang.org/en/master/File.html#class-file-data-mode-specified-as-an-integer

Note that although there is a constant File::BINARY, setting its value in an integer stream mode has no effect; this is because, as documented in File::Constants, the File::BINARY value disables line code conversion, but does not change the external encoding.

Make sure the following tasks are checked

In rubyGH-9202, I commented that `temp_file.binmode` is redundant. But I
was wrong. We need `temp_file.binmode` even when we specify
`File::BINARY`. Sorry.
@kou kou mentioned this pull request Jan 18, 2026
4 tasks
Copy link
Member

@hsbt hsbt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@hsbt hsbt merged commit b764b28 into ruby:master Jan 19, 2026
85 checks passed
@kou kou deleted the atomic-write-binmode branch January 19, 2026 07:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants