Skip to content

Replace URI.encode, removed in Ruby 3.0, with Fog::OpenStack.escape - #554

Open
tas50 wants to merge 1 commit into
fog:masterfrom
tas50:ruby3-fix-uri-encode
Open

Replace URI.encode, removed in Ruby 3.0, with Fog::OpenStack.escape#554
tas50 wants to merge 1 commit into
fog:masterfrom
tas50:ruby3-fix-uri-encode

Conversation

@tas50

@tas50 tas50 commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

URI.encode (an alias for URI.escape) was removed in Ruby 3.0, so all thirteen call sites below raise NoMethodError on Ruby 3 and later. The library still loads fine — I confirmed require "fog/openstack" works on Ruby 4.0.6 — it fails only when the request is actually made:

NoMethodError: undefined method 'encode' for module URI

Why Fog::OpenStack.escape rather than CGI.escape

This gem already ships the right helper. Fog::OpenStack.escape is documented as "CGI.escape, but without special treatment on spaces" and is what roughly twenty storage call sites already use. The thirteen here are simply the ones that never got migrated to it.

CGI.escape is not a safe replacement, because it encodes a space as +:

Fog::OpenStack.escape("sp ace")   # => "sp%20ace"   correct in a path
CGI.escape("sp ace")              # => "sp+ace"     a literal '+' in a path

+ only means "space" in a query string. In a URL path it is a literal plus, so any resource name containing a space would be silently corrupted. There is a spec pinning this down.

This overlaps #527, which found the same thirteen call sites — credit there for spotting it — but converts them to CGI.escape. I have left a comment on that PR with the detail.

Two call sites that need care

delete_multiple_objects escapes the joined "#{container}/#{name}", so the separator has to survive. It passes '/' as an extra excluded character, matching how Storage::Files#get_url already does this. A blanket replacement here would emit container%2Fobject and break Swift bulk delete outright.

The compute service requests escaped their optional params like this:

optional_params = optional_params.each { |k, v| optional_params[k] = URI.encode(v) } if optional_params

That assigns back into the hash while iterating it, mutating the caller's hash as a side effect. transform_values escapes the values without touching the original. There is a spec for that too.

base64 development dependency

Neither the spec nor the unit suite can load at all on Ruby 3.4+ without this:

cannot load such file -- base64 (LoadError)

base64 became a bundled gem in Ruby 3.4, and webmock requires it without declaring a dependency on it. Declaring it is required for any of this to be testable on a current Ruby, so it is included here.

Tests

spec/escaping_spec.rb adds ten specs: the escaping behaviour of each group of call sites, the space and separator cases that distinguish Fog::OpenStack.escape from CGI.escape, the non-mutation of the caller hash, and a guard that URI.encode does not reappear anywhere in lib/.

They build a Real instance with allocate and stub #request, so no credentials or network are needed.

10 runs, 13 assertions, 0 failures, 0 errors, 0 skips

RuboCop is unchanged on lib/ (0 offences before and after on the touched files) and clean on the new spec.

State of the rest of the suite

Being straightforward about this, since the numbers look alarming out of context.

On master both suites fail to load on Ruby 4.0.6 with the base64 LoadError. With that fixed they run, and report pre-existing failures:

Suite On master With this PR
rake tests:spec does not load 110 runs, 79 errors, 10 skips
rake tests:unit does not load 53 runs, 52 errors

None of those errors are in files this PR touches, and all ten new specs pass inside the full run. They are almost entirely one separate issue: the suites use Minitest's bare expectation style (x.must_equal y), which Minitest 6 removed in favour of _(x).must_equal y. minitest is unpinned in the gemspec, so a current bundle resolves 6.0.6 and those calls become NoMethodError.

That is a real blocker for Ruby 3.4/4.x support, but it is a large mechanical change across the whole test suite and unrelated to URL escaping, so I have deliberately kept it out of this PR. Happy to open it separately if useful.

URI.encode (an alias for URI.escape) was removed in Ruby 3.0, so every
one of these call sites raises NoMethodError on Ruby 3 and later. The
library still loads; it fails when the request is actually made.

This gem already has the right helper for the job. Fog::OpenStack.escape
is documented as 'CGI.escape, but without special treatment on spaces'
and is what the storage requests have always used. The thirteen call
sites here are simply the ones that never got migrated to it.

CGI.escape is not a suitable replacement: it encodes a space as '+',
which is only correct in a query string. In a URL path a '+' is a
literal plus, so any resource name containing a space would be
corrupted.

Two call sites need care:

  - delete_multiple_objects escapes the joined 'container/object'
    string, so the separator has to survive. It passes '/' as an
    extra excluded character, matching how Storage::Files#get_url
    already does this.

  - The compute service requests escaped their optional params by
    assigning back into the hash while iterating it, which mutated the
    caller's hash. transform_values escapes the values without that
    side effect.

Also declares a development dependency on base64. It became a bundled
gem in Ruby 3.4, and webmock requires it without depending on it, so
without this neither the spec nor the unit suite can even load on Ruby
3.4 or later.

Adds spec/escaping_spec.rb covering the escaping behaviour of each
group of call sites, the space and separator cases that distinguish
Fog::OpenStack.escape from CGI.escape, and a guard that URI.encode does
not reappear anywhere in lib.

Signed-off-by: Tim Smith <tsmith84@proton.me>
Comment thread spec/escaping_spec.rb
captured
end

describe 'URL escaping' do

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metrics/BlockLength: Block has too many lines. [65/25]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants