Skip to content

Add Config::base_url to resolve relative links (#287) - #288

Open
jaredlynskey wants to merge 1 commit into
jugglerchris:mainfrom
jaredlynskey:base-url-resolution
Open

Add Config::base_url to resolve relative links (#287)#288
jaredlynskey wants to merge 1 commit into
jugglerchris:mainfrom
jaredlynskey:base-url-resolution

Conversation

@jaredlynskey

Copy link
Copy Markdown

Implements the request in #287.

What it does

Adds Config::base_url(...), which resolves relative hrefs while rendering:

let markdown = html2text::config::plain()
    .base_url("https://example.com/a/page.html")
    .string_from_read(html, 80)?;

<a href="/login"> renders as https://example.com/login instead of /login.

Behaviour

Resolution follows RFC 3986 §5.3:

reference result
https://other.example/x, mailto:…, data:…, tel:… unchanged
//cdn.example/x.js inherits the base scheme
/login origin + path
c.html, ../c.html merged with the base directory, dot segments removed
#frag, ?q=1 attached to the base path

With no base URL set the output is byte-identical to before — the existing 128 tests pass unchanged, and the option defaults to None.

On the implementation

Resolution is done in-crate (src/url_resolve.rs, ~110 lines) rather than by adding the url crate. Only reference resolution is needed and this crate's dependency list is deliberately small, so pulling in a full URL parser for it seemed the wrong trade — but I'm happy to switch to url behind an optional feature if you'd prefer that; it would be a small change.

I put the resolution at the point the href is read in process_dom_node, so it happens once during tree building rather than in the decorators. That keeps every decorator and output format getting resolved links for free. If you'd rather it sat elsewhere I'm glad to move it.

Tests

  • 8 unit tests in url_resolve covering each reference form, non-HTTP schemes, an unusable base, and dot segments at the root
  • 3 integration tests through the normal test_html_conf path

One worth calling out: /../../etc initially resolved to https://example.cometc, because .. popped the leading empty segment and took the root separator with it. RFC 3986 §5.2.4 discards those instead, which is what it now does — there's a test pinning it.

cargo test passes (139 lib + 3 doc), cargo fmt --check is clean, and it builds with --all-features and --no-default-features.

An <a href=/login> renders as /login, which only means something to a
reader who has the page it came from. When the output is consumed away from
that context -- feeding a page to an LLM, or storing the Markdown -- those
links cannot be followed.

Config::base_url(...) resolves them per RFC 3986 section 5.3. Absolute URLs
and non-HTTP schemes (mailto:, data:, tel:) are passed through untouched, and
with no base URL set the output is unchanged.

Resolution is implemented in-crate rather than by taking a dependency on the
url crate, since only reference resolution is needed here.

Closes jugglerchris#287
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant