url

tape
import url from "url";

let page: url.Url = url.parse("https://example.org/docs/guide.html", null) or { return 1; };
let link: url.Url = url.parse("../api/url.html", &page) or { return 1; };
io.println(link.href());     // https://example.org/api/url.html

url parses URLs the way browsers do: by the WHATWG URL Standard. It passes all 896 of the web-platform-tests’ URL parsing cases, so a link resolves in your program to what it resolves to in a browser: odd spacing, backslashes, .. segments, upper-case hosts, IPv4 in hex, IPv6, non-ASCII domain names and all.

Input is UTF-8 text. Parsing allocates the URL’s strings; they are freed with the Url.

Parsing

FunctionDoes
parse(input, base) -> Url or UrlErrorParse input, resolved against base when it is relative; null for no base
empty() -> UrlA URL with nothing in it, to fall back on
u.href() -> stringThe whole URL, normalized
tape
let u: url.Url = url.parse("  HTTP://Example.COM:80/a/./b/../c  ", null) or {
    io.println("not a URL: {err}");
    return 1;
};
io.println(u.href());        // http://example.com/a/c

Parsing normalizes: the scheme and host are lower-cased, a port that is the scheme’s default is dropped, . and .. segments are resolved, spaces around the URL are trimmed, and characters a URL can’t carry are percent-encoded (a b in a path becomes a%20b).

When a URL doesn’t parse, the error says why:

UrlErrorWhen
NoBaseA relative URL (/a, ../b, page.html) with no base
MissingHosthttp://, https://user@/: a web URL needs a host
InvalidHostA host with a forbidden character, a bad IP address, a domain that can’t be converted
InvalidPortA port above 65535 or with something other than digits

Pass the page’s URL as the base. Every kind of relative link works:

tape
let page: url.Url = url.parse("https://example.org/docs/guide/index.html?lang=fi#intro", null) or { return 1; };

url.parse("../api/url.html", &page)           // https://example.org/docs/api/url.html
url.parse("/search?q=x", &page)               // https://example.org/search?q=x
url.parse("//cdn.example.org/app.js", &page)  // https://cdn.example.org/app.js
url.parse("?lang=en", &page)                  // https://example.org/docs/guide/index.html?lang=en
url.parse("#top", &page)                      // https://example.org/docs/guide/index.html?lang=fi#top
url.parse("mailto:me@example.org", &page)     // absolute: the base is ignored

Each line gives a Url or UrlError; handle it with or as above.

Reading the parts

The getters return the parts as a browser’s URL object shows them:

MethodFor https://u:p@example.org:8080/a/b?q=1#top
protocol()https:
host_port()example.org:8080 (the API’s host)
hostname()example.org
port_text()8080; empty when there is no port
pathname()/a/b
search()?q=1; empty when there is no query
hash()#top; empty when there is no fragment

The Url struct’s fields hold the same, without the punctuation: scheme ("https"), username, password, host, port (a ?u16, null when there is none or it is the default), path (the segments, a collections.Vec_string), query and fragment. A URL like mailto:me@example.org has an opaque path: opaque is true and the path is in opaque_path. has_host, has_query and has_fragment tell an empty part (http://h/?) from a missing one.

tape
let p: u16 = u.port or { url.default_port(u.scheme) or { 0 } };
FunctionDoes
is_special(scheme) -> boolIs the scheme one of http, https, ws, wss, ftp, file? Those parse with hosts and paths; others are kept more as written
default_port(scheme) -> ?u1680, 443 or 21; null for file and other schemes

Non-ASCII domain names are stored in their ASCII form, as browsers send them: http://bücher.example/ has the host xn--bcher-kva.example.

The origin

u.origin() is the URL’s origin as text: scheme://host plus :port when the port isn’t the default. Two URLs with the same origin are the same site as far as a browser’s security rules go.

tape
io.println(page.origin());   // https://example.org

file: URLs, data:, mailto: and other schemes without a host have an opaque origin, "null". A blob: URL has the origin of the http(s) URL inside it.

Percent-encoding

FunctionDoes
percent_encode(s, set) -> stringEach byte of s in set as %XX, the rest as is
percent_decode(s) -> stringEach %XX as its byte; a % without two hex digits stays

The sets are the Standard’s, from fewest bytes encoded to most:

EncodeSetEncodes (besides control characters and everything non-ASCII)
C0Controlnothing more
Fragmentspace " < > `
Queryspace " # < >
SpecialQueryQuery and '
PathQuery and ? ^ ` { }
UserinfoPath and / : ; = @ [ \ ] `
ComponentUserinfo and $ % & + ,

To put user text into a URL, encode it with Component (like JavaScript’s encodeURIComponent):

tape
let term: string = url.percent_encode("café & bar", url.EncodeSet.Component);
let q: url.Url = url.parse("/search?q={term}", &page) or { return 1; };
io.println(q.search());      // ?q=caf%C3%A9%20%26%20bar

percent_decode never fails, and its result is bytes: it may not be valid UTF-8 (utf8.validate checks).

For RFC 3986’s encoding (everything but letters, digits and -._~), use encoding’s percent_encode.

Punycode

FunctionDoes
punycode_encode(label) -> string or UrlErrorbücher → bcher-kva
punycode_decode(s) -> string or UrlErrorbcher-kva → bücher; InvalidPunycode when it isn’t Punycode

These work on one label, without the xn-- prefix. parse already converts hosts; use these to show a host to a person.

Limits

  • Non-ASCII domain names are mapped for the common cases (case, full-width forms, the scripts’ capitals) but not with Unicode’s full IDNA tables, so a rare domain may be converted differently from a browser.
  • There are no setters (set_pathname), no URLSearchParams and no application/x-www-form-urlencoded parser yet: build a new URL string and parse it.
  • http has its own simpler parse_url for the requests it makes.

Last modified: