URL

Goals

The URL standard takes the following approach towards making URLs fully interoperable:

• Align RFC 3986 and RFC 3987 with contemporary implementations and obsolete the RFCs in the process. (E.g., spaces, other "illegal" code points, query encoding, equality, canonicalization, are all concepts not entirely shared, or defined.) URL parsing needs to become as solid as HTML parsing. [RFC3986] [RFC3987]

• Standardize on the term URL. URI and IRI are just confusing. In practice a single algorithm is used for both so keeping them distinct is not helping anyone. URL also easily wins the search result popularity contest.

• Supplanting Origin of a URI [sic]. [RFC6454]

• Define URL’s existing JavaScript API in full detail and add enhancements to make it easier to work with. Add a new URL object as well for URL manipulation without usage of HTML elements. (Useful for JavaScript worker environments.)

• Ensure the combination of parser, serializer, and API guarantee idempotence. For example, a non-failure result of a parse-then-serialize operation will not change with any further parse-then-serialize operations applied to it. Similarly, manipulating a non-failure result through the API will not change from applying any number of serialize-then-parse operations to it.

As the editors learn more about the subject matter the goals might increase in scope somewhat.

1. Infrastructure

This specification depends on Infra. [INFRA]

Some terms used in this specification are defined in the following standards and specifications:

• Encoding [ENCODING]
• File API [FILEAPI]
• HTML [HTML]
• Unicode IDNA Compatibility Processing [UTS46]
• Web IDL [WEBIDL]

To serialize an integer, represent it as the shortest possible decimal number.

1.1. Writing

A validation error indicates a mismatch between input and valid input. User agents, especially conformance checkers, are encouraged to report them somewhere.

const url = new URL("https:foo.html", "https://example.org/");

const url = new URL("💩");

const url = new URL("💩", "mailto:user@example.org");

const url = new URL("/c:/path/to/file", "file:///c:/");

1.2. Parsers

The EOF code point is a conceptual code point that signifies the end of a string or code point stream.

A pointer for a string input is an integer that points to a code point within input. Initially it points to the start of input. If it is −1 it points nowhere. If it is greater than or equal to input’s code point length, it points to the EOF code point.

When a pointer is used, c references the code point the pointer points to as long as it does not point nowhere. When the pointer points to nowhere c cannot be used.

When a pointer is used, remaining references the code point substring from the pointer + 1 to the end of the string, as long as c is not the EOF code point. When c is the EOF code point remaining cannot be used.

If "mailto:username@example" is a string being processed and a pointer points to @, c is U+0040 (@) and remaining is "example".

If the empty string is being processed and a pointer points to the start and is then decreased by 1, using c or remaining would be an error.

1.3. Percent-encoded bytes

A percent-encoded byte is a string consisting of U+0025 (%) followed by two ASCII hex digits.

It is generally a good idea for sequences of percent-encoded bytes to be such that, when percent-decoded and then passed to UTF-8 decode without BOM or fail, they do not end up as failure. How important this is depends on where the percent-encoded bytes are used. E.g., for the host parser not following this advice is fatal, whereas for URL rendering the percent-encoded bytes would not be rendered percent-decoded.

A percent-encode set is a set of code points.

The C0 control percent-encode set is a percent-encode set consisting of C0 controls and all code points greater than U+007E (~).

The fragment percent-encode set is a percent-encode set consisting of the C0 control percent-encode set and U+0020 SPACE, U+0022 ("), U+003C (<), U+003E (>), and U+0060 (`).

The query percent-encode set is a percent-encode set consisting of the C0 control percent-encode set and U+0020 SPACE, U+0022 ("), U+0023 (#), U+003C (<), and U+003E (>).

The query percent-encode set cannot be defined in terms of the fragment percent-encode set due to the omission of U+0060 (`).

The special-query percent-encode set is a percent-encode set consisting of the query percent-encode set and U+0027 (').

The path percent-encode set is a percent-encode set consisting of the query percent-encode set and U+003F (?), U+005E (^), U+0060 (`), U+007B ({), and U+007D (}).

The userinfo percent-encode set is a percent-encode set consisting of the path percent-encode set and U+002F (/), U+003A (:), U+003B (;), U+003D (=), U+0040 (@), U+005B ([) to U+005D (]), inclusive, and U+007C (|).

The component percent-encode set is a percent-encode set consisting of the userinfo percent-encode set and U+0024 ($) to U+0026 (&), inclusive, U+002B (+), and U+002C (,).

This is used by HTML for registerProtocolHandler(), and could also be used by other standards to percent-encode data that can then be embedded in a URL’s path, query, or fragment; or in an opaque host. Using it with UTF-8 percent-encode gives identical results to JavaScript’s encodeURIComponent() [sic]. [HTML] [ECMA-262]

The application/x-www-form-urlencoded percent-encode set is a percent-encode set consisting of the component percent-encode set and U+0021 (!), U+0027 (') to U+0029 RIGHT PARENTHESIS, inclusive, and U+007E (~).

The application/x-www-form-urlencoded percent-encode set contains all code points, except the ASCII alphanumeric, U+002A (*), U+002D (-), U+002E (.), and U+005F (_).

2. Security considerations

The security of a URL is a function of its environment. Care is to be taken when rendering, interpreting, and passing URLs around.

When rendering and allocating new URLs "spoofing" needs to be considered. An attack whereby one host or URL can be confused for another. For instance, consider how 1/l/I, m/rn/rri, 0/O, and а/a can all appear eerily similar. Or worse, consider how U+202A LEFT-TO-RIGHT EMBEDDING and similar code points are invisible. [UTR36]

When passing a URL from party A to B, both need to carefully consider what is happening. A might end up leaking data it does not want to leak. B might receive input it did not expect and take an action that harms the user. In particular, B should never trust A, as at some point URLs from A can come from untrusted sources.

3. Hosts (domains and IP addresses)

At a high level, a host, valid host string, host parser, and host serializer relate as follows:

• The host parser takes an arbitrary scalar value string and returns either failure or a host.

• A host can be seen as the in-memory representation.

• A valid host string defines what input would not trigger a validation error or failure when given to the host parser. I.e., input that would be considered conforming or valid.

• The host serializer takes a host and returns an ASCII string. (If that string is then parsed, the result will equal the host that was serialized.)