@commonshost/manifest/README.md

# @commonshost/manifest 📑

**HTTP/2 Server Push Manifest** is a declarative configuration syntax for web servers, CDNs, and tools that offers web developers access to this powerful new protocol feature.

```js
[
  {
    get: '/index.html',
    push: '/**/*.{js,css}'
  }
]
```

## JSON Schema

See: [./schema.json](./schema.js)

## Manifest Format

The idea is simple: specify a list of **trigger and action tuples**. The syntax is flexible and forgiving, with shorthand notation for ease of use.

A **manifest** is a JSON array containing rules.

```js
[]
```

Each **rule** is a JSON object containing a `get` and `push` property. These are the **trigger** and **action** respectively.

```js
[
  {get, push},
  {get, push}, ...
]
```

**Triggers** are *URI Template* patterns or file path *globs*. They match the response's pathname, or the resource on disk (in the case of static sites).

```js
[
  {get: '/index.html', push},
  {get: 'https://example.net/', push}, ...
]
```

**Actions** are the resources to be pushed. These are also file path *globs* or *URI templates*.

```js
[
  {get: '/index.html', push: '/app.js'},
  {get: '/app.js', push: '/lib.js'}, ...
]
```

**Globs**, and **extglobs**, are POSIX/Bash-compatible expressions to match file paths. They support wildcards, negations, expansions, and more. See [micromatch](https://github.com/micromatch/micromatch).

```js
[
  {get: '/index.html',
   push: '/{css,js}/**/*'}, ...
]
```

**URI Templates** are strings formatted to match URLs. They support for pathname expansion, order-independent query string fields, and wildcards. See [RFC 6570](https://tools.ietf.org/html/rfc6570).

```js
[
  {get: 'https://example.net/shop{/brand}',
   push: 'https://example.net/banners{/brand}.png'},
  {get: 'https://example.net{/topic}',
   push: 'https://api.example.net/topics{/productId}.json'}, ...
]
```

In **shorthand** notation, both actions and triggers can be expressed as JSON strings, or arrays of strings.

```js
[
  {get: ['/*.html', '!/legacy.html'],
   push: ['/images/*.{png,jpg}', '!/images/huge.jpg']}, ...
]
```

In **normalised** notation, they are arrays containing objects with `uri` and/or `glob` properties, which are in turn either strings or arrays of strings.

The **priority** number is a property of the normalised push action and ranges from `0` to `255`. Defaults to `16` as per the HTTP/2 spec.

```js
[
  {
    get: [
      {
        uri: [
          'https://example.net/',
          'https://example.net/about', ...
        ],
        glob: [
          '/images/*.{jpg,webp}',
          '/styles/**/*.css', ...
        ]
      }, ...
    ],
    push: [
      {
        glob: [
          '/fonts/*.ttf', ...
        ],
        uri: [
          'https://cdn.example.net/logo.svg', ...
        ],
        priority: 100
      }, ...
    ]
  }, ...
]
```

Take a look at the [test cases](./test) for more normalisation examples.

## FAQ

### What problem does this solve?

One of the features introduced by HTTP/2 is Server Push. Difficulty in configuring servers has [inhibited widespread adoption](https://youtu.be/wR1gF5Lhcq0?t=48m14s) by web developers. This declarative **HTTP/2 Server Push Manifest** format offers a server- and language-agnostic abstraction.

### Why declarative rules?

Declarative rules are passive, i.e. they do not require arbitrary code execution. This is ideal for processing by edge services like a CDN nodes, or other static file servers.

### Which software supports HTTP/2 Manifests?

This is currently supported by [@commonshost/server](https://www.npmjs.com/package/@commonshost/server), a Node.js based static web server. The [commons.host](https://commons.host) static hosting & CDN service also supports it, as it is based on the same server. Build-tool support (i.e. dependency graph tracing) is currently in development.

*Implementors: Please add your implementation to this paragraph by opening a pull request.*

### Aren't CDNs and web servers using preload link headers?

Yes, and those too could be generated based on a Server Push manifest. For example, [@commonshost/server](https://www.npmjs.com/package/@commonshost/server) automatically outputs `Link: https://...; rel=preload` headers for clients connecting via HTTP/1.1, and uses `PUSH_PROMISE` frames for HTTP/2 clients.

But those `Link` headers are arguably a leftover from the HTTP/1 era. Legacy CDN, proxy, and other middlebox support are often limiting factors.

In practice it is unlikely that you can send more than a handful HTTP/1 `Link` headers. It is much more efficient to use `PUSH_PROMISE` frames, which are compressed, virtually unlimited in number, and offer more functionality.

### Can I still programmatically push assets?

Sure, that is an implementation issue. Your backend web application could emit `PUSH_PROMISE` frames which an intermediary CDN edge combines with a push manifest.

### Does this work with a CDN?

CDNs are ideally suited for server push manifests. Edge nodes can make full use of the *think time* between the receipt of a browser request and the response from an upstream, origin server.

### What about browser support?

The manifest is not applicable to browsers. No real considerations.

All current popular web browsers (Cr/Fx/Sf/Eg) support HTTP/2 Server Push. While some complications, and occasionally bugs, exist, these will be solved faster if more people start using HTTP/2 Server Push.

### Which files can be pushed?

Anything goes. All you need is a URL or file name.

### Should this be an RFC?

Probably not, since it is an implementation detail for servers and other tools. Parts of it rely on RFCs (URI Templates, HTTP/2 spec) and other "open standards" like JSON (it's complicated) or Bash extglobs (good luck referencing the exact ISO POSIX spec!).

### Why JSON?

JSON is widely supported by programming languages used on the web.

### Why publish this as a spec?

So that, hopefully, other server and tool builders avoid reinventing the wheel. Makes it easier to build tools that target the manifest format rather than specific server or CDN implementations.

### Is JavaScript required?

No, the spec is a language-neutral JSON Schema. The JavaScript code is simply a proof-of-concept implementation.

### Can I use HTTP/2 Server Push with CORS assets?

Yes, in theory. The manifest allows pushing URIs.

In practice, browser support is still limited. This area is highly experimental with several related proposals currently under consideration at the IETF HTTP-WG.

### Are there security considerations?

Servers and CDNs coalescing multiple origins must take care that a manifest's origin is authorised to push CORS assets from other origins.

### What about caching?

Caching and other headers to be included in a `PUSH_PROMISE` frame are not currently part of this spec. Share your thoughts in the issue tracker if you would like to see support for this feature.

### What about HTTP methods, status codes, etc?

Currently only pushes in response to a `GET` request are described in the manifest. However there is nothing technically preventing a future version of the spec from supporting other methods like `POST` or `PUT`, or other (pseudo-)headers. Please share your interest in these use-cases on the issue tracker.

### Will the server push too much stuff?

That depends on your manifest. With great power comes great responsibility. Protip: Use negated globs (e.g. `!/**.*.map`) to exclude files.

### What if the browser already has some assets in its cache?

When the server pushes a stream, the browser can quickly close it to avoid wasting resources. Unfortunately this still requires a round-trip.

Consider using HTTP/2 Cache Digests. This spec allows browsers to send a condensed list of its cache contents to the server. The server can use this to avoid sending redundant assets.

### What about auto-push?

Several attempts have been made at automatically pushing assets, by build-tools scanning for dependencies or by intellgent servers analysing traffic patterns. The result of these can be encoded as manifest rules for portability or caching.

### What about stream dependencies and priorities?

Dependencies can be inferred by recursively tracing the manifest rules. For example resource A has a rule that pushes resource B, which pushes resource C in another rule. Now C is a dependency of B which is a dependency of A.

Priorities are explicitly specified on each rule using the `priority` key. Its value is an integer from 0 to 255 and defaults to 16. This follows the HTTP/2 spec. All pushed resources in a rule share the same priority.

### I thought HTTP/2 priority was a client-only feature?

Correct, oh wise one. The priority of HTTP/2 streams is a *suggestion* by the client that the server *may* follow. Ultimately, however, it is the server that controls the order of its data frames. This manifest spec introduces a way for web developers to instruct the server on the priority of streams.

### I (do/don't) like this

Not really a question. But feel free to share your experience, thoughts, and suggestions on the issue tracker. Open discussion and feedback is most welcome.

## API

```js
const {validate, normalise, schema} = require('@commonshost/manifest')

const manifest = [
  {
    get: '/index.html',
    push: ['/app.js', '/design.css']
  }
]
```

Check if the syntax is valid.

```js
try {
  validate(manifest)
  // Returns `true` or throws a validation error
} catch (error) {
  console.error(error)
}
```

Transform the shorthand rules into a fully exploded format.

```js
normalise(manifest)

// [
//   {
//     get: [{glob: ['/index.html']}],
//     push: [{glob: ['/app.js', '/design.css'], priority: 16}]
//   }
// ]
```

Use the JSON Schema with your validator of choice.

```js
const validator = new MyValidator()
validator.validate(manifest, schema)
```

## Colophon

Made with 💘 by Sebastiaan Deckers in 🇸🇬 Singapore.