# Factory Function LibAV instances are created by an asynchronous factory function, `LibAV.LibAV`. In many cases, it can be called with no option, e.g., `libav = await LibAV.LibAV()`. `LibAV.LibAV` takes an optional argument in which loading options may be provided. The loading options and their default values are: ``` { "noworker": false, "nowasm": false, "yesthreads": false, "nothreads": false, "base": , "toImport": , "factory": , "variant": , "wasmurl": } ``` `nowasm` forces libav.js to load only asm.js code, not WebAssembly code. By default, it will determine what the browser supports and choose accordingly, so this is overridable here mainly for testing purposes. The other no/yes options affect the execution mode of libav.js. libav.js can run in one of three modes: `"direct"` (synchronous), `"worker"`, or `"threads"`. After creating a libav.js instance, the mode can be found in `libav.libavjsMode`. By default, libav.js will use the `"worker"` mode if Web Workers are available, and `"direct"` otherwise. libav.js never uses the `"threads"` mode by default, though this may change in the future. If `noworker` is set or Web Workers are not available, Web Workers will be disabled, so libav.js will run in the main thread (i.e., will run in `"direct"` mode). This is synchronous, so usually undesirable. Note that if you're loading libav.js *in* a worker, it may be reasonable to set `noworker`, and make libav.js synchronous with your worker thread. If `yesthreads` is set (and `nothreads` is not set) and threads are supported (see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer ), then a threaded version of libav.js will be loaded. This will significantly improve the performance of some encoders and decoders. However, threads are disabled by default, as their benefit or otherwise depends on the precise behavior of your code, and some browsers have a fairly low limit to the number of worker threads an entire page is allowed to have. Note that separate instances of libav.js, created by separate calls to `LibAV.LibAV`, will be in separate threads as long as workers are used, regardless of the value of `yesthreads`, and thus `yesthreads` is only needed if you need concurrency *within* a libav.js instance. libav.js automatically detects which WebAssembly features are available, so even if you set `yesthreads` to `true`, a version without threads may be loaded. To know which version will be loaded, call `LibAV.target`. It will return `"asm"` if only asm.js is used, `"wasm"` for baseline, or `"thr"` for threads. These strings correspond to the filenames to be loaded, so you can use them to preload and cache the large WebAssembly files. `LibAV.target` takes the same optional argument as `LibAV.LibAV`. The `base` option can be used in these options in place of `LibAV.base`, and will override `LibAV.base` if set. The `variant` option can be set to load a different variant than the one given by the URL you loaded libav.js itself from, which in turn can be used to load multiple variants at the same time. The `toImport`, `factory`, and `wasmurl` options are documented in the “bundlers” section of the [main README](../README.md), as they are mainly of interest to bundlers. The tests used to determine which features are available are also exported, as `LibAV.isWebAssemblySupported` and `LibAV.isThreadingSupported`. NOTE: libav.js used to have a SIMD build as well. This was dropped because none of the constituent libraries actually support WebAssembly SIMD, so it substantially increased the size and time of builds to no benefit. The `LibAV.LibAV` factory returns (a promise resolving to) a libav instance, which is an object exposing libav and libav.js's API as methods. # libav Instance Methods Most of libav.js's API is libav's API, and for such functions, you can consult FFmpeg's documentation. Not every function is exposed, of course; see [funcs.json](../funcs.json) for a list of exposed functions or to add new functions. Functions that use double-pointers are exposed as `_js` metafunctions that take and return single pointers. Most structs are exposed as raw pointers (numbers), and their parts can be accessed using accessor functions named `Struct_member` and `Struct_member_s`. For instance, to read `frame_size` from an `AVCodecContext`, use `await AVCodecContext_frame_size(ctx)`, and to write it, use `await AVCodecContext_frame_size_s(ctx, frame_size)`. There are also libav.js-specific JavaScript objects for many of them, documented in libav.types.d.ts. Further examples are available in the `samples` directory of https://github.com/ennuicastr/libavjs-webcodecs-polyfill , which uses libav.js along with WebCodecs (or its own polyfill of WebCodecs), so shows how to marry these two technologies. The following additional functionality is provided by libav.js itself, divided here by the libav component it belongs to. Please read [../libav.types.in.d.ts](libav.types.in.d.ts) for type declarations. # Data types libav frames and packets can be represented as numbers, pointers to actual `AVFrame` or `AVPacket` instances, but most functions will instead copy out their data into libav.js `Frame` or `Packet` objects. One important similarity in `Frame` and `Packet` objects is the `libavjsTransfer` member. If you send a `Frame` or `Packet` into libav.js (and it happens to be running a worker), then any ArrayBuffers given in the `libavjsTransfer` array will be transferred. By default, all included ArrayBuffers are in `libavjsTransfer`, so all will be transferred. This works in both directions, so if you get a `Frame` from libav.js and then send it back again, you will lose access to its underlying data. For the rest of the details of these types, see the TypeScript type documentation. # AVFormat ## Muxing ### `ff_init_muxer` ``` ff_init_muxer( opts: { oformat?: number, format_name?: string, filename?: string, device?: boolean, open?: boolean, codecpars?: boolean }, streamCtxs: [number, number, number][] ): Promise<[number, number, number, number[]]> ``` Initializes a muxer all at once, opening the file and initializing the format context. You can provide the format as a libav numerical code (`oformat`) or as a name (`format_name`), the filename to write to (which can of course be a device) (`filename`), and/or create it as a writer device automatically (`device`). You have the option not to open the file (`open=false`, the default), in which case you will need to provide your own `pb`. For the streams to mux, each stream is in the form `[number, number, number]`, consisting of the codec context, `time_base_num`, and `time_base_den`, respectively. If `opts.codecpars` is set, use a codec parameters (codecpar), not a codec context. Returns `[output context (oc), format, writer context (pb), stream contexts]`. Usually called as `[oc, fmt, pb] = await ff_init_muxer(...)`. To write, you must first use `libav.avformat_write_header`, and after writing all packets, you must use `libav.av_write_trailer`. You may write packets with `libav.ff_write_multi` (below), or directly using libav APIs. ### `ff_write_multi` ``` ff_write_multi( oc: number, pkt: number, inPackets: (Packet | number)[], interleave?: boolean ): Promise ``` Write packets to an output context. You need to not just provide the packet(s) (`inPackets`), but allocate space for packets in libav format, so there's somewhere to write them to temporarily (`pkt`). Packets may be in libav.js `Packet` format, or `AVPacket` pointers. Use `av_packet_alloc` (and eventually, `av_packet_free`) for that, or get it from one of the AVCodec metafunctions. `interleave` means that it will use `av_interleaved_write_frame`, and if `interleave===false`, it will use `av_write_frame` instead. `interleave` defaults to true, and this is usually the right option, but if your input is already interleaved, you should set this to false. ### `ff_free_muxer` ``` ff_free_muxer(oc: number, pb: number): Promise ``` Frees the context generated by `ff_init_muxer`. ## Demuxing ### `ff_init_demuxer_file` ``` ff_init_demuxer_file( filename: string, fmt?: string ): Promise<[number, Stream[]]> ``` Initializes a demuxer from the given filename, which can be a reader device. Optionally takes the expected format as a string. Returns `[format context (fmt_ctx), streams]`. Streams are of the `Stream` type. Note that there's no equivalent of `ff_free_muxer`, because `ff_init_demuxer_file` only initializes one libav object. Free `fmt_ctx` with `avformat_close_input_js`. ### `ff_read_frame_multi` ``` ff_read_frame_multi( fmt_ctx: number, pkt: number, opts?: { limit?: number, // OUTPUT limit, in bytes unify?: boolean, // If true, unify the packets into a single stream (called 0), so that the output is in the same order as the input copyoutPacket?: string // Version of ff_copyout_packet to use } ): Promise<[number, Record]> ``` Read some packets from a format context. Like `ff_write_multi`, you must provide `pkt`. By default, this will read as much as it can, which is typically the entire file. If using a device in this default mode, you will need to feed it data, and won't get anything back until it's done. To limit how much data is sent, use the `limit` option. `limit` limits how much data `ff_read_frame_multi` outputs to one packet more than the number of bytes you request. For reading from a reader device, see also `ff_reader_dev_waiting` and `ff_reader_dev_send`. Returns `[result, packets]`. The result is the result code from the underlying read; `0` is success, but you can also expect `-libav.EAGAIN` (if you hit a limit) or `libav.AVERROR_EOF` (at the end of the file). The returned packets are in a record indexed by the stream index. Those indices are the indices of the `Stream` objects in the `Stream[]` array given by `ff_init_demuxer_file`. Alternatively, you can use `unify`, in which case all packets will be in input order in a single array, `packets[0]`. There are multiple versions of `ff_copyout_packet`, only one of which is actually used to copy out packets. See the documentation of `ff_copyout_packet` below for how to use the `copyoutPacket` option. ## Data manipulation ### `ff_copyout_packet` and variants ``` ff_copyout_packet(pkt: number): Promise ``` Variants: `ff_copyout_packet_ptr` Copy a packet from internal libav memory (`pkt`) as a libav.js object. The `ff_copyout_packet_ptr` function is also available, and copies the packet into a separate `AVPacket` pointer, instead of actually copying out any data. This is a good compromise if you're building pipelines, e.g. reading then decoding, to avoid copying data back and forth when that data is just going back into libav.js. Be careful, though! `ff_read_frame_multi` reads from every stream, and if you're only using data from one of them, copied packets using `ff_copyout_packet_ptr` will leak memory! Use `ff_copyout_packet_ptr` carefully. Metafunctions that use `ff_copyout_packet` internally, namely `ff_read_frame_multi`, have a configuration option, `copyoutPacket`, to specify which version of `ff_copyout_packet` to use. It is a string option, accepting the following values: `"default", "ptr"`. ### `ff_copyin_packet` ``` ff_copyin_packet(pktPtr: number, packet: Packet | number): Promise ``` Copy a packet as a libav.js object (`packet`) into libav memory (`pktPtr`). Also works to duplicate a packet that is already an `AVPacket` pointer as a number. # AVCodec ## Encoding ### `ff_init_encoder` ``` ff_init_encoder( name: string, opts?: { ctx?: AVCodecContextProps, options?: Record } ): Promise<[number, number, number, number, number]> ``` Allocate and initialize an encoder (whether audio or video). Sometimes, all that is sufficient is the name, in libav style, e.g. `libopus` or `aac`. Common features like bitrate are in `ctx` (see the description of `AVCodecContextProps` in libav.types.in.d.ts). `options` is for codec-specific options. Returns a *lot* of things, some of which aren't always needed: `[codec, codec context (c), frame, packet (pkt), frame size]`. Usually called as `[, c, frame, pkt, frame_size] = await ff_init_encoder(...)`. ### `ff_encode_multi` ``` ff_encode_multi( ctx: number, frame: number, pkt: number, inFrames: (Frame | number)[], fin?: boolean ): Promise ``` Encode multiple frames into packets. Set `fin` if these are the last frames; otherwise the arguments should be obvious. Note that it's fine to set `inFrames` to `[]` to encode no frames, typically to set `fin`. The frames may be `AVFrame` pointers, as numbers. ### `ff_free_encoder` ``` ff_free_encoder( c: number, frame: number, pkt: number ): Promise ``` Free the things allocated by `ff_init_encoder`. ## Decoding ### `ff_init_decoder` ``` ff_init_decoder( name: string | number, config?: { codecpar?: number | CodecParameters, time_base?: [number, number] } ): Promise<[number, number, number, number]> ``` Initialize a decoder. `name` can be a string name (e.g. `"libopus"`) *or* an internal identifier. Usually, an internal identifier would come from a `Stream` from `ff_init_demuxer_file`, in which case `codecpar` and `time_base` would also come from that `Stream`. Returns a *lot* of things, some of which aren't always needed: `[codec, codec context (c), packet (pkt), frame]`. Usually called as `[, c, pkt, frame] = ff_init_decoder(...)`. ### `ff_decode_multi` ``` ff_decode_multi( ctx: number, pkt: number, frame: number, inPackets: (Packet | number)[], config?: boolean | { fin?: boolean, ignoreErrors?: boolean, copyoutFrame?: string } ): Promise ``` Decode multiple packets, which may be libav.js `Packet`s or `AVPacket` pointers. `config` can be set to `true` as `fin`, which means these are the last packets. Alternatively, `config` can be set to an object, and `ignoreErrors` will attempt to continue decoding in the case of errors. There are multiple versions of `ff_copyout_frame`, only one of which is actually used to copy out frames. See the documentation of `ff_copyout_frame` below for how to use the `copyoutFrame` option. ### `ff_decode_filter_multi` Combination of `ff_decode_multi` and `ff_filter_multi`. Documented with `ff_filter_multi`, below. ### `ff_free_decoder` ``` ff_free_decoder( c: number, pkt: number, frame: number ): Promise ``` Free the things allocated by `ff_init_decoder`. ## Data manipulation ### `ff_copyout_frame` and variants ``` ff_copyout_frame(frame: number): Promise ``` Variants: `ff_copyout_frame_video`, `ff_copyout_frame_video_packed`, `ff_copyout_frame_video_imagedata`, `ff_copyout_frame_ptr` Copy a frame out of internal libav memory (`frame`) as a libav.js object. `ff_copyout_frame` supports video frames, but if you know a frame is a video frame, you can bypass the check by using `ff_copyout_frame_video` instead. When video frames are copied out as `Frame`s, the frame's data is copied as a single `Uint8Array`, and the layout of the pixel data within that data is given by a `layout` array. Each frame in the layout is an element of the array (a PlaneLayout), and each PlaneLayout consists of an `offset` and a `stride`. The offset is the location of this plane in the data, and the stride is the number of bytes between lines, which can be fairly arbitrary for memory alignment reasons. `ff_copyout_frame_video_packed` copies out the same as `ff_copyout_frame_video`, but with all the data packed (i.e., with the minimum possible stride). To copy out video frames directly as `ImageData` objects instead of libav.js `Frame`s at all, use `ff_copyout_frame_video_imagedata`. `ImageData` is only available in browsers. Further, `ff_copyout_frame_video_imagedata` does *not* convert the image format, so video frames must already be in RGBA (*not* RGB32!) format to use it. The `ff_copyout_frame_ptr` function is also available, and copies the frame into a separate `AVFrame` pointer, instead of actually copying out any data. This is a good compromise if you're building pipelines, e.g. decoding and then filtering, to avoid copying data back and forth when that data is just going back into libav.js. Metafunctions that use `ff_copyout_frame` internally, namely `ff_decode_multi` and `ff_filter_multi`, have a configuration option, `copyoutFrame`, to specify which version of `ff_copyout_frame` to use. It is a string option, accepting the following values: `"default", "video", "video_packed", "ImageData", "ptr"`. ### `ff_copyin_frame` ``` ff_copyin_frame(framePtr: number, frame: Frame | number): Promise ``` Copy a libav.js Frame object (`frame`) into libav memory (`framePtr`). Also works if `frame` is another `AVFrame` pointer, e.g. as created by `ff_copyout_frame_ptr`. # AVFilter ### `ff_init_filter_graph` ``` ff_init_filter_graph( filters_descr: string, input: FilterIOSettings | FilterIOSettings[], output: FilterIOSettings | FilterIOSettings[] ): Promise<[number, number | number[], number | number[]]>; ``` Create a filter graph. Most of the context is `filters_descr` is either a simple description (as would be passed to `-vf` or `-af` in ffmpeg) or a complex description (as would be passed to `-filter_complex`). `FilterIOSettings` describes things like the sample rate/frame rate and sample format/pixel format. A filter graph can have multiple inputs and multiple outputs, in which case `input` and `output` can be arrays. If there is one input, the input pad is named `in`. Otherwise, the input pads are named `in0`, `in1`, etc. If there is one output, the output pad is named `out`. Otherwise, the output pads are named `out0`, `out1`, etc. Returns `[filter graph, source context(s), sink context(s)]`. If there are multiple inputs (sources), then source contexts is an array, and if there are multiple outputs (sinks), then sink contexts is an array. ### `ff_filter_multi` ``` ff_filter_multi@sync( srcs: number | number[], buffersink_ctx: number, framePtr: number, inFrames: (Frame | number)[] | (Frame | number)[][], config?: boolean | { fin?: boolean, copyoutFrame?: string } ): @promise@Frame[]@; ``` Filter one or more frames. Takes one or more source contexts (`srcs`), but *only one sink context* (`buffersink_ctx`). If you need to filter with multiple outputs, you need to use the lower-level libav functions. `inFrames` should be an array of frames if there's only one source, or an array of arrays if there are multiple sources. Frames may be libav.js `Frame`s or `AVFrame` pointers as numbers. Also requires a frame pointer (`framePtr`) to use for temporary storage, which can be allocated directly or by an encoding/decoding metafunction. `config` can be used to pass configuration options, or pass `true` as `config` as an equivalent of `config.fin`. Set `fin` if these are the last input frames. There are multiple versions of `ff_copyout_frame`, only one of which is actually used to copy out frames. See the documentation of `ff_copyout_frame` above for how to use the `copyoutFrame` option. ### `ff_decode_filter_multi` ``` ff_decode_filter_multi( ctx: number, buffersrc_ctx: number, buffersink_ctx: number, pkt: number, frame: number, inPackets: (Packet | number)[], config?: boolean | { fin?: boolean, ignoreErrors?: boolean, copyoutFrame?: string } ): Promise ``` Combination of `ff_decode_multi` and `ff_filter_multi` in a single function. Useful to avoid the overhead of copying data and waiting for Promises between decoding and filtering. `await ff_decode_filter_multi(ctx, src, sink, pkt, frame, packets, config)` is equivalent to `await ff_filter_multi(src, sink, frame, await ff_decode_multi(ctx, pkt, frame, packets), config)`. That is, this really just combines the two calls. However, internally, neither frame copying nor Promises are used. # Filesystem The `readFile`, `writeFile`, `unlink`, and `mkdev` functions are provided directly from Emscripten's filesystem module. In addition, functions to create streaming devices are provided. These are further documented in [IO.md](IO.md). ## Reader device ### `mkreaderdev` ``` mkreaderdev(name: string, mode?: number): Promise ``` Make a reader device. This is used to stream data, and acts like a Unix character device. The mode is usually unnecessary. ### `mkblockreaderdev` ``` mkblockreaderdev(name: string, size: number): Promise ``` Similar to `mkreaderdev`, but `mkreaderdev` creates a character device (streaming device), whereas `mkblockreaderdev` creates a block device, so it will have a size and random access. To intercept read requests from the block reader device(s), you must set `libav.onblockread` to a function `(name: string, position: number, length: number) => void`. Because `onblockread` is a callback, it is usually possible to create a block reader device and its callback and then use the API as if no devices are used. ### `ff_reader_dev_send` ``` ff_reader_dev_send(name: string, data: Uint8Array): Promise ``` Send data to a reader device. There is no limit imposed by libav.js to how much data a reader device can buffer. ### `ff_block_reader_dev_send` ``` ff_block_reader_dev_send( name: string, position: number, data: Uint8Array ): Promise ``` Similar to `ff_reader_dev_send`, but for block reader devices, and has a position specified for the data. ### `ff_reader_dev_waiting` ``` ff_reader_dev_waiting(name?: string): Promise ``` Returns `true` if one or more reader or block reader devices are waiting for data. Give the filename to check if a *specific* reader device is waiting, or no name to check if *any* reader device is waiting. Typically, this is used along with `ff_init_demuxer_file` and `ff_read_frame_multi` to feed in data. If instead of awaiting those promises, you store the promises aside, you can loop with `while (await libav.ff_reader_dev_waiting())` and send data, *then* await the file-reading promise. In this way, you can control the transfer of input data without having to send entire files or predict how much data might be needed. ## Writer device ### `mkwriterdev` ``` mkwriterdev(name: string, mode?: number): Promise ``` Make a writer device. This is generally used to stream data, but some formats aren't streamable, so this acts a bit more like a Unix *block* device. To receive data from the writer device(s), you must set `libav.onwrite` to a function `(name: string, position: number, buffer: Uint8Array) => void`. ### `mkstreamwriterdev` ``` mkstreamwriterdev(name: string, mode? number): Promise ``` Make a stream writer device. Identical to a writer device except that seeking is not allowed, so libav treats it like a stream. Most formats either don't care or simply won't work with a stream, but certain formats (like `wav` and `matroska`) will behave differently if the output is a stream than if it's a block device. Receive data in the same way as with `mkwriterdev`. # CLI If a variant is used that employs the `cli` fragment, then the entire `ffmpeg` and `ffprobe` CLIs are exposed, as well as the various libav interfaces. ### `ffmpeg` ``` ffmpeg(...args: (string | string[])[]): Promise ``` Runs the `ffmpeg` CLI tool. `args` can be each string argument to `ffmpeg`, or an array of strings, or any combination thereof. Returns `ffmpeg`'s exit code. NOTE: ffmpeg 6.0 and later require threads for the ffmpeg CLI. libav.js *does* support the ffmpeg CLI on unthreaded environments, but to do so, it uses a thread compatibility layer, [emfiberthreads](https://github.com/Yahweasel/emfiberthreads). This is just a compatibility layer, and is expected not only to be slower than true multithreading, but to be slower than single-threaded operation. ### `ffprobe` ``` ffprobe(...args: (string | string[])[]): Promise ``` Like `ffmpeg`, but for the `ffprobe` tool.