ArrayBuffer

The ArrayBuffer object, TypedArray views, and DataView views are JavaScript interfaces for manipulating binary data. These objects have existed for a long time as part of a separate specification (published in February 2011). ES6 incorporated them into the ECMAScript specification and added new methods. They all handle binary data using array syntax, so they are collectively called binary arrays. The original design purpose of this interface is related to the WebGL project. WebGL refers to the communication interface between the browser and the graphics card. To meet the demand for large amounts of real-time data exchange between JavaScript and the graphics card, their data communication must be binary rather than the traditional text format. Transferring a 32-bit integer in text format requires both the JavaScript script and the graphics card to perform format conversion, which is very time-consuming. If there were a mechanism that could directly manipulate bytes, like in C, sending a 4-byte 32-bit integer to the graphics card in binary form without any conversion, script performance would be greatly improved.

Binary arrays were born in this context. They are very similar to C arrays, allowing developers to directly manipulate memory using array indices, greatly enhancing JavaScript's ability to handle binary data. This makes it possible for developers to perform binary communication with the operating system's native interfaces through JavaScript.

Binary arrays consist of three types of objects.

(1) ArrayBuffer object: Represents a segment of binary data in memory that can be manipulated through "views". Views implement the array interface, which means you can use array methods to manipulate memory.

(2) TypedArray views: Include 9 types of views, such as Uint8Array (unsigned 8-bit integer) array view, Int16Array (16-bit integer) array view, Float32Array (32-bit floating point) array view, and so on.

(3) DataView view: Allows custom composite format views. For example, the first byte is Uint8 (unsigned 8-bit integer), the second and third bytes are Int16 (16-bit integer), and from the fourth byte onward is Float32 (32-bit floating point), etc. It also allows custom byte order.

In simple terms, the ArrayBuffer object represents raw binary data, TypedArray views are used to read and write simple types of binary data, and DataView views are used to read and write complex types of binary data.

TypedArray views support a total of 9 data types (DataView views support all 8 types except Uint8C).

Data Type	Byte Length	Meaning	Corresponding C Type
Int8	1	8-bit signed integer	signed char
Uint8	1	8-bit unsigned integer	unsigned char
Uint8C	1	8-bit unsigned integer (with overflow clamping)	unsigned char
Int16	2	16-bit signed integer	short
Uint16	2	16-bit unsigned integer	unsigned short
Int32	4	32-bit signed integer	int
Uint32	4	32-bit unsigned integer	unsigned int
Float32	4	32-bit floating point	float
Float64	8	64-bit floating point	double

Note that binary arrays are not actual arrays, but array-like objects.

Many browser APIs use binary arrays to manipulate binary data. Here are some of them.

• Canvas
• Fetch API
• File API
• WebSockets
• XMLHttpRequest

ArrayBuffer Object

Overview

The ArrayBuffer object represents a segment of memory that stores binary data. It cannot be read or written directly; it can only be read and written through views (TypedArray views and DataView views). The purpose of views is to interpret binary data in a specified format.

ArrayBuffer is also a constructor that can allocate a contiguous memory region for storing data.

const buf = new ArrayBuffer(32);

The code above generates a 32-byte memory region where each byte defaults to 0. As you can see, the parameter of the ArrayBuffer constructor is the required memory size (in bytes).

To read and write this memory, you need to specify a view for it. Creating a DataView view requires providing an ArrayBuffer object instance as a parameter.

const buf = new ArrayBuffer(32);
const dataView = new DataView(buf);
dataView.getUint8(0) // 0

The code above creates a DataView view for a 32-byte memory segment, then reads 8 bits of binary data from the beginning in unsigned 8-bit integer format. The result is 0 because the original ArrayBuffer object defaults all bits to 0.

Another type of view, TypedArray, differs from DataView in that it is not a single constructor but a group of constructors representing different data formats.

const buffer = new ArrayBuffer(12);

const x1 = new Int32Array(buffer);
x1[0] = 1;
const x2 = new Uint8Array(buffer);
x2[0]  = 2;

x1[0] // 2

The code above creates two views for the same memory segment: a 32-bit signed integer (Int32Array constructor) and an 8-bit unsigned integer (Uint8Array constructor). Since both views correspond to the same memory, modifying the underlying memory through one view will affect the other view.

The TypedArray view constructors can accept not only ArrayBuffer instances as parameters but also regular arrays as parameters, directly allocating memory to generate the underlying ArrayBuffer instance while simultaneously assigning values to that memory.

const typedArray = new Uint8Array([0,1,2]);
typedArray.length // 3

typedArray[0] = 5;
typedArray // [5, 1, 2]

The code above uses the Uint8Array constructor of TypedArray views to create a new unsigned 8-bit integer view. As you can see, Uint8Array directly uses a regular array as a parameter, and the assignment of the underlying memory is completed at the same time.

ArrayBuffer.prototype.byteLength

The byteLength property of an ArrayBuffer instance returns the byte length of the allocated memory region.

const buffer = new ArrayBuffer(32);
buffer.byteLength
// 32

If the memory region to be allocated is very large, allocation may fail (due to insufficient contiguous free memory), so it is necessary to check whether allocation was successful.

if (buffer.byteLength === n) {
  // success
} else {
  // failure
}

ArrayBuffer.prototype.slice()

ArrayBuffer instances have a slice method that allows copying part of the memory region to generate a new ArrayBuffer object.

const buffer = new ArrayBuffer(8);
const newBuffer = buffer.slice(0, 3);

The code above copies the first 3 bytes of the buffer object (from 0 to just before byte 3) to generate a new ArrayBuffer object. The slice method actually involves two steps: first, allocating a new segment of memory, and second, copying the original ArrayBuffer object to it.

The slice method accepts two parameters: the first parameter indicates the starting byte number for copying (inclusive), and the second parameter indicates the ending byte number (exclusive). If the second parameter is omitted, it defaults to the end of the original ArrayBuffer object.

Besides the slice method, the ArrayBuffer object does not provide any direct methods for reading or writing memory. It only allows views to be built on top of it, and then reading and writing is done through the views.

ArrayBuffer.isView()

ArrayBuffer has a static method isView that returns a boolean value indicating whether the parameter is a view instance of ArrayBuffer. This method is roughly equivalent to checking whether the parameter is a TypedArray instance or a DataView instance.

const buffer = new ArrayBuffer(8);
ArrayBuffer.isView(buffer) // false

const v = new Int32Array(buffer);
ArrayBuffer.isView(v) // true

TypedArray Views

Overview

An ArrayBuffer object, as a memory region, can store multiple types of data. The same memory segment can be interpreted differently depending on the data type -- this is called a "view". ArrayBuffer has two types of views: TypedArray views and DataView views. The former's array members are all of the same data type, while the latter's array members can be different data types.

Currently, TypedArray views include 9 types in total, each being a constructor.

• Int8Array: 8-bit signed integer, 1 byte in length.
• Uint8Array: 8-bit unsigned integer, 1 byte in length.
• Uint8ClampedArray: 8-bit unsigned integer, 1 byte in length, with different overflow handling.
• Int16Array: 16-bit signed integer, 2 bytes in length.
• Uint16Array: 16-bit unsigned integer, 2 bytes in length.
• Int32Array: 32-bit signed integer, 4 bytes in length.
• Uint32Array: 32-bit unsigned integer, 4 bytes in length.
• Float32Array: 32-bit floating point, 4 bytes in length.
• Float64Array: 64-bit floating point, 8 bytes in length.

The arrays generated by these 9 constructors are collectively called TypedArray views. They are very similar to regular arrays: they all have a length property, individual elements can be accessed using the bracket operator ([]), and all array methods can be used on them. The main differences between regular arrays and TypedArray arrays are as follows.

• All members of a TypedArray array are of the same type.
• Members of a TypedArray array are contiguous, with no empty slots.
• The default value of TypedArray array members is 0. For example, new Array(10) returns a regular array without any members -- just 10 empty slots; new Uint8Array(10) returns a TypedArray array where all 10 members are 0.
• A TypedArray array is just a layer of view and does not store data itself. Its data is stored in the underlying ArrayBuffer object, and the buffer property must be used to access the underlying object.

Constructors

TypedArray arrays provide 9 constructors for generating array instances of the corresponding type.

Constructors have multiple uses.

(1) TypedArray(buffer, byteOffset=0, length?)

Multiple views can be built on the same ArrayBuffer object based on different data types.

// Create an 8-byte ArrayBuffer
const b = new ArrayBuffer(8);

// Create an Int32 view pointing to b, starting at byte 0, until the end of the buffer
const v1 = new Int32Array(b);

// Create a Uint8 view pointing to b, starting at byte 2, until the end of the buffer
const v2 = new Uint8Array(b, 2);

// Create an Int16 view pointing to b, starting at byte 2, with length 2
const v3 = new Int16Array(b, 2, 2);

The code above generates three views on an 8-byte memory segment (b): v1, v2, and v3.

The view constructor accepts three parameters:

• First parameter (required): The underlying ArrayBuffer object for the view.
• Second parameter (optional): The starting byte number for the view, defaulting to 0.
• Third parameter (optional): The number of data elements the view contains, defaulting to the end of the memory region.

Therefore, v1, v2, and v3 overlap: v1[0] is a 32-bit integer pointing to bytes 0-3; v2[0] is an 8-bit unsigned integer pointing to byte 2; v3[0] is a 16-bit integer pointing to bytes 2-3. Any modification to memory by one view will be reflected in the other two views.

Note that byteOffset must be consistent with the data type being created, otherwise an error will be thrown.

const buffer = new ArrayBuffer(8);
const i16 = new Int16Array(buffer, 1);
// Uncaught RangeError: start offset of Int16Array should be a multiple of 2

In the code above, a new 8-byte ArrayBuffer object is created, then a signed 16-bit integer view is created starting at the first byte, resulting in an error. This is because a signed 16-bit integer requires two bytes, so the byteOffset parameter must be divisible by 2.

If you want to interpret an ArrayBuffer object starting from any byte, you must use the DataView view, because TypedArray views only provide 9 fixed interpretation formats.

(2) TypedArray(length)

Views can also be generated by directly allocating memory without going through an ArrayBuffer object.

const f64a = new Float64Array(8);
f64a[0] = 10;
f64a[1] = 20;
f64a[2] = f64a[0] + f64a[1];

The code above generates a Float64Array array with 8 members (64 bytes total), then assigns values to each member sequentially. In this case, the parameter of the view constructor is the number of members. As you can see, assignment operations on view arrays are identical to regular array operations.

(3) TypedArray(typedArray)

The constructor of a TypedArray array can accept another TypedArray instance as a parameter.

const typedArray = new Int8Array(new Uint8Array(4));

In the code above, the Int8Array constructor accepts a Uint8Array instance as a parameter.

Note that the newly generated array only copies the values from the parameter array; the underlying memory is different. The new array will allocate a new segment of memory to store data, rather than building a view on the original array's memory.

const x = new Int8Array([1, 1]);
const y = new Int8Array(x);
x[0] // 1
y[0] // 1

x[0] = 2;
y[0] // 1

In the code above, array y is generated using array x as a template. When x changes, y does not change.

If you want to construct different views based on the same memory, you can use the following approach.

const x = new Int8Array([1, 1]);
const y = new Int8Array(x.buffer);
x[0] // 1
y[0] // 1

x[0] = 2;
y[0] // 2

(4) TypedArray(arrayLikeObject)

The constructor parameter can also be a regular array, which directly generates a TypedArray instance.

const typedArray = new Uint8Array([1, 2, 3, 4]);

Note that in this case, the TypedArray view will allocate new memory and will not build a view on the original array's memory.

The code above generates an unsigned 8-bit integer TypedArray instance from a regular array.

TypedArray arrays can also be converted back to regular arrays.

const normalArray = [...typedArray];
// or
const normalArray = Array.from(typedArray);
// or
const normalArray = Array.prototype.slice.call(typedArray);

Array Methods

Regular array methods and properties are fully applicable to TypedArray arrays.

• TypedArray.prototype.copyWithin(target, start[, end = this.length])
• TypedArray.prototype.entries()
• TypedArray.prototype.every(callbackfn, thisArg?)
• TypedArray.prototype.fill(value, start=0, end=this.length)
• TypedArray.prototype.filter(callbackfn, thisArg?)
• TypedArray.prototype.find(predicate, thisArg?)
• TypedArray.prototype.findIndex(predicate, thisArg?)
• TypedArray.prototype.forEach(callbackfn, thisArg?)
• TypedArray.prototype.indexOf(searchElement, fromIndex=0)
• TypedArray.prototype.join(separator)
• TypedArray.prototype.keys()
• TypedArray.prototype.lastIndexOf(searchElement, fromIndex?)
• TypedArray.prototype.map(callbackfn, thisArg?)
• TypedArray.prototype.reduce(callbackfn, initialValue?)
• TypedArray.prototype.reduceRight(callbackfn, initialValue?)
• TypedArray.prototype.reverse()
• TypedArray.prototype.slice(start=0, end=this.length)
• TypedArray.prototype.some(callbackfn, thisArg?)
• TypedArray.prototype.sort(comparefn)
• TypedArray.prototype.toLocaleString(reserved1?, reserved2?)
• TypedArray.prototype.toString()
• TypedArray.prototype.values()

For the usage of all the methods above, please refer to the array methods introduction, which will not be repeated here.

Note that TypedArray arrays do not have a concat method. If you want to merge multiple TypedArray arrays, you can use the following function.

function concatenate(resultConstructor, ...arrays) {
  let totalLength = 0;
  for (let arr of arrays) {
    totalLength += arr.length;
  }
  let result = new resultConstructor(totalLength);
  let offset = 0;
  for (let arr of arrays) {
    result.set(arr, offset);
    offset += arr.length;
  }
  return result;
}

concatenate(Uint8Array, Uint8Array.of(1, 2), Uint8Array.of(3, 4))
// Uint8Array [1, 2, 3, 4]

Additionally, TypedArray arrays, like regular arrays, have the Iterator interface deployed, so they can be iterated.

let ui8 = Uint8Array.of(0, 1, 2);
for (let byte of ui8) {
  console.log(byte);
}
// 0
// 1
// 2

Byte Order

Byte order refers to how numeric values are represented in memory.

const buffer = new ArrayBuffer(16);
const int32View = new Int32Array(buffer);

for (let i = 0; i < int32View.length; i++) {
  int32View[i] = i * 2;
}

The code above generates a 16-byte ArrayBuffer object, then builds a 32-bit integer view on it. Since each 32-bit integer occupies 4 bytes, a total of 4 integers can be written: 0, 2, 4, and 6.

If a 16-bit integer view is then built on this data, completely different results can be read.

const int16View = new Int16Array(buffer);

for (let i = 0; i < int16View.length; i++) {
  console.log("Entry " + i + ": " + int16View[i]);
}
// Entry 0: 0
// Entry 1: 0
// Entry 2: 2
// Entry 3: 0
// Entry 4: 4
// Entry 5: 0
// Entry 6: 6
// Entry 7: 0

Since each 16-bit integer occupies 2 bytes, the entire ArrayBuffer object is now divided into 8 segments. Then, because x86 architecture computers use little-endian byte order, where the more significant bytes are stored at higher memory addresses and the less significant bytes are stored at lower memory addresses, the results above are obtained.

For example, a 4-byte hexadecimal number 0x12345678 has "12" as the most significant byte and "78" as the least significant byte. Little-endian byte order places the least significant byte first, so the storage order is 78563412; big-endian byte order is the complete opposite, placing the most significant byte first, so the storage order is 12345678. Currently, almost all personal computers use little-endian byte order, so TypedArray arrays also use little-endian byte order for reading and writing data internally, or more precisely, they read and write data according to the byte order set by the host operating system.

This does not mean that big-endian byte order is unimportant. In fact, many network devices and certain operating systems use big-endian byte order. This creates a serious problem: if data is in big-endian byte order, TypedArray arrays cannot parse it correctly because they can only handle little-endian byte order! To solve this problem, JavaScript introduced the DataView object, which allows setting byte order, as will be discussed in detail below.

Here is another example.

// Assume a buffer segment contains the following bytes [0x02, 0x01, 0x03, 0x07]
const buffer = new ArrayBuffer(4);
const v1 = new Uint8Array(buffer);
v1[0] = 2;
v1[1] = 1;
v1[2] = 3;
v1[3] = 7;

const uInt16View = new Uint16Array(buffer);

// The computer uses little-endian byte order
// so the first two bytes equal 258
if (uInt16View[0] === 258) {
  console.log('OK'); // "OK"
}

// Assignment operations
uInt16View[0] = 255;    // bytes become [0xFF, 0x00, 0x03, 0x07]
uInt16View[0] = 0xff05; // bytes become [0x05, 0xFF, 0x03, 0x07]
uInt16View[1] = 0x0210; // bytes become [0x05, 0xFF, 0x10, 0x02]

The following function can be used to determine whether the current view uses little-endian or big-endian byte order.

const BIG_ENDIAN = Symbol('BIG_ENDIAN');
const LITTLE_ENDIAN = Symbol('LITTLE_ENDIAN');

function getPlatformEndianness() {
  let arr32 = Uint32Array.of(0x12345678);
  let arr8 = new Uint8Array(arr32.buffer);
  switch ((arr8[0]*0x1000000) + (arr8[1]*0x10000) + (arr8[2]*0x100) + (arr8[3])) {
    case 0x12345678:
      return BIG_ENDIAN;
    case 0x78563412:
      return LITTLE_ENDIAN;
    default:
      throw new Error('Unknown endianness');
  }
}

In summary, compared to regular arrays, the biggest advantage of TypedArray arrays is the ability to directly manipulate memory without data type conversion, making them much faster.

BYTES_PER_ELEMENT Property

Each view constructor has a BYTES_PER_ELEMENT property indicating the number of bytes this data type occupies.

Int8Array.BYTES_PER_ELEMENT // 1
Uint8Array.BYTES_PER_ELEMENT // 1
Uint8ClampedArray.BYTES_PER_ELEMENT // 1
Int16Array.BYTES_PER_ELEMENT // 2
Uint16Array.BYTES_PER_ELEMENT // 2
Int32Array.BYTES_PER_ELEMENT // 4
Uint32Array.BYTES_PER_ELEMENT // 4
Float32Array.BYTES_PER_ELEMENT // 4
Float64Array.BYTES_PER_ELEMENT // 8

This property can also be accessed on TypedArray instances, i.e., TypedArray.prototype.BYTES_PER_ELEMENT.

Conversion Between ArrayBuffer and Strings

For converting between ArrayBuffer and strings, use the native TextEncoder and TextDecoder methods. For clarity, the following code includes TypeScript type signatures.

/**
 * Convert ArrayBuffer/TypedArray to String via TextDecoder
 *
 * @see https://developer.mozilla.org/en-US/docs/Web/API/TextDecoder
 */
function ab2str(
  input: ArrayBuffer | Uint8Array | Int8Array | Uint16Array | Int16Array | Uint32Array | Int32Array,
  outputEncoding: string = 'utf8',
): string {
  const decoder = new TextDecoder(outputEncoding)
  return decoder.decode(input)
}

/**
 * Convert String to ArrayBuffer via TextEncoder
 *
 * @see https://developer.mozilla.org/zh-CN/docs/Web/API/TextEncoder
 */
function str2ab(input: string): ArrayBuffer {
  const view = str2Uint8Array(input)
  return view.buffer
}

/** Convert String to Uint8Array */
function str2Uint8Array(input: string): Uint8Array {
  const encoder = new TextEncoder()
  const view = encoder.encode(input)
  return view
}

In the code above, the second parameter outputEncoding of ab2str() specifies the output encoding. It generally keeps the default value (utf-8). For other possible values, refer to the official documentation (opens new window) or Node.js documentation (opens new window).

Overflow

The range of values that different view types can hold is fixed. Exceeding this range results in overflow. For example, an 8-bit view can only hold an 8-bit binary value. If a 9-bit value is placed in it, overflow occurs.

The overflow handling rule for TypedArray arrays is simply to discard the overflowing bits and then interpret according to the view type.

const uint8 = new Uint8Array(1);

uint8[0] = 256;
uint8[0] // 0

uint8[0] = -1;
uint8[0] // 255

In the code above, uint8 is an 8-bit view, and 256 in binary is a 9-bit value 100000000, which causes overflow. According to the rule, only the last 8 bits are kept, i.e., 00000000. The interpretation rule for a uint8 view is unsigned 8-bit integer, so 00000000 is 0.

Negative numbers are represented internally using "two's complement", which means taking the corresponding positive value, performing a NOT operation, then adding 1. For example, the positive value corresponding to -1 is 1. After the NOT operation, we get 11111110, and adding 1 gives the two's complement form 11111111. uint8 interprets 11111111 as an unsigned 8-bit integer, returning a result of 255.

A simple conversion rule can be expressed as follows.

• Positive overflow: When the input value is greater than the maximum value of the current data type, the result equals the minimum value of the current data type plus the remainder, minus 1.
• Negative overflow (underflow): When the input value is less than the minimum value of the current data type, the result equals the maximum value of the current data type minus the absolute value of the remainder, plus 1.

The "remainder" mentioned above is the result of the modulo operation, i.e., the result of the % operator in JavaScript.

12 % 4 // 0
12 % 5 // 2

In the code above, 12 divided by 4 has no remainder, while dividing by 5 gives a remainder of 2.

See the following example.

const int8 = new Int8Array(1);

int8[0] = 128;
int8[0] // -128

int8[0] = -129;
int8[0] // 127

In the example above, int8 is a signed 8-bit integer view with a maximum value of 127 and a minimum value of -128. When the input value is 128, it represents a positive overflow of 1. According to the rule "minimum value plus the remainder (the remainder of 128 divided by 127 is 1), minus 1", it returns -128. When the input value is -129, it represents a negative overflow of 1. According to the rule "maximum value minus the absolute value of the remainder (the absolute value of the remainder of -129 divided by -128 is 1), plus 1", it returns 127.

The overflow rule for Uint8ClampedArray views is different from the rules above. It specifies that any positive overflow results in the maximum value of the current data type, which is 255; and any negative overflow results in the minimum value of the current data type, which is 0.

const uint8c = new Uint8ClampedArray(1);

uint8c[0] = 256;
uint8c[0] // 255

uint8c[0] = -1;
uint8c[0] // 0

In the example above, uint8C is a Uint8ClampedArray view. Positive overflow always returns 255, and negative overflow always returns 0.

TypedArray.prototype.buffer

The buffer property of a TypedArray instance returns the ArrayBuffer object corresponding to the entire memory region. This property is read-only.

const a = new Float32Array(64);
const b = new Uint8Array(a.buffer);

In the code above, the a view object and the b view object correspond to the same ArrayBuffer object, i.e., the same memory segment.

TypedArray.prototype.byteLength, TypedArray.prototype.byteOffset

The byteLength property returns the memory length occupied by the TypedArray array in bytes. The byteOffset property returns the byte from which the TypedArray array starts in the underlying ArrayBuffer object. Both properties are read-only.

const b = new ArrayBuffer(8);

const v1 = new Int32Array(b);
const v2 = new Uint8Array(b, 2);
const v3 = new Int16Array(b, 2, 2);

v1.byteLength // 8
v2.byteLength // 6
v3.byteLength // 4

v1.byteOffset // 0
v2.byteOffset // 2
v3.byteOffset // 2

TypedArray.prototype.length

The length property indicates how many members the TypedArray array contains. Note the difference between the length property and the byteLength property: the former is the member count, while the latter is the byte count.

const a = new Int16Array(8);

a.length // 8
a.byteLength // 16

TypedArray.prototype.set()

The set method of TypedArray arrays is used to copy arrays (regular arrays or TypedArray arrays), i.e., to completely copy one segment of content into another memory segment.

const a = new Uint8Array(8);
const b = new Uint8Array(8);

b.set(a);

The code above copies the contents of array a into array b. This is a whole-memory copy, which is much faster than copying members one by one.

The set method can also accept a second parameter indicating from which member of the b object to start copying the a object.

const a = new Uint16Array(8);
const b = new Uint16Array(10);

b.set(a, 2)

The b array in the code above has two more members than the a array, so copying starts from b[2].

TypedArray.prototype.subarray()

The subarray method creates a new view for a portion of a TypedArray array.

const a = new Uint16Array(8);
const b = a.subarray(2,3);

a.byteLength // 16
b.byteLength // 2

The first parameter of the subarray method is the starting member index, and the second parameter is the ending member index (exclusive). If omitted, it includes all remaining members. So a.subarray(2,3) in the code above means b contains only one member a[2], with a byte length of 2.

TypedArray.prototype.slice()

The slice method of TypeArray instances can return a new TypedArray instance at a specified position.

let ui8 = Uint8Array.of(0, 1, 2);
ui8.slice(-1)
// Uint8Array [ 2 ]

In the code above, ui8 is an instance of the 8-bit unsigned integer array view. Its slice method can return a new view instance from the current view.

The parameter of the slice method represents the specific position in the original array from which to start generating the new array. A negative value indicates a reverse position, i.e., -1 is the last position, -2 is the second-to-last position, and so on.

TypedArray.of()

All constructors of TypedArray arrays have a static method of that converts parameters into a TypedArray instance.

Float32Array.of(0.151, -8, 3.7)
// Float32Array [ 0.151, -8, 3.7 ]

The following three methods all generate the same TypedArray array.

// Method 1
let tarr = new Uint8Array([1,2,3]);

// Method 2
let tarr = Uint8Array.of(1,2,3);

// Method 3
let tarr = new Uint8Array(3);
tarr[0] = 1;
tarr[1] = 2;
tarr[2] = 3;

TypedArray.from()

The static method from accepts an iterable data structure (such as an array) as a parameter and returns a TypedArray instance based on this structure.

Uint16Array.from([0, 1, 2])
// Uint16Array [ 0, 1, 2 ]

This method can also convert one type of TypedArray instance into another type.

const ui16 = Uint16Array.from(Uint8Array.of(0, 1, 2));
ui16 instanceof Uint16Array // true

The from method can also accept a function as its second parameter, which iterates over each element, similar to the map method.

Int8Array.of(127, 126, 125).map(x => 2 * x)
// Int8Array [ -2, -4, -6 ]

Int16Array.from(Int8Array.of(127, 126, 125), x => 2 * x)
// Int16Array [ 254, 252, 250 ]

In the example above, the from method did not cause overflow, which indicates that the iteration is not performed on the original 8-bit integer array. In other words, from copies the TypedArray array specified by the first parameter into another memory segment, processes it, and then converts the result into the specified array format.

Composite Views

Since view constructors can specify starting positions and lengths, different types of data can be stored sequentially within the same memory segment. This is called a "composite view".

const buffer = new ArrayBuffer(24);

const idView = new Uint32Array(buffer, 0, 1);
const usernameView = new Uint8Array(buffer, 4, 16);
const amountDueView = new Float32Array(buffer, 20, 1);

The code above divides a 24-byte ArrayBuffer object into three parts:

• Bytes 0 to 3: one 32-bit unsigned integer
• Bytes 4 to 19: sixteen 8-bit integers
• Bytes 20 to 23: one 32-bit floating point number

This data structure can be described in C as follows:

struct someStruct {
  unsigned long id;
  char username[16];
  float amountDue;
};

DataView View

If data includes multiple types (such as HTTP data from a server), in addition to building composite views on ArrayBuffer objects, you can also operate through DataView views.

The DataView view provides more operation options and supports setting byte order. By design, the various TypedArray views of ArrayBuffer objects are intended for sending data to local devices like network cards and sound cards, so using the local byte order is sufficient. The DataView view, on the other hand, is designed to process data from network devices, so big-endian or little-endian byte order can be set independently.

The DataView view itself is a constructor that accepts an ArrayBuffer object as a parameter to generate a view.

DataView(ArrayBuffer buffer [, byte start position [, length]]);

Here is an example.

const buffer = new ArrayBuffer(24);
const dv = new DataView(buffer);

DataView instances have the following properties, with the same meanings as the same-named methods on TypedArray instances.

• DataView.prototype.buffer: Returns the corresponding ArrayBuffer object
• DataView.prototype.byteLength: Returns the occupied memory byte length
• DataView.prototype.byteOffset: Returns the byte from which the current view starts in the corresponding ArrayBuffer object

DataView instances provide 8 methods for reading memory.

• getInt8: Reads 1 byte, returns an 8-bit integer.
• getUint8: Reads 1 byte, returns an unsigned 8-bit integer.
• getInt16: Reads 2 bytes, returns a 16-bit integer.
• getUint16: Reads 2 bytes, returns an unsigned 16-bit integer.
• getInt32: Reads 4 bytes, returns a 32-bit integer.
• getUint32: Reads 4 bytes, returns an unsigned 32-bit integer.
• getFloat32: Reads 4 bytes, returns a 32-bit floating point number.
• getFloat64: Reads 8 bytes, returns a 64-bit floating point number.

All these get methods take a byte index as a parameter (which cannot be negative, otherwise an error is thrown), indicating from which byte to start reading.

const buffer = new ArrayBuffer(24);
const dv = new DataView(buffer);

// Read an 8-bit unsigned integer from byte 1
const v1 = dv.getUint8(0);

// Read a 16-bit unsigned integer from byte 2
const v2 = dv.getUint16(1);

// Read a 16-bit unsigned integer from byte 4
const v3 = dv.getUint16(3);

The code above reads the first 5 bytes of the ArrayBuffer object, including one 8-bit integer and two 16-bit integers.

When reading two or more bytes at once, the data storage method must be specified -- whether it is little-endian or big-endian byte order. By default, DataView's get methods interpret data using big-endian byte order. If you need to use little-endian byte order, you must specify true as the second parameter of the get method.

// Little-endian byte order
const v1 = dv.getUint16(1, true);

// Big-endian byte order
const v2 = dv.getUint16(3, false);

// Big-endian byte order
const v3 = dv.getUint16(3);

The DataView view provides 8 methods for writing to memory.

• setInt8: Writes 1 byte of 8-bit integer.
• setUint8: Writes 1 byte of 8-bit unsigned integer.
• setInt16: Writes 2 bytes of 16-bit integer.
• setUint16: Writes 2 bytes of 16-bit unsigned integer.
• setInt32: Writes 4 bytes of 32-bit integer.
• setUint32: Writes 4 bytes of 32-bit unsigned integer.
• setFloat32: Writes 4 bytes of 32-bit floating point number.
• setFloat64: Writes 8 bytes of 64-bit floating point number.

All these set methods accept two parameters: the first is the byte index indicating where to start writing, and the second is the data to write. For methods that write two or more bytes, a third parameter is needed: false or undefined means writing in big-endian byte order, and true means writing in little-endian byte order.

// Write a 32-bit integer with value 25 at byte 1 in big-endian byte order
dv.setInt32(0, 25, false);

// Write a 32-bit integer with value 25 at byte 5 in big-endian byte order
dv.setInt32(4, 25);

// Write a 32-bit floating point with value 2.5 at byte 9 in little-endian byte order
dv.setFloat32(8, 2.5, true);

If you are unsure about the byte order of the computer you are using, you can use the following method to determine it.

const littleEndian = (function() {
  const buffer = new ArrayBuffer(2);
  new DataView(buffer).setInt16(0, 256, true);
  return new Int16Array(buffer)[0] === 256;
})();

If it returns true, it is little-endian byte order; if it returns false, it is big-endian byte order.

Applications of Binary Arrays

Many Web APIs use ArrayBuffer objects and their view objects.

AJAX

Traditionally, AJAX operations on servers could only return text data, with the responseType property defaulting to text. XMLHttpRequest version 2 (XHR2) allows servers to return binary data, which falls into two cases. If the type of returned binary data is known, the return type (responseType) can be set to arraybuffer; if unknown, it should be set to blob.

let xhr = new XMLHttpRequest();
xhr.open('GET', someUrl);
xhr.responseType = 'arraybuffer';

xhr.onload = function () {
  let arrayBuffer = xhr.response;
  // ···
};

xhr.send();

If you know the returned data consists of 32-bit integers, you can handle it as follows.

xhr.onreadystatechange = function () {
  if (req.readyState === 4 ) {
    const arrayResponse = xhr.response;
    const dataView = new DataView(arrayResponse);
    const ints = new Uint32Array(dataView.byteLength / 4);

    xhrDiv.style.backgroundColor = "#00FF00";
    xhrDiv.innerText = "Array is " + ints.length + "uints long";
  }
}

Canvas

The binary pixel data output by the Canvas element on a web page is a TypedArray array.

const canvas = document.getElementById('myCanvas');
const ctx = canvas.getContext('2d');

const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);
const uint8ClampedArray = imageData.data;

Note that uint8ClampedArray in the code above, while being a TypedArray array, has a view type that is a specialized type for Canvas elements called Uint8ClampedArray. The characteristic of this view type is that it is specifically designed for colors, interpreting each byte as an unsigned 8-bit integer that can only take values from 0 to 255, and automatically filters out high-bit overflow during operations. This brings great convenience to image processing.

For example, if pixel color values are set to Uint8Array type, then when multiplied by a gamma value, the calculation must be:

u8[i] = Math.min(255, Math.max(0, u8[i] * gamma));

Because the Uint8Array type automatically converts results greater than 255 (such as 0xFF+1) to 0x00, image processing must be done this way. This is cumbersome and affects performance. If color values are set to Uint8ClampedArray type, the calculation is much simpler.

pixels[i] *= gamma;

The Uint8ClampedArray type ensures that values less than 0 are set to 0 and values greater than 255 are set to 255. Note that IE 10 does not support this type.

WebSocket

WebSocket can send or receive binary data through ArrayBuffer.

let socket = new WebSocket('ws://127.0.0.1:8081');
socket.binaryType = 'arraybuffer';

// Wait until socket is open
socket.addEventListener('open', function (event) {
  // Send binary data
  const typedArray = new Uint8Array(4);
  socket.send(typedArray.buffer);
});

// Receive binary data
socket.addEventListener('message', function (event) {
  const arrayBuffer = event.data;
  // ···
});

Fetch API

Data retrieved by the Fetch API is an ArrayBuffer object.

fetch(url)
.then(function(response){
  return response.arrayBuffer()
})
.then(function(arrayBuffer){
  // ...
});

File API

If you know the binary data type of a file, you can also read the file as an ArrayBuffer object.

const fileInput = document.getElementById('fileInput');
const file = fileInput.files[0];
const reader = new FileReader();
reader.readAsArrayBuffer(file);
reader.onload = function () {
  const arrayBuffer = reader.result;
  // ···
};

Below is an example of processing a bmp file. Assuming the file variable is a file object pointing to a bmp file, first read the file.

const reader = new FileReader();
reader.addEventListener("load", processimage, false);
reader.readAsArrayBuffer(file);

Then, define the callback function for processing the image: first build a DataView view on the binary data, then create a bitmap object to store the processed data, and finally display the image in a Canvas element.

function processimage(e) {
  const buffer = e.target.result;
  const datav = new DataView(buffer);
  const bitmap = {};
  // Specific processing steps
}

When processing the image data specifically, first handle the bmp file header. For the format and definition of each file header, please refer to relevant documentation.

bitmap.fileheader = {};
bitmap.fileheader.bfType = datav.getUint16(0, true);
bitmap.fileheader.bfSize = datav.getUint32(2, true);
bitmap.fileheader.bfReserved1 = datav.getUint16(6, true);
bitmap.fileheader.bfReserved2 = datav.getUint16(8, true);
bitmap.fileheader.bfOffBits = datav.getUint32(10, true);

Next, process the image metadata section.

bitmap.infoheader = {};
bitmap.infoheader.biSize = datav.getUint32(14, true);
bitmap.infoheader.biWidth = datav.getUint32(18, true);
bitmap.infoheader.biHeight = datav.getUint32(22, true);
bitmap.infoheader.biPlanes = datav.getUint16(26, true);
bitmap.infoheader.biBitCount = datav.getUint16(28, true);
bitmap.infoheader.biCompression = datav.getUint32(30, true);
bitmap.infoheader.biSizeImage = datav.getUint32(34, true);
bitmap.infoheader.biXPelsPerMeter = datav.getUint32(38, true);
bitmap.infoheader.biYPelsPerMeter = datav.getUint32(42, true);
bitmap.infoheader.biClrUsed = datav.getUint32(46, true);
bitmap.infoheader.biClrImportant = datav.getUint32(50, true);

Finally, process the pixel information of the image itself.

const start = bitmap.fileheader.bfOffBits;
bitmap.pixels = new Uint8Array(buffer, start);

At this point, all data from the image file has been fully processed. The next step can involve image transformation, format conversion, or display in a Canvas web element, as needed.

SharedArrayBuffer

JavaScript is single-threaded, and Web Workers introduced multi-threading: the main thread is used for user interaction, and Worker threads are used for computational tasks. Each thread's data is isolated, communicating through postMessage(). Here is an example.

// Main thread
const w = new Worker('myworker.js');

In the code above, the main thread creates a new Worker thread. This thread has a communication channel with the main thread. The main thread sends messages to the Worker thread via w.postMessage and listens for the Worker thread's responses through the message event.

// Main thread
w.postMessage('hi');
w.onmessage = function (ev) {
  console.log(ev.data);
}

In the code above, the main thread first sends a message hi, then prints the Worker thread's response after receiving it.

The Worker thread also listens for the message event to receive messages from the main thread and respond.

// Worker thread
onmessage = function (ev) {
  console.log(ev.data);
  postMessage('ho');
}

Data exchange between threads can be in various formats, not just strings -- it can also be binary data. This exchange uses a copy mechanism, where one process copies the data it needs to share and passes it to another process via the postMessage method. If the data volume is large, this type of communication is obviously less efficient. It is easy to imagine that a shared memory region could be set aside, accessible by both the main thread and Worker thread for reading and writing. This would greatly improve efficiency and simplify collaboration (unlike the cumbersome postMessage approach).

ES2017 introduced SharedArrayBuffer (opens new window), which allows Worker threads and the main thread to share the same memory block. The SharedArrayBuffer API is identical to ArrayBuffer, with the only difference being that the latter cannot share data.

// Main thread

// Create 1KB of shared memory
const sharedBuffer = new SharedArrayBuffer(1024);

// The main thread sends the shared memory address
w.postMessage(sharedBuffer);

// Build a view on the shared memory for writing data
const sharedArray = new Int32Array(sharedBuffer);

In the code above, the parameter of the postMessage method is a SharedArrayBuffer object.

The Worker thread retrieves data from the event's data property.

// Worker thread
onmessage = function (ev) {
  // The data shared by the main thread is the 1KB shared memory
  const sharedBuffer = ev.data;

  // Build a view on the shared memory for convenient reading and writing
  const sharedArray = new Int32Array(sharedBuffer);

  // ...
};

Shared memory can also be created in a Worker thread and sent to the main thread.

Like ArrayBuffer, SharedArrayBuffer itself cannot be read or written directly. Views must be built on it, and then reading and writing are done through the views.

// Allocate memory space for 100,000 32-bit integers
const sab = new SharedArrayBuffer(Int32Array.BYTES_PER_ELEMENT * 100000);

// Build a 32-bit integer view
const ia = new Int32Array(sab);  // ia.length == 100000

// Create a new prime number generator
const primes = new PrimeGenerator();

// Write 100,000 prime numbers into this memory space
for ( let i=0 ; i < ia.length ; i++ )
  ia[i] = primes.next();

// Send this shared memory to the Worker thread
w.postMessage(ia);

The Worker thread processes the data as follows after receiving it.

// Worker thread
let ia;
onmessage = function (ev) {
  ia = ev.data;
  console.log(ia.length); // 100000
  console.log(ia[37]); // outputs 163, because it is the 38th prime number
};

Atomics Object

The biggest problem with multi-threaded shared memory is how to prevent two threads from modifying the same address simultaneously, or in other words, when one thread modifies shared memory, there must be a mechanism for other threads to synchronize. The SharedArrayBuffer API provides the Atomics object, which ensures that all shared memory operations are "atomic" and can be synchronized across all threads.

What does "atomic operation" mean? In modern programming languages, a single ordinary instruction, after being processed by the compiler, becomes multiple machine instructions. If running in a single thread, this is not a problem. In a multi-threaded environment with shared memory, problems arise because during the execution of this group of machine instructions, instructions from other threads may be inserted, leading to incorrect results. See the following example.

// Main thread
ia[42] = 314159;  // original value 191
ia[37] = 123456;  // original value 163

// Worker thread
console.log(ia[37]);
console.log(ia[42]);
// Possible result
// 123456
// 191

In the code above, the original order in the main thread is to assign to position 42 first, then position 37. However, the compiler and CPU may change the execution order of these two operations for optimization (since they are independent of each other), assigning to position 37 first, then position 42. When execution is partway through, the Worker thread may come to read the data, resulting in printing 123456 and 191.

Here is another example.

// Main thread
const sab = new SharedArrayBuffer(Int32Array.BYTES_PER_ELEMENT * 100000);
const ia = new Int32Array(sab);

for (let i = 0; i < ia.length; i++) {
  ia[i] = primes.next(); // Place primes into ia
}

// Worker thread
ia[112]++; // Incorrect
Atomics.add(ia, 112, 1); // Correct

In the code above, having the Worker thread directly modify shared memory with ia[112]++ is incorrect. This statement is compiled into multiple machine instructions, and there is no guarantee that instructions from other processes will not be inserted between them. Imagine if two threads simultaneously execute ia[112]++ -- the results they get are likely both incorrect.

The Atomics object was proposed to solve this problem. It can ensure that the multiple machine instructions corresponding to an operation are always run as a whole without being interrupted. In other words, the operations it involves can all be seen as atomic single operations, which avoids thread competition and improves the safety of operations when multiple threads share memory. Therefore, ia[112]++ should be rewritten as Atomics.add(ia, 112, 1).

The Atomics object provides several methods.

(1) Atomics.store(), Atomics.load()

The store() method is used to write data to shared memory, and the load() method is used to read data from shared memory. Compared to direct read/write operations, their advantage is guaranteeing the atomicity of read/write operations.

Additionally, they solve another problem: multiple threads use a certain position in shared memory as a flag. Once the value at that position changes, a specific operation is performed. In this case, the assignment operation at that position must execute after all preceding operations that might modify memory have finished; and the value-reading operation at that position must execute before all subsequent operations that might read that position begin. The store and load methods can achieve this -- the compiler will not rearrange the execution order of machine instructions for optimization.

Atomics.load(array, index)
Atomics.store(array, index, value)

The store method accepts three parameters: a SharedBuffer view, a position index, and a value. It returns the value of sharedArray[index]. The load method only accepts two parameters: a SharedBuffer view and a position index, and also returns the value of sharedArray[index].

// Main thread main.js
ia[42] = 314159;  // original value 191
Atomics.store(ia, 37, 123456);  // original value 163

// Worker thread worker.js
while (Atomics.load(ia, 37) == 163);
console.log(ia[37]);  // 123456
console.log(ia[42]);  // 314159

In the code above, the main thread's Atomics.store assignment to position 42 is guaranteed to occur before the assignment to position 37. As long as position 37 equals 163, the Worker thread will not exit the loop. The values read from positions 37 and 42 are guaranteed to be after the Atomics.load operation.

Here is another example.

// Main thread
const worker = new Worker('worker.js');
const length = 10;
const size = Int32Array.BYTES_PER_ELEMENT * length;
// Create a shared memory segment
const sharedBuffer = new SharedArrayBuffer(size);
const sharedArray = new Int32Array(sharedBuffer);
for (let i = 0; i < 10; i++) {
  // Write 10 integers to shared memory
  Atomics.store(sharedArray, i, 0);
}
worker.postMessage(sharedBuffer);

In the code above, the main thread uses Atomics.store() to write data. Below is the Worker thread using Atomics.load() to read data.

// worker.js
self.addEventListener('message', (event) => {
  const sharedArray = new Int32Array(event.data);
  for (let i = 0; i < 10; i++) {
    const arrayValue = Atomics.load(sharedArray, i);
    console.log(`The item at array index ${i} is ${arrayValue}`);
  }
}, false);

(2) Atomics.exchange()

If a Worker thread needs to write data, it can use the Atomics.store() method mentioned above, or it can use the Atomics.exchange() method. The difference is that Atomics.store() returns the written value, while Atomics.exchange() returns the replaced value.

// Worker thread
self.addEventListener('message', (event) => {
  const sharedArray = new Int32Array(event.data);
  for (let i = 0; i < 10; i++) {
    if (i % 2 === 0) {
      const storedValue = Atomics.store(sharedArray, i, 1);
      console.log(`The item at array index ${i} is now ${storedValue}`);
    } else {
      const exchangedValue = Atomics.exchange(sharedArray, i, 2);
      console.log(`The item at array index ${i} was ${exchangedValue}, now 2`);
    }
  }
}, false);

The code above changes the values at even positions in shared memory to 1 and the values at odd positions to 2.

(3) Atomics.wait(), Atomics.wake()

Using a while loop to wait for notifications from the main thread is not very efficient. If used on the main thread, it would cause freezing. The Atomics object provides wait() and wake() methods for wait-and-notify. These two methods are equivalent to locking memory: when one thread is performing an operation, other threads are put to sleep (establishing a lock). When the operation is complete, those sleeping threads are woken up (releasing the lock).

// Worker thread
self.addEventListener('message', (event) => {
  const sharedArray = new Int32Array(event.data);
  const arrayIndex = 0;
  const expectedStoredValue = 50;
  Atomics.wait(sharedArray, arrayIndex, expectedStoredValue);
  console.log(Atomics.load(sharedArray, arrayIndex));
}, false);

In the code above, the Atomics.wait() method effectively tells the Worker thread that as long as the given condition is met (sharedArray at position 0 equals 50), the Worker thread enters sleep at this line.

Once the main thread changes the value at the specified position, it can wake up the Worker thread.

// Main thread
const newArrayValue = 100;
Atomics.store(sharedArray, 0, newArrayValue);
const arrayIndex = 0;
const queuePos = 1;
Atomics.wake(sharedArray, arrayIndex, queuePos);

In the code above, position 0 of sharedArray is changed to 100, then Atomics.wake() is executed to wake up one thread from the sleep queue at position 0 of sharedArray.

The usage format of the Atomics.wait() method is as follows.

Atomics.wait(sharedArray, index, value, timeout)

Its four parameters have the following meanings.

• sharedArray: The view array of the shared memory.
• index: The position in the view data (starting from 0).
• value: The expected value at that position. Once the actual value equals the expected value, sleep begins.
• timeout: An integer representing the time in milliseconds after which to automatically wake up. This parameter is optional and defaults to Infinity, meaning indefinite sleep that can only be woken by Atomics.wake().

The return value of Atomics.wait() is a string with three possible values. If sharedArray[index] is not equal to value, it returns the string not-equal; otherwise, it enters sleep. If woken by Atomics.wake(), it returns the string ok; if woken due to timeout, it returns the string timed-out.

The usage format of the Atomics.wake() method is as follows.

Atomics.wake(sharedArray, index, count)

Its three parameters have the following meanings.

• sharedArray: The view array of the shared memory.
• index: The position in the view data (starting from 0).
• count: The number of Worker threads to wake up, defaulting to Infinity.

Once the Atomics.wake() method wakes up sleeping Worker threads, it lets them continue running.

See the following example.

// Main thread
console.log(ia[37]);  // 163
Atomics.store(ia, 37, 123456);
Atomics.wake(ia, 37, 1);

// Worker thread
Atomics.wait(ia, 37, 163);
console.log(ia[37]);  // 123456

In the code above, position 37 of the view array ia originally has a value of 163. The Worker thread uses Atomics.wait(), specifying that as long as ia[37] equals 163, it enters sleep. The main thread uses Atomics.store() to write 123456 into ia[37], then uses Atomics.wake() to wake up the Worker thread.

Additionally, for a lock-based memory implementation based on the wait and wake methods, see Lars T Hansen's js-lock-and-condition (opens new window) library.

Note that the browser's main thread should not be set to sleep, as this would cause the user to lose responsiveness. Moreover, the main thread will actually refuse to enter sleep.

(4) Arithmetic Methods

Certain operations on shared memory must not be interrupted, meaning that during the operation, other threads cannot be allowed to modify values in memory. The Atomics object provides several arithmetic methods to prevent data from being modified.

Atomics.add(sharedArray, index, value)

Atomics.add adds value to sharedArray[index] and returns the old value of sharedArray[index].

Atomics.sub(sharedArray, index, value)

Atomics.sub subtracts value from sharedArray[index] and returns the old value of sharedArray[index].

Atomics.and(sharedArray, index, value)

Atomics.and performs a bitwise and operation between value and sharedArray[index], stores the result in sharedArray[index], and returns the old value.

Atomics.or(sharedArray, index, value)

Atomics.or performs a bitwise or operation between value and sharedArray[index], stores the result in sharedArray[index], and returns the old value.

Atomics.xor(sharedArray, index, value)

Atomic.xor performs a bitwise xor operation between vaule and sharedArray[index], stores the result in sharedArray[index], and returns the old value.

(5) Other Methods

The Atomics object also has the following methods.

• Atomics.compareExchange(sharedArray, index, oldval, newval): If sharedArray[index] equals oldval, writes newval and returns oldval.
• Atomics.isLockFree(size): Returns a boolean value indicating whether the Atomics object can handle memory locking of a certain size. If it returns false, the application needs to implement its own locking.

One use of Atomics.compareExchange is to read a value from a SharedArrayBuffer, perform some operation on it, and after the operation is complete, check whether the original value in the SharedArrayBuffer has changed (i.e., been modified by another thread). If it has not been modified, write it back to the original position; otherwise, read the new value and start the operation over from the beginning.