Description
When converting between different kinds of byte arrays like Buffer and Uint8Array, it is very easy to forget offset and length.
Consider this example:
function makeUint8Array() {
const data: Buffer = getNodeBufferFromSomewhere()
return new Uint8Array(data.buffer)
}Seems fine, right? You've got some buffer, converting it to a Uint8Array and returning that. TypeScript is happy with it and it probably works fine during testing. But one day your production server starts randomly including its TLS private key in client responses... 💥
What you should have written is this:
function makeUint8Array() {
const data: Buffer = getNodeBufferFromSomewhere()
return new Uint8Array(data.buffer, data.byteOffset, data.byteLength)
}The internal ArrayBuffer in a Node.js Buffer could be a larger slice of memory than just the memory that the Buffer represents. Often they are equivalent, sometimes they're not. TypeScript doesn't care because an ArrayBuffer is an ArrayBuffer.
The reason I'm proposing this rule is because I made this mistake once, understood what was going wrong, learned from it ... and then made the exact same mistake again a few months later. It's just easy to forget those extra parameters.
When dealing with ArrayBufferViews with BYTES_PER_ELEMENT !== 1 we also have to be careful to divide the byteLength by the number of bytes per element to get the length.
function makeUint32Array() {
const data: Buffer = getNodeBufferFromSomewhere()
return new Uint32Array(data.buffer, data.byteOffset, data.byteLength / Uint32Array.BYTES_PER_ELEMENT)
}Fail
// missing byteOffset and length parameters
const data!: Buffer
new Uint8Array(data.buffer)// missing length parameter
const data!: Buffer
new Uint8Array(data.buffer, data.byteOffset)// passing byteLength directly to a length property that expects something other than bytes
const data!: Buffer
new Uint32Array(data.buffer, data.byteOffset, data.byteLength)// passing byteLength divided by the wrong BYTES_PER_ELEMENT
const data!: Buffer
new Uint32Array(data.buffer, data.byteOffset, data.byteLength / Uint16Array.BYTES_PER_ELEMENT)// converting from TypedArray to Buffer by accessing buffer but without byteOffset and byteLength
const data!: Uint8Array
Buffer.from(data.buffer)// converting from TypedArray to Buffer by accessing buffer but without byteLength
const data!: Uint8Array
Buffer.from(data.buffer, data.byteOffset)// converting from TypedArray to Buffer using length instead of byteLength
const data!: Uint32Array
Buffer.from(data.buffer, data.byteOffset, data.length)// making a copy of the entire backing ArrayBuffer instead of the relevant portion
const data!: Buffer
data.buffer.slice()Pass
// passing byteLength and byteOffset correctly
const data!: Buffer
new Uint8Array(data.buffer, data.byteLength, data.byteOffset)// passing byteLength divided by the correct BYTES_PER_ELEMENT property
const data!: Buffer
new Uint32Array(data.buffer, data.byteOffset, data.byteLength / Uint32Array.BYTES_PER_ELEMENT)// passing some unknown expression to the length parameter - we assume they know what they're doing
const data!: Buffer
new Uint32Array(data.buffer, data.byteOffset, data.byteLength / SOME_CONSTANT)// converting from Uint8Array to Buffer correctly
const data!: Uint8Array
Buffer.from(data)// converting from Uint8Array to Buffer via ArrayBuffer correctly
const data!: Uint8Array
Buffer.from(data.buffer, data.byteOffset, data.byteLength)// making a copy of the relevant portion of the backing ArrayBuffer of some Buffer
const data!: Buffer
data.buffer.slice(data.byteOffset, data.byteOffset + data.byteLength)Additional Info
No response
How does funding with Polar work?
Pay now to fund the work behind this issue.
Get updates on progress being made.
Maintainer is rewarded once the issue is completed.
Backer
You're funding impactful open source efforts
Contributor
You want to contribute to this effort
Maintainer
You want to get funding like this too