Skip to content

Instantly share code, notes, and snippets.

@Jozo132
Last active November 15, 2023 07:47
Show Gist options
  • Star 15 You must be signed in to star a gist
  • Fork 5 You must be signed in to fork a gist
  • Save Jozo132/2c0fae763f5dc6635a6714bb741d152f to your computer and use it in GitHub Desktop.
Save Jozo132/2c0fae763f5dc6635a6714bb741d152f to your computer and use it in GitHub Desktop.
JavaScript (Node.js) IEEE 754 Single precision Floating-Point (32-bit) binary conversion from and to both Hex and Bin
/* ##### float32encoding.js #####
MIT License
- Forked 'toFloat' from https://gist.github.com/laerciobernardo/498f7ba1c269208799498ea8805d8c30
- Forked 'toHex' from stackoverflow answer https://stackoverflow.com/a/47187116/10522253
- Modifyed by: Jozo132 (https://github.com/Jozo132)
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
const Float32ToHex = (float32) => {
const getHex = i => ('00' + i.toString(16)).slice(-2);
var view = new DataView(new ArrayBuffer(4))
view.setFloat32(0, float32);
return Array.apply(null, { length: 4 }).map((_, i) => getHex(view.getUint8(i))).join('');
}
const Float32ToBin = (float32) => {
const HexToBin = hex => (parseInt(hex, 16).toString(2)).padStart(32, '0');
const getHex = i => ('00' + i.toString(16)).slice(-2);
var view = new DataView(new ArrayBuffer(4))
view.setFloat32(0, float32);
return HexToBin(Array.apply(null, { length: 4 }).map((_, i) => getHex(view.getUint8(i))).join(''));
}
const HexToFloat32 = (str) => {
var int = parseInt(str, 16);
if (int > 0 || int < 0) {
var sign = (int >>> 31) ? -1 : 1;
var exp = (int >>> 23 & 0xff) - 127;
var mantissa = ((int & 0x7fffff) + 0x800000).toString(2);
var float32 = 0
for (i = 0; i < mantissa.length; i += 1) { float32 += parseInt(mantissa[i]) ? Math.pow(2, exp) : 0; exp-- }
return float32 * sign;
} else return 0
}
const BinToFloat32 = (str) => {
var int = parseInt(str, 2);
if (int > 0 || int < 0) {
var sign = (int >>> 31) ? -1 : 1;
var exp = (int >>> 23 & 0xff) - 127;
var mantissa = ((int & 0x7fffff) + 0x800000).toString(2);
var float32 = 0
for (i = 0; i < mantissa.length; i += 1) { float32 += parseInt(mantissa[i]) ? Math.pow(2, exp) : 0; exp-- }
return float32 * sign;
} else return 0
}
// Full example
var test_value = -0.3;
console.log(`Input value (${test_value}) => hex (${Float32ToHex(test_value)}) [${Math.ceil(Float32ToHex(test_value).length / 2)} bytes] => float32 (${HexToFloat32(Float32ToHex(test_value))})`);
console.log(`Input value (${test_value}) => binary (${Float32ToBin(test_value)}) [${Float32ToBin(test_value).length} bits] => float32 (${BinToFloat32(Float32ToBin(test_value))})`);
/* DEBUG OUTPUT:
Input value (-0.3) => hex (be99999a) [4 bytes] => float32 (-0.30000001192092896)
Input value (-0.3) => binary (10111110100110011001100110011010) [32 bits] => float32 (-0.30000001192092896)
*/
/* ##### float32encoding.min.js #####
MIT License
- Forked 'toFloat' from https://gist.github.com/laerciobernardo/498f7ba1c269208799498ea8805d8c30
- Forked 'toHex' from stackoverflow answer https://stackoverflow.com/a/47187116/10522253
- Modifyed by: Jozo132 (https://github.com/Jozo132)
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
const Float32ToHex = float32 => { const getHex = i => ('00' + i.toString(16)).slice(-2); var view = new DataView(new ArrayBuffer(4)); view.setFloat32(0, float32); return Array.apply(null, { length: 4 }).map((_, i) => getHex(view.getUint8(i))).join(''); }
const Float32ToBin = float32 => parseInt(Float32ToHex(float32), 16).toString(2).padStart(32, '0');
const ToFloat32 = num => { if (num > 0 || num < 0) { var sign = (num >>> 31) ? -1 : 1; var exp = (num >>> 23 & 0xff) - 127; var mantissa = ((num & 0x7fffff) + 0x800000).toString(2); var float32 = 0; for (i = 0; i < mantissa.length; i += 1) { float32 += parseInt(mantissa[i]) ? Math.pow(2, exp) : 0; exp-- } return float32 * sign; } else return 0 }
const HexToFloat32 = str => ToFloat32(parseInt(str, 16));
const BinToFloat32 = str => ToFloat32(parseInt(str, 2));
// ------ FULL EXAMPLE ------
var value = -0.3; // JS number variable
// FLOAT32 <===> HEX
var f32_hex = Float32ToHex(value); // JS number => HEX string of a Float32 standard number
var f32_hex_inverse = HexToFloat32(f32_hex); // HEX string of a Float32 standard number => JS number
// FLOAT32 <===> BIN
var f32_bin = Float32ToBin(value); // JS number => HEX string of a Float32 standard number
var f32_bin_inverse = BinToFloat32(f32_bin); // HEX string of a Float32 standard number => JS number
console.log(`Input value (${value}) => hex (${f32_hex}) [${Math.ceil(f32_hex.length / 2)} bytes] => float32 (${f32_bin_inverse})`);
console.log(`Input value (${value}) => binary (${f32_bin}) [${f32_bin.length} bits] => float32 (${f32_bin_inverse})`);
/* DEBUG OUTPUT:
Input value (-0.3) => hex (be99999a) [4 bytes] => float32 (-0.30000001192092896)
Input value (-0.3) => binary (10111110100110011001100110011010) [32 bits] => float32 (-0.30000001192092896)
*/
@guest271314
Copy link

Can the code be used to convert the content of <data> tags within output of mkv2xml to Float32Array for use with Web Audio API WebAudio/web-audio-api-v2#63?

@Jozo132
Copy link
Author

Jozo132 commented Feb 9, 2020

Can the code be used to convert the content of <data> tags within output of mkv2xml to Float32Array for use with Web Audio API WebAudio/web-audio-api-v2#63?

Hello, I think you probably need something like this:

let stringBuffer = `0000000...` // the inital string buffer
let buff = Buffer.from(stringBuffer.replace(/(?:\r\n|\r|\n)/g, ''), 'hex') // string converted to actual buffer
let output = new Float32Array(buff) // buffer converted to Float32Array

Hope this helps

@guest271314
Copy link

In the browser, without using nodejs?

This site https://www.scadacore.com/tools/programming-calculators/online-hex-converter/ outputs the correct value for "9201c93b" input. Yet none of the conversion formulas using JavaScript result in audio output reflecting the original file.

@Jozo132
Copy link
Author

Jozo132 commented Feb 9, 2020

In the browser, without using nodejs?

This site https://www.scadacore.com/tools/programming-calculators/online-hex-converter/ outputs the correct value for "9201c93b" input. Yet none of the conversion formulas using JavaScript result in audio output reflecting the original file.

Using my code you can do this:

let input = '9201c93b4201a13b4201a13b9201c93be001703be001703b00000000000072bb'
let array = input.match(/.{1,8}/g).map(x => x.match(/.{1,2}/g).reverse().join('')).map(x => HexToFloat32(x)) 

@Jozo132
Copy link
Author

Jozo132 commented Feb 9, 2020

If possible, you should import the Buffer library, because it can increase performance up to 5x, compared to my quick solution

const ToFloat32 = num => { if (num > 0 || num < 0) { var sign = (num >>> 31) ? -1 : 1; var exp = (num >>> 23 & 0xff) - 127; var mantissa = ((num & 0x7fffff) + 0x800000).toString(2); var float32 = 0; for (i = 0; i < mantissa.length; i += 1) { float32 += parseInt(mantissa[i]) ? Math.pow(2, exp) : 0; exp-- } return float32 * sign; } else return 0 }
const HexToFloat32 = str => ToFloat32(parseInt(str, 16));

const input = `0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
9201c93b4201a13b4201a13b9201c93be001703be001703b00000000000072bb
004017bd009080bd1a018d3c7f813f3d8981443de201f13b00c0ddbce001703b
0601833ccf81e73c9201c93b000000006a01353ce381713c1a018d3c4001a03a
4201a13be201f13b6a01353c4201213ce201f13b19810c3c9201c93b6a01353c
e381713cba015d3c19810c3c4201213c9201c93be201f13b9201c93be201f13b
6a01353ce201f13b19810c3c4201a13b9201c93b9201c93be201f13b9201c93b
4001203b9201c93b4001203b9201c93be001703b4001203b000000004001203b
6a01353c0080c9bc00602bbd9201c93b2e21173d6041303d00805dbc004021bd
e001703ba7a1533d0181803d7e41bf3c00401cbd00208dbd2e01973c4c41263d
ba015d3c4201a13b6a41b53c6b61353d56412b3d9201c93b00c0e7bc0601833c
cf81e73c6041303df801fc3c000000001a018d3c6a01353c1a018d3c0000fcbc
00a049bd00200dbd008049bc0601833ce201f13b4001a03a19810c3c9241c93c
ba015d3c4201213ca781d33c0000f2bb008035bc4201a13b000072bb0601833c
19810c3c000022bbe381713c9201c93b0000cabb0000a4ba1a018d3ce4c1f13c
0601033d9201c93b0080bfbc4201a13ce001703b9201493c4201a13b00000dbc
0080c9bc4201213ccf81e73c00805dbccf81e73c008049bc008035bc5641ab3c
7e41bf3c1a010d3d4001a03a008035bc4001203be4c1f13cf801fc3c4001a03a
0000a4ba008035bc0000a1bb0000a4ba5641ab3c00408dbc0601833c00000000
2e01973c19810c3c0000000000000000000000009381493d4201213ccf81e73c
9201c93b008049bc75613a3d0000000038211c3d6a01353c1a018d3ca781d33c
ba015d3c2e01973c000000002e01973c008049bc00c062bd0000f2bbbb81dd3c
e4c1f13ce201f13bba015d3c4201a13be381713c00000000000000002e01973c
bb81dd3c008035bc7e41bf3c9201c93b008035bce4c1f13c0000a4bae001703b
e201f13b0040a1bc19810c3c19810c3c0000f2bb4201213c4201a13bba015d3c
ba015d3c008035bc9201c93b4001a03a5641ab3c9201c93b0000a4ba000072bb
0000a1bb1a018d3c6a01353c000000004201a13be001703be381713ce001703b
004097bc1a010d3d00000000000021bc0601833c00805dbc4201a13c0000a4ba
4201a13bba015d3c9201c93b4201a13c0080c9bcbb81dd3c1a010d3d0080b5bc
6a41b53c00c0d3bce4c1f13c9381493d00409cbdeee1763d9201c93b0000cabb
0b81853d0080c9bc4201213ce001703b4001203bcf81e73c004097bc0000a1bb
e4c1f13c00805dbc4c41263de201f13b0080c9bc75613a3d00c0e7bc1a018d3c
e201f13b0080b5bc9201493ce4c1f13c000072bc0040a1bc0601833c2e01973c
008035bce001703bc5a1623d00e06cbd0000000038211c3dba015d3c0080b5bc
0080abbcc5a1623d000022bb000000000080abbc19810c3ccf81e73c004097bc
e201f13b4201a13ba781d33c0000f2bb19810c3c00c0ddbc000072bbc5a1623d
000021bc0080b5bca781d33c4001a03a0601833c0601833c004083bc000021bc
19810c3c1a018d3c000000004201a13c000022bb5641ab3c0000a4ba0040a1bc
9241c93c4201213c4001a03a0000a4ba9241c93c0040a1bcbb81dd3c8981443d
004097bda7a1533d4001a03a0080c9bc1a118d3d0080bfbc00805dbc4001a03a
00a03fbd4c41a63d008035bd000072bbcf81e73c00401cbd0601033d002003bd
7e41bf3c00c0d3bc2e01973c9201493c002012bdf801fc3c9201c93b4201213c
4c41263d9201493c00000dbc000072bb56412b3d4001a03ae381713c0601033d
f801fc3c00000dbc7e41bf3c002012bdf8017c3d4001a03a2e01973c9241c93c
008035bd8981443d7e41bf3ce201f13b2421123d00602bbd0080abbc1011883d
19810c3c008049bcba015d3c000022bbe201f13b0601833cf801fc3c00200dbd
e4c1f13c0601033d004097bd0601833d56412b3d00e0a8bde201f13b2e21173d
4221213d4201a13b00805dbc00c0d3bc29a1943d00408dbcba015d3c2e01973c
004021bd1a118d3d4001203b00e067bd2e31973d004017bd000021bc0181803d
00a044bd1a118d3d4001a03a00c0d8bdfd81fe3d0000fcbc1a018d3c7e41bf3c
0080bfbcdac16c3d9201c93b0080bfbc2e01973cba015d3c5641ab3c4001203b
00e06cbdc5a1623d9201493c4001203be001703b00408dbc7f813f3d4001a03a
00408dbc5641ab3c19810c3c4001203b0080b5bc4201a13c56412b3d4001203b
6a01353c4201a13c009085bdb631db3d00e06cbd1a018d3cbba15d3d00f0b2bd
e4e1713d2421923d00a0c9bd2421123da7a1533d0060b0bde4f1f13d00b094bd
e4e1713dba015d3c00409cbdca41e53d000072bde4c1f13c66e1b23d00a0cebd
00602bbd4001203e00401cbd000072bb0000000000c0d3bc1a010d3d6041303d
e001703b00200dbd00000000002003bd4201a13c0601833c5641ab3c2e21173d`


const iterations = 1000
setTimeout(() => {
    let start = +new Date()
    for (let i = 0; i < iterations; i++)
        input.match(/.{1,8}/g).map(x => Buffer.from(x, 'hex').readFloatLE()) // ~0.3 ms
    console.log('Time to execute: ', (+new Date() - start) / iterations, 'ms')
}, 100)
setTimeout(() => {
    let start = +new Date()
    for (let i = 0; i < iterations; i++)
        input.match(/.{1,8}/g).map(x => x.match(/.{1,2}/g).reverse().join('')).map(x => HexToFloat32(x)) // ~1.4 ms
    console.log('Time to execute: ', (+new Date() - start) / iterations, 'ms')
}, 2000)

@guest271314
Copy link

guest271314 commented Feb 9, 2020

That results in a flattened array having length 11264 where the length of the Float32Array from decodeAudioData() is 22528 (11264*2). The original hexadecimal string length is 90112 (11264*8) https://plnkr.co/edit/VenuQyeKErEy66PHtpp6?p=preview.

The original code which encodes to hexadecimal https://github.com/vi/mkvparse/blob/master/mkv2xml.

Can improve performance once have a working example of the conversion algorithm.

How to import Buffer module without using nodejs?

@guest271314
Copy link

The concept is roughly to write/read (MatroskaTM <=> XML) segments (128 element Float32Arrays for use with AudioWorkletProcessor) PCM encoded audio files. Eventually with the ability to substitute the N times larger PCM for various other codecs, e.g., Opus, Vorbis, et al. to overcome using decodeAudioData() due to memory consumption (and the main thread).

@Jozo132
Copy link
Author

Jozo132 commented Feb 9, 2020

How to import Buffer module without using nodejs?

You will need to convert this Buffer library with browserify and attach the script to the web page

@Jozo132
Copy link
Author

Jozo132 commented Feb 10, 2020

Did you try running my code? I'm not sure how the output should look like and what the requirements are.
Some benchmarks need to be done to see if this solution works reliably.

@guest271314
Copy link

Ok.

The conversion should be possible without using nodejs or a library.

Using HexToFloat32 alone does not output the expected result.

@Jozo132
Copy link
Author

Jozo132 commented Feb 10, 2020

Using HexToFloat32 alone does not output the expected result.

What is the expected result? There are 4 encoding types for Float, each giving a very different output

@guest271314
Copy link

Did you try running my code?

Yes. See the linked plnkr at https://gist.github.com/Jozo132/2c0fae763f5dc6635a6714bb741d152f#gistcomment-3171409. (Note, Firefox does not currently support decoding PCM in Matroska container, Chrome or Chromium needs to be used to verify results.)

All of the necessary input code is included.

The expected result is a Float32Array that has the same values or outputs the same or similar audio as the Float32Array from getChannelData(0) of the AudioBuffer created by Web Audio API decodeAudioData(). That is, for the resulting Float32Array to be indistinguishable from the Float32Array from decodeAudioData() as to audio output from a buffer source or AudioWorkletProcessor (WebAudio/web-audio-api-v2#61 (comment)).

@guest271314
Copy link

We know that parsing and playback of the input is possible due to code provided by the author of mkvparse

$ cat mediarecorder_pcm.xml | xml2 | grep '/mkv2xml/Segment/Cluster/SimpleBlock/data=' | cut -f2-2 -d= | xxd -r -p | ffplay -f f32le -ar 22050 -ac 1 -

That should also be possible using native code shipped with the FOSS browser. Or, conclusively determine that the conversion is not possible using code shipped with the browser.

Getting Bus error (core dumped) at the terminal following running several un-related tests. Will probably have to re-install OS before trying to import Buffer module. Though am trying to not use a library or import non-native modules to achieve the requirement, else could use Native Messaging.

@Jozo132
Copy link
Author

Jozo132 commented Feb 10, 2020

In the browser, without using nodejs?
This site https://www.scadacore.com/tools/programming-calculators/online-hex-converter/ outputs the correct value for "9201c93b" input. Yet none of the conversion formulas using JavaScript result in audio output reflecting the original file.

Could you please copy-paste the actual desired output? I'm blind here.
If "9201c93b" is one float32 number, there can only be 11264 elements in the given array, just as I have given the answer.

@guest271314
Copy link

Your code does in fact output the expected result.

const audioBuffer = new AudioBuffer({length: 11264, numberOfChannels: 1, sampleRate: 22050});
audioBuffer.getChannelData(0).set(new Float32Array(result));
const source = new AudioBufferSourceNode(ac, {buffer: audioBuffer});
source.connect(ac.destination);
source.start();

@guest271314
Copy link

When new AudioContext() is executed without specifying sampleRate the sampleRate is set to 44100 by default which affects the result of decodeAudioData(). When adjusting the code to const ac = new AudioContext({sampleRate:22050, numberOfChannels:1}); both results have length of 11264. The AudioBufferSourceNode created when sampleRate is 44100 will play the AudioBuffer which length of 22528.

Thanks for sharing the code and helping here.

@guest271314
Copy link

Can you explain the purpose of the RegExp in this part

.match(/.{1,8}/g).map(x => x.match(/.{1,2}/g).reverse().join('')

of the code?

@guest271314
Copy link

What is the corresponding reverse of the RegExp combinations, that is, Float32Array to hexadecimal?

@Jozo132
Copy link
Author

Jozo132 commented Feb 10, 2020

Can you explain the purpose of the RegExp in this part

.match(/.{1,8}/g).map(x => x.match(/.{1,2}/g).reverse().join('')

of the code?

Simply put it splits up each 8 characters for hex value of each float32 in an array, then reverses pairs for little endian decoding and puts them back together for every float32 value.

'01234567ABCDEF01' --> [ '67452301', '01EFCDAB']

So in the end you can just map every array item to the correct Float32 value

@Jozo132
Copy link
Author

Jozo132 commented Feb 10, 2020

What is the corresponding reverse of the RegExp combinations, that is, Float32Array to hexadecimal?

Inverse function to convert Float32Array to hex little endian buffer string is pretty simple:

let myFloatArray = [ 0.0 , 0.1 , 0.2 , 0.3 ]
let buffer_string = myFloatArray.map(f => Float32ToHex(f).match(/.{1,2}/g).reverse().join('')).join('')

@black7375
Copy link

Hello, this code is pretty good.
What is the license?

@Jozo132
Copy link
Author

Jozo132 commented Feb 12, 2022

What is the license?

Hello, I guess we can go with MIT and keep the original head references at the top.
Is that OK?

@black7375
Copy link

Yes. thank you. 👍
I was thinking to be used as a simple terminal tool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment