Skip to content

Instantly share code, notes, and snippets.

@manifestinteractive
Last active April 3, 2023 08:02
Show Gist options
  • Star 11 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save manifestinteractive/ec216bcc457e432a0838a3757a34b310 to your computer and use it in GitHub Desktop.
Save manifestinteractive/ec216bcc457e432a0838a3757a34b310 to your computer and use it in GitHub Desktop.
LZ-based Compression Algorithm for Salesforce Commerce Cloud (SFCC)

LZ-based Compression Algorithm

Helper Script for Salesforce Commerce Cloud (SFCC) to compress and decompress Objects and Strings. This library uses LZ-String to work around SFCC Character Limits by using Compression, as mentioned in this Trailhead Module:

Build Your Career as a Salesforce B2C Commerce Technical Architect > Project Documentation for Salesforce B2C Commerce Technical Architects > Create a Data Model.

SFCC Session Limits:

  • Strings are limited to 2000 characters
  • Session is limited to 4000 characters in total size

Installation

Copy lzStringHelper.js into your client cartridge scripts helper folder cartridge/scripts/helpers/lzStringHelper.js

Usage

Compress

Compressing UTF-8 Strings and Objects

// Require LZString Helper
var LZString = require('*/cartridge/scripts/helpers/lzStringHelper');

// Fake SFCC Object
var obj = {
  zipcode: '33705',
  storeId: '0083',
  storeName: 'Store Name',
  orderNo: '10000123',
  paymentMethod: 'CREDIT_CARD'
};

// Fake String
var str = 'Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.';

// Save Object to Session ( this will automatially be converted to a string )
session.custom.myCompressedObject = LZString.compress(obj);

// Save String to Session
session.custom.myCompressedString = LZString.compress(str);

/**
 * COMPRESSION RESULTS:
 *
 * session.custom.myCompressedObject:
 *
 * "㞂⁞ॠฌ惶İ⦈ׂ̰惬`⬈Ð耳耮瀄鈉ꈮ렁솱晓䁲Șୢ鴀攮䠀ᇴᇚ눪?〈쳙銀䳬䥅큀㭲Ť鎐Ũ醀愀䨁䐀諐ʠ὚쁁宎䀅"
 *
 * Decompressed: 112 Bytes
 * Compressed: 56 Bytes
 * Reduction: 50%
 *
 * session.custom.myCompressedString:
 *
 * "ಇ끎੠똀䂖p㎁嵠ጐۜ㉜ű聃⠠‚᠆㄀㬤⚠ӵᎉ櫣ꁳ᠐눒꣝ౌ䋠ꉅЄ限ዣ먔璈ꈥ颀⍰ႅᓃ鈌⣅淪≄䁇ᓄǐ쀊ꑂᴸ놈䥁쀁룹슑医烁킁‑肠䡀x䆂ထ쏐삡揩䋐쏩ᦂ웻጑㢢℉ꔋ?䏉䫐ぃ㫧렀褵ₒ쌦定꧐쉀≀Xﮢ易쇎蝠ꀠ蜈汁∰퇃ᖢ䭣ᤈꃀƘꏲ䔑턗⌑囥뎸ń틔ࠞ숆쏁㔘蓅죑癰琮償᎚ⱀ⢟┏֢⇰酘澌أĀ㰞灞㶍棆䭨곘ᄽ埋Ņऒ㠍ૢ"
 *
 * Decompressed: 447 Bytes
 * Compressed: 160 Bytes
 * Reduction: 64%
 */

Decompress

Decompressing UTF-8 Strings and Objects

// Require LZString Helper
var LZString = require('*/cartridge/scripts/helpers/lzStringHelper');

// Decompress Session Object
var obj = LZString.decompress(session.custom.myCompressedObject);

// Decompress Session String
var str = LZString.decompress(session.custom.myCompressedString);

/**
 * RETURNS JSON:
 * 
 * obj:
 *
 * {
 *   "zipcode": "33705",
 *   "storeId": "0083",
 *   "storeName": "Store Name",
 *   "orderNo": "10000123",
 *   "paymentMethod": "CREDIT_CARD"
 * }
 *
 * str:
 *
 * "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum."
 */

Working with UTF-16

If you need to work with UTF-16 characters, use the following ( NOTE: This UTF-16 Compression is 6.66% larger than those produced by UTF-8 compress ):

// Save Object to Session
session.custom.myCompressedObject = LZString.compressToUTF16(obj);

// Save String to Session
session.custom.myCompressedString = LZString.compressToUTF16(str);
// Decompress Session Object
var obj = LZString.decompressFromUTF16(session.custom.myCompressedObject);

// Decompress Session String
var str = LZString.decompressFromUTF16(session.custom.myCompressedString);

Working with Base64

If you need to work with Base64 characters, use the following ( NOTE: This Base64 Compression is 166% larger than those produced by UTF-8 compress, but can still significantly compressed some JSON objects. ):

// Save Object to Session
session.custom.myCompressedObject = LZString.compressToBase64(obj);

// Save String to Session
session.custom.myCompressedString = LZString.compressToBase64(str);
// Decompress Session Object
var obj = LZString.decompressFromBase64(session.custom.myCompressedObject);

// Decompress Session String
var str = LZString.decompressFromBase64(session.custom.myCompressedString);

Working with Encoded URI Components

If you need to work with Encoded URI characters, use the following ( NOTE: This Encoded URI Compression is similiar to Base64 and 166% larger than those produced by UTF-8 compress, but can still significantly compressed some JSON objects. ):

// Save Object to Session
session.custom.myCompressedObject = LZString.compressToEncodedURIComponent(obj);

// Save String to Session
session.custom.myCompressedString = LZString.compressToEncodedURIComponent(str);
// Decompress Session Object
var obj = LZString.decompressFromEncodedURIComponent(session.custom.myCompressedObject);

// Decompress Session String
var str = LZString.decompressFromEncodedURIComponent(session.custom.myCompressedString);

Working with Uint8Array

If you need to work with Uint8Array, use the following:

// Save Object to Session
session.custom.myCompressedObject = LZString.compressToUint8Array(obj);

// Save String to Session
session.custom.myCompressedString = LZString.compressToUint8Array(str);
// Decompress Session Object
var obj = LZString.decompressFromUint8Array(session.custom.myCompressedObject);

// Decompress Session String
var str = LZString.decompressFromUint8Array(session.custom.myCompressedString);
/**
* LZ-based Compression Algorithm
* This is a modified version of the following URL for SFCC
* @see https://github.com/pieroxy/lz-string/blob/master/libs/lz-string.js
*/
module.exports = (function() {
var f = String.fromCharCode;
var keyStrBase64 = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=';
var keyStrUriSafe = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+-$';
var baseReverseDic = {};
/**
* Core Compress Function
*
* @param {Mixed} uncompressed String or Object to Compress
* @param {Integer} bitsPerChar Bits per Character
* @param {Integer} getCharFromInt Get Character from Integer
* @returns {String}
*/
function compress(uncompressed, bitsPerChar, getCharFromInt) {
if (uncompressed == null) return '';
if (typeof uncompressed !== 'string') {
try {
uncompressed = JSON.stringify(uncompressed)
} catch (err) {
return err.name + ': ' + err.message;
}
}
var i, value,
contextDictionary = {},
contextDictionaryToCreate = {},
contextC = '',
contextWC = '',
contextW = '',
contextEnlargeIn = 2,
contextDictSize = 3,
contextNumBits = 2,
contextData = [],
contextDataVal = 0,
contextDataPosition = 0,
ii;
for (ii = 0; ii < uncompressed.length; ii += 1) {
contextC = uncompressed.charAt(ii);
if (!Object.prototype.hasOwnProperty.call(contextDictionary, contextC)) {
contextDictionary[contextC] = contextDictSize++;
contextDictionaryToCreate[contextC] = true;
}
contextWC = contextW + contextC;
if (Object.prototype.hasOwnProperty.call(contextDictionary, contextWC)) {
contextW = contextWC;
} else {
if (Object.prototype.hasOwnProperty.call(contextDictionaryToCreate, contextW)) {
if (contextW.charCodeAt(0) < 256) {
for (i = 0; i < contextNumBits; i++) {
contextDataVal = (contextDataVal << 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
}
value = contextW.charCodeAt(0);
for (i = 0; i < 8; i++) {
contextDataVal = (contextDataVal << 1) | (value & 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = value >> 1;
}
} else {
value = 1;
for (i = 0; i < contextNumBits; i++) {
contextDataVal = (contextDataVal << 1) | value;
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = 0;
}
value = contextW.charCodeAt(0);
for (i = 0; i < 16; i++) {
contextDataVal = (contextDataVal << 1) | (value & 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = value >> 1;
}
}
contextEnlargeIn--;
if (contextEnlargeIn == 0) {
contextEnlargeIn = Math.pow(2, contextNumBits);
contextNumBits++;
}
delete contextDictionaryToCreate[contextW];
} else {
value = contextDictionary[contextW];
for (i = 0; i < contextNumBits; i++) {
contextDataVal = (contextDataVal << 1) | (value & 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = value >> 1;
}
}
contextEnlargeIn--;
if (contextEnlargeIn == 0) {
contextEnlargeIn = Math.pow(2, contextNumBits);
contextNumBits++;
}
contextDictionary[contextWC] = contextDictSize++;
contextW = String(contextC);
}
}
if (contextW !== '') {
if (Object.prototype.hasOwnProperty.call(contextDictionaryToCreate, contextW)) {
if (contextW.charCodeAt(0) < 256) {
for (i = 0; i < contextNumBits; i++) {
contextDataVal = (contextDataVal << 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
}
value = contextW.charCodeAt(0);
for (i = 0; i < 8; i++) {
contextDataVal = (contextDataVal << 1) | (value & 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = value >> 1;
}
} else {
value = 1;
for (i = 0; i < contextNumBits; i++) {
contextDataVal = (contextDataVal << 1) | value;
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = 0;
}
value = contextW.charCodeAt(0);
for (i = 0; i < 16; i++) {
contextDataVal = (contextDataVal << 1) | (value & 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = value >> 1;
}
}
contextEnlargeIn--;
if (contextEnlargeIn == 0) {
contextEnlargeIn = Math.pow(2, contextNumBits);
contextNumBits++;
}
delete contextDictionaryToCreate[contextW];
} else {
value = contextDictionary[contextW];
for (i = 0; i < contextNumBits; i++) {
contextDataVal = (contextDataVal << 1) | (value & 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = value >> 1;
}
}
contextEnlargeIn--;
if (contextEnlargeIn == 0) {
contextEnlargeIn = Math.pow(2, contextNumBits);
contextNumBits++;
}
}
value = 2;
for (i = 0; i < contextNumBits; i++) {
contextDataVal = (contextDataVal << 1) | (value & 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = value >> 1;
}
while (true) {
contextDataVal = (contextDataVal << 1);
if (contextDataPosition == bitsPerChar - 1) {
contextData.push(getCharFromInt(contextDataVal));
break;
} else contextDataPosition++;
}
return contextData.join('');
}
/**
* Core Decompress Function
*
* @param {Integer} length String length
* @param {Integer} resetValue Reset Value
* @param {Function} getNextValue Callback Handler
* @returns {Mixed} Uncompressed String or Object
*/
function decompress(length, resetValue, getNextValue) {
var dictionary = [],
next,
enlargeIn = 4,
dictSize = 4,
numBits = 3,
entry = '',
result = [],
i,
w,
bits, resB, maxPower, power,
c,
data = {
val: getNextValue(0),
position: resetValue,
index: 1
};
for (i = 0; i < 3; i += 1) {
dictionary[i] = i;
}
bits = 0;
maxPower = Math.pow(2, 2);
power = 1;
while (power != maxPower) {
resB = data.val & data.position;
data.position >>= 1;
if (data.position == 0) {
data.position = resetValue;
data.val = getNextValue(data.index++);
}
bits |= (resB > 0 ? 1 : 0) * power;
power <<= 1;
}
switch (next = bits) {
case 0:
bits = 0;
maxPower = Math.pow(2, 8);
power = 1;
while (power != maxPower) {
resB = data.val & data.position;
data.position >>= 1;
if (data.position == 0) {
data.position = resetValue;
data.val = getNextValue(data.index++);
}
bits |= (resB > 0 ? 1 : 0) * power;
power <<= 1;
}
c = f(bits);
break;
case 1:
bits = 0;
maxPower = Math.pow(2, 16);
power = 1;
while (power != maxPower) {
resB = data.val & data.position;
data.position >>= 1;
if (data.position == 0) {
data.position = resetValue;
data.val = getNextValue(data.index++);
}
bits |= (resB > 0 ? 1 : 0) * power;
power <<= 1;
}
c = f(bits);
break;
case 2:
return '';
}
dictionary[3] = c;
w = c;
result.push(c);
while (true) {
if (data.index > length) {
return '';
}
bits = 0;
maxPower = Math.pow(2, numBits);
power = 1;
while (power != maxPower) {
resB = data.val & data.position;
data.position >>= 1;
if (data.position == 0) {
data.position = resetValue;
data.val = getNextValue(data.index++);
}
bits |= (resB > 0 ? 1 : 0) * power;
power <<= 1;
}
switch (c = bits) {
case 0:
bits = 0;
maxPower = Math.pow(2, 8);
power = 1;
while (power != maxPower) {
resB = data.val & data.position;
data.position >>= 1;
if (data.position == 0) {
data.position = resetValue;
data.val = getNextValue(data.index++);
}
bits |= (resB > 0 ? 1 : 0) * power;
power <<= 1;
}
dictionary[dictSize++] = f(bits);
c = dictSize - 1;
enlargeIn--;
break;
case 1:
bits = 0;
maxPower = Math.pow(2, 16);
power = 1;
while (power != maxPower) {
resB = data.val & data.position;
data.position >>= 1;
if (data.position == 0) {
data.position = resetValue;
data.val = getNextValue(data.index++);
}
bits |= (resB > 0 ? 1 : 0) * power;
power <<= 1;
}
dictionary[dictSize++] = f(bits);
c = dictSize - 1;
enlargeIn--;
break;
case 2:
var output = result.join('');
try {
return JSON.parse(output)
} catch (err) {
return output
}
}
if (enlargeIn == 0) {
enlargeIn = Math.pow(2, numBits);
numBits++;
}
if (dictionary[c]) {
entry = dictionary[c];
} else {
if (c === dictSize) {
entry = w + w.charAt(0);
} else {
return null;
}
}
result.push(entry);
dictionary[dictSize++] = w + entry.charAt(0);
enlargeIn--;
w = entry;
if (enlargeIn == 0) {
enlargeIn = Math.pow(2, numBits);
numBits++;
}
}
}
/**
* Get Base Dictionary Value
*
* @param {String} alphabet Alphabet to use for Compression
* @param {String} character Current Character
* @returns {String} Dictionary Character
*/
function getBaseValue(alphabet, character) {
if (!baseReverseDic[alphabet]) {
baseReverseDic[alphabet] = {};
for (var i = 0; i < alphabet.length; i++) {
baseReverseDic[alphabet][alphabet.charAt(i)] = i;
}
}
return baseReverseDic[alphabet][character];
}
return {
/**
* Compress UTF-8 Strings
* Can be decompressed with `decompress`
* @param {Mixed} uncompressed String or Object to Compress
* @returns {String}
*/
compress: function(uncompressed) {
return compress(uncompressed, 16, function(a) {
return f(a);
});
},
/**
* Compress to Base64
*
* Produces ASCII UTF-16 strings representing the original string encoded in Base64.
* Can be decompressed with `decompressFromBase64`
* This works by using only 6bits of storage per character.
* The strings produced are therefore 166% bigger than those produced by compress.
* It can still reduce significantly some JSON compressed objects.
*
* @param {Mixed} uncompressed String or Object to Compress
* @returns {String}
*/
compressToBase64: function(uncompressed) {
if (uncompressed == null) return '';
var res = compress(uncompressed, 6, function(a) {
return keyStrBase64.charAt(a);
});
switch (res.length % 4) {
default:
case 0:
return res;
case 1:
return res + '===';
case 2:
return res + '==';
case 3:
return res + '=';
}
},
/**
* Compress to Encoded URI Component
*
* Produces ASCII strings representing the original string encoded in Base64 with a few tweaks to make these URI safe.
* Hence, you can send them to the server without thinking about URL encoding them. This saves bandwidth and CPU.
* These strings can be decompressed with `decompressFromEncodedURIComponent`.
* See the bullet point above for considerations about size.
*
* @param {Mixed} uncompressed String or Object to Compress
* @returns {String}
*/
compressToEncodedURIComponent: function(uncompressed) {
if (uncompressed == null) return '';
return compress(uncompressed, 6, function(a) {
return keyStrUriSafe.charAt(a);
});
},
/**
* Compress to Uint8Array
*
* Produces an uint8Array. Can be decompressed with `decompressFromUint8Array`
* @param {Mixed} uncompressed String or Object to Compress
* @returns {Uint8Array}
*/
compressToUint8Array: function(uncompressed) {
var compressed = compress(uncompressed);
var buf = new Uint8Array(compressed.length * 2);
for (var i = 0, TotalLen = compressed.length; i < TotalLen; i++) {
var currentValue = compressed.charCodeAt(i);
buf[i * 2] = currentValue >>> 8;
buf[i * 2 + 1] = currentValue % 256;
}
return buf;
},
/**
* Compress to UTF16
*
* Produces "valid" UTF-16 strings in the sense that all browsers can store them safely.
* So they can be stored in localStorage on all browsers tested.
* Can be decompressed with `decompressFromUTF16`
* This works by using only 15bits of storage per character.
* The strings produced are therefore 6.66% bigger than those produced by `compress`
*
* @param {*} uncompressed String or Object to Compress
* @returns {String}
*/
compressToUTF16: function(uncompressed) {
if (uncompressed == null) return '';
return compress(uncompressed, 15, function(a) {
return f(a + 32);
}) + ' ';
},
/**
* Decompress output generated from `compress`
*
* @param {String} compressed Compress String
* @returns {Mixed} Decompressed String or Object
*/
decompress: function(compressed) {
if (compressed == null) return '';
if (compressed == '') return null;
return decompress(compressed.length, 32768, function(index) {
return compressed.charCodeAt(index);
});
},
/**
* Decompress output generated from `compressToBase64`
*
* @param {String} compressed Compress String
* @returns {Mixed} Decompressed String or Object
*/
decompressFromBase64: function(compressed) {
if (compressed == null) return '';
if (compressed == '') return null;
return decompress(compressed.length, 32, function(index) {
return getBaseValue(keyStrBase64, compressed.charAt(index));
});
},
/**
* Decompress output generated from `compressToEncodedURIComponent`
*
* @param {String} compressed Compress String
* @returns {Mixed} Decompressed String or Object
*/
decompressFromEncodedURIComponent: function(compressed) {
if (compressed == null) return '';
if (compressed == '') return null;
compressed = compressed.replace(/ /g, '+');
return decompress(compressed.length, 32, function(index) {
return getBaseValue(keyStrUriSafe, compressed.charAt(index));
});
},
/**
* Decompress output generated from `compressToUint8Array`
*
* @param {String} compressed Compress String
* @returns {Mixed} Decompressed String or Object
*/
decompressFromUint8Array: function(compressed) {
if (compressed === null || compressed === undefined) {
return decompress(compressed);
} else {
var buf = new Array(compressed.length / 2);
for (var i = 0, TotalLen = buf.length; i < TotalLen; i++) {
buf[i] = compressed[i * 2] * 256 + compressed[i * 2 + 1];
}
var result = [];
buf.forEach(function(c) {
result.push(f(c));
});
return decompress(result.join(''));
}
},
/**
* Decompress output generated from `compressToUTF16`
*
* @param {String} compressed Compress String
* @returns {Mixed} Decompressed String or Object
*/
decompressFromUTF16: function(compressed) {
if (compressed == null) return '';
if (compressed == '') return null;
return decompress(compressed.length, 16384, function(index) {
return compressed.charCodeAt(index) - 32;
});
}
};
})();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment