Skip to content

Instantly share code, notes, and snippets.

@devinrhode2
Last active March 11, 2021 20:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save devinrhode2/a2bac31663287d8d107ed7d1b7112f2d to your computer and use it in GitHub Desktop.
Save devinrhode2/a2bac31663287d8d107ed7d1b7112f2d to your computer and use it in GitHub Desktop.
normalize a 0xa0 non-breaking space character to ascii compatible encoding. For error

Unable to parse JSON and generate JSONx: illegal character 0xa0 at offset 7164 of

DataPower issue

// Error "Unable to parse JSON and generate JSONx: illegal character 0xa0 at offset 7164 of" from DataPower (sending email)
/*
results = {};
[
['wonky', String.fromCharCode(160)],
['basic', ' '],
].forEach(([label, str]) => {
['NFC', 'NFD', 'NFKC', 'NFKD'].forEach((uform) => {
[
['esca', escape],
['enco', encodeURIComponent],
['btoa',btoa]
].forEach(([methodName, method]) => {
results[label+uform] = results[label+uform] || []
results[label+uform].push(
method(str.normalize(uform))
)
})
})
})
results
*/
/*
results = {
aA_guide: ['escape', 'encodeURIComp', 'btoa']
};
[
['wonky', String.fromCharCode(160)],
['aThing', 'à'],
['basic', ' '],
].forEach(([label, originalStr]) => {
['NFC', 'NFD', 'NFKC', 'NFKD'].forEach((uform) => {
results[label+uform] = [
escape,
encodeURIComponent,
btoa
].map((method) => {
var str;
try {
str = method(originalStr.normalize(uform))
} finally {
return str
}
})
})
})
results
*/
results = {
//aA_guide: ['escape', 'encodeURIComp']
};
[
['wonky', String.fromCharCode(160)+'à'],
].forEach(([label, originalStr]) => {
['NFC', 'NFD', 'NFKC', 'NFKD'].forEach((uform) => {
results[label+uform] = [
escape,
encodeURIComponent
].map((method) => {
var str;
try {
str = method(originalStr.normalize(uform))
} finally {
return str
}
})
})
})
results
@devinrhode2
Copy link
Author

results = {
  aA_guide: ['escape', 'encodeURIComp', 'btoa']
};
nbsp = String.fromCharCode(160);
[
  ['_nbsp', `2020${nbsp}TaxReturn.pdf`],
  ['acc-a', "àmélie's W2.pdf"],
].forEach(([label, originalStr]) => {
  ['_NFC', '_NFD', 'NFKC', 'NFKD'].forEach((uform) => {
    results[label+'_'+uform] = [
      escape,
      encodeURIComponent,
      btoa
    ].map((method) => {
      var str;
      try {
        str = method(originalStr.normalize(uform.replace('_', '')))
      } finally {
        return str
      }
    })
  })
})
results

NFKC seems best to avoid causing DataPower to crash with error of "Unable to parse JSON and generate JSONx: illegal character 0xa0 at offset 7164 of"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment