Skip to content

Instantly share code, notes, and snippets.

@iampeterbanjo
Created May 31, 2019 14:13
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save iampeterbanjo/0443de891403ded8160034830d78dd44 to your computer and use it in GitHub Desktop.
Save iampeterbanjo/0443de891403ded8160034830d78dd44 to your computer and use it in GitHub Desktop.
Pull data processing pipeline
/*
* Prototype of a data processing pipeline
* that uses strings as an example.
* the idea is to be fault tolerant by processing
* each word and so allow filtering of bad data
* instead of a batch processing approach
* which can cause the pipeline to halt on errors
* without any good data make it through to
* completion.
*/
const capitalizeWord = word => {
return word[0].toUpperCase() + word.slice(1);
}
const reverseWord = word => {
return word.split('').reverse().join('');
}
const isString = (wordMaybe) => typeof wordMaybe === 'string';
const process = (word) => [word].map(reverseWord).map(capitalizeWord)[0];
const transform = (list) => {
return list.reduce((acc, word) => {
if(isString(word)) {
acc.push(process(word))
} else {
console.warn(`${word} is not a string`);
}
return acc;
}, []);
}
const list = ['cat', 'dog', 'mouse', '01', 23, 45]
const result = transform(list);
console.log(result); // =>
/**
​​​23 is not a string​​​
​​​45 is not a string​​​
[ 'Tac', 'God', 'Esuom', '10' ] 
*/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment