Skip to content

Instantly share code, notes, and snippets.

@indraai
Last active March 18, 2019 02:38
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save indraai/624a6bd0f040af8ffb1a5696cea21e53 to your computer and use it in GitHub Desktop.
Generic Javascript function for removing duplicate sentences from a string.
// author INDRA.ai
// Copyright 2019
// This is a generic function to remove duplicate sentences from a string.
// Example:
/*
const paragraph = "Hello there. Hello there. How are you."
console.log(removeDuplicateSentences(paragraph);
*/
// MIT License
function removeDuplicateSentences(input=false) {
if (!input) return 'What did you say?';
const inputArr = input.match(/[^\.\!\?]*[\.\!\?]/gm);
const matchArr = [];
const finalArr = [];
inputArr.forEach((val,idx) => {
val = val.trim();
const matchVal = val.replace(/(http|https):\/\/\S*/gi, '').replace(/\W/gi, '').toLowerCase().trim();
if (matchArr.includes(matchVal)) return;
matchArr.push(matchVal);
finalArr.push(val);
});
return finalArr.join(' ')
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment