Skip to content

Instantly share code, notes, and snippets.

@khanhtran
Created March 4, 2020 00:48
Show Gist options
  • Save khanhtran/51658355b502ef966db04b8292fc09de to your computer and use it in GitHub Desktop.
Save khanhtran/51658355b502ef966db04b8292fc09de to your computer and use it in GitHub Desktop.
def requiredColumnsRegs = [/ISSN|ISBN10/,/eISSN|ISSN13/,/Title/]
for (record in records) {
try {
if (!haveRequiredColumns(requiredColumnsRegs, record)) {
record.value['drop-file'] = true
}
output.write(record)
} catch (e) {
// Write a record to the error pipeline
log.error(e.toString(), e)
error.write(record, e.toString())
}
}
def haveRequiredColumns(requiredColumnsRegs, record) {
for (column in record.value['originalFields'].keySet()) {
if (!anyMatch(column, requiredColumnsRegs)) {
return false;
}
}
return true
}
def anyMatch(str, regexList) {
for (reg in regexList) {
if (str ==~ reg) {
return true
}
}
return false
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment