Skip to content

Instantly share code, notes, and snippets.

@talolard
Last active February 28, 2019 12:07
Show Gist options
  • Save talolard/757e812d195de6aed3152333218441d8 to your computer and use it in GitHub Desktop.
Save talolard/757e812d195de6aed3152333218441d8 to your computer and use it in GitHub Desktop.
replacingWordsWithTags.ipynb
Display the source blob
Display the rendered blob
Raw
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Problem\n",
"Given a string and a and a collection of annotations, we'd like to replace the annotated words in the string with the tags of the annotation. E.g. given\n",
"\"Hello I like NLP\" \n",
"and the annotations\n",
"```\n",
"[ {\n",
" \"start\":0, end:5, \"tag\":\"FOO\",\n",
" },\n",
" {\n",
" \"start\":13, end:16, \"tag\":\"BAR\",\n",
" },\n",
" ]\n",
"```\n",
"We want to get back \"FOO I like BAR!\" \n",
" \n",
"## Insight\n",
"Naively replacing words inside the string is hard, because we keep modifying the string length with each replacement. Instead, we can build a copy of the string, iterating over the annotations \n",
" "
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"S = \"Hello I like NLP!\" \n",
"Annotations = [ {\n",
" \"start\":0, \"end\":5, \"tag\":\"FOO\",\n",
" },\n",
" {\n",
" \"start\":13, \"end\":16, \"tag\":\"BAR\",\n",
" },\n",
" ]\n"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"def replaceStringWithTags(original,annos):\n",
" prev= 0\n",
" annos.sort(key=lambda x:x[\"start\"]) # Sort the annotations\n",
" newString =\"\"\n",
" for anno in annos:\n",
" newString += original[prev:anno[\"start\"]] + anno[\"tag\"]\n",
" prev = anno[\"end\"]\n",
" newString += original[prev:]\n",
" return newString\n",
" \n",
" "
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'FOO I like BAR!'"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"replaceStringWithTags(S,Annotations)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.2"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment