Parse Cochrane library citations export to tabular format

cochrane_citation_parser.py

# coding: utf-8

# Load up the citation text file as per this link
#
# http://onlinelibrary.wiley.com/cochranelibrary/search/advanced?hiddenFields.strategySortBy=last-modified-date;desc&hiddenFields.showStrategies=true&hiddenFields.containerId=12670271652807976185&hiddenFields.originalContainerId=15960432450033216107&hiddenFields.etag=8064622093550521747
#
# And saved here as `citation-export.txt`

# Test example
testData = """
Record #1 of 1442
ID: CD009642
AU: Kranke Peter
AU: Jokinen Johanna
AU: Pace Nathan Leon
AU: Schnabel Alexander
AU: Hollmann Markus W
AU: Hahnenkamp Klaus
AU: Eberhart Leopold HJ
AU: Poepping Daniel M
AU: Weibel Stephanie
TI: Continuous intravenous perioperative lidocaine infusion for postoperative pain and recovery
SO: Cochrane Database of Systematic Reviews
YR: 2015
NO: 7
PB: John Wiley & Sons, Ltd
KY: Anesthetics, Intravenous [administration & dosage];Anesthetics, Local [administration & dosage];Lidocaine [administration & dosage];Pain Measurement;Pain, Postoperative [drug therapy];Recovery of Function;Humans[checkword]
DOI: 10.1002/14651858.CD009642.pub2
AB: Background: The management of postoperative pain and recovery is still unsatisfactory in clinical practice. Opioids used for postoperative analgesia are frequently associated with adverse effects including nausea and constipation. These adverse effects prevent smooth postoperative recovery. On the other hand not all patients may be suited to, and take benefit from, epidural analgesia used to enhance postoperative recovery. The non-opioid lidocaine was investigated in several studies for its use in multi-modal management strategies to reduce postoperative pain and enhance recovery.Objectives: The aim of this review was to assess the effects (benefits and risks) of perioperative intravenous lidocaine infusion compared to placebo/no treatment or compared to epidural analgesia on postoperative pain and recovery in adults undergoing various surgical procedures.Search methods: We searched the Cochrane Central Register of Controlled Trials (CENTRAL, Issue 5 2014), MEDLINE (January 1966 to May 2014), EMBASE (1980 to May 2014), CINAHL (1982 to May 2014), and reference lists of articles. We searched the trial registry database ClinicalTrials.gov, contacted researchers in the field, and handsearched journals and congress proceedings. We did not apply any language restrictions.Selection criteria: We included randomized controlled trials comparing the effect of continuous perioperative intravenous lidocaine infusion either with placebo, or no treatment, or with epidural analgesia in adults undergoing elective or urgent surgery under general anaesthesia. The intravenous lidocaine infusion must have been started intraoperatively prior to incision and continued at least until the end of surgery.Data collection and analysis: Trial quality was independently assessed by two authors according to the methodological procedures specified by the Cochrane Collaboration. Data were extracted by two independent authors. We collected trial data on postoperative pain, recovery of gastrointestinal function, length of hospital stay, postoperative nausea and vomiting (PONV), opioid consumption, patient satisfaction, surgical complication rates, and adverse effects of the intervention.Main results: We included 45 trials involving 2802 participants. Two trials compared intravenous lidocaine versus epidural analgesia. In all the remaining trials placebo or no treatment was used as a comparator. Trials involved participants undergoing open abdominal (12), laparoscopic abdominal (13), or various other surgical procedures (20).The risk of bias was low with respect to selection bias (random sequence generation), performance bias, attrition bias, and detection bias in more than 50% of the included studies. For allocation concealment and selective reporting the quality assessment yielded low risk of bias for only approximately 20% of the included studies.We found evidence of effect for intravenous lidocaine on the reduction of postoperative pain (visual analogue scale, 0 to 10 cm) compared to placebo or no treatment at 'early time points (one to four hours)' (mean difference (MD) −0.84 cm, 95% confidence interval (CI) −1.10 to −0.59; low-quality evidence) and at 'intermediate time points (24 hours)' (MD −0.34 cm, 95% CI −0.57 to −0.11; low-quality evidence) after surgery. However, no evidence of effect was found for lidocaine to reduce pain at 'late time points (48 hours)' (MD -0.22 cm, 95% CI −0.47 to 0.03; low-quality evidence). Pain reduction was most obvious at 'early time points' in participants undergoing laparoscopic abdominal surgery (MD −1.14, 95% CI −1.51 to −0.78; low-quality evidence) and open abdominal surgery (MD −0.72, 95% CI −0.96 to −0.47; moderate-quality evidence). No evidence of effect was found for lidocaine to reduce pain in participants undergoing all other surgeries (MD −0.30, 95% CI −0.89 to 0.28; low-quality evidence). Quality of evidence is limited due to inconsistency and indirectness (small trial sizes).Evidence of effect was found for lidocaine on gastrointestinal recovery regarding the reduction of the time to first flatus (MD −5.49 hours, 95% CI −7.97 to −3.00; low-quality evidence), time to first bowel movement (MD −6.12 hours, 95% CI −7.36 to −4.89; low-quality evidence), and the risk of paralytic ileus (risk ratio (RR) 0.38, 95% CI 0.15 to 0.99; low-quality evidence). However, no evidence of effect was found for lidocaine on shortening the time to first defaecation (MD −9.52 hours, 95% CI −23.24 to 4.19; very low-quality evidence).Furthermore, we found evidence of positive effects for lidocaine administration on secondary outcomes such as reduction of length of hospital stay, postoperative nausea, intraoperative and postoperative opioid requirements. There was limited data on the effect of IV lidocaine on adverse effects (e.g. death, arrhythmias, other heart rate disorders or signs of lidocaine toxicity) compared to placebo treatment as only a limited number of studies systematically analysed the occurrence of adverse effects of the lidocaine intervention.The comparison of intravenous lidocaine versus epidural analgesia revealed no evidence of effect for lidocaine on relevant outcomes. However, the results have to be considered with caution due to imprecision of the effect estimates.Authors' conclusions: There is low to moderate evidence that this intervention, when compared to placebo, has an impact on pain scores, especially in the early postoperative phase, and on postoperative nausea. There is limited evidence that this has further impact on other relevant clinical outcomes, such as gastrointestinal recovery, length of hospital stay, and opioid requirements. So far there is a scarcity of studies that have systematically assessed the incidence of adverse effects; the optimal dose; timing (including the duration of the administration); and the effects when compared with epidural anaesthesia.
US: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009642.pub2/abstract


Record #2 of 1442
ID: CD001893
AU: Guay Joanne
AU: Nishimori Mina
AU: Kopp Sandra
TI: Epidural local anaesthetics versus opioid-based analgesic regimens for postoperative gastrointestinal paralysis, vomiting and pain after abdominal surgery
SO: Cochrane Database of Systematic Reviews
YR: 2016
NO: 7
PB: John Wiley & Sons, Ltd
KY: Anesthesia, Epidural;Abdomen [surgery];Analgesics, Opioid [therapeutic use];Anesthetics, Local [therapeutic use];Flatulence;Gastrointestinal Diseases [drug therapy] [etiology];Gastrointestinal Transit [drug effects];Pain, Postoperative [drug therapy];Paralysis [drug therapy] [etiology];Postoperative Complications [drug therapy];Postoperative Nausea and Vomiting [drug therapy];Randomized Controlled Trials as Topic;Humans[checkword]
DOI: 10.1002/14651858.CD001893.pub2
AB: Background: Gastrointestinal paralysis, nausea and vomiting and pain are major clinical problems following abdominal surgery. Anaesthetic and analgesic techniques that reduce pain and postoperative nausea and vomiting (PONV), while preventing or reducing postoperative ileus, may reduce postoperative morbidity, duration of hospitalization and hospital costs. This review was first published in 2001 and was updated by new review authors in 2016.Objectives: To compare effects of postoperative epidural analgesia with local anaesthetics versus postoperative systemic or epidural opioids in terms of return of gastrointestinal transit, postoperative pain control, postoperative vomiting, incidence of anastomotic leak, length of hospital stay and costs after abdominal surgery.Search methods: We identified trials by conducting computerized searches of the Cochrane Central Register of Controlled Trials (CENTRAL) (2014, Issue 12), MEDLINE (from 1950 to December 2014) and EMBASE (from 1974 to December 2014) and by checking the reference lists of trials retained. When we reran the search in February 2016, we added 16 potential new studies of interest to the list of ‘Studies awaiting classification' and will incorporate these studies into formal review findings during the next review update.Selection criteria: We included parallel randomized controlled trials comparing effects of postoperative epidural local anaesthetic versus regimens based on systemic or epidural opioids.Data collection and analysis: We rated the quality of studies by using the Cochrane 'Risk of bias' tool. Two review authors independently extracted data and judged the quality of evidence according to the GRADE (Grades of Recommendation, Assessment, Development and Evaluation Working Group) scale.Main results: We included 128 trials with 8754 participants in the review, and 94 trials with 5846 participants in the analysis. Trials included in the review were funded as follows: charity (n = 19), departmental resources (n = 8), governmental sources (n = 15) and industry (in part or in total) (n = 15). The source of funding was not specified for the other studies.Results of 22 trials including 1138 participants show that an epidural containing a local anaesthetic will decrease the time required for return of gastrointestinal transit as measured by time to first flatus after an abdominal surgery (standardized mean difference (SMD) -1.28, 95% confidence interval (CI) -1.71 to -0.86; high quality of evidence; equivalent to 17.5 hours). The effect is proportionate to the concentration of local anaesthetic used. A total of 28 trials including 1559 participants reported a decrease in time to first faeces (stool) (SMD -0.67, 95% CI -0.86 to -0.47; low quality of evidence; equivalent to 22 hours). Thirty-five trials including 2731 participants found that pain on movement at 24 hours after surgery was also reduced (SMD -0.89, 95% CI -1.08 to -0.70; moderate quality of evidence; equivalent to 2.5 on scale from 0 to 10). From findings of 22 trials including 1154 participants we did not find a difference in the incidence of vomiting within 24 hours (risk ratio (RR) 0.84, 95% CI 0.57 to 1.23; low quality of evidence). From investigators in 17 trials including 848 participants we did not find a difference in the incidence of gastrointestinal anastomotic leak (RR 0.74, 95% CI 0.41 to 1.32; low quality of evidence). Researchers in 30 trials including 2598 participants noted that epidural analgesia reduced length of hospital stay for an open surgery (SMD -0.20, 95% CI -0.35 to -0.04; very low quality of evidence; equivalent to one day). Data on costs were very limited.Authors' conclusions: An epidural containing a local anaesthetic, with or without the addition of an opioid, accelerates the return of gastrointestinal transit (high quality of evidence). An epidural containing a local anaesthetic with an opioid decreases pain after abdominal surgery (moderate quality of evidence). We did not find a difference in the incidence of vomiting or anastomotic leak (low quality of evidence). For open surgery, an epidural containing a local anaesthetic would reduce the length of hospital stay (very low quality of evidence).
US: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD001893.pub2/abstract


Record #3 of 1442
ID: CD009016
AU: Madrid Eva
AU: Urrútia Gerard
AU: Roqué i Figuls Marta
AU: Pardo-Hernandez Hector
AU: Campos Juan Manuel
AU: Paniagua Pilar
AU: Maestre Luz
AU: Alonso-Coello Pablo
TI: Active body surface warming systems for preventing complications caused by inadvertent perioperative hypothermia in adults
SO: Cochrane Database of Systematic Reviews
YR: 2016
NO: 4
PB: John Wiley & Sons, Ltd
KY: Air;Blood Loss, Surgical;Body Surface Area;Body Temperature Regulation;Cardiovascular Diseases [prevention & control];Cold Temperature [adverse effects];Heating [instrumentation] [methods];Hypothermia [etiology] [prevention & control];Intraoperative Complications [prevention & control];Randomized Controlled Trials as Topic;Surgical Wound Infection [prevention & control];Humans[checkword]
DOI: 10.1002/14651858.CD009016.pub2
AB: Background: Inadvertent perioperative hypothermia is a phenomenon that can occur as a result of the suppression of the central mechanisms of temperature regulation due to anaesthesia, and of prolonged exposure of large surfaces of skin to cold temperatures in operating rooms. Inadvertent perioperative hypothermia has been associated with clinical complications such as surgical site infection and wound-healing delay, increased bleeding or cardiovascular events. One of the most frequently used techniques to prevent inadvertent perioperative hypothermia is active body surface warming systems (ABSW), which generate heat mechanically (heating of air, water or gels) that is transferred to the patient via skin contact.Objectives: To assess the effectiveness of pre- or intraoperative active body surface warming systems (ABSW), or both, to prevent perioperative complications from unintended hypothermia during surgery in adults.Search methods: We searched the Cochrane Central Register of Controlled Trials (CENTRAL; Issue 9, 2015); MEDLINE (PubMed) (1964 to October 2015), EMBASE (Ovid) (1980 to October 2015), and CINAHL (Ovid) (1982 to October 2015).Selection criteria: We included randomized controlled trials (RCTs) that compared an ABSW system aimed at maintaining normothermia perioperatively against a control or against any other ABSW system. Eligible studies also had to include relevant clinical outcomes other than measuring temperature alone.Data collection and analysis: Several authors, by pairs, screened references and determined eligibility, extracted data, and assessed risks of bias. We resolved disagreements by discussion and consensus, with the collaboration of a third author.Main results: We included 67 trials with 5438 participants that comprised 79 comparisons. Forty-five RCTs compared ABSW versus control, whereas 18 compared two different types of ABSW, and 10 compared two different techniques to administer the same type of ABSW. Forced-air warming (FAW) was by far the most studied intervention.Trials varied widely regarding whether the interventions were applied alone or in combination with other active (based on a different mechanism of heat transfer) and/or passive methods of maintaining normothermia. The type of participants and surgical interventions, as well as anaesthesia management, co-interventions and the timing of outcome measurement, also varied widely. The risk of bias of included studies was largely unclear due to limitations in the reports. Most studies were open-label, due to the nature of the intervention and the fact that temperature was usually the principal outcome. Nevertheless, given that outcome measurement could have been conducted in a blinded manner, we rated the risk of detection and performance bias as high.The comparison of ABSW versus control showed a reduction in the rate of surgical site infection (risk ratio (RR) 0.36, 95% confidence interval (CI) 0.20 to 0.66; 3 RCTs, 589 participants, low-quality evidence). Only one study at low risk of bias observed a beneficial effect with forced-air warming on major cardiovascular complications (RR 0.22, 95% CI 0.05 to 1.00; 1 RCT with 12 events, 300 participants, low-quality evidence) in people at high cardiovascular risk. We found no beneficial effect for mortality. ABSW also reduced blood loss during surgery but the magnitude of this effect seems to be irrelevant (MD -46.17 mL, 95% CI -82.74 to -9.59; I² = 78%; 20 studies, 1372 participants). The same conclusion applies to total fluids infused during surgery (MD -144.49 mL, 95% CI -221.57 to -67.40; I² = 73%; 24 studies, 1491 participants). These effects did not translate into a significant reduction in the number of participants being transfused or the average amount of blood transfused. ABSW was associated with a reduction in shivering (RR 0.39, 95% CI 0.28 to 0.54; 29 studies, 1922 participants) and in thermal comfort (standardized mean difference (SMD) 0.76, 95% CI 0.29 to 1.24; I² = 77%, 4 trials, 364 participants).For the comparison between different types of ABSW system or modes of administration of a particular type of ABSW, we found no evidence for the superiority of any system in terms of clinical outcomes, except for extending systemic warming to the preoperative period in participants undergoing major abdominal surgery (one study at low risk of bias).There were limited data on adverse effects (the most relevant being thermal burns). While some trials included a narrative report mentioning that no adverse effects were observed, the majority made no reference to it. Nothing so far suggests that ABSW involves a significant risk to patients.Authors' conclusions: Forced-air warming seems to have a beneficial effect in terms of a lower rate of surgical site infection and complications, at least in those undergoing abdominal surgery, compared to not applying any active warming system. It also has a beneficial effect on major cardiovascular complications in people with substantial cardiovascular disease, although the evidence is limited to one study. It also improves patient's comfort, although we found high heterogeneity among trials. While the effect on blood loss is statistically significant, this difference does not translate to a significant reduction in transfusions. Again, we noted high heterogeneity among trials for this outcome. The clinical relevance of blood loss reduction is therefore questionable. The evidence for other types of ABSW is scant, although there is some evidence of a beneficial effect in the same direction on chills/shivering with electric or resistive-based heating systems. Some evidence suggests that extending systemic warming to the preoperative period could be more beneficial than limiting it only to during surgery. Nothing suggests that ABSW systems pose a significant risk to patients.The difficulty in observing a clinically-relevant beneficial effect with ABSW in outcomes other than temperature may be explained by the fact that many studies applied concomitant procedures that are routinely in place as co-interventions to prevent hypothermia, whether passive or active warming systems based in other physiological mechanisms (e.g. irrigation fluid or gas warming), as well as a stricter control of temperature in the context of the study compared with usual practice. These may have had a beneficial effect on the participants in the control group, leading to an underestimation of the net benefit of ABSW.
US: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009016.pub2/abstract

"""



# In[4]:

from pyparsing import *
# http://stackoverflow.com/questions/9361521/pyparsing-capturing-groups-of-arbitrary-text-with-given-headers-as-nested-lists?rq=1

ParserElement.defaultWhitespaceChars=(" \t")
NL = LineEnd().suppress()
END = StringEnd()

number = Word(nums)

record_id = Suppress(Literal("Record #") + number + Literal("of") + number)
record_id.parseString( """Record #3 of 1442""" )

label = lineStart + Word(alphas) + FollowedBy(":")

kv = label + Suppress(":") + restOfLine
kv.parseString("ID: CD009016")


# text isn't just anything! don't accept header line, and stop at the end of the input string
# text=Group(~record_id + ~END + restOfLine)

overall_structure = ZeroOrMore(Group(Suppress(record_id) +
                                     Group(ZeroOrMore(Group(kv)))))
overall_structure.ignore(NL)

# from pprint import pprint
# print(overall_structure.parseString(testData).asList())
file_text = open("citation-export.txt").read()
len(file_text)

# Clean
import unicodedata
def remove_control_characters(s):
    return "".join(ch for ch in s if unicodedata.category(ch)[0]!="C" or ch == "\n")

file_text = (remove_control_characters(file_text))
results = overall_structure.parseString(file_text).asList()


# len(results)
# results[1]

# Now loop through results and convert into something which will fit into an excel table

def tidy_authors(result):
    """
    Pass in result (the second part)
    Enumerate tags
    Append the first author
    Append a combined author list
    """
    i = 1
    new_result = []
    authors = []

    for i in range(len(result)):
        keyval = result[i]
        if keyval[0] == "AU":
            keyval = ["AU" + str(i), keyval[1]]
            if i == 1:
                first_author = ["AU_first", keyval[1]]
            authors.append(keyval[1])
            ++i
        else:
            new_result.append(keyval)
    new_result.append(first_author)
    new_result.append(["AU_all", ",".join(authors)])
    return(new_result)

results_clean = []
headers = {}
# Loop through results
for j in range(len(results)):
    foo = results[j][0]
    foo_tidy = tidy_authors(foo)
    foo_dict = {item[0]: item[1].strip() for item in foo_tidy}
    results_clean.append(foo_dict)
    keys = foo_dict.keys()
    if len(keys) > len(headers):
        headers = keys | headers
    # print(len(foo_dict))

print(headers)
print(len(results_clean))
# Now write out to tab-separated values

# Now convert to tab separated values
import csv
toTSV = results_clean

with open('cochrane-export.tsv', 'w') as tsvfile:
    writer = csv.DictWriter(tsvfile, fieldnames=headers, dialect=csv.excel_tab)

    writer.writeheader()
    # writer.writerow(foo_dict)
    writer.writerows(toTSV)