Skip to content

Instantly share code, notes, and snippets.

@jmcarp
Last active March 30, 2023 03:07
Show Gist options
  • Save jmcarp/7105045 to your computer and use it in GitHub Desktop.
Save jmcarp/7105045 to your computer and use it in GitHub Desktop.
Extract text from PDF document using PDFMiner
"""
Extract PDF text using PDFMiner. Adapted from
http://stackoverflow.com/questions/5725278/python-help-using-pdfminer-as-a-library
"""
from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter#process_pdf
from pdfminer.pdfpage import PDFPage
from pdfminer.converter import TextConverter
from pdfminer.layout import LAParams
from cStringIO import StringIO
def pdf_to_text(pdfname):
# PDFMiner boilerplate
rsrcmgr = PDFResourceManager()
sio = StringIO()
codec = 'utf-8'
laparams = LAParams()
device = TextConverter(rsrcmgr, sio, codec=codec, laparams=laparams)
interpreter = PDFPageInterpreter(rsrcmgr, device)
# Extract text
fp = file(pdfname, 'rb')
for page in PDFPage.get_pages(fp):
interpreter.process_page(page)
fp.close()
# Get text from StringIO
text = sio.getvalue()
# Cleanup
device.close()
sio.close()
return text
@mayurgite
Copy link

I am also facing same error.

@ahmedkhemiri95
Copy link

You can find a great information and lab for PDF Processing with python here:
https://medium.com/@ahmedkhemiri24/pdf-preprocessing-with-python-19829752af9f

@AyeshaSarwar
Copy link

I am trying to convert pdf to text using pdfminer. No matter what settings, I use, my numbers, digits, dates are replaced by spaces in the output file. Does anybody know what is it due to?

@AyeshaSarwar
Copy link

The code, I am using right now is:

from pdfminer.pdfparser import PDFParser
from pdfminer.pdfdocument import PDFDocument
from pdfminer.pdfpage import PDFPage
from pdfminer.pdfpage import PDFTextExtractionNotAllowed
from pdfminer.pdfinterp import PDFResourceManager
from pdfminer.pdfinterp import PDFPageInterpreter
from pdfminer.pdfdevice import PDFDevice
from pdfminer.layout import LAParams
from pdfminer.converter import PDFPageAggregator
import pdfminer

Open a PDF file.

fp = open('doc.pdf', 'rb')

Create a PDF parser object associated with the file object.

parser = PDFParser(fp)

Create a PDF document object that stores the document structure.

Password for initialization as 2nd parameter

document = PDFDocument(parser)

Check if the document allows text extraction. If not, abort.

if not document.is_extractable:
raise PDFTextExtractionNotAllowed

Create a PDF resource manager object that stores shared resources.

rsrcmgr = PDFResourceManager()

Create a PDF device object.

device = PDFDevice(rsrcmgr)

BEGIN LAYOUT ANALYSIS

Set parameters for analysis.

laparams = LAParams(all_texts=True)

Create a PDF page aggregator object.

device = PDFPageAggregator(rsrcmgr, laparams=laparams)

Create a PDF interpreter object.

interpreter = PDFPageInterpreter(rsrcmgr, device)

def parse_obj(lt_objs):
file = open('doc.txt', "a+")

# loop over the object list
for obj in lt_objs:

    # if it's a textbox, print text and location
    if isinstance(obj, pdfminer.layout.LTTextBoxHorizontal):
        post_text = obj.get_text().replace('\n', ' ')
        file.write(post_text)

    # if it's a container, recurse
    elif isinstance(obj, pdfminer.layout.LTFigure):
        parse_obj(obj._objs)
file.close()

loop over all pages in the document

for page in PDFPage.create_pages(document):

# read the page into a layout object
interpreter.process_page(page)
layout = device.get_result()

# extract text from this object
parse_obj(layout._objs)

@ahmedkhemiri95
Copy link

ahmedkhemiri95 commented May 9, 2020

I'm so glad to announce that I released Python Multiple and Large PDF Documents Text Extraction solution on Github:
https://github.com/ahmedkhemiri95/PDFs-TextExtract

@moulya-somasundara
Copy link

How do you extract a URL present in a PDF? For ex. if you are trying to extract the URL present in the left hand panel of the pdf version of a LinkedIn Profile using PDF miner?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment