Skip to content

Instantly share code, notes, and snippets.

@jmcarp
Last active March 30, 2023 03:07
Show Gist options
  • Save jmcarp/7105045 to your computer and use it in GitHub Desktop.
Save jmcarp/7105045 to your computer and use it in GitHub Desktop.
Extract text from PDF document using PDFMiner
"""
Extract PDF text using PDFMiner. Adapted from
http://stackoverflow.com/questions/5725278/python-help-using-pdfminer-as-a-library
"""
from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter#process_pdf
from pdfminer.pdfpage import PDFPage
from pdfminer.converter import TextConverter
from pdfminer.layout import LAParams
from cStringIO import StringIO
def pdf_to_text(pdfname):
# PDFMiner boilerplate
rsrcmgr = PDFResourceManager()
sio = StringIO()
codec = 'utf-8'
laparams = LAParams()
device = TextConverter(rsrcmgr, sio, codec=codec, laparams=laparams)
interpreter = PDFPageInterpreter(rsrcmgr, device)
# Extract text
fp = file(pdfname, 'rb')
for page in PDFPage.get_pages(fp):
interpreter.process_page(page)
fp.close()
# Get text from StringIO
text = sio.getvalue()
# Cleanup
device.close()
sio.close()
return text
@namitha-sharma
Copy link

I am new to python. I am currently using eclipse IDE PyDev for python.I am not able to use pdfminer in eclipse. I have added the path of pdf miner to environment variable in my windows 7,just in case if it works, but still no luck. I am getting unsolved import error.Can you please help with my queries.
1)Which IDE is best for Python .(I want to work on pdf extraction and excel sheets)
2)how do I use PDFMiner.

Thanks In Advance.

@mkwoodburn
Copy link

What version of python area you using? Python 3 or Python 2? If you are using python 3 you will need to pip install pdfminer.six.

@murlikrishna03192
Copy link

hay, i want to extract pdf text page by page from pdf file. if i use pdfminer it converts whole pdf into text then it gives the result is their any possibilities to get the text of each page separately from pdf

@mingyun
Copy link

mingyun commented Jan 5, 2017

run your code in windows7 python2.7,but get error

    self.outfp.write(text.encode(self.codec, 'ignore'))
TypeError: unicode argument expected, got 'str'

@jtlz2
Copy link

jtlz2 commented Jul 27, 2017

Also (now) getting TypeError: unicode argument expected, got 'str' on macOS...

@eyxu
Copy link

eyxu commented Aug 24, 2017

from pdfminer.pdfparser import PDFParser
from pdfminer.pdfdocument import PDFDocument
from pdfminer.pdfpage import PDFPage
from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter
from pdfminer.converter import PDFPageAggregator
from pdfminer.layout import LAParams, LTTextBox, LTTextLine
from pdfminer.pdfdevice import PDFDevice

rsrcmgr = PDFResourceManager()
laparams = LAParams()
laparams.char_margin = 0.5
laparams.word_margin = 0.5

device = PDFPageAggregator(rsrcmgr, laparams=laparams)

interpreter = PDFPageInterpreter(rsrcmgr, device)

for page in PDFPage.get_pages(fp):
    interpreter.process_page(page)
    layout = device.get_result()

For some pdf files, it appears that the device.get_result() returns an object whose _objs has 0 length. The pdf files contains a table with cells having text or numbers. (I used to have pdfminer3k, under that package, these pdf files will get exception for zlib error.)

@gijbelst
Copy link

gijbelst commented Oct 9, 2017

@mingyun, @jtlz2,

Try changing: from cStringIO import StringIO to from io import BytesIO

@ozgurdugmeci
Copy link

from io import BytesIO
from io import StringIO

from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter
from pdfminer.converter import TextConverter
from pdfminer.layout import LAParams
from pdfminer.pdfpage import PDFPage

PDFMiner boilerplate

rsrcmgr = PDFResourceManager()
sio = StringIO()
codec = 'utf-8'
laparams = LAParams()
device = TextConverter(rsrcmgr, sio, codec=codec, laparams=laparams)
interpreter = PDFPageInterpreter(rsrcmgr, device)

fp = open('C:\Users\Ozgur\Desktop\otivit\kitaplar\pdf2\çavdar-tarlasında-çocuklar-salinger.pdf', 'rb')
for page in PDFPage.get_pages(fp):
interpreter.process_page(page)
fp.close()
text = sio.getvalue()
text=text.replace(chr(272)," ")
print(type(text))
f = open('C:\Users\Ozgur\Desktop\otivit\kitaplar\txt2\yes.txt','w')
f.write(text)

print("hello")

@CherylJacob
Copy link

Is it possible to extract only a specific section from the PDF ?

@vikasAravi
Copy link

Yes it's possible to extract a specific section from pdf by using find method in python

@vikasAravi
Copy link

Unicode argument error mean
You extract the code from the pdf and the page extracted from the pdf is in byte form so convert that to string:)

@CherylJacob
Copy link

Hello, Is there a way I can make changes to the pdfminer so that it ignores the tables in the pdf doc while converting the pdf to text?

@RAZelzner
Copy link

Yes it's possible to extract a specific section from pdf by using find method in python

How would the be the Code, if i want to extract from pdf-page 147 until pdf-page 200?

@madux
Copy link

madux commented Jan 10, 2019

HERE IS THE CODE

from io import BytesIO
from io import StringIO

from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter
from pdfminer.converter import TextConverter
from pdfminer.layout import LAParams
from pdfminer.pdfpage import PDFPage

rsrcmgr = PDFResourceManager()
sio = BytesIO()
codec = 'utf-8'
laparams = LAParams()
device = TextConverter(rsrcmgr, sio, codec=codec, laparams=laparams)
interpreter = PDFPageInterpreter(rsrcmgr, device)

fp = open('C:\Python27\MINER\strin.pdf', 'rb')
for page in PDFPage.get_pages(fp):
interpreter.process_page(page)
fp.close()
text = sio.getvalue()
#text=text.replace(chr(272)," ")
print(type(text))
f = open('C:\Python27\MINER\hob.txt','w')
f.write(text)

print("hello")

@heenamakwana
Copy link

I am trying to convert PDF to TEXT in Python 3.6. But I got error by using from pdfminer.pdfpage import PDFPage. I have no idea how to use pdfminer.six. The error is following:
Traceback (most recent call last):
File "", line 1, in
File "/home/system/anaconda3/lib/python3.6/site-packages/pdfminer/pdfpage.py", line 5, in
from .pdftypes import PDFObjectNotFound
ImportError: cannot import name 'PDFObjectNotFound'

Please reply.

@mayurgite
Copy link

I am also facing same error.

@ahmedkhemiri95
Copy link

You can find a great information and lab for PDF Processing with python here:
https://medium.com/@ahmedkhemiri24/pdf-preprocessing-with-python-19829752af9f

@AyeshaSarwar
Copy link

I am trying to convert pdf to text using pdfminer. No matter what settings, I use, my numbers, digits, dates are replaced by spaces in the output file. Does anybody know what is it due to?

@AyeshaSarwar
Copy link

The code, I am using right now is:

from pdfminer.pdfparser import PDFParser
from pdfminer.pdfdocument import PDFDocument
from pdfminer.pdfpage import PDFPage
from pdfminer.pdfpage import PDFTextExtractionNotAllowed
from pdfminer.pdfinterp import PDFResourceManager
from pdfminer.pdfinterp import PDFPageInterpreter
from pdfminer.pdfdevice import PDFDevice
from pdfminer.layout import LAParams
from pdfminer.converter import PDFPageAggregator
import pdfminer

Open a PDF file.

fp = open('doc.pdf', 'rb')

Create a PDF parser object associated with the file object.

parser = PDFParser(fp)

Create a PDF document object that stores the document structure.

Password for initialization as 2nd parameter

document = PDFDocument(parser)

Check if the document allows text extraction. If not, abort.

if not document.is_extractable:
raise PDFTextExtractionNotAllowed

Create a PDF resource manager object that stores shared resources.

rsrcmgr = PDFResourceManager()

Create a PDF device object.

device = PDFDevice(rsrcmgr)

BEGIN LAYOUT ANALYSIS

Set parameters for analysis.

laparams = LAParams(all_texts=True)

Create a PDF page aggregator object.

device = PDFPageAggregator(rsrcmgr, laparams=laparams)

Create a PDF interpreter object.

interpreter = PDFPageInterpreter(rsrcmgr, device)

def parse_obj(lt_objs):
file = open('doc.txt', "a+")

# loop over the object list
for obj in lt_objs:

    # if it's a textbox, print text and location
    if isinstance(obj, pdfminer.layout.LTTextBoxHorizontal):
        post_text = obj.get_text().replace('\n', ' ')
        file.write(post_text)

    # if it's a container, recurse
    elif isinstance(obj, pdfminer.layout.LTFigure):
        parse_obj(obj._objs)
file.close()

loop over all pages in the document

for page in PDFPage.create_pages(document):

# read the page into a layout object
interpreter.process_page(page)
layout = device.get_result()

# extract text from this object
parse_obj(layout._objs)

@ahmedkhemiri95
Copy link

ahmedkhemiri95 commented May 9, 2020

I'm so glad to announce that I released Python Multiple and Large PDF Documents Text Extraction solution on Github:
https://github.com/ahmedkhemiri95/PDFs-TextExtract

@moulya-somasundara
Copy link

How do you extract a URL present in a PDF? For ex. if you are trying to extract the URL present in the left hand panel of the pdf version of a LinkedIn Profile using PDF miner?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment