PDFMiner
PDFMiner
Python PDF parser and analyzer
What's It?
PDFMiner is a suite of programs that help extracting and analyzing text data of PDF documents. Unlike other PDF-related tools, it allows to obtain the exact location of texts in a page, as well as other extra information such as font information or ruled lines. It includes a PDF converter that can transform PDF files into other text formats (such as HTML). It has an extensible PDF parser that can be used for other purposes instead of text analysis.
Features:
- Written entirely in Python. (version 2.4 or newer required)
- PDF-1.7 specification support. (well, almost)
- Non-ASCII languages and vertical writing scripts support.
- Various font types (Type1, TrueType, Type3, and CID) support.
- Basic encryption (RC4) support.
- PDF to HTML conversion (with a sample converter web app).
- Outline (TOC) extraction.
- Tagged contents extraction.
- Infer text running by using clustering technique.
Download:
http://www.unixuser.org/~euske/python/pdfminer/pdfminer-dist-20090711.tar.gz (1.8Mbytes)
Discussion: (for questions and comments, post here)
http://groups.google.com/group/pdfminer-users/
View the source:
http://code.google.com/p/pdfminerr/source/browse/trunk/pdfminer
Online Demonstration: (pdf -> html conversion webapp)
http://pdf2html.tabesugi.net:8080/
How to Install
- Install Python 2.4 or newer.
- Download the PDFMiner source .
- Extract it.
- Run
setup.pyto install:
# python setup.py install
- Do the following test:
$ pdf2txt.py samples/simple1.pdf Hello World Hello World
- Done!
For non-ASCII languages
In order to handle non-ASCII languages (e.g. Japanese), you need to install an additional data called CMap , which is distributed from Adobe.
Here is how:
- Get a CMap archive file from http://www.unixuser.org/~euske/pub/CMap.tar.bz2
- Expand the archive and put the
CMapdirectory under the directory wherepdfmineris installed. (Normally this should be something like/usr/lib/python2.5/site-packages.) For example:$ cd /usr/lib/python2.5/site-packages $ tar jxf CMap.tar.bz2
- Do the following. (this is optional, but highly recommended)
This may take several minutes.$ python -m pdfminer.cmap
How to Use
PDFMiner comes with two handy tools: pdf2txt.py and dumppdf.py .
pdf2txt.py
pdf2txt.py extracts text contents from a PDF file. It extracts all the texts that are to be rendered programmatically, It cannot recognize texts drawn as images that would require optical character recognition. It also extracts the corresponding locations, font names, font sizes, writing direction (horizontal or vertical) for each text portion. You need to provide a password for protected PDF documents when its access is restricted. You cannot extract any text from a PDF document which does not have extraction permission.
For non-ASCII languages, you can specify the output encoding (such as UTF-8).
Note: Not all characters in a PDF can be safely converted to Unicode.
Examples:
$ pdf2txt.py samples/naacl06-shinyama.pdf -o output.html (extract text as an HTML file whose filename is output.html) $ pdf2txt.py -c GBK samples/jo.pdf -o output.html (extract a Chinese HTML file in vertical writing, CMap is required) $ pdf2txt.py -P mypassword secret.pdf -o output.txt (extract a text from an encrypted PDF file)
Options:
-o filename
-p pageno[,pageno,...]
-c codec
-t type
-
html: HTML format. (Default) -
text: TEXT format. -
sgml: SGML format. -
tag: "Tagged PDF" format. A tagged PDF has its own contents annotated with HTML-like tags. pdf2txt tries to extract its content streams rather than inferring its text locations. Tags used here are defined in the PDF specification (See §10.7 "Tagged PDF ").
-M char_margin
-L line_margin
-W word_margin
Each value is specified not as an actual length, but as a proportion of the length to the size of each character in question. The default values are M = 1.0, L = 0.3, and W = 0.2, respectively.
| → | ← M | ||||
Q u i |
c k |
b r o w |
n f o x |
↓ | |
| → | ← W | L | |||
|
↑ |
-s scale
-m maxpages
-P password
-C CMap directory
CMAP_PATH environment variable.
-d
dumppdf.py
dumppdf.py dumps the internal contents of a PDF file in pseudo-XML format. This program is primarily for debugging purpose, but it's also possible to extract some meaningful contents (such as images).
Examples:
$ dumppdf.py -a foo.pdf (dump all the headers and contents, except stream objects) $ dumppdf.py -T foo.pdf (dump the table of contents) $ dumppdf.py -r -i6 foo.pdf > pic.jpeg (extract a JPEG image)
Options:
-a
-i objno,objno, ...
-i options are accepted.
-p pageno,pageno, ...
-p options are accepted. Note that page numbers start from one, not zero.
-r (raw)
-b (binary)
-t (text)
With -r option, the "raw" stream contents are dumped without decompression. With -b option, the decompressed contents are dumped as a binary blob. With -t option, the decompressed contents are dumped in a text format, similar to repr() manner. When -r or -b option is given, no stream header is displayed for the ease of saving it to a file.
-T
-P password
-d
Changes
- 2009/07/11: Improvement in layout analysis. Thanks to Lubos Pintes.
- 2009/05/17: Bugfixes, massive code restructuring, and simple graphic element support added. setup.py is supported.
- 2009/03/30: Text output mode added.
- 2009/03/25: Encoding problems fixed. Word splitting option added.
- 2009/02/28: Robust handling of corrupted PDFs. Thanks to Troy Bollinger.
- 2009/02/01: Various bugfixes. Thanks to Hiroshi Manabe.
- 2009/01/17: Handling a trailer correctly that contains both /XrefStm and /Prev entries.
- 2009/01/10: Handling Type3 font metrics correctly.
- 2008/12/28: Better handling of word spacing. Thanks to Christian Nentwich.
- 2008/09/06: A sample pdf2html webapp added.
- 2008/08/30: ASCII85 encoding filter support.
- 2008/07/27: Tagged contents extraction support.
- 2008/07/10: Outline (TOC) extraction support.
- 2008/06/29: HTML output added. Reorganized the directory structure.
- 2008/04/29: Bugfix for Win32. Thanks to Chris Clark.
- 2008/04/27: Basic encryption and LZW decoding support added.
- 2008/01/07: Several bugfixes. Thanks to Nick Fabry for his contribution.
- 2007/12/31: Initial release.
- 2004/12/24: Start writing the code out of boredom...
Related Projects
Terms and Conditions
Copyright (c) 2004-2009 Yusuke Shinyama <yusuke at cs dot nyu dot edu>
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Yusuke Shinyama