Extracting Tables from PDFs: A Guide Using Python with Tabula and Camelot#

The Need for PDF Table Extraction#

The demand for extracting structured data from PDFs stems from several key areas:

• Data Analysis: Researchers, analysts, and data scientists frequently encounter reports or datasets embedded within PDFs. Extracting this data into a structured format is the first step before any analysis can begin.
• Automation: Automating processes that rely on data from PDF reports (e.g., updating databases, generating summaries, comparing figures) requires reliable programmatic extraction. Manual copy-pasting is time-consuming and prone to errors.
• Reporting and Compliance: Organizations may receive financial statements, invoices, or regulatory documents in PDF format. Extracting key figures is essential for internal reporting and compliance checks.
• Digitization of Information: Converting legacy or published PDF documents into searchable, structured datasets makes the information more accessible and usable.

Traditional text extraction tools often fail with tables because they might read across columns or misinterpret borders and spacing. Libraries like Tabula and Camelot employ more sophisticated techniques to identify the boundaries and structure of tables within a PDF page.

Introducing Tabula and Camelot#

Tabula-py and Camelot are specialized Python libraries built for the task of PDF table extraction.

• Tabula-py: This library is a Python wrapper for the Java-based Tabula library. It is particularly effective at extracting tables from native PDFs, which are documents created directly from a digital source where text and structure information are still present (though potentially challenging to access). Tabula works well when table cell boundaries are clearly defined or inferred from spacing.
• Camelot: Camelot is a Python library that offers more flexibility, designed to handle a wider variety of PDFs, including scanned documents and PDFs with complex layouts or unclear borders. It provides different “flavors” of parsing strategies (“Lattice” and “Stream”) to tackle various table structures. Camelot often requires more tuning and parameter adjustments for optimal results but can succeed where simpler tools fail.

Both libraries provide the extracted data in a structured format, commonly as a list of pandas DataFrames, making integration with standard Python data science workflows seamless.

Getting Started: Installation#

Before using Tabula-py or Camelot, installation is necessary. Both have some dependencies.

1
pip install tabula-py

1
pip install camelot-py[cv] opencv-python pandas

Using Tabula-py for Table Extraction#

Tabula-py provides a straightforward function, read_pdf(), to extract tables.

1
import tabula
2
import pandas as pd
3

4
pdf_file = "example_report.pdf" # Replace with your PDF file path
5

6
try:
7
    # Extract tables from all pages, allow multiple tables per page
8
    list_of_dfs = tabula.read_pdf(pdf_file, pages='all', multiple_tables=True, guess=True)
9

10
    if list_of_dfs:
11
        print(f"Found {len(list_of_dfs)} tables.")
12
        # Access and display the first table
13
        first_table = list_of_dfs[0]
14
        print("\nFirst extracted table (first 5 rows):")
15
        print(first_table.head().to_markdown(index=False)) # Using to_markdown for clear printing
16

17
        # Example: Save all tables to separate CSV files
18
        for i, df in enumerate(list_of_dfs):
19
            df.to_csv(f"table_{i+1}.csv", index=False)
20
            print(f"Saved table {i+1} to table_{i+1}.csv")
21

22
    else:
23
        print("No tables found in the PDF.")
24

25
except Exception as e:
26
    print(f"An error occurred: {e}")
27

28
# Example of direct conversion to CSV
29
# try:
30
#     tabula.convert_into(pdf_file, "all_tables.csv", output_format="csv", pages='all', multiple_tables=True)
31
#     print("\nAll tables converted and saved to all_tables.csv")
32
# except Exception as e:
33
#     print(f"Error during direct conversion: {e}")

Using Camelot for Table Extraction#

Camelot offers two primary parsing “flavors”:

• Lattice: Assumes the table has a grid-like structure with clearly defined lines separating cells. This is suitable for tables with borders.
• Stream: Suitable for tables without borders, where cells are separated by whitespace. It works by grouping text elements based on their proximity.

Camelot’s strength lies in its ability to handle less structured or scanned PDFs, often requiring tuning parameters to achieve the best results.

1
import camelot
2
import pandas as pd
3

4
pdf_file = "example_scanned_report.pdf" # Replace with your PDF file path
5
# Consider trying 'lattice' first, then 'stream' if needed
6
extraction_flavor = 'lattice' # or 'stream'
7

8
try:
9
    # Extract tables using the specified flavor
10
    tables = camelot.read_pdf(pdf_file, pages='all', flavor=extraction_flavor)
11

12
    print(f"Found {tables.n} tables using '{extraction_flavor}' flavor.")
13

14
    if tables.n > 0:
15
        # Access and display the first table
16
        first_table = tables[0]
17
        print("\nFirst extracted table (first 5 rows):")
18
        print(first_table.df.head().to_markdown(index=False))
19

20
        print("\nParsing Report for first table:")
21
        print(first_table.parsing_report)
22

23
        # Example: Save all tables to separate CSV files
24
        for i, table in enumerate(tables):
25
            table.to_csv(f"camelot_table_{i+1}.csv", index=False)
26
            print(f"Saved table {i+1} to camelot_table_{i+1}.csv")
27

28
    else:
29
        print("No tables found in the PDF using the current settings.")
30

31
except Exception as e:
32
    print(f"An error occurred: {e}")
33
    print("Ensure Ghostscript is installed and in your PATH.")

Choosing Between Tabula and Camelot#

Selecting the appropriate library depends largely on the nature of the PDF document:

• For native PDFs with clear tables: Start with Tabula-py. Its guess=True option often works effectively out-of-the-box.
• For scanned PDFs or complex native PDFs: Camelot is generally a better choice. Experiment with both ‘lattice’ and ‘stream’ flavors. Use Camelot’s plotting feature to understand why extraction might be failing and tune parameters like table_areas, row_tol, etc.

In some cases, trying one library and falling back to the other if the first fails is a practical approach.

Troubleshooting Common Issues#

• Dependencies not found: Ensure Java (for Tabula-py) and Ghostscript (for Camelot) are installed correctly and are accessible via your system’s PATH. Installation instructions vary by operating system.
• No tables detected or incorrect extraction:
  • Verify the PDF actually contains tables.
  • Try different pages specifications.
  • For Tabula, try setting guess=False and manually specifying the table area using the area parameter (requires identifying coordinates, potentially using the online Tabula tool or a PDF reader’s coordinate display).
  • For Camelot, try the other flavor (‘lattice’ vs ‘stream’). Use the plot() function to visualize what Camelot sees. Manually specify the table_areas if needed. Adjust parameters like row_tol or column_tol.
• Merged cells or complex headers: These are challenging for both libraries. Post-processing the extracted DataFrame with pandas might be necessary to handle merged cells or multi-row headers.
• Multi-page tables: Both libraries extract tables page by page. Combining parts of a table spread across multiple pages typically requires custom code to concatenate the relevant DataFrames based on header identification or page order.

Real-World Application: Automating Financial Data Extraction#

Consider a scenario where an analyst needs to regularly extract quarterly revenue and expense tables from PDF reports published by various companies. Manually copying and pasting this data from dozens of PDFs is inefficient and error-prone.

Using Python with Tabula-py or Camelot can automate this process:

1. A script identifies the PDF reports to be processed.
2. For each PDF, the script uses tabula.read_pdf() or camelot.read_pdf(), potentially with page numbers or area specifications refined for the known report structure.
3. The script processes the resulting DataFrames to identify the specific tables containing revenue and expense data (e.g., by searching for keywords in column headers).
4. Relevant data points are extracted from these tables.
5. The extracted data is consolidated into a master spreadsheet or database.
6. The script runs on a schedule or is triggered when new reports are available.

This automation significantly reduces the time spent on data collection, frees the analyst for higher-value tasks like analysis, and improves data accuracy by eliminating manual transcription errors. The choice between Tabula and Camelot would depend on the consistency and structure of the PDF reports from different companies; Camelot might be preferred if the reports vary significantly in layout or include scanned components.

Key Takeaways#

• Extracting tables from PDFs programmatically saves significant time and improves accuracy compared to manual methods.
• Python libraries Tabula-py and Camelot are powerful tools for this task.
• Tabula-py is generally well-suited for native PDFs with clear table structures and requires Java.
• Camelot offers more flexibility for scanned or complex PDFs using ‘lattice’ and ‘stream’ parsing flavors and requires Ghostscript.
• Both libraries return extracted data as pandas DataFrames, facilitating further analysis.
• Choosing the right library depends on the PDF’s characteristics (native vs. scanned, layout complexity).
• Troubleshooting often involves ensuring dependencies are installed, trying different parsing parameters, and potentially specifying table areas manually.
• Real-world applications include automating data collection from financial reports, research papers, and other documents distributed in PDF format.