When data is spread over multiple pagesâlike product listings, event schedules, job postings, or forum threadsâyou want to capture everything rather than just Page 1. Lutraâs ability to handle multi-page scraping saves time and ensures you end up with a complete data set for deeper analysis.
Identify the Page Pattern
Navigate to the site and locate at least two different page URLs.
Compare the URLs to see how the page parameter changes. For example:
http://example.com/products?page=1http://example.com/products?page=2
Verify continuity: confirm that simply incrementing the page number (e.g.,
?page=3) continues to reveal new data.
Example patterns:
?page=1,?page=2&pg=1,&pg=2/page/1/,/page/2/
Start a New Chat with Lutra
Open Lutra and create a new chat session.
Paste the URLs for the first few page (e.g., page 1 and 2).
Explain what data you want to scrape (lists, tables, product info, etc.).
Example instruction:
âPlease scrape the list of products, their prices, and links from these two pages:
http://example.com/products?page=1
http://example.com/products?page=2
Notice the page parameter. I want you to scrape pages 1 through 5 using that same format.â
Tip: If Lutraâs extracted data looks off, instruct it with feedback:
âYou missed the product ASIN code. Please be sure to extract that, try again.â
Test on a Small Number of Pages
Start small by having Lutra scrape just Pages 1â3.
Check the results in the Chat view or in your chosen outputâthis is a great time to ask Lutra to place the data in a Google Sheet or Excel file.
Provide feedback: if Lutra misses any fields or if you need more or less data, let Lutra know and it can refine its approach.
Refine until the output matches your expectations.
Tip: You can say, âSave the scraped data in a new Google Sheet with columns for Product Name, Price, and Link.â This lets you easily review the data before scraping more pages.
Scale Up
Define the page range: once Lutra succeeds on a few pages, specify how far you want to goâfor example, pages 1 through 20.
Ensure consistency: confirm the website continues to follow the same URL pattern for all pages.
Run the broader scrape: Lutra will systematically request each page according to the pattern you provided.
Export your data: if you havenât already, you can now have Lutra write all the scraped data into one CSV, Excel file, or Google Sheet for final checks.
Tip: If youâre not sure how many pages exist, you can estimate a maximum, or ask Lutra to stop scraping once it detects no new data.
Save as a Playbook
Click âCreate Playbookâ in Lutra after youâre satisfied with the multi-page scraping approach.
Reuse or schedule the Playbook to automate multi-page scraping in the future, without re-specifying the same instructions.
Organizing the Data in a Spreadsheet
Once Lutra finishes scraping, you can do one of the following:
Save to a New Google Sheet
In the same chat, simply say:
âPlease put the final data into a Google Sheet with columns: Product Name, Price, Link.â
Lutra will create a new sheet or update an existing one for easy reviewing.
Download an Excel File
Ask Lutra to export the scraped results as an Excel file. It can then provide you with a downloadable link or attachment.
You can open the file in Excel (or any compatible spreadsheet program) to quickly filter, sort, or format the data.
Use CSV or JSON
If you prefer a more developer-friendly format, Lutra can generate a CSV or JSON file that you can import into databases or other tools.
Reviewing your scraped data in spreadsheet format makes it easy to spot-check for completeness and accuracy. If you notice anything missing or incorrect, provide feedback to Lutra directly in the chat, and it can adjust how itâs scraping or how itâs formatting the output.