Puppeteer recursive crawl
Run the following example to perform a recursive crawl of a website using PuppeteerCrawler
.
tip
To run this example on the Apify Platform, select the apify/actor-node-puppeteer-chrome
image for your Dockerfile.
Run on
import { PuppeteerCrawler } from 'crawlee';
const crawler = new PuppeteerCrawler({
async requestHandler({ request, page, enqueueLinks, log }) {
const title = await page.title();
log.info(`Title of ${request.url}: ${title}`);
await enqueueLinks({
globs: ['http?(s)://www.iana.org/**'],
});
},
maxRequestsPerCrawl: 10,
});
await crawler.addRequests(['https://www.iana.org/']);
await crawler.run();