Crawl multiple URLs
This example crawls the specified list of URLs.
- Cheerio Crawler
- Puppeteer Crawler
- Playwright Crawler
Run on
import { CheerioCrawler } from 'crawlee';
const crawler = new CheerioCrawler({
// Function called for each URL
async requestHandler({ request, $, log }) {
const title = $('title').text();
log.info(`URL: ${request.url}\nTITLE: ${title}`);
},
});
// Run the crawler with initial request
await crawler.run(['http://www.example.com/page-1', 'http://www.example.com/page-2', 'http://www.example.com/page-3']);
tip
To run this example on the Apify Platform, select the apify/actor-node-puppeteer-chrome
image for your Dockerfile.
Run on
import { PuppeteerCrawler } from 'crawlee';
const crawler = new PuppeteerCrawler({
// Function called for each URL
async requestHandler({ request, page, log }) {
const title = await page.title();
log.info(`URL: ${request.url}\nTITLE: ${title}`);
},
});
// Run the crawler with initial request
await crawler.run(['http://www.example.com/page-1', 'http://www.example.com/page-2', 'http://www.example.com/page-3']);
tip
To run this example on the Apify Platform, select the apify/actor-node-playwright-chrome
image for your Dockerfile.
Run on
import { PlaywrightCrawler } from 'crawlee';
const crawler = new PlaywrightCrawler({
// Function called for each URL
async requestHandler({ request, page, log }) {
const title = await page.title();
log.info(`URL: ${request.url}\nTITLE: ${title}`);
},
});
// Run the crawler with initial request
await crawler.run(['http://www.example.com/page-1', 'http://www.example.com/page-2', 'http://www.example.com/page-3']);