Skip to main content
Version: 3.5

Add data to dataset

This example saves data to the default dataset. If the dataset doesn't exist, it will be created. You can save data to custom datasets by using Dataset.open()

Run on
import { Dataset, CheerioCrawler } from 'crawlee';

const crawler = new CheerioCrawler({
// Function called for each URL
async requestHandler({ request, body }) {
// Save data to default dataset
await Dataset.pushData({
url: request.url,
html: body,
});
},
});

await crawler.addRequests([
'http://www.example.com/page-1',
'http://www.example.com/page-2',
'http://www.example.com/page-3',
]);

// Run the crawler
await crawler.run();

Each item in this dataset will be saved to its own file in the following directory:

{PROJECT_FOLDER}/storage/datasets/default/