Skip to main content
Version: 4.0 (RC)

Remote browser services

Instead of launching a local browser, Crawlee can connect to a remote browser service like Browserbase, Browserless, Steel, or any service that exposes a WebSocket/CDP endpoint. The crawler manages session rotation and the request lifecycle the same way it does locally — only the browser itself runs elsewhere.

Use this when you need IPs in specific regions, want to offload CPU/memory from your runner, or need stealth features the service provides.

How it works​

Set the crawler's remoteBrowser option with the connection details. The crawler builds its regular BrowserPool around a remote variant of its own browser plugin (RemotePlaywrightPlugin or RemotePuppeteerPlugin), so the connection is always for the matching browser — there's no plugin to construct and no way to mismatch the pool with the crawler. The plugin owns everything remote: resolving the endpoint per launch, connecting with the library's connect(), and releasing the session when the browser disconnects.

Basic usage​

The simplest form is a static connection URL. Use this when the service exposes a single endpoint and doesn't need per-session setup.

src/main.ts
import { PlaywrightCrawler } from 'crawlee';

const token = process.env.BROWSERLESS_TOKEN!;

const crawler = new PlaywrightCrawler({
    // Connect to a remote browser instead of launching locally. The crawler builds the right
    // pool for its browser — you only supply the connection details.
    remoteBrowser: {
        endpoint: `wss://production-sfo.browserless.io?token=${token}`,
        // Optional — respect the service's concurrent session limit.
        maxOpenBrowsers: 5,
    },
    async requestHandler({ page, request, log }) {
        const title = await page.title();
        log.info(`${request.loadedUrl} — "${title}"`);
    },
});

await crawler.run(['https://crawlee.dev']);

endpoint can also be a function returning { url, context }, called once per browser launch. Pair it with a release callback (it receives the context) to clean up sessions on the service side when the browser closes, crashes, or the pool is destroyed.

maxOpenBrowsers caps the number of concurrent remote browsers — set it to the service's concurrent-session limit to avoid 429 errors. The pool enforces it inside newPage(), which waits for a free slot rather than overshooting.

Self-hosted​

Some services ship a Docker image you can run locally or on your own infrastructure. For example, Browserless has an open-source Chromium image:

docker run -p 3000:3000 -e CONCURRENT=4 ghcr.io/browserless/chromium

Point the pool at the local endpoint with endpoint: 'ws://localhost:3000'.

Custom provider​

For services with a session-create / session-release lifecycle, extend RemoteBrowserProvider and pass the instance as the endpoint. connect() runs once per browser launch and returns the connection URL plus an optional context object passed back to release(). maxOpenBrowsers set on the provider is adopted by the crawler's pool.

src/main.ts
import { RemoteBrowserProvider } from '@crawlee/browser-pool';
import { PlaywrightCrawler } from 'crawlee';

const apiKey = process.env.BROWSERBASE_API_KEY!;
const projectId = process.env.BROWSERBASE_PROJECT_ID!;

class BrowserbaseProvider extends RemoteBrowserProvider<{ id: string }> {
    // Respect the service's concurrent session limit to avoid 429s.
    override maxOpenBrowsers = 5;

    async connect() {
        const response = await fetch('https://api.browserbase.com/v1/sessions', {
            method: 'POST',
            headers: { 'x-bb-api-key': apiKey, 'Content-Type': 'application/json' },
            body: JSON.stringify({ projectId }),
        });

        if (!response.ok) {
            throw new Error(`Failed to create session: ${response.status} ${response.statusText}`);
        }

        const session = (await response.json()) as { id: string; connectUrl: string };
        return { url: session.connectUrl, context: { id: session.id } };
    }

    override async release({ id }: { id: string }) {
        await fetch(`https://api.browserbase.com/v1/sessions/${id}`, {
            method: 'POST',
            headers: { 'x-bb-api-key': apiKey, 'Content-Type': 'application/json' },
            body: JSON.stringify({ status: 'REQUEST_RELEASE' }),
        });
    }
}

const crawler = new PlaywrightCrawler({
    // Pass the provider as the `endpoint`; the crawler's pool calls connect()/release() per browser.
    remoteBrowser: {
        endpoint: new BrowserbaseProvider(),
    },
    async requestHandler({ page, request, log }) {
        const title = await page.title();
        log.info(`${request.loadedUrl} — "${title}"`);
    },
});

await crawler.run(['https://crawlee.dev']);

Puppeteer​

PuppeteerCrawler accepts the same remoteBrowser option. Puppeteer connects over CDP:

src/main.ts
import { PuppeteerCrawler } from 'crawlee';

const token = process.env.BROWSERLESS_TOKEN!;

const crawler = new PuppeteerCrawler({
    // PuppeteerCrawler connects over CDP. Same `remoteBrowser` option, matching browser guaranteed.
    remoteBrowser: {
        endpoint: `wss://production-sfo.browserless.io?token=${token}`,
    },
    async requestHandler({ page, request, log }) {
        const title = await page.title();
        log.info(`${request.loadedUrl} — "${title}"`);
    },
});

await crawler.run(['https://crawlee.dev']);

For Playwright you can choose the protocol via the remoteBrowser.protocol option: 'cdp' (default, connectOverCDP()) or 'playwright' (connect(), Playwright's own WebSocket protocol).

Tuning or sharing the pool​

remoteBrowser builds a pool the crawler owns and tears down. To tune the pool, or to share one across multiple crawlers, pass the same remoteBrowser to the pool factory for your crawler (playwrightBrowserPool(), puppeteerBrowserPool()) and hand the result over as browserPool instead — a pool supplied that way is never destroyed by the crawler, so you control its lifecycle. Use remoteBrowser or browserPool, not both.

Declare it with await using and the browsers (and their remote sessions) are released once you are done with the pool:

import { PlaywrightCrawler, playwrightBrowserPool } from 'crawlee';

await using browserPool = playwrightBrowserPool({
    remoteBrowser: { endpoint: 'wss://production-sfo.browserless.io?token=xxx', maxOpenBrowsers: 2 },
    maxOpenPagesPerBrowser: 5,
});

await new PlaywrightCrawler({ browserPool, requestHandler: async () => { /* ... */ } }).run(['https://crawlee.dev']);
await new PlaywrightCrawler({ browserPool, requestHandler: async () => { /* ... */ } }).run(['https://apify.com']);

Building the pool from @crawlee/browser-pool directly works too, and lets you rotate between several services — the pool round-robins over its plugins:

import { BrowserPool, RemotePlaywrightPlugin } from '@crawlee/browser-pool';
import playwright from 'playwright';

const browserPool = new BrowserPool({
    browserPlugins: [
        new RemotePlaywrightPlugin(playwright.chromium, { endpoint: 'wss://production-sfo.browserless.io?token=xxx' }),
        new RemotePlaywrightPlugin(playwright.chromium, { endpoint: 'wss://connect.browserbase.com?apiKey=xxx' }),
    ],
});
Requires Node.js 24

The await using syntax needs Node.js 24 or later. On Node.js 22 call destroy() yourself instead — it is what the disposal hook calls anyway.

Limitations​

  • headless and launchOptions don't apply. The remote service controls headless mode and browser flags; configure them on the service side.
  • useIncognitoPages is forced to true for Playwright remote connections — connect() / connectOverCDP() don't accept persistent contexts. For state shared across requests, use the SessionPool.
  • userDataDir has no effect — there's no local profile when the browser runs remotely. Use the service's persistence API (e.g. Browserbase Contexts, Steel Profiles).

Further reading​