Skip to main content
Version: 4.0 (RC)

Remote browser services

Instead of launching a local browser, Crawlee can connect to a remote browser service like Browserbase, Browserless, Steel, or any service that exposes a WebSocket/CDP endpoint. The crawler manages session rotation and the request lifecycle the same way it does locally — only the browser itself runs elsewhere.

Use this when you need IPs in specific regions, want to offload CPU/memory from your runner, or need stealth features the service provides.

How it works​

Set the crawler's remoteBrowser option with the connection details. The crawler builds a RemoteBrowserPool around its own browser plugin, so the connection is always for the matching browser — there's no plugin to construct and no way to mismatch the pool with the crawler. The pool (an IBrowserPool wrapping the regular BrowserPool) owns everything remote: resolving the endpoint, releasing sessions when browsers close, and capping how many remote browsers run at once.

Basic usage​

The simplest form is a static connection URL. Use this when the service exposes a single endpoint and doesn't need per-session setup.

src/main.ts
import { PlaywrightCrawler } from 'crawlee';

const token = process.env.BROWSERLESS_TOKEN!;

const crawler = new PlaywrightCrawler({
    // Connect to a remote browser instead of launching locally. The crawler builds the right
    // pool for its browser — you only supply the connection details.
    remoteBrowser: {
        endpoint: `wss://production-sfo.browserless.io?token=${token}`,
        // Optional — respect the service's concurrent session limit.
        maxOpenBrowsers: 5,
    },
    async requestHandler({ page, request, log }) {
        const title = await page.title();
        log.info(`${request.loadedUrl} — "${title}"`);
    },
});

await crawler.run(['https://crawlee.dev']);

endpoint can also be a function returning { url, context }, called once per browser launch. Pair it with a release callback (it receives the context) to clean up sessions on the service side when the browser closes, crashes, or the pool is destroyed.

maxOpenBrowsers caps the number of concurrent remote browsers — set it to the service's concurrent-session limit to avoid 429 errors. The pool enforces it inside newPage(), which waits for a free slot rather than overshooting.

Self-hosted​

Some services ship a Docker image you can run locally or on your own infrastructure. For example, Browserless has an open-source Chromium image:

docker run -p 3000:3000 -e CONCURRENT=4 ghcr.io/browserless/chromium

Point the pool at the local endpoint with endpoint: 'ws://localhost:3000'.

Custom provider​

For services with a session-create / session-release lifecycle, extend RemoteBrowserProvider and pass the instance as the pool's endpoint. connect() runs once per browser launch and returns the connection URL plus an optional context object passed back to release(). maxOpenBrowsers set on the provider is adopted by the pool.

src/main.ts
import { RemoteBrowserProvider } from '@crawlee/browser-pool';
import { PlaywrightCrawler } from 'crawlee';

const apiKey = process.env.BROWSERBASE_API_KEY!;
const projectId = process.env.BROWSERBASE_PROJECT_ID!;

class BrowserbaseProvider extends RemoteBrowserProvider<{ id: string }> {
    // Respect the service's concurrent session limit to avoid 429s.
    override maxOpenBrowsers = 5;

    async connect() {
        const response = await fetch('https://api.browserbase.com/v1/sessions', {
            method: 'POST',
            headers: { 'x-bb-api-key': apiKey, 'Content-Type': 'application/json' },
            body: JSON.stringify({ projectId }),
        });

        if (!response.ok) {
            throw new Error(`Failed to create session: ${response.status} ${response.statusText}`);
        }

        const session = (await response.json()) as { id: string; connectUrl: string };
        return { url: session.connectUrl, context: { id: session.id } };
    }

    override async release({ id }: { id: string }) {
        await fetch(`https://api.browserbase.com/v1/sessions/${id}`, {
            method: 'POST',
            headers: { 'x-bb-api-key': apiKey, 'Content-Type': 'application/json' },
            body: JSON.stringify({ status: 'REQUEST_RELEASE' }),
        });
    }
}

const crawler = new PlaywrightCrawler({
    // Pass the provider as the `endpoint`; the crawler's pool calls connect()/release() per browser.
    remoteBrowser: {
        endpoint: new BrowserbaseProvider(),
    },
    async requestHandler({ page, request, log }) {
        const title = await page.title();
        log.info(`${request.loadedUrl} — "${title}"`);
    },
});

await crawler.run(['https://crawlee.dev']);

Puppeteer​

PuppeteerCrawler works the same way — build the pool with a PuppeteerPlugin. Puppeteer connects over CDP:

src/main.ts
import { PuppeteerCrawler } from 'crawlee';

const token = process.env.BROWSERLESS_TOKEN!;

const crawler = new PuppeteerCrawler({
    // PuppeteerCrawler connects over CDP. Same `remoteBrowser` option, matching browser guaranteed.
    remoteBrowser: {
        endpoint: `wss://production-sfo.browserless.io?token=${token}`,
    },
    async requestHandler({ page, request, log }) {
        const title = await page.title();
        log.info(`${request.loadedUrl} — "${title}"`);
    },
});

await crawler.run(['https://crawlee.dev']);

For Playwright you can choose the protocol via the remoteBrowser.connection.protocol option: 'cdp' (default, connectOverCDP()) or 'playwright' (connect(), Playwright's own WebSocket protocol).

Sharing a pool across crawlers​

remoteBrowser builds a pool the crawler owns and tears down. To share one remote pool across multiple crawlers, construct a RemoteBrowserPool yourself and pass it as the browserPool option instead — a pool supplied that way is never destroyed by the crawler, so you control its lifecycle. Use remoteBrowser or browserPool, not both.

Declare it with await using and the browsers (and their remote sessions) are released once you are done with the pool:

import { PlaywrightPlugin, RemoteBrowserPool } from '@crawlee/browser-pool';
import { PlaywrightCrawler } from 'crawlee';
import playwright from 'playwright';

await using browserPool = new RemoteBrowserPool({
    browserPlugins: [new PlaywrightPlugin(playwright.chromium)],
    endpoint: 'wss://production-sfo.browserless.io?token=xxx',
    maxOpenBrowsers: 2,
});

await new PlaywrightCrawler({ browserPool, requestHandler: async () => { /* ... */ } }).run(['https://crawlee.dev']);
await new PlaywrightCrawler({ browserPool, requestHandler: async () => { /* ... */ } }).run(['https://apify.com']);
Requires Node.js 24

The await using syntax needs Node.js 24 or later. On Node.js 22 call destroy() yourself instead — it is what the disposal hook calls anyway.

Limitations​

  • headless and launchOptions don't apply. The remote service controls headless mode and browser flags; configure them on the service side.
  • useIncognitoPages is forced to true for Playwright remote connections — connect() / connectOverCDP() don't accept persistent contexts. For state shared across requests, use the SessionPool.
  • userDataDir has no effect — there's no local profile when the browser runs remotely. Use the service's persistence API (e.g. Browserbase Contexts, Steel Profiles).

Further reading​