BrowserPool <Options, BrowserPlugins, BrowserControllerReturn, LaunchContextReturn, PageOptions, PageReturn>

The BrowserPool class is the most important class of the browser-pool module. It manages opening and closing of browsers and their pages and its constructor options allow easy configuration of the browsers' and pages' lifecycle.

The most important and useful constructor options are the various lifecycle hooks. Those allow you to sequentially call a list of (asynchronous) functions at each stage of the browser / page lifecycle.

Example:

import { BrowserPool, PlaywrightPlugin } from '@crawlee/browser-pool';
import playwright from 'playwright';

const browserPool = new BrowserPool({
    browserPlugins: [new PlaywrightPlugin(playwright.chromium)],
    preLaunchHooks: [(pageId, launchContext) => {
        // do something before a browser gets launched
        launchContext.launchOptions.headless = false;
    }],
    postLaunchHooks: [(pageId, browserController) => {
        // manipulate the browser right after launch
        console.dir(browserController.browser.contexts());
    }],
    prePageCreateHooks: [(pageId, browserController) => {
        if (pageId === 'my-page') {
            // make changes right before a specific page is created
        }
    }],
    postPageCreateHooks: [async (page, browserController) => {
        // update some or all new pages
        await page.evaluate(() => {
            // now all pages will have 'foo'
            window.foo = 'bar'
        })
    }],
    prePageCloseHooks: [async (page, browserController) => {
        // collect information just before a page closes
        await page.screenshot();
    }],
    postPageCloseHooks: [(pageId, browserController) => {
        // clean up or log after a job is done
        console.log('Page closed: ', pageId)
    }]
});

Hierarchy

• TypedEmitter<BrowserPoolEvents<BrowserControllerReturn, PageReturn>>
  • BrowserPool

Implements

• IBrowserPool<PageReturn>

Index

Constructors

• constructor

Properties

• activeBrowserControllers
• browserPlugins
• closeInactiveBrowserAfterMillis
• fingerprintCache
• fingerprintGenerator
• fingerprintInjector
• fingerprintOptions
• maxOpenBrowsers
• maxOpenPagesPerBrowser
• operationTimeoutMillis
• pageCounter
• pageIds
• pages
• pageToBrowserController
• postLaunchHooks
• postPageCloseHooks
• postPageCreateHooks
• preLaunchHooks
• prePageCloseHooks
• prePageCreateHooks
• retireBrowserAfterPageCount
• retiredBrowserControllers
• startingBrowserControllers
• useFingerprints
• defaultMaxListeners

Methods

• [asyncDispose]
• addListener
• closeAllBrowsers
• closePage
• destroy
• emit
• eventNames
• extractPageState
• getBrowserControllerByPage
• getMaxListeners
• getPage
• getPageId
• hasActiveBrowserWithFreeCapacity
• hasFreeBrowserSlot
• injectPageState
• listenerCount
• listeners
• newPage
• newPageInNewBrowser
• newPageWithEachPlugin
• off
• on
• once
• prependListener
• prependOnceListener
• rawListeners
• removeAllListeners
• removeListener
• retireAllBrowsers
• retireBrowserByPage
• retireBrowserController
• setMaxListeners

Constructors

constructor

• new BrowserPool<Options, BrowserPlugins, BrowserControllerReturn, LaunchContextReturn, PageOptions, PageReturn>(options): BrowserPool<Options, BrowserPlugins, BrowserControllerReturn, LaunchContextReturn, PageOptions, PageReturn>

• Overrides TypedEmitter<BrowserPoolEvents<BrowserControllerReturn, PageReturn>>.constructor

  Parameters

  • options: Options & BrowserPoolHooks<BrowserControllerReturn, LaunchContextReturn, PageReturn>

  Returns BrowserPool<Options, BrowserPlugins, BrowserControllerReturn, LaunchContextReturn, PageOptions, PageReturn>

Properties

activeBrowserControllers

activeBrowserControllers: Set<BrowserControllerReturn> = ...

browserPlugins

browserPlugins: BrowserPlugins

closeInactiveBrowserAfterMillis

closeInactiveBrowserAfterMillis: number

optionalfingerprintCache

fingerprintCache?: QuickLRU<string, BrowserFingerprintWithHeaders>

optionalfingerprintGenerator

fingerprintGenerator?: FingerprintGenerator

optionalfingerprintInjector

fingerprintInjector?: FingerprintInjector

fingerprintOptions

fingerprintOptions: FingerprintOptions

maxOpenBrowsers

maxOpenBrowsers: number

maxOpenPagesPerBrowser

maxOpenPagesPerBrowser: number

operationTimeoutMillis

operationTimeoutMillis: number

pageCounter

pageCounter: number = 0

pageIds

pageIds: WeakMap<PageReturn, string> = ...

pages

pages: Map<string, PageReturn> = ...

pageToBrowserController

pageToBrowserController: WeakMap<PageReturn, BrowserControllerReturn> = ...

postLaunchHooks

postLaunchHooks: PostLaunchHook<BrowserControllerReturn>[]

postPageCloseHooks

postPageCloseHooks: PostPageCloseHook<BrowserControllerReturn>[]

postPageCreateHooks

postPageCreateHooks: PostPageCreateHook<BrowserControllerReturn, PageReturn>[]

preLaunchHooks

preLaunchHooks: PreLaunchHook<LaunchContextReturn>[]

prePageCloseHooks

prePageCloseHooks: PrePageCloseHook<BrowserControllerReturn, PageReturn>[]

prePageCreateHooks

prePageCreateHooks: PrePageCreateHook<BrowserControllerReturn, PageOptions>[]

retireBrowserAfterPageCount

retireBrowserAfterPageCount: number

retiredBrowserControllers

retiredBrowserControllers: Set<BrowserControllerReturn> = ...

startingBrowserControllers

startingBrowserControllers: Set<BrowserControllerReturn> = ...

optionaluseFingerprints

useFingerprints?: boolean

staticexternalinheriteddefaultMaxListeners

defaultMaxListeners: number

Inherited from TypedEmitter.defaultMaxListeners

Methods

[asyncDispose]

• [asyncDispose](): Promise<void>

• Returns Promise<void>

externalinheritedaddListener

• addListener<U>(event, listener): this

• Inherited from TypedEmitter.addListener

  Parameters

  • externalevent: U

  • externallistener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]

  Returns this

closeAllBrowsers

• closeAllBrowsers(): Promise<void>

• Closes all managed browsers without waiting for pages to close.

  ---

  Returns Promise<void>

closePage

• closePage(page, options): Promise<void>

• Implementation of IBrowserPool.closePage

  Releases a page back to the pool. The page is closed and, if the optional error is a SessionError, the browser controller that served the page is retired so that its tainted state (cookies, storage, etc.) cannot leak into future sessions.

  This is the primary way the crawler should return pages to the pool.

  ---

  Parameters

  • page: PageReturn

    The page to release.

  • optionaloptions: { error?: Error }

  • optionalerror: Error

    The error that caused the page to be released, if any.

  Returns Promise<void>

destroy

• destroy(): Promise<void>

• Closes all managed browsers and tears down the pool.

  ---

  Returns Promise<void>

externalinheritedemit

• emit<U>(event, ...args): boolean

• Inherited from TypedEmitter.emit

  Parameters

  • externalevent: U

  • externalrest...args: Parameters<BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]>

  Returns boolean

externalinheritedeventNames

• eventNames<U>(): U[]

• Inherited from TypedEmitter.eventNames

  Returns U[]

extractPageState

• extractPageState(page): Promise<PageState>

• Implementation of IBrowserPool.extractPageState

  Extracts the relevant state (currently just cookies) from a page via its owning BrowserController. Returns empty state when the page is no longer associated with a controller.

  As with BrowserPool.injectPageState, cookies are isolated per page only when the pool is configured with useIncognitoPages: true. With the default useIncognitoPages: false, the extracted cookies include those set by any sibling page sharing the same browser.

  ---

  Parameters

  • page: PageReturn

  Returns Promise<PageState>

getBrowserControllerByPage

• getBrowserControllerByPage(page): undefined | BrowserControllerReturn

• Retrieves a BrowserController for a given page. This is useful when you're working only with pages and need to access the browser manipulation functionality.

  You could access the browser directly from the page, but that would circumvent BrowserPool and most likely cause weird things to happen, so please always use BrowserController to control your browsers. The function returns undefined if the browser is closed.

  ---

  Parameters

  • page: PageReturn

    Browser plugin page

  Returns undefined | BrowserControllerReturn

externalinheritedgetMaxListeners

• getMaxListeners(): number

• Inherited from TypedEmitter.getMaxListeners

  Returns number

getPage

• getPage(id): undefined | PageReturn

• If you provided a custom ID to one of your pages or saved the randomly generated one, you can use this function to retrieve the page. If the page is no longer open, the function will return undefined.

  ---

  Parameters

  • id: string

  Returns undefined | PageReturn

getPageId

• getPageId(page): undefined | string

• Page IDs are used throughout BrowserPool as a method of linking events. You can use a page ID to track the full lifecycle of the page. It is created even before a browser is launched and stays with the page until it's closed.

  ---

  Parameters

  • page: PageReturn

  Returns undefined | string

hasActiveBrowserWithFreeCapacity

• hasActiveBrowserWithFreeCapacity(): boolean

• Returns true if any active browser has room for another page.

  ---

  Returns boolean

hasFreeBrowserSlot

• hasFreeBrowserSlot(): boolean

• Returns true if the pool can accept a new browser launch without exceeding BrowserPoolOptions.maxOpenBrowsers. Counts starting, active, and retired browsers.

  ---

  Returns boolean

injectPageState

• injectPageState(page, state): Promise<void>

• Implementation of IBrowserPool.injectPageState

  Injects state into a page via its owning BrowserController.

  No-op when the page is no longer associated with a controller.

  Note that cookies are isolated per page only when the pool is configured with useIncognitoPages: true — each page then gets its own browser context. With the default useIncognitoPages: false, all pages in a browser share a single context, so injected cookies are visible to every page served by that browser.

  ---

  Parameters

  • page: PageReturn

  • state: PageState

  Returns Promise<void>

externalinheritedlistenerCount

• listenerCount(type): number

• Inherited from TypedEmitter.listenerCount

  Parameters

  • externaltype: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>

  Returns number

externalinheritedlisteners

• listeners<U>(type): BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U][]

• Inherited from TypedEmitter.listeners

  Parameters

  • externaltype: U

  Returns BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U][]

newPage

• newPage(options): Promise<PageReturn>

• Implementation of IBrowserPool.newPage

  Opens a new page in one of the running browsers or launches a new browser and opens a page there, if no browsers are active, or their page limits have been exceeded.

  Session injection (best-effort): When a session is provided, this implementation uses it as a cache key for browser fingerprints (when fingerprinting is enabled) and reads session.proxyInfo.url / session.proxyInfo.ignoreTlsErrors as defaults for proxyUrl and ignoreTlsErrors respectively. Explicit proxyUrl / ignoreTlsErrors values in the options take precedence.

  Beyond fingerprint caching and proxy configuration, no other session properties are consumed — cookie and header injection remain the crawler's responsibility.

  ---

  Parameters

  • options: BrowserPoolNewPageOptions<PageOptions, BrowserPlugins[number]> = {}

  Returns Promise<PageReturn>

newPageInNewBrowser

• newPageInNewBrowser(options): Promise<PageReturn>

• Unlike newPage, newPageInNewBrowser always launches a new browser to open the page in. Use the launchOptions option to configure the new browser.

  ---

  Parameters

  • options: BrowserPoolNewPageInNewBrowserOptions<PageOptions, BrowserPlugins[number]> = {}

  Returns Promise<PageReturn>

newPageWithEachPlugin

• newPageWithEachPlugin(optionsList): Promise<PageReturn[]>

• Opens new pages with all available plugins and returns an array of pages in the same order as the plugins were provided to BrowserPool. This is useful when you want to run a script in multiple environments at the same time, typically in testing or website analysis.

  Example:

  const browserPool = new BrowserPool({
      browserPlugins: [
          new PlaywrightPlugin(playwright.chromium),
          new PlaywrightPlugin(playwright.firefox),
          new PlaywrightPlugin(playwright.webkit),
      ]
  });
  
  const pages = await browserPool.newPageWithEachPlugin();
  const [chromiumPage, firefoxPage, webkitPage] = pages;

  ---

  Parameters

  • optionsList: Omit<BrowserPoolNewPageOptions<PageOptions, BrowserPlugins[number]>, browserPlugin>[] = []

  Returns Promise<PageReturn[]>

externalinheritedoff

• off<U>(event, listener): this

• Inherited from TypedEmitter.off

  Parameters

  • externalevent: U

  • externallistener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]

  Returns this

externalinheritedon

• on<U>(event, listener): this

• Inherited from TypedEmitter.on

  Parameters

  • externalevent: U

  • externallistener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]

  Returns this

externalinheritedonce

• once<U>(event, listener): this

• Inherited from TypedEmitter.once

  Parameters

  • externalevent: U

  • externallistener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]

  Returns this

externalinheritedprependListener

• prependListener<U>(event, listener): this

• Inherited from TypedEmitter.prependListener

  Parameters

  • externalevent: U

  • externallistener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]

  Returns this

externalinheritedprependOnceListener

• prependOnceListener<U>(event, listener): this

• Inherited from TypedEmitter.prependOnceListener

  Parameters

  • externalevent: U

  • externallistener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]

  Returns this

externalinheritedrawListeners

• rawListeners<U>(type): BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U][]

• Inherited from TypedEmitter.rawListeners

  Parameters

  • externaltype: U

  Returns BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U][]

externalinheritedremoveAllListeners

• removeAllListeners(event): this

• Inherited from TypedEmitter.removeAllListeners

  Parameters

  • externaloptionalevent: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>

  Returns this

externalinheritedremoveListener

• removeListener<U>(event, listener): this

• Inherited from TypedEmitter.removeListener

  Parameters

  • externalevent: U

  • externallistener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]

  Returns this

retireAllBrowsers

• retireAllBrowsers(): void

• Removes all active browsers from the pool. The browsers will be closed after all their pages are closed.

  ---

  Returns void

retireBrowserByPage

• retireBrowserByPage(page): void

• Removes a browser from the pool. It will be closed after all its pages are closed.

  ---

  Parameters

  • page: PageReturn

  Returns void

retireBrowserController

• retireBrowserController(browserController): void

• Removes a browser controller from the pool. The underlying browser will be closed after all its pages are closed.

  ---

  Parameters

  • browserController: BrowserControllerReturn

  Returns void

externalinheritedsetMaxListeners

• setMaxListeners(n): this

• Inherited from TypedEmitter.setMaxListeners

  Parameters

  • externaln: number

  Returns this