BrowserPool <Options, BrowserPlugins, BrowserControllerReturn, LaunchContextReturn, PageOptions, PageReturn>

The BrowserPool class is the most important class of the browser-pool module. It manages opening and closing of browsers and their pages and its constructor options allow easy configuration of the browsers' and pages' lifecycle.

The most important and useful constructor options are the various lifecycle hooks. Those allow you to sequentially call a list of (asynchronous) functions at each stage of the browser / page lifecycle.

Example:

import { BrowserPool, PlaywrightPlugin } from '@crawlee/browser-pool';
import playwright from 'playwright';

const browserPool = new BrowserPool({
    browserPlugins: [new PlaywrightPlugin(playwright.chromium)],
    preLaunchHooks: [(pageId, launchContext) => {
        // do something before a browser gets launched
        launchContext.launchOptions.headless = false;
    }],
    postLaunchHooks: [(pageId, browserController) => {
        // manipulate the browser right after launch
        console.dir(browserController.browser.contexts());
    }],
    prePageCreateHooks: [(pageId, browserController) => {
        if (pageId === 'my-page') {
            // make changes right before a specific page is created
        }
    }],
    postPageCreateHooks: [async (page, browserController) => {
        // update some or all new pages
        await page.evaluate(() => {
            // now all pages will have 'foo'
            window.foo = 'bar'
        })
    }],
    prePageCloseHooks: [async (page, browserController) => {
        // collect information just before a page closes
        await page.screenshot();
    }],
    postPageCloseHooks: [(pageId, browserController) => {
        // clean up or log after a job is done
        console.log('Page closed: ', pageId)
    }]
});

Hierarchy

• TypedEmitter<BrowserPoolEvents<BrowserControllerReturn, PageReturn>>
  • BrowserPool

Index

Constructors

• constructor

Properties

• activeBrowserControllers
• browserPlugins
• closeInactiveBrowserAfterMillis
• fingerprintCache
• fingerprintGenerator
• fingerprintInjector
• fingerprintOptions
• maxOpenPagesPerBrowser
• operationTimeoutMillis
• pageCounter
• pageIds
• pageToBrowserController
• pages
• postLaunchHooks
• postPageCloseHooks
• postPageCreateHooks
• preLaunchHooks
• prePageCloseHooks
• prePageCreateHooks
• retireBrowserAfterPageCount
• retiredBrowserControllers
• useFingerprints
• defaultMaxListeners

Methods

• addListener
• closeAllBrowsers
• destroy
• emit
• eventNames
• getBrowserControllerByPage
• getMaxListeners
• getPage
• getPageId
• listenerCount
• listeners
• newPage
• newPageInNewBrowser
• newPageWithEachPlugin
• off
• on
• once
• prependListener
• prependOnceListener
• rawListeners
• removeAllListeners
• removeListener
• retireAllBrowsers
• retireBrowserByPage
• retireBrowserController
• setMaxListeners

Constructors

constructor

• new BrowserPool<Options, BrowserPlugins, BrowserControllerReturn, LaunchContextReturn, PageOptions, PageReturn>(options: Options & BrowserPoolHooks<BrowserControllerReturn, LaunchContextReturn, PageReturn>): BrowserPool<Options, BrowserPlugins, BrowserControllerReturn, LaunchContextReturn, PageOptions, PageReturn>

• Overrides TypedEmitter<BrowserPoolEvents<BrowserControllerReturn, PageReturn>>.constructor

  Type parameters

  • Options: BrowserPoolOptions<BrowserPlugin<CommonLibrary, undefined | Dictionary, CommonBrowser, unknown, CommonPage>> = BrowserPoolOptions<BrowserPlugin<CommonLibrary, undefined | Dictionary, CommonBrowser, unknown, CommonPage>>
  • BrowserPlugins: BrowserPlugin<CommonLibrary, undefined | Dictionary, CommonBrowser, unknown, CommonPage>[] = InferBrowserPluginArray<Options[browserPlugins]>
  • BrowserControllerReturn: BrowserController<CommonLibrary, undefined | Dictionary, CommonBrowser, unknown, CommonPage> = ReturnType<BrowserPlugins[number][createController]>
  • LaunchContextReturn: LaunchContext<CommonLibrary, undefined | Dictionary, CommonBrowser, unknown, CommonPage> = ReturnType<BrowserPlugins[number][createLaunchContext]>
  • PageOptions = Parameters<BrowserControllerReturn[newPage]>[0]
  • PageReturn: CommonPage = UnwrapPromise<ReturnType<BrowserControllerReturn[newPage]>>

  Parameters

  • options: Options & BrowserPoolHooks<BrowserControllerReturn, LaunchContextReturn, PageReturn>

  Returns BrowserPool<Options, BrowserPlugins, BrowserControllerReturn, LaunchContextReturn, PageOptions, PageReturn>

Properties

activeBrowserControllers

activeBrowserControllers: Set<BrowserControllerReturn> = ...

browserPlugins

browserPlugins: BrowserPlugins

closeInactiveBrowserAfterMillis

closeInactiveBrowserAfterMillis: number

optionalfingerprintCache

fingerprintCache?: QuickLRU<string, BrowserFingerprintWithHeaders>

optionalfingerprintGenerator

fingerprintGenerator?: FingerprintGenerator

optionalfingerprintInjector

fingerprintInjector?: FingerprintInjector

fingerprintOptions

fingerprintOptions: FingerprintOptions

maxOpenPagesPerBrowser

maxOpenPagesPerBrowser: number

operationTimeoutMillis

operationTimeoutMillis: number

pageCounter

pageCounter: number = 0

pageIds

pageIds: WeakMap<PageReturn, string> = ...

pageToBrowserController

pageToBrowserController: WeakMap<PageReturn, BrowserControllerReturn> = ...

pages

pages: Map<string, PageReturn> = ...

postLaunchHooks

postLaunchHooks: PostLaunchHook<BrowserControllerReturn>[]

postPageCloseHooks

postPageCloseHooks: PostPageCloseHook<BrowserControllerReturn>[]

postPageCreateHooks

postPageCreateHooks: PostPageCreateHook<BrowserControllerReturn, PageReturn>[]

preLaunchHooks

preLaunchHooks: PreLaunchHook<LaunchContextReturn>[]

prePageCloseHooks

prePageCloseHooks: PrePageCloseHook<BrowserControllerReturn, PageReturn>[]

prePageCreateHooks

prePageCreateHooks: PrePageCreateHook<BrowserControllerReturn, PageOptions>[]

retireBrowserAfterPageCount

retireBrowserAfterPageCount: number

retiredBrowserControllers

retiredBrowserControllers: Set<BrowserControllerReturn> = ...

optionaluseFingerprints

useFingerprints?: boolean

staticexternaldefaultMaxListeners

defaultMaxListeners: number

Inherited from TypedEmitter.defaultMaxListeners

Methods

externaladdListener

• addListener<U>(event: U, listener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]): this

• Inherited from TypedEmitter.addListener

  Type parameters

  • U: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>

  Parameters

  • externalevent: U

  • externallistener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]

  Returns this

closeAllBrowsers

• closeAllBrowsers(): Promise<void>

• Closes all managed browsers without waiting for pages to close.

  ---

  Returns Promise<void>

destroy

• destroy(): Promise<void>

• Closes all managed browsers and tears down the pool.

  ---

  Returns Promise<void>

externalemit

• emit<U>(event: U, ...args: Parameters<BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]>): boolean

• Inherited from TypedEmitter.emit

  Type parameters

  • U: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>

  Parameters

  • externalevent: U

  • externalrest...args: Parameters<BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]>

  Returns boolean

externaleventNames

• eventNames<U>(): U[]

• Inherited from TypedEmitter.eventNames

  Type parameters

  • U: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>

  Returns U[]

getBrowserControllerByPage

• getBrowserControllerByPage(page: PageReturn): undefined | BrowserControllerReturn

• Retrieves a BrowserController for a given page. This is useful when you're working only with pages and need to access the browser manipulation functionality.

  You could access the browser directly from the page, but that would circumvent BrowserPool and most likely cause weird things to happen, so please always use BrowserController to control your browsers. The function returns undefined if the browser is closed.

  ---

  Parameters

  • page: PageReturn

    Browser plugin page

  Returns undefined | BrowserControllerReturn

externalgetMaxListeners

• getMaxListeners(): number

• Inherited from TypedEmitter.getMaxListeners

  Returns number

getPage

• getPage(id: string): undefined | PageReturn

• If you provided a custom ID to one of your pages or saved the randomly generated one, you can use this function to retrieve the page. If the page is no longer open, the function will return undefined.

  ---

  Parameters

  • id: string

  Returns undefined | PageReturn

getPageId

• getPageId(page: PageReturn): undefined | string

• Page IDs are used throughout BrowserPool as a method of linking events. You can use a page ID to track the full lifecycle of the page. It is created even before a browser is launched and stays with the page until it's closed.

  ---

  Parameters

  • page: PageReturn

  Returns undefined | string

externallistenerCount

• listenerCount(type: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>): number

• Inherited from TypedEmitter.listenerCount

  Parameters

  • externaltype: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>

  Returns number

externallisteners

• listeners<U>(type: U): BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U][]

• Inherited from TypedEmitter.listeners

  Type parameters

  • U: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>

  Parameters

  • externaltype: U

  Returns BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U][]

newPage

• newPage(options?: BrowserPoolNewPageOptions<PageOptions, BrowserPlugins[number]>): Promise<PageReturn>

• Opens a new page in one of the running browsers or launches a new browser and opens a page there, if no browsers are active, or their page limits have been exceeded.

  ---

  Parameters

  • options: BrowserPoolNewPageOptions<PageOptions, BrowserPlugins[number]> = {}

  Returns Promise<PageReturn>

newPageInNewBrowser

• newPageInNewBrowser(options?: BrowserPoolNewPageInNewBrowserOptions<PageOptions, BrowserPlugins[number]>): Promise<PageReturn>

• Unlike newPage, newPageInNewBrowser always launches a new browser to open the page in. Use the launchOptions option to configure the new browser.

  ---

  Parameters

  • options: BrowserPoolNewPageInNewBrowserOptions<PageOptions, BrowserPlugins[number]> = {}

  Returns Promise<PageReturn>

newPageWithEachPlugin

• newPageWithEachPlugin(optionsList?: Omit<BrowserPoolNewPageOptions<PageOptions, BrowserPlugins[number]>, browserPlugin>[]): Promise<PageReturn[]>

• Opens new pages with all available plugins and returns an array of pages in the same order as the plugins were provided to BrowserPool. This is useful when you want to run a script in multiple environments at the same time, typically in testing or website analysis.

  Example:

  const browserPool = new BrowserPool({
      browserPlugins: [
          new PlaywrightPlugin(playwright.chromium),
          new PlaywrightPlugin(playwright.firefox),
          new PlaywrightPlugin(playwright.webkit),
      ]
  });
  
  const pages = await browserPool.newPageWithEachPlugin();
  const [chromiumPage, firefoxPage, webkitPage] = pages;

  ---

  Parameters

  • optionsList: Omit<BrowserPoolNewPageOptions<PageOptions, BrowserPlugins[number]>, browserPlugin>[] = []

  Returns Promise<PageReturn[]>

externaloff

• off<U>(event: U, listener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]): this

• Inherited from TypedEmitter.off

  Type parameters

  • U: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>

  Parameters

  • externalevent: U

  • externallistener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]

  Returns this

externalon

• on<U>(event: U, listener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]): this

• Inherited from TypedEmitter.on

  Type parameters

  • U: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>

  Parameters

  • externalevent: U

  • externallistener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]

  Returns this

externalonce

• once<U>(event: U, listener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]): this

• Inherited from TypedEmitter.once

  Type parameters

  • U: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>

  Parameters

  • externalevent: U

  • externallistener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]

  Returns this

externalprependListener

• prependListener<U>(event: U, listener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]): this

• Inherited from TypedEmitter.prependListener

  Type parameters

  • U: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>

  Parameters

  • externalevent: U

  • externallistener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]

  Returns this

externalprependOnceListener

• prependOnceListener<U>(event: U, listener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]): this

• Inherited from TypedEmitter.prependOnceListener

  Type parameters

  • U: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>

  Parameters

  • externalevent: U

  • externallistener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]

  Returns this

externalrawListeners

• rawListeners<U>(type: U): BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U][]

• Inherited from TypedEmitter.rawListeners

  Type parameters

  • U: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>

  Parameters

  • externaltype: U

  Returns BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U][]

externalremoveAllListeners

• removeAllListeners(event?: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>): this

• Inherited from TypedEmitter.removeAllListeners

  Parameters

  • externaloptionalevent: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>

  Returns this

externalremoveListener

• removeListener<U>(event: U, listener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]): this

• Inherited from TypedEmitter.removeListener

  Type parameters

  • U: keyof BrowserPoolEvents<BrowserControllerReturn, PageReturn>

  Parameters

  • externalevent: U

  • externallistener: BrowserPoolEvents<BrowserControllerReturn, PageReturn>[U]

  Returns this

retireAllBrowsers

• retireAllBrowsers(): void

• Removes all active browsers from the pool. The browsers will be closed after all their pages are closed.

  ---

  Returns void

retireBrowserByPage

• retireBrowserByPage(page: PageReturn): void

• Removes a browser from the pool. It will be closed after all its pages are closed.

  ---

  Parameters

  • page: PageReturn

  Returns void

retireBrowserController

• retireBrowserController(browserController: BrowserControllerReturn): void

• Removes a browser controller from the pool. The underlying browser will be closed after all its pages are closed.

  ---

  Parameters

  • browserController: BrowserControllerReturn

  Returns void

externalsetMaxListeners

• setMaxListeners(n: number): this

• Inherited from TypedEmitter.setMaxListeners

  Parameters

  • externaln: number

  Returns this