Skip to main content
Version: 3.3

enqueueLinks

Callable

  • This function enqueues the urls provided to the RequestQueue provided. If you want to automatically find and enqueue links, you should use the context-aware enqueueLinks function provided on the crawler contexts.

    Optionally, the function allows you to filter the target links' URLs using an array of globs or regular expressions and override settings of the enqueued Request objects.

    Example usage

    await enqueueLinks({
      urls: aListOfFoundUrls,
      requestQueue,
      selector: 'a.product-detail',
      globs: [
          'https://www.example.com/handbags/*',
          'https://www.example.com/purses/*'
      ],
    });

    Parameters

    • options: { baseUrl: undefined | string; exclude: undefined | (GlobInput | RegExpInput)[]; forefront: undefined | boolean; globs: undefined | GlobInput[]; label: undefined | string; limit: undefined | number; pseudoUrls: undefined | PseudoUrlInput[]; regexps: undefined | RegExpInput[]; requestQueue: RequestQueue; selector: undefined | string; strategy: undefined | EnqueueStrategy | all | same-domain | same-hostname | same-origin; transformRequestFunction: undefined | RequestTransform; urls: string[]; userData: undefined | Dictionary<any> }

      All enqueueLinks() parameters are passed via an options object.

      • baseUrl: undefined | string
      • exclude: undefined | (GlobInput | RegExpInput)[]
      • forefront: undefined | boolean
      • globs: undefined | GlobInput[]
      • label: undefined | string
      • limit: undefined | number
      • pseudoUrls: undefined | PseudoUrlInput[]
      • regexps: undefined | RegExpInput[]
      • requestQueue: RequestQueue

        A request queue to which the URLs will be enqueued.

      • selector: undefined | string
      • strategy: undefined | EnqueueStrategy | all | same-domain | same-hostname | same-origin
      • transformRequestFunction: undefined | RequestTransform
      • urls: string[]

        An array of URLs to enqueue.

      • userData: undefined | Dictionary<any>

    Returns Promise<BatchAddRequestsResult>

    Promise that resolves to BatchAddRequestsResult object.