Tavily Crawl
tavily_crawlFull Description
Crawl a website starting from a URL. Extracts content from pages with configurable depth and breadth.
Parameters (1 required, 10 optional)
urlstringThe root URL to begin the crawl
allow_externalbooleanWhether to return external links in the final response
Trueextract_depthstringAdvanced extraction retrieves more data, including tables and embedded content, with higher success but may increase latency
basicadvancedbasicformatstringThe format of the extracted web page content. markdown returns content in markdown format. text returns plain text and may increase latency.
markdowntextmarkdowninclude_faviconbooleanWhether to include the favicon URL for each result
FalseinstructionsstringNatural language instructions for the crawler. Instructions specify which types of pages the crawler should return.
limitintegerTotal number of links the crawler will process before stopping
50max_breadthintegerMax number of links to follow per level of the tree (i.e., per page)
20max_depthintegerMax depth of the crawl. Defines how far from the base URL the crawler can explore.
1select_domainsarrayRegex patterns to restrict crawling to specific domains or subdomains (e.g., ^docs\.example\.com$)
[]select_pathsarrayRegex patterns to select only URLs with specific path patterns (e.g., /docs/.*, /api/v1.*)
[]