# Web crawler API for AI agents

Scrollport gives AI agents 2 tools to crawl a website, from Firecrawl, through one connection. Prices are per page or request, depending on the tool, paid per use from one prepaid wallet with no subscription, provider accounts or API keys. Agents use them over MCP, the CLI or the HTTP API.

Updated 15 September 2026

## Research and extract web data with Scrollport

[Get started](https://scrollport.com/start)

Use verified tools with pay-per-use pricing. Provider access is included. Tools that work in your own apps ask you to connect that account.

Connect your first agent to receive $1 trial credit.

## Compare Web crawler API tools

Prices are per page or request, depending on the tool, so compare what each tool returns and a typical run's cost before choosing.

| Tool | Provider | What it returns | Price |
| --- | --- | --- | --- |
| [Firecrawl Crawl](https://scrollport.com/tools/providers/firecrawl/site-crawl) | Firecrawl | Read a bounded set of pages from a public website. | $0.0025/page |
| [Firecrawl Map](https://scrollport.com/tools/providers/firecrawl/site-map) | Firecrawl | Discover page URLs across a public website. | $0.0025/request |

## 2 available tools

Crawl a website

$0.0025/page

### Firecrawl Crawl

Read a bounded set of pages from a public website.

Firecrawl

Crawl a website

$0.0025/request

### Firecrawl Map

Discover page URLs across a public website.

Firecrawl

## Questions about Web crawler API

### What is the difference between a website crawler API and a web scraping API?

A web scraping API fetches and extracts one page you name. A website crawler API follows links within one site to collect many pages, within the limits you set. Scrollport lists single-page tools on its web scraping API page.

### How much do web crawler API tools cost?

Prices are per page or request, depending on the tool. You pay per use from one prepaid wallet with no subscription. Your agent sees the exact price before each run, and connecting your first agent adds $1 of trial credit.

### What is the best web crawler API for an AI agent?

It depends on what you need back and how much you will run it. Firecrawl Crawl (Firecrawl, $0.0025/page): Read a bounded set of pages from a public website. Firecrawl Map (Firecrawl, $0.0025/request): Discover page URLs across a public website. Your agent can inspect the closest match and see its exact price before it runs.

## Guide to web crawler API

Collect content from a bounded set of pages within a public website. Use the result as one defined step in a wider agent workflow workflow.

### Define the request

Tell your agent the outcome, audience, source material and constraints that define a useful result. It can inspect the available tools and ask for anything essential that is missing.

### Compare the tools

The current catalog supplies this job through Firecrawl. Inspect the tool before running because availability and accepted fields remain contract-specific.

### Check the result

Ask your agent to retain source context, explain important gaps and check that the result answers the original request before using it in the wider workflow. Retain source links and verify coverage, returned fields and freshness.

### Common questions

**What should I check before using web crawler API tools?**

Collect content from a bounded set of pages within a public website. The current catalog supplies this job through Firecrawl. Inspect the tool before running because availability and accepted fields remain contract-specific. Ask your agent to retain source context, explain important gaps and check that the result answers the original request before using it in the wider workflow.

**How do I use web crawler API tools with Scrollport?**

Give the setup prompt to your agent. Scrollport includes provider access, adds $1 in trial credit when you connect your first agent, and shows the live price before every run.

---

Canonical page: https://scrollport.com/tools/web-crawler-api