API Docs

Technical guide for developers

Build applications using Minifetch

Overview

Minifetch is an SEO audit tool that also gives you the underlying data as a portable API you can use with your AI agents and automation workflows. Every fetch is built around a core principle: ethical data extraction that identifies itself transparently, respects robots.txt, and never hammers target servers. It's why sites that block aggressive scrapers often work just fine with Minifetch.

The Minifetch API Client has a Quick Start for Javascript or TypeScript if you're looking to add it to your project quickly.

This documentation covers the the main endpoints available for developers. It also covers details about accessing the Minifetch API as well as example data. All API endpoints return responses in JSON format.

Features
Use Cases

Quick Start

Minifetch is pay-per-fetch. No setup fee, no monthly fee, no charges for blocked or errored pages. Prices per fetch are listed on each API endpoint below. Pick your payment method:

1. Credit Card

Sign up on the dashboard to get free starter credits, then top up with your credit card.

2. USDC on Base or Solana

Pay directly from your crypto wallet via the x402 protocol. No account needed.

Now click whichever access method fits your setup:

Browser

Visit the Minifetch homepage or hit any API endpoint URL directly. Free to start if you signed up, then top up with credit card after. You can also use your browser plugin crypto wallet. The default on the homepage is metadata extraction, with the option to extract links from pages as well. Great for trying out the service. If you want full API access, try the other options below.

Minifetch API Client NEW

The minifetch-api client is an NPM package purpose-built for Javascript or TypeScript. The simplest way to integrate Minifetch into a Node.js app. Works with both API keys and USDC. Recommended for agent builders.

Curl + API Key

Sign up, create an API key from the dashboard, and query endpoints from the CLI: 

curl "https://minifetch.com/api/v1/extract/url-preview?url=https://anthropic.com" \
            -H "Authorization: Bearer [your_api_key]"
Runtime Agent Access

For agents that need to pay at runtime without human sign-up. If you're building an agent yourself, the API Client above is usually the better starting point.

Coinbase Payments MCP gives AI assistants like Claude Desktop a built-in wallet to discover and pay for x402 services with no code. One command, email sign-in, no private key. 

npx @coinbase/payments-mcp

The installer configures your MCP client (Claude Desktop, Claude Code, Gemini CLI, etc). Then ask your assistant to use the Coinbase Payments MCP to discover Minifetch endpoints on the x402 Bazaar.

Agentic Wallets (awal) is Coinbase's wallet infrastructure for autonomous agents: non-custodial, keys isolated, with programmable spending limits and gasless USDC trades on Base. Recommended for programmatic and agent-to-agent usage. 

npx awal

Once funded, your agent can call Minifetch with any x402-compatible client (e.g. @x402/fetch) or skip the client entirely by adding awal's pay-for-service skill. 

npx skills add coinbase/agentic-wallet-skills

API Base URL

The Minifetch API client will handle this for you. All API requests will be made to:

https://minifetch.com

Endpoint: URL Check (Free)

GET/api/v1/free/preflight/url-check
Description

This free endpoint checks if a URL is allowed to be crawled according to the website's robots.txt file. Use this before making paid requests to the /extract/ endpoints to avoid spending credits on uncrawlable URLs.

Use Case

Call this endpoint before the /extract/ endpoints.

Price

Free

Request Parameters:
Parameter Type Required Description
url string Yes The target URL to check against robots.txt
Example Request:
GET https://minifetch.com/api/v1/free/preflight/url-check?url=https://example.com
Example Response Data:
{
  "success": true,
  "results": [
    {
      "data": {
        "url": "https://example.com",
        "allowed": true,
        "message": "robots.txt not found, defaulting to allowed",
        "crawlDelay": 1
      }
    }
  ]
}
Status Codes:

Endpoint: Extract URL Metadata (Paid)

GET/api/v1/extract/url-metadata
GET/api/v1/x402/extract/url-metadata
Description

This endpoint fetches and extracts rich structured metadata from a URL: title, description, meta tags, open graph tags, twitter tags, hreflang, json-ld, citations, images, headings, response headers, and more. The structured output doubles as an instant SEO audit — every field maps directly to a ranking signal or competitive intelligence data point.

Use Cases

SEO Research, Content Indexing, Ad Enhancement, Building AI Training Datasets

Price

$0.002

Request Parameters:
Parameter Type Required Description
url string Yes The target URL from which to extract metadata
verbosity string No Defaults to "standard". Set to "full" for larger response.
includeResponseBody boolean No If set to true, includes the full HTML response body as string in result.
Example Request:
GET https://minifetch.com/api/v1/extract/url-metadata?url=https://example.com&includeResponseBody=true
Example Response Data:
Status Codes:

Endpoint: Extract URL Preview (Paid)

GET/api/v1/extract/url-preview
GET/api/v1/x402/extract/url-preview
Description

This endpoint fetches and extracts a light, token-efficient preview of a URL: title, description, and image (only).

Use Cases

AI Agents & RAG Pipelines

Price

$0.001

Request Parameters:
Parameter Type Required Description
url string Yes The target URL from which to extract the preview
Example Request:
GET https://minifetch.com/api/v1/extract/url-preview?url=https://github.com
Example Response Data:
Status Codes:

Endpoint: Extract URL Content (Paid)

GET/api/v1/extract/url-content
GET/api/v1/x402/extract/url-content
Description

Extracts a clean, LLM-ready, token-efficient content summary as markdown from a URL. Removes ads, nav, scripts. Much more efficient than raw HTML fetches.

Use Cases

AI Agents & RAG Pipelines

Price

$0.002

Request Parameters:
Parameter Type Required Description
url string Yes The target URL from which to extract content
includeMediaUrls boolean No If set to "true", includes image and video URLs in the result
Example Request:
GET https://minifetch.com/api/v1/extract/url-content?url=https://apnews.com/article/time-person-of-year-2025-77ec65c6792bc99ec2ce1919c5f421ea&includeMediaUrls=true
Example Response Data:
Status Codes:

Libraries & SDKs

Minifetch.com is the hosted version of a free and open-source npm package called url-metadata that has been serving the NPM community for 10+ years. During that time some users expressed a preference for a hosted service for various reasons, mostly technical hurdles with coding or accidentally triggering blocks from websites when using the open-source package.

For direct integration with your codebase, consider:

minifetch-apiNEW — the official Minifetch API client for Javascript/ Typescript

npm install minifetch-api --save

url-metadata — the open-source npm package (10+ years, 3.6M downloads) that powers Minifetch

npm install url-metadata --save

Both from the same creator with 3.6 Million+ downloads over 10 years.
Add them to your project with confidence.

🤖 Agent Skills (SKILL.md)

Task-specific step-by-step guides for AI agents using the Minifetch API:

View all skills →

Caching & Rate Limiting

Minifetch implements several measures to ensure responsible and efficient operation:

These measures help us maintain an ethical scraping approach while providing reliable service.

Service Limitations

Minifetch only extracts publicly available metadata and content from pages accessible without authentication and Javascript execution.

What Minifetch does NOT do:

What Minifetch does NOT do currently but may offer in the future as an add-on:

Contact

If you have questions or need assistance:
join our Discord server.

To leave feedback or join the bulk credits waitlist:
use our feedback form.

While you're at it, give us a follow on X.com (Twitter):
@minifetch