Skip to main content
Some pages need JavaScript to show anything. arakun fetch and arakun crawl load them in Cloudflare Browser Run, a real browser on Cloudflare’s network, and give back Markdown an agent can read. They work without a running Arakun.

Set up a token

1

Create a Cloudflare API token

Give it the Browser Run - Edit permission. The cf sign-in doesn’t carry it.
2

Store it with its account

The token is read from stdin and kept in the system keyring.
Or set CLOUDFLARE_BROWSER_RUN_TOKEN and CLOUDFLARE_ACCOUNT_ID instead.

Fetch a page

Crawl a site

This prints the site’s pages as Markdown, one after another, as they are read.
The most pages to read. The default is 20.
How many links deep to go from the first page.
Run each page’s JavaScript before reading it.
Read only pages whose address matches. Repeat it for more patterns.
Skip pages whose address matches. Repeat it for more patterns.
A crawl gives up after 30 minutes.

Limits

  • What you can fetch depends on your Workers plan. On a rate limit, Arakun waits and tries again a few times.
  • It does not get past captchas.
  • arakun crawl honors robots.txt and Content Signals.