Skip to content

Tutorial

got-scraping and Cloudflare Turnstile: A Node.js Form Tutorial

Post a Cloudflare Turnstile form from Node.js with got-scraping: consistent browser headers, a cookie jar, a token from an API, and what replaces it now.

By 4 min readPublished Updated

got-scraping cannot run the Cloudflare Turnstile widget, because it is an HTTP client, not a browser. What it can do is fetch the form with browser-like headers, keep the site’s cookies, and post the form back with a token that a solving API produced from the widget’s sitekey. That is enough for most Turnstile-protected forms. This tutorial builds it, and ends with what to use now that got-scraping is deprecated.

Only automate sites you are allowed to. See responsible captcha automation.

got-scraping is end of life

Before you build on it: got-scraping’s README says the package is deprecated and “will no longer receive updates or support”, and recommends impit, a client with a fetch interface built on Rust’s reqwest. Existing crawlers keep working, and the pattern below carries over to impit unchanged: fetch the form, read the sitekey, get a token, post the form with the same session.

Set up

Terminal window
npm install got-scraping tough-cookie
export ZEROCAPTCHA_API=… # the API's base URL
export ZEROCAPTCHA_KEY=… # your API key

got-scraping is ESM only, so save the script as newsletter.mjs. It needs Node.js 20 or later for the built-in fetch used to call the API.

The script

import { gotScraping } from "got-scraping";
import { CookieJar } from "tough-cookie";
const API = process.env.ZEROCAPTCHA_API;
const KEY = process.env.ZEROCAPTCHA_KEY;
const PAGE = "https://shop.example.com/newsletter";
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
async function call(method, body, headers = {}) {
const response = await fetch(`${API}/${method}`, {
method: "POST",
headers: { "Content-Type": "application/json", ...headers },
body: JSON.stringify({ clientKey: KEY, ...body }),
signal: AbortSignal.timeout(15_000),
});
const reply = await response.json();
if (reply.errorId) throw new Error(`${method}: ${reply.errorCode}`);
return reply;
}
async function solveTurnstile(websiteURL, websiteKey, action, cdata) {
// The widget's data-action and data-cdata go in metadata, only when it sets them: many sites
// check both when they verify the token.
const task = { type: "TurnstileTaskProxyless", websiteURL, websiteKey, metadata: {} };
if (action) task.metadata.action = action;
if (cdata) task.metadata.cdata = cdata;
// One Idempotency-Key per task: a retried create with it returns the same task.
const { taskId } = await call("createTask", { task }, { "Idempotency-Key": crypto.randomUUID() });
const deadline = Date.now() + 180_000;
while (Date.now() < deadline) {
await sleep(2_000);
const result = await call("getTaskResult", { taskId });
if (result.status === "ready") return result.solution.token;
}
throw new Error(`task ${taskId}: no token within 180 seconds`);
}
// One visitor: the same cookies and the same generated headers on every request.
const session = { cookieJar: new CookieJar(), sessionToken: {} };
const page = await gotScraping({ url: PAGE, ...session });
const sitekey = page.body.match(/data-sitekey="([^"]+)"/)?.[1];
if (!sitekey) throw new Error("No Cloudflare Turnstile sitekey in the HTML");
const action = page.body.match(/data-action="([^"]+)"/)?.[1];
const cdata = page.body.match(/data-cdata="([^"]+)"/)?.[1];
const token = await solveTurnstile(PAGE, sitekey, action, cdata);
const submitted = await gotScraping({
url: PAGE,
method: "POST",
form: { email: "me@example.com", "cf-turnstile-response": token },
...session,
});
console.log(submitted.statusCode, submitted.url);

Run it with node newsletter.mjs. The form here posts back to its own URL; if yours has an action attribute, post to that URL instead.

What each piece does

  • sessionToken. got-scraping generates browser-like headers for each request. Headers made with the same sessionToken object never change, so the form page and the submission look like one browser, not two.
  • cookieJar. Whatever cookies the form page sets, such as a session ID or a CSRF cookie, go back with the submission. Many sites refuse a form whose session they did not issue.
  • form. got sends it as application/x-www-form-urlencoded, like a browser form, with the token under cf-turnstile-response, the name the widget itself uses.
  • The regular expressions are enough for a tutorial. For real pages, parse the HTML with cheerio and read $("[data-sitekey]").attr("data-sitekey"), which copes with attribute order and quoting.

When the sitekey is not in the HTML

If the page renders the widget from JavaScript with turnstile.render(), the sitekey may only exist in a script. Look in the page’s scripts for a string starting 0x4AAAA, as find a Cloudflare Turnstile sitekey explains, or use a browser crawler such as Crawlee’s PlaywrightCrawler.

What headers cannot do

Browser-like headers and TLS settings make a request look like a browser’s, which matters on sites that screen clients before any form is shown. They do not produce a Turnstile token: the site checks the token with Cloudflare when the form arrives, and a missing or made-up token fails. And if the page itself answers “Just a moment…”, that is a Cloudflare challenge page, which needs a cf_clearance cookie rather than a token: see Cloudflare challenge page vs Cloudflare Turnstile.

Keep the token fresh

A Cloudflare Turnstile token works once, for 300 seconds. Solve right before the submission, as the script does, and solve again if the site rejects the form for any reason. Cloudflare Turnstile token expiry has the details.

Sources

The team that builds and runs the ZeroCaptcha API. Articles are drafted with AI tools, then checked against the API's code and the primary sources each one cites.

Questions

Is got-scraping still maintained?

No. Its README says the package is deprecated and will no longer receive updates or support, and recommends impit, a fetch-style client, for new projects.

Do browser-like headers get past Cloudflare Turnstile?

No. Headers help a request look like a browser, but a Turnstile-protected form still checks a token on the server. The token has to come from a widget that passed, or from a solving API.

Why use a cookie jar for a Turnstile form?

Many sites tie the form to the session that loaded it. Sending the form page's cookies back with the submission makes it look like the same visitor.

Read next

This article is part of the Cloudflare Turnstile solver hub. Every task is charged only when a token is ready.

Get an API key