SDK

Install

npm i @scrapa/sdk

Read a page

import { Scrapa } from "@scrapa/sdk";

const scrapa = new Scrapa(process.env.SCRAPA_API_KEY!);

const page = await scrapa.scrape({
  url: "https://example.com",
  formats: ["markdown", "links"],
});

console.log(page.engine, page.markdown);

What your editor autocompletes is what the server accepts: the parameter types come from the same schemas that validate the request.

What it cost

const page = await scrapa.scrape(
  { url: "https://example.com" },
  { onUsage: ({ cost, creditsRemaining }) => log(cost, creditsRemaining) },
);

// …or before you spend anything:
const { cost, maxCost } = await scrapa.quote({ url: "https://example.com" });

A method hands you the page itself, so the meter arrives through a callback, once per call, rather than a mutable field two concurrent scrapes would fight over.

When something says no

import { ScrapaError } from "@scrapa/sdk";

try {
  await scrapa.scrape({ url });
} catch (error) {
  if (!(error instanceof ScrapaError)) throw error;
  switch (error.outcome) {
    case "blocked":
      // time ran out, nobody said no: give the page more time
      if (error.code === "NAV_TIMEOUT") return scrapa.scrape({ url, timeoutMs: 60_000 });
      // the site said no: try another pool, country or engine
      return scrapa.scrape({ url, engine: "browser" });
    case "denied": // our policy said no, so retrying cannot help
      return null;
    case "broken": // ours, already refunded: worth one retry
      return scrapa.scrape({ url });
  }
  throw error; // a bad field, a missing key, a spent quota
}

outcome tells you what to do next: blocked, denied or broken. One code breaks the rule: NAV_TIMEOUT comes as blocked, but nobody said no. Time ran out, so raise timeoutMs instead of trying another engine or country. cost is on the error too, and it is zero on all three, unless your own browser.actions script ran in the page first: then blocked and denied keep the browser price and its bandwidth. Broken is always refunded.

What it retries

Only what a repeat can fix. A 429, 408 or 425 means the call never ran, so the client sends it again, and waits as long as the answer asks when that is 10 seconds or less. A read, a quote or a cancel is also sent again after no answer or a 5xx. A scrape, a search, a new batch and a new session are not: one that timed out on your side may already have read the page, and a second try would be charged for the same page twice. You get the error and decide.