Web Scraping Layers for AI & Automation

back to blog

How to run a competitor SEO audit with Ahrefs, Moz, Similarweb, and Ubersuggest data

ParseBird·14 Oct 2026

Key Takeaways

Why use four SEO data sources instead of one? Each tool measures something different. Ahrefs is strongest on backlinks and Domain Rating, Moz on Domain Authority and spam score, Similarweb on total traffic and channel mix, and Ubersuggest on keyword volume, CPC, and difficulty. Combining them gives a fuller competitor picture than any single free tool.

Do these scrapers need paid Ahrefs, Moz, Similarweb, or Ubersuggest accounts? No. They read the data each vendor shows in its free tools and public lists, such as Ahrefs' free website checkers, Moz's free Domain Analysis, and Similarweb's public website pages.

Can I measure a competitor's visibility in AI search? Yes. The Ahrefs Scraper can return AI visibility, AI Overview, and AI Mode data with citation counts by model, and the Similarweb Scraper returns AI chatbot referral traffic, top chatbots, and top prompts where available.

Are failed lookups charged? No. In all four actors, a domain or keyword with no data returns a row with an error field and is not charged.

What each SEO data source measures

A competitor SEO audit answers four questions: how authoritative is each site, how much traffic does it get and from where, which keywords and pages drive it, and who links to it. Each of the four vendors answers a different part.

QuestionBest sourceKey fields
How strong is the link profile?Ahrefs Scraperdomain_rating, backlinks, refdomains, top_backlinks
What is the Domain Authority and spam risk?MOZ Scraperdomain_authority, page_authority, spam_score, linking_root_domains
How much traffic, and from which channels?Similarweb ScrapertotalVisits, searchOrganicTraffic, socialTraffic, referralTraffic
Which keywords are worth targeting?Ubersuggest Scrapervolume, cpc, seoDifficulty, searchIntent

Domain Rating (Ahrefs) and Domain Authority (Moz) are both 0 to 100 link-strength scores, but they use different link indexes and formulas. Report them side by side rather than treating them as interchangeable.

Which is more accurate, Ahrefs Domain Rating or Moz Domain Authority? Neither is a Google metric, so neither is "accurate" in an absolute sense. They are relative scores for comparing sites. Use one consistently for ranking competitors and the other as a cross-check. Big disagreements between them are often worth a manual look.

Start with a list of competitor domains and pull authority from both sources.

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
competitors = ["make.com", "n8n.io", "zapier.com", "pipedream.com"]

ahrefs = client.actor("parsebird/ahrefs-scraper").call(run_input={
    "urls": competitors,
    "include_web_authority": True,
    "include_traffic": True,
    "include_backlinks": True,
    "include_ai_visibility": True,
})

moz = client.actor("parsebird/moz-scraper").call(run_input={
    "urls": competitors,
    "include_overview": True,
    "include_authority": True,
})

The Ahrefs Scraper returns one row per lookup type per domain, with type set to values such as authority, traffic, backlinks, or ai_visibility. The Moz Scraper returns an overview row (Domain Authority, homepage Page Authority, spam score, linking root domains, ranking keywords) and, with include_authority, a domain_authority row with top pages, top linking domains, and 60 days of discovered and lost linking domains.

What does Moz spam score mean? Spam score is the percentage of sites with similar features that Moz has found to be penalized or banned by Google. A high spam score on a competitor, or on a site linking to you, is a flag to review its links, not proof of a penalty.

Step 2: Traffic and channel mix with the Similarweb Scraper

Authority shows potential. Traffic shows results. The Similarweb Scraper returns estimated visits and the channel split for each domain:

similarweb = client.actor("parsebird/similarweb-scraper").call(run_input={
    "urls": competitors,
    "include_base_data": True,
    "include_similar_sites": True,
    "output_mode": "aggregated",
})
FieldWhat it tells you
totalVisits, monthlyVisitsOverall traffic scale and trend
searchOrganicTraffic, searchPaidTrafficShare of traffic from SEO and from paid search
socialTraffic, referralTraffic, mailTrafficShare from non-search channels
genAiTraffic, aiTopChatbots, aiChatbotShareShare from ChatGPT and other AI assistants, and which ones
topKeywordsSearch terms sending traffic
similar_sitesCompetitors you may have missed

The channel fields are shares between 0 and 1. For example, searchOrganicTraffic: 0.21 means about 21% of visits come from organic search. Multiply by totalVisits for an estimated visit count.

similar_sites is useful at the start of an audit. It often surfaces competitors that are not on your original list.

Can I see a competitor's traffic from ChatGPT and other AI chatbots? Where Similarweb has the data, yes. genAiTraffic, aiTrafficVisits, aiTopChatbots, aiChatbotShare, and aiTopPrompts show how much traffic comes from AI assistants and which ones. Combined with Ahrefs AI visibility data (total_ai_citations and citations_by_model), this shows whether a competitor is being cited and visited from AI search.

Step 3: Keyword metrics with the Ubersuggest Scraper

Once you know which competitors matter, look at the keywords. The Ubersuggest Scraper returns keyword metrics, SERPs, top pages, and backlink overviews in one run:

ubersuggest = client.actor("parsebird/ubersuggest-scraper").call(run_input={
    "keywords": ["workflow automation", "zapier alternative", "no code automation"],
    "domainsOrUrls": competitors,
    "country": "us",
    "language": "en",
    "includeSerp": True,
    "maxSerpResults": 10,
    "includeTopPages": True,
})

Each row has a rowType: keyword_metric (volume, CPC, competition, SEO difficulty, search intent, monthly trend), serp_result (position, URL, domain, estimated clicks, domain authority), top_page (a competitor's top pages by traffic), or backlinks_overview. Ubersuggest covers 238 countries, which helps for multi-market audits.

Joining the four SEO datasets into one competitor table

All four actors key rows by domain, so the join is straightforward:

function competitorTable(domains, { ahrefs, moz, similarweb }) {
  return domains.map((domain) => {
    const a = ahrefs.find((r) => r.domain === domain && r.type === "authority") ?? {};
    const m = moz.find((r) => r.domain === domain && r.type === "overview") ?? {};
    const s = similarweb.find((r) => r.domain === domain) ?? {};
    return {
      domain,
      ahrefsDR: a.domain_rating,
      mozDA: m.domain_authority,
      spamScore: m.spam_score,
      monthlyVisits: s.totalVisits,
      organicShare: s.searchOrganicTraffic,
      estOrganicVisits: s.totalVisits && s.searchOrganicTraffic ? Math.round(s.totalVisits * s.searchOrganicTraffic) : null,
      aiShare: s.genAiTraffic,
    };
  });
}

The result is one row per competitor with authority, traffic, and channel mix side by side. Add Ubersuggest's top_page rows to see which pages drive each competitor's traffic. To keep it updated, schedule the runs monthly. The competitor monitoring guide covers scheduling and alerts.

SEO scraper pricing

ActorApproximate price per 1,000 results (free plan, at time of writing)
Ahrefs Scraper$8.99
MOZ Scraper$6.99
Similarweb Scraper$6.99, plus $1.99 per 1,000 in-depth lookups
Ubersuggest Scraper$1.09

Paid Apify plans get lower rates, and lookups that return no data are not charged. See each actor page for current pricing.

FAQ

Is this the same data as a paid Ahrefs or Semrush subscription? No. The scrapers return what the vendors show in their free tools and public lists. That covers authority, traffic estimates, top pages, top keywords, and sample backlinks, which is enough for competitor benchmarking but not a full backlink export.

Can I check thousands of domains at once? Yes. All four actors take arrays of domains or URLs. For large lists, split them into several runs so a single slow domain does not hold up the batch.

Can I get the top websites in a category? Yes. The Ahrefs Scraper can return AhrefsTop rankings by category and country (up to 1,000 sites), and the MOZ Scraper can return the Moz Top 500.

Is it legal to scrape Ahrefs, Moz, Similarweb, and Ubersuggest? The actors read data the vendors display publicly in free tools. Check each vendor's terms of service before reselling or redistributing the data.

Browse the SEO Tools category for more search data tools.