Batch API

The Similarweb Batch API is designed for handling large-scale data extraction tasks. Instead of making individual requests for each piece of data, you submit large-scale jobs that specify the data you need, and you receive the results when the job is complete. This approach is suitable for scenarios where you require extensive historical data, long-term trend analysis, or periodic data updates.

 

The Benefits of Batch API

• Efficient Data Retrieval: Retrieve large datasets of up to one million domains in a single request, reducing the overhead of multiple API calls. Reports can be sent directly to your own database systems including S3 and Snowflake.
• Scheduled Updates: Schedule batch jobs to run at specific times or intervals, fully automating data retrieval tasks.
• Historical Analysis: Access historical data of up to five years for in-depth trend analysis and long-term insights.
• Optimized for Large-Scale Needs: Ideal for applications that require extensive data sets, such as market research or historical trend analysis.

Who Has Access to Batch API?

Batch API is available only to paid Similarweb users and comes included with 3 plans:

• Business plan: A great option for companies looking for both platform and API access to support their growth. Comes with 25,000 monthly data credits that can be used to query the Batch API and our Standard (REST) API.
• Enterprise plan: Perfect for larger organizations looking to analyze multiple datasets at scale. Comes with 50,000 monthly data credits that can be used to query the Batch API and our Standard (REST) API.
• API-only plan: Our most affordable and flexible plan for API-first companies. Comes with 10,000 monthly data credits and can be upgraded to 50,000 and 100,000 depending on usage.

Batch API is also available as an add-on for clients who purchased the Team plan. Reach out to your Account Manager for more details.

 

Batch API Datasets

Here’s a high level overview of all the available datasets. For a deeper understanding and metric exploration, use the Data Exporter.

Website Intelligence Datasets

• Traffic and Engagement - Traffic metrics for any website, including visits, engagements, ranking and more. Metrics representing Total, Mobile and desktop.
• Website Traffic & Marketing Channels - Monitor website traffic sources and their impact on total site visits, by marketing channel. Channels include: direct, organic & paid search, referrals, mail, social media, display ads.
• Website Referrals - Incoming and outgoing traffic data from websites
• Website Audience - Audience and demographic analysis for cross-browser behavior of a website's visitors/audience
• Popular Pages - Discover the top pages and best performing content on a domain with page level data.
• Top Geos - Analyze the top countries driving traffic to websites on desktop over selected periods.
• Similar Sites - Identify top websites similar to a domain with detailed similarity scores.

Search Datasets

• Site Keywords - keyword that drive traffic to specific websites, broken down by total, paid and organic channels
• Keywords - deep dive into keywords by volume, CPC, difficulty and zero clicks

Technology Dataset

• Website Technographics - Offers comprehensive insights into more than 5,000 diverse technologies used across the web.

Company Datasets

• Company Firmographics - View in depth firmographics, size and performance data of businesses. 
• Website Details - Company and ecommerce meta description and analysis, including estimated online revenue and e-commerce type.

App Datasets

• App Engagement - View app usage, audience and ranking metrics to evaluate app performance and benchmark
• App Rankings - Returns the iOS / Android app rank from Google Play or Apple store for a certain app
• App Retention - Evaluate long-term user retention on Android to assess marketing and engagement strategies
• Website / App Cross Usage - Discover user overlap and cross-usage between mobile apps and websites. Available for android only.
•  

Ecommerce Datasets  Uncover consumer shopping behavior to optimize your brand performance

• Brands and categories - analyze selected brands or categories to understand top performers, trending products, and more.
• Top brands in category - discover the unique product views, revenue, units sold, and conversion rate for the top brands in the selected domain and category. Query results include up to 80 brands. 
• Top products in category - unearth the unique product views, revenue, and units sold by the selected brand in the selected domain and category.
• Top keywords in category - discover the unique clicks per keyword on products in a category, with repeat clicks in a short session counted as one. You can also find out organic click share, paid click share, the brand with the most keyword-driven clicks, and more.
• Top products in brands - the same as top products in category, but focusing on selected brands.
• Top keywords in brands - the same as top products in category, but focusing on selected brands.

Keyword analysis - find out the number of searches for a specific keyword, the organic and paid ratio for clicks from a keyword, and more.

 

Batch API Documentation

The Similarweb API Documentation offers a detailed walkthrough of how to set up the Batch API, its different capabilities (some of which are mentioned above), error code handling, and more.

 

Click here for documentation links that explore the Batch API capabilities.

 

Batch API Example

You can explore and build the body of the API call using the Data Exporter, just choose the dataset you need and add the domains, countries, granularity, dates and metrics. A more comprehensive guide can be found here.

Below is an example of a Batch API query using the Traffic and Engagement table. Here is a link to the API Recipe.

 

https://api.similarweb.com/batch/v4/request-report
curl --location 'https://api.similarweb.com/batch/v4/request-report' \
--header 'api-key: {{API_KEY}}' \
--header 'Content-Type: application/json' \
--data '{
  "delivery_information": {
    "response_format": "csv"
  },
  "report_query": {
    "tables": [
      {
        "vtable": "traffic_and_engagement",
        "granularity": "monthly",
        "filters": {
          "domains": [
            "similarweb.com"
          ],
          "countries": [
            "WW",
            "GB",
            "US"
          ],
          "include_subdomains": true
        },
        "metrics": [
          "desktop_share",
          "desktop_average_visit_duration",
          "country_rank",
          "mobile_share"
        ],
        "start_date": "2024-01",
        "end_date": "2024-03",
      }
    ]
  },
  "report_name": "Traffic Analysis"
}'