How to Use SimilarWeb Batch API

 

  

Overview

 Introduction to the SimilarWeb Batch API Beta

With the SimilarWeb Batch API, you can easily extract valuable digital traffic data at scale for any analysis purpose. Our new and enhanced API has additional capabilities incorporated to retrieve data reports in the most efficient manner possible.

You can now group your requests so your code runs faster and more efficiently. The Batch API allows you to make requests for up to 1M domains for multiple countries and 19 traffic and engagement metrics (find them detailed below).

API Request Cycle

Similar_API_final.png

Step 1. Make a POST request

To retrieve a SimilarWeb data report, make a POST request with an attached JSON file that delineates the desired domains, countries, metrics, start and end date and granularity for the report.

The POST request will return a unique report ID that you can use to make a GET request to get updates on the report completion, and once the report is ready, it will return a link to download the report.

To request a data report, call the following endpoint which will return a report ID:

Method:
POST https://api.similarweb.com/v3/batch/websites/request-report/{{your_api_key}}

For the payload, use a valid JSON file with the following structure:

curl -X POST \

https://api.similarweb.com/websites/request-report/{{your_api_key}}
 \

  -H 'content-type: multipart/form-data; \

  -F request=@{{/path/to/json/file}}.json

Your JSON file must include the following parameters:

 

Parameter

Required

Description

domains

X

Characters in domain name can include letters, numbers, dashes and hyphens. One request can include up to 1M domains.

countries

X

Countries with standard 2-letter ISO encoding.

metrics

X

See list of metrics supported below.

start & end date

X

For daily granularity, format the start-and-end date like this: YYYY-MM-DD.


For monthly granularity, format the start-and-end date like this- YYYY-MM.

granularity

X

Valid values are:
‘daily’, ‘monthly’.

Include_subdomains

 

Boolean, valid values ‘true’, ‘false’.
If you’d like to omit data from any linked subdomains, please set to false. If omitted, default is set to true and returns data for main- and subdomains.

 

Your JSON file should be structured like this:

 {

   domains": [

      "example.com”,

       "example.com”,

      "example.com”,

       ],

   "countries": ["US", DE”, “AU”],

   "metrics": [

       "all_traffic_visits",

       "all_traffic_pages_per_visit",

       "all_traffic_average_visit_duration",

       "all_traffic_bounce_rate",

       "desktop_visits",

       "desktop_pages_per_visit",

       "desktop_average_visit_duration",

       "desktop_bounce_rate",

       "mobile_visits",

       "mobile_pages_per_visit",

       "mobile_bounce_rate",

       "mobile_average_visit_duration"

   ],

   "start_date": "2019-07-01",

   "end_date": "2019-07-10",

  "granularity": "daily"

}

 

You will receive a response with your report ID, which is valid for up to 30 days.
Response Example:

{

"report_id":"unique_report_id"

}

 

Step 2. Make a GET Request

Once you’ve received your report ID, make a GET request to retrieve the download link to your report.

Use the following endpoint to poll the status of your request. Once the request is completed you will receive a unique link that will allow you to download the report. Links to generated reports are valid for 30 days.

Method:
GET

https://api.similarweb.com/v3/batch/websites/request-status/{{your_api_key}}/{{generated_report_id}}

Response Structure

  • Status: Pending/Validating/Bad Request/Processing/Internal/Completed
  • Error: In case of bad request, we will include error code and description
  • Notations: If there are any system exceptions or additional information, our systems will note them here.
  • Download_URL: When status returns “completed”, the response will include a link to the report.
  • Used_quota: when status returns “completed”, the response will include how many data points the generated report consumed.

Response Handling

Status

 

Response

Description

Pending

Request received and queued for processing.

Validating

Request is being validated. Validation includes data access, syntax and requested data validity.

Bad Request

The request has failed due to a validation or data access error. Based on the error codes and notations, amend the request before retrying.

Processing

The request has been validated and our service is retrieving the data to build your report.

Internal Error

Our service has encountered an unexpected condition that prevents it from fulfilling the request. Please retry and if the issue persists, contact support.

Completed

The request has been successfully processed and has produced a response. See download_URL field to download your report.

Download_url

Link to results

  

Example Requests:

Completed Request

{

 “download_url”: “https://bulk-api-reports.com/juniqueJobID&Signature&amz-security-token&Expires=****”,

 “status”: “completed”,

 “used_quota”: “8520.0"

}


Bad Request Example 


{

 “errors”: “Request failed due to invalid metrics. Please refer to the SimilarWeb API documentation for metrics supported via bulk. Invalid metrics: all_traffic_visitsadfad”,

 “status”: “bad_request”

}

Supported Metrics

 

Metric Name

Description

Supported Granularity*

Type

Description

all_traffic_visits

Total Traffic  Visits (Desktop + Mobile)

Monthly, Weekly, Daily

Integer

Returns estimated number of visits for the domain.

all_traffic_pages_per_visit

Total Traffic Pages/Visits

Monthly, Weekly, Daily

Integer

Returns the average page views per visit for the given domain.

all_traffic_average_visit_duration

Total Traffic Average Visit Duration

Monthly, Weekly, Daily

Integer

Returns the average visit duration on the given domain (in seconds).

all_traffic_bounce_rate

Total Traffic Bounce Rate

Monthly, Weekly, Daily

Integer

Returns the bounce rate for the given domain.

desktop_vs_mobile_split

Desktop vs Mobile Split

Monthly, Weekly, Daily

Integer

Returns the desktop vs mobile web traffic share split

desktop_visits

Desktop Visits

Monthly, Weekly, Daily

Integer

Returns estimated number of desktop visits for the domain.

desktop_unique_visitors

Desktop Unique Visitors

Monthly

Integer

Returns monthly unique desktop visitors

 desktop_pages_per_visit

Desktop Pages per Visit

Monthly, Weekly, Daily

Integer

Returns the average page views per desktop visit for the given domain.

desktop_average_visit_duration

Desktop Average Visit Duration

Monthly, Weekly, Daily

Integer

Returns the average desktop visit duration on the given domain (in seconds)

desktop_bounce_rate

Desktop Bounce Rate

Monthly, Weekly, Daily

Integer

Returns the desktop bounce rate for the given domain.

mobile_visits

Mobile Web Visits

Monthly, Weekly, Daily

Integer

Returns estimated number of mobile web visits for the domain.

 mobile_unique_visitors

Mobile Web Unique Visitors

Monthly, Weekly, Daily

Integer

Returns monthly unique mobile web visitors

mobile_pages_per_visit

Mobile Web Pages/Visit

Monthly, Weekly, Daily

Integer

Returns the average page views per mobile web visit for the given domain.

mobile_average_visit_duration

Average Visit Duration (Mobile Web Only)

Monthly, Weekly, Daily

Integer

Returns the average mobile web visit duration on the given domain (in seconds).

mobile_bounce_rate

Bounce Rate (Mobile Web Only)

Monthly, Weekly, Daily

Integer

Returns the mobile web bounce rate for the given domain.

global_rank

Global Rank

Monthly

Integer

Returns SimilarWeb's monthly Global Rank for a given domain

 

country_rank

Country Rank

Monthly

Integer

Returns SimilarWeb's monthly Country Rank for a given domain

category_rank

Category Rank

Monthly

Integer

Returns the Category of a given domain and its Global Rank within its given category.

 

Error Responses

 

Error Type

Response Type

Response Message

Quota has been reached (for part of the request)

Bad Request

Your request exceeds the data points in your subscription. Please remove X domains/ reduce the request by X or contact your account manager

If domains are missing from the request

Bad Request

Request failed due to a missing parameter. Please specify the domains you're interested in.

If country is missing from the request

Bad Request

Request failed due to a missing parameter. Please specify the countries you're interested in.

If countries are requested that are not in the user's subscription

Bad Request

Request failed due to subscription error. You do not have access to these countries specified in the request: %Country%

If metrics are missing from the request

Bad Request

Request failed due to a missing parameter. Please specify the metrics you\’re interested in.

Not found- If metrics requested are not supported

Bad Request

Request failed due to invalid metrics. Please refer to the SimilarWeb API documentation (link) for metrics supported via bulk.

If timeframe is missing from the request and granularity isn't set

Bad Request

Start and end date set to last month by default due to no input.

Unauthorized

Bad Request

API key provided is not valid.

Request Failed

Bad Request

The parameters were valid but the request failed. Please try again.

Connection error

Bad Request

The request failed to make a connection to the SimilarWeb API. Please try again.


Utilities 

Historical Data Endpoint

Use this GET method to retrieve start and end dates for each metric:

For daily granularity: https://api.similarweb.com/v3/batch/describe/daily

For weekly granularity: https://api.similarweb.com/v3/batch/describe/weekly

For monthly granularity: https://api.similarweb.com/v3/batch/describe/monthly

For quota usage

Use this GET method to keep track of how many credits you have left:  

https://api.similarweb.com/v3/batch/credits/{{api_key}}

 

 

 

 

 




Was this article helpful?
0 out of 0 found this helpful