Search request

Forming a search request entails crafting a search string, and adding a parameter object to control the results.

Search request formats

The API accepts multiple different search request formats. They have slightly different form, depending on the use case.

{
  "searchterm": "spotify",
  "params": {
    "requestedarticles": 20
  }
}

{
  "expressions": [
    {
      "linemode": "R",
      "searchline": "spotify"
    }
  ],
  "params": {
    "requestedarticles": 20
  }
}

{
  "expressions": [
    {
      "linemode": "R",
      "searchline": {
        "searchterm": "header:spotify",
        "filters": [
          {
            "type": "lang",
            "id": "no"
          },
          {
            "type": "url",
            "id": "vg.no"
          }
        ]
      }
    }
  ],
  "params": {
    "requestedarticles": 10
  }
}

Search syntax

Here we describe the search syntax.

Although most of the examples is phrased in terms of how to match something, for instance a particular word or phrase, you can just as easily negate the term, and thereby exclude content, simply by prefixing the term with the ANDNOT operator.

The section on boolean expressions contains a more thorough list of options on how to take the basic building blocks and construct more complex searches.

The response contains information about any syntax errors.

Basic terms and phrases

spotify

Search for spotify somewhere in all the text, i.e. either in the header, the summary, the body, or in any of the captions.
music spotify

Search for articles containing spotify somewhere and music somewhere.
Spotify

Searching for a term with an upper case letter will match words with the same casing. Using only lower case letters matches any casing. While spotify will match both Spotify and spotify, Spotify only matches Spotify.
ANDNOT spotify

Search for articles not mentioning spotify.
'Michael Jordan'

Search for Michael followed immediately by Jordan, i.e. the phrase 'Michael Jordan'.
"Michael Jordan"

You can also use double quotes to denote phrases.
'Jeanne d\'Arc'

If the phrase contains a quote, the quote must be escaped, otherwise 'Jeanne d' would be interpreted as the phrase. Adding \ before the quote means the ' character will be interpreted as being part of the phrase.
"Jeanne d'Arc"

Alternatively, if there are only single quotes in the phrase, you can use double quotes around it. In this case escaping is not needed. Conversely if there are double quotes in the phrase, you can use single quotes around it.

Searching in specific sections

By default all content is matched against, but this can be changed by prefixing the term or phrase by the name of a section:

header:spotify

Search for spotify in the header.
summary:spotify

Search for spotify in the summary.
body:spotify

Search for spotify in the body text.
caption:spotify

Search for spotify in any of the captions.
author:johnson

Search for johnson in authors' names.
domain:bbc

Search for articles containing bbc in its domain name.
url:bbc.co.uk/news

Search for articles containing bbc.co.uk/news in its url.

Exact, prefix and wildcard matching

The more advanced search string functionality have some limitations. There is no support for prefix wildcards, and there must be at least three characters before any * or ?. Phrases and restrictions, however, as detailed above, can be used freely.

domain:=bbc.co.uk

Search for the exact phrase, in this case articles under exactly bbc.co.uk, not our-bbc.co.uk.
spot*

Search for words beginning with spot.
harbo*r

Search for words beginning with harbo and ending with r. There can be zero or more characters in between.
visuali?ation

Search for words with any character at the position of the question mark. This will match both visualization and visualisation.
header:^London

Search headers beginning with London.

Proximity

There are several ways to match based on terms occurring together in the text:

'Trump Biden'~5

Search for Trump and Biden with at most 5 words between them.
SPAN/5(Trump Pence, Biden Obama)

Search for either Trump or Pence with either Biden or Obama at most 5 words apart. This is a generalized version of the proximity search, ~.

The order of the words does not matter, Obama can come before Trump in the text, or it could be the other way around.
OSPAN/5(Trump Pence, Biden Obama)

An ordered version of SPAN, where Trump or Pence must occur before Biden or Obama in the text.
SPAN/5(Trump Pence, Biden Obama, election)

A three-argument span search. Search for either Trump or Pence with either Biden or Obama and the word election, all with at most 5 words between any of the terms.

Frequency

Matches can also be done based on how often a term occurs:

London[3..]

Match articles where the word London occurs at least 3 times.
London[..7]

Match articles where the word London occurs no more than 7 times.
London[3..7]

Match articles where the word London occurs at least 3 times, but no more than 7 times.

Ranks

Sites are given both a global and country based rank of their importance as a source of news, where the most important site is given a ranking of 1, the second most important a ranking of 2, and so on.

globalrank<200

Match only articles among sites ranked within the top 200 globally.
globalrank>200

Match only articles among sites ranked outside of the top 200 globally.
countryrank<50

Match only articles among sites ranked within the top 50 in their country.
countryrank>50

Match only articles among sites ranked outside of the top 50 in their country.

Language and subjects

It is possible to filter on other attributes of the article other than their text content, such as the langauge it is written in and metadata set on the site level, for instance the country of the publisher.

Except for the language codes, these filters applies to what has historically been named subjects. A subject is a piece of metadata, there is for instance a subject for the location Norway and another subject for the media type of print. The subjects are divided into categories, like geo for locations, containing for instance Norway and Sweden. Sites, and subsequently articles, are then assigned subjects and those are searchable.

To take a concrete example, 1203 is the id corresponding to the subject of Norway, thus searching for geo:1203 will only return articles on sites where the publisher is from Norway, for instance VG, which is a Norwegian site.

Since the numeric ids are not for humans to parse, the filters are usually obtained through interacting with the suggestion server, which along with the ids provide names of the subjects in the users language for display.

lang:en

Filter on a language code.
geo:1372

Filter on the geographical area represented by the id.
site:318

Filter on a site.
media:575

Filter on the media type represented by the id.
content:501

Filter on the content provider type represented by the id.
rank:11

Filter on the rank category represented by the id.

A newer, more fine-grained alternative is to use the countryrank and globalrank operators. The rank subjects can be considered deprecated.
subject:3333

Filter on a generic subject id, irrespective of the category.

Topics and entities

ent:1000291644

Filter on entity represented by id.
topic:111000000

Filter on topic represented by id.
crunchbase:123456

Filter on entities using their Crunchbase ID.
permid:123456

Filter on entities using their PermID.
lei:12345678901234567890

Filter on legal entities using their LEI (Legal Entity Identifier).

Sites, sources and articles

Each article is published on a site, for instance in a local newspaper or on a website. Each of these sites have an associated id. The article also has an id, but this id is only unique for the site it was published on.

If you want to fetch specific articles, those articles must therefore be accessed using both a site id and an article id. This combination is unique.

site:11

Match on a site. Id can be fetched from the suggestion server.
site:11,14,110

Match on several sites. Ids can be fetched from the suggestion server.
ANDNOT site:11

Exclude a site. Id can be fetched from the suggestion server.
article:11-123

Match on a specific article under the site. The site id comes first, then the article id. In this case 11 is the site id, and 123 is the article id.
article:123

Match on an article id. Combine with site id to target unique articles.
article:2,12,45,65

Match on any of the article ids. Articles having an id in the list will be matched. Combine with site id to target many unique articles.

If searching only based on ids, it is more efficient to use the articles param.
id:11_123

Match on a specific article under the site. The site id comes first, then the article id. In this case 11 is the site id, and 123 is the article id.

If searching only based on ids, it is more efficient to use the articles param.

In addition to sites, there are also sources. A source is a page where links to articles are harvested. One site can have multiple sources. A news website can have a front page and a separate page with sports content. Source ids are globally unique.

source:233

Match a source.
firstsource:8758

Match based on the first source the article was observed on.

Profiles

Searches that are stored as profiles can be used as a building block in the query.

profile:123343

Expands to search stored in profile with id 123343.
profile:123343:1680159851

Expands to search stored in a previous version of profile 123343, where the second number is the version timestamp, that can be found using the profile history endpoint in the API.
profile:123343 ANDNOT profile:123343:1680159851

Return articles matching the current version of the profile but not the older version of it.
profile:123343:1680159851 ANDNOT profile:123343

Conversely, return articles matching the old version of the profile but not the new version. This way, you can see hits that you have filtered away.

Collections

Under construction.

Misc

charcount<2000

Match only articles where the number of letters in the header, summary and body text combined amounts to less than 2000.
charcount>500

Match only articles where the number of letters in the header, summary and body text combined, exceeds 500.

Boolean expressions

Use boolean operators to build complex expressions.

football AND england

Both terms must match.
football england

By default, listing multiple terms is equivalent to an AND operator. All the terms must match.
(url:telegraph.co.uk OR url:bbc.co.uk)

Matches either term. It is usually a good idea to be explicit with parentheses and operator precedence.
football ANDNOT tottenham
football -tottenham

A shorter form of the negation operator is available. Use a dash before the term.
ANDNOT site:11

The ANDNOT operator can also be used on its own.

Parameters

If you want to add more fields to the response, tweak the default response, restrict the search to a given time interval, or do similar things, the params is the place to look.

link main object

Settings related to the text and matches of each article.

In order return text from the articles, either main.header, main.summary or main.body must be set to a non-zero value. All are off by default.
- link main.header int, 0, 1 or 2
  
  Include a header field in the response with the articles headers if the value is non-zero. If main.matches is set, and the value is 2, phrases matching the search string will be enclosed in <match> tags.
- link main.summary int, 0, 1 or 2
  
  Include a summary field in the response with the articles summaries if the value is non-zero. If main.matches is set, and the value is 2, phrases matching the search string will be enclosed in <match> tags.
- link main.text int, 0, 1 or 2
  
  Include a body field in the response with the articles body text if the value is non-zero. If main.matches is set, and the value is 2, phrases matching the search string will be enclosed in <match> tags.
- link main.matches boolean, default false
  
  Include matches and matches_table fields in the response, with information about phrases matching the search string.
- link main.quotes int
  
  Include a quotes field in the response, containing text surrounding each of the mathcing phrases. If main.matches is set, and the value is 2, phrases matching the search string will be enclosed in <match> tags.
- link main.quotesfrom int
  
  The sections to include quotes from.
- link main.shortheaderlength int
  
  Include a short_header field in the response, with a shortened version of the header, of at most main.shortheaderlength characters.
- link main.shortsummarylength int
  
  Include a short_summary field in the response, with a shortened version of the summary, of at most main.shortsummarylength characters.
- link main.shortbodylength int
  
  Include a short_body field in the response, with a shortened version of the body text, of at most main.shortbodylength characters.
- link main.shortfirstsrcname int
- link main.othersources boolean
  
  Include a sources field in the response with information about all the sources the article was observed on.
- link main.shortothersrcname int
- link main.colorbar boolean
- link main.all boolean
- link main.inherit boolean
link do_not_move_to_summary boolean, default false

Do not move part of the body text to the summary field.

link readership boolean

error_outline Requires license

Include similarweb.readership and similarweb.article_readership fields in the response, with readership estimates.
link allsubject string

Add a topics field to the response, with information about the articles subjects. Passing 0 returns all of the associated subjects. Passing a string with comma separated ids will only return subjects that is also in the list.
link textrazor int

error_outline Requires license
Add data about topics and entities to the response. The parameter is interpreted as a four digit binary number, each digit toggling the inclusion or exclusion of data.

Setting any of the first three digits adds a topics_and_entities field to each document in the response. Each bit sets what data to include:
1. Topics
2. Entities
3. Entities without a wikidata id
Setting the fourth bit adds entity ids to match tags, using ent attributes, <match ent="1,23...">. Since these tags are activated by main.matches, you should set that as well. It also adds two dictionaries, wikinames and wikidescriptions, with human readable names and descriptions of the entities.

Some useful values:
- Setting it to 7 will include all the available data, but no highlighting information.
- Setting it 8 will include only highlighting information.
- Setting it to 15 will include everything.
link allmeta boolean

Add a meta_data field to the response.

link articles list of objects

Only return matching articles with particular ids. The number of results is upper bounded by the requestedarticles parameter, not the length of this list.
- link articles[].id_site int
- link articles[].id_article int
link select_by_ids boolean

Discard the search string and fetch articles based only on the ids listed in articles. This parameter improves efficiency when getting specific articles based only on ids, as the search string is of no interest.
link excludearticles list of objects

List of articles to explicitly exclude from the results.
- link excludearticles[].id_site int
- link excludearticles[].id_article int

link sort_oldest_first boolean, default false

Instead of returning articles sorted from newest to oldest, return the oldest articles first.

link groupidentical boolean, default false

Group identical articles. If you enable this you will only receive one article from the group of identicals in your response. If you also enable include_ogae_identical you will also receive the identical articles in an array in identical_documents.
link identical object

Content to include for the identical documents. Adds an identical_documents field to the response with identical documents. The format is the same as for main, but with an additional identical.inherit option, which copies all of the settings, meaning the same fields are returned for the identical articles as those for the main article.
- link identical.inherit boolean
  
  Use the same settings as main.
link identical_sorting list of objects

Sort articles in group. It is specified as a list of object with field to sort by and in which order. Articles are first sorted by the first item in the list, then the second item, etc.
- link identical_sorting[].field int
  Field to sort on.
  1. Paywall
  2. Global rank
- link identical_sorting[].order int
  How to order the results. Defaults to ascending order.
  1. Ascending order
  2. Descending order
link include_equalgroup_id int

Include the articles OGAE group id in an equalgroup field in the response.
link include_ogae_identical string
link strict_time_interval boolean

When including identical articles, remove any articles not published within the time interval given by the oldest and newest parameters.
link max_similar_articles int, default 250

Maximum number of similar articles to return. If the number of similar articles exceeds the limit, the resulting list is capped.

link mark_matches_url int

link quotewishlength int, default 50
link max_article_length int

Number of characters to truncate the summary and text fields in the response to. The limit is on the sum of the two fields. If the summary field exists, the truncation is first applied to it. If the length of the summary field is less than max_article_length, the remaining characters are fetched from the body text.

link external_article_provider int

Include providers organization specific ids in the search results.
link track_id_site int

Get a tracking articles matching articles. Used in conjunction with track_id_article.
link track_id_article int