API Documentation

Driftnet's Internet Scan data contains reports gathered by visiting open internet services using their IP and port.

Internet scan data is returned by the scan/protocols endpoint. The simplest way to search internet scan data is by IP address.

curl -s -H 'Authorization: Bearer <your-api-token>' \
     'https://api.driftnet.io/v1/scan/protocols?ip=8.8.8.8' \
  | jq . \
  | less -S

{
  "page": 0,
  "pages": 22,
  "result_count": 2174,
  "results": [
    {
      "date": "2019-05-13",
      "id": "hJgzWy25TfuNfhvgMzs8Tw",
      "items": [
        {
          "context": "",
          "is_metadata": true,
          "type": "ip",
          "value": "8.8.8.8"
        },
        {
          "context": "",
          "is_metadata": true,
          "type": "port-tcp",
          "value": "443"
        },
        ...

In this example, there are 2174 results for IP address 8.8.8.8. The API returns results in batches of 100, so there are 22 pages in total. The newest results are returned first. To retrieve another page of results, use the page= parameter. Page numbering starts at zero.

Each result takes the form of a report. A report has a date stamp, a unique ID, and a collection of items.

Each item contains:

• type: The type of data being displayed, e.g. ip or host.
• context: The context in which the value was seen, e.g. cert-dns-name for an ip or host seen inside an X.509 certificate.
• value: The actual data value.
• is_metadata: true if the data came from inside the collection system (e.g. an enrichment), false if it was collected from the external environment.

The ip parameter can take a CIDR range. Setting ip=8.8.8.0/24 would have searched the entire /24 address range, i.e. all addresses from 8.8.8.0 to 8.8.8.255 inclusive.

By default, a search by IP will only match cases where the IP being searched is the one that was scanned. You might also want to look for a type of ip in any context, i.e. including results with some other reference (usually an X.509 certificate) to the IP address searched for. To get these results, add the indirect=true qualifier:

curl -s -H 'Authorization: Bearer <your-api-token>' \
     'https://api.driftnet.io/v1/scan/protocols?ip=8.8.8.8&indirect=true' \
  | jq . \
  | less -S

{
  "page": 0,
  "pages": 90,
  "result_count": 8922,
  "results": [
    {
      "date": "2019-05-13",
      "id": "AR1NAVUQSMuy7NMQweNPgw",
      "items": [
        {
          "context": "",
          "is_metadata": true,
          "type": "ip",
          "value": "45.160.122.135"
        },
        {
          "context": "",
          "is_metadata": true,
          "type": "port-tcp",
          "value": "853"
        },
        ...

With the indirect=true parameter set we include results which present a certificate that is valid for IP 8.8.8.8, but which are not directly hosted on IP 8.8.8.8.

Driftnet supports much more complicated searches. For this, you can use expression= queries.

Fundamentals

The basis of an expression search is a type and a value separated by a colon:

For example, to find all results with a server-banner containing cherrypy, you can call

curl -s -H 'Authorization: Bearer <your-api-token>' \
      --get --data-urlencode 'expression=server-banner:cherrypy' \
     'https://api.driftnet.io/v1/scan/protocols' \
  | jq . \
  | less -S

You can search for almost any type field that you discover in the UI. Some other commonly-searched type fields include:

• http-header: To search within returned HTTP headers.
• host: To search for a hostname, seen anywhere. Hostname matches are right-anchored, so expression=host:example.com will match foo.bar.example.com, etc. As a special case, to search for a host field within a URL, also set host_in_url=true.
• title: To search for the HTML title from a surveyed page.
• issuer and subject: TLS certificate issuer/subject fields.
• url: To search for a URL, seen anywhere. URL searches also support host queries, so expression=url:example.com will match https://foo.bar.example.com/abc/def, etc.

To get a degree of sloppy matching, use the slop= parameter. For instance, setting slop=1 would allow a query for university london to match university of london.

If you don't know which type field to look in, you can use the search-term type:

curl -s -H 'Authorization: Bearer <your-api-token>' \
      --get --data-urlencode 'expression=search-term:cherrypy' \
     'https://api.driftnet.io/v1/scan/protocols' \
  | jq . \
  | less -S

Using the search-term type is slow compared to searching with a stricter type, so try not to use this approach routinely.

Quoting Values

You should use double quotes when your value contains multiple words / spaces for instance to search for the entity Tencent Cloud Computing you would use expression=entity:"Tencent Cloud Computing",

curl -s -H 'Authorization: Bearer <your-api-token>' \
      --get --data-urlencode 'expression=entity:"Tencent Cloud Computing"' \
     'https://api.driftnet.io/v1/scan/protocols' \
  | jq . \
  | less -S

Combining Terms

You can combine <type>:<value> terms using the operators and, or and not. These operators precede a <type>:<value> term as shown in the following examples:

• To search a host AND port together:
  host:driftnet.io and port-tcp:443
• To search for one product tag OR another:
  product-tag:dahua or product-tag:hikvision
• To search for an entity but exclude two ports:
  entity:"Tencent Cloud Computing" not port-tcp:22 and not port-tcp:2222

curl -s -H 'Authorization: Bearer <your-api-token>' \
      --get --data-urlencode 'expression=entity:"Tencent Cloud Computing" not port-tcp:22 and not port-tcp:2222' \
     'https://api.driftnet.io/v1/scan/protocols' \
  | jq . \
  | less -S

Value Modifier Operators

For textual fields, <type>:<value> terms will match words inside the text. In some situations you will want to have more control over how the value is interpreted, and there are two operators you can use in these situations: asterisk (*) for prefix searching and equals (=) for keyword searching.

Ending a value in an asterisk will use the value as a prefix. Replacing the colon between the type and value with an equals will only return results where the full value matches exactly. For example:

• To search for hikvision as a prefix inside the product-tag field:product-tag:hikvision*
• To search for an exact complete product-tag:product-tag="hikvision dvr ds-7208hwi-sh"

curl -s -H 'Authorization: Bearer <your-api-token>' \
      --get --data-urlencode 'expression=product-tag="hikvision dvr ds-7208hwi-sh"' \
     'https://api.driftnet.io/v1/scan/protocols' \
  | jq . \
  | less -S

Special Keywords

There are two special keyword “types” in the API. Firstly,protocol, which can be used to search for the context of the server-banner type. This context actually contains the particular protocol that Driftnet identified.

curl -s -H 'Authorization: Bearer <your-api-token>' \
      --get --data-urlencode 'expression=protocol=modbus' \
     'https://api.driftnet.io/v1/scan/protocols' \
  | jq . \
  | less -S

Secondly, http-header-name can be used to search for the context of the http-header type, i.e. for a particular HTTP header name.

curl -s -H 'Authorization: Bearer <your-api-token>' \
      --get --data-urlencode 'expression=http-header-name=X-Clacks-Overhead' \
     'https://api.driftnet.io/v1/scan/protocols' \
  | jq . \
  | less -S

Both of these fields are case-sensitive, and both must be searched as full keywords (i.e. using the = operator).

Some types must also be searched as keywords. For instance, geo-country type:

curl -s -H 'Authorization: Bearer <your-api-token>' \
      --get --data-urlencode 'expression=geo-country=GB' \
     'https://api.driftnet.io/v1/scan/protocols' \
  | jq . \
  | less -S

Operator Precedence

The operator precedence is not > and > or. For example,

is equal to

To make sure you run the query you intended, it is a good idea to always use parentheses. The intent of the above query is properly captured by:

curl -s -H 'Authorization: Bearer <your-api-token>' \
      --get --data-urlencode 'expression=product-tag:dahua and (geo-country=RU or geo-country=IR)' \
     'https://api.driftnet.io/v1/scan/protocols' \
  | jq . \
  | less -S

Special Characters

A backslash is used to escaped special characters. Note that a double backslash \\ will be unescaped to a single backslash.

Within a quoted string escape double quotes:

curl -s -H 'Authorization: Bearer <your-api-token>' \
      --get --data-urlencode 'expression=http-header="inline; filename=\"index.html\""' \
     'https://api.driftnet.io/v1/scan/protocols' \
  | jq . \
  | less -S

Within an unquoted string escape opening and closing brackets ( ).

curl -s -H 'Authorization: Bearer <your-api-token>' \
      --get --data-urlencode 'expression=product-info=\(Fedora\) ' \
     'https://api.driftnet.io/v1/scan/protocols' \
  | jq . \
  | less -S

Context Restriction

It is possible to restrict searches to both a type and a context. Use a semicolon to specify the context, and specify a full keyword:

For example, to search a http-header type with a specific context Server and a particular value,

curl -s -H 'Authorization: Bearer <your-api-token>' \
      --get --data-urlencode 'expression=http-header;Server="NimPlant C2 Server"' \
     'https://api.driftnet.io/v1/scan/protocols' \
  | jq . \
  | less -S

Usually, you"ll need to use the = (keyword) operator when including the context. The exceptions are for IP and host. For ip searches, you should always use the : operator — e.g. to look up the IP address scanned,

curl -s -H 'Authorization: Bearer <your-api-token>' \
      --get --data-urlencode 'expression=ip;:8.8.8.8' \
     'https://api.driftnet.io/v1/scan/protocols' \
  | jq . \
  | less -S

For host searches you should use : if you want to include subdomains (expression=host;:example.com), or = if you don't (expression=host;=example.com).

To time-filter the results, use the from= and to= parameters. These accept dates in the format YYYY-MM-DD.

To filter on any arbitrary type, you may filter=type:value, for instance filter=port-tcp:443 to restrict to TCP port 443. As an alternative to type filtering, you might just include want to include additional terms in your expression.

Driftnet stores results as a time series. Often, you only want to know the most recent result for an {ip, port} pair. Set most_recent=true, and voilà.

curl -s -H 'Authorization: Bearer <your-api-token>' \
      --get --data-urlencode 'expression=ip;:8.8.8.8' --data-urlencode 'most_recent=true' \
     'https://api.driftnet.io/v1/scan/protocols' \
  | jq . \
  | less -S

If you search for example.com, do you also want to match URLs of the form scheme://sub.example.com/path/to/somewhere? It depends on your use-case. By default, this feature is off, but you can enable it by setting host_in_url=true:

curl -s -H 'Authorization: Bearer <your-api-token>' \
      --get --data-urlencode 'expression=host:google.com' --data-urlencode 'host_in_url=true' \ 
     'https://api.driftnet.io/v1/scan/domains' \
  | jq . \
  | less -S

Often, you want to get a quick rollup summary of a particular field. The API enables this with the summarize= parameter. This call will get all scan results including TLS certificates issued to the University of Oxford, and summarize the ports they were seen on:

curl -s -H 'Authorization: Bearer <your-api-token>' \
     --get --data-urlencode 'expression=subject:"university oxford"' --data-urlencode 'slop=1' \
     --data-urlencode 'summarize=port-tcp' \
     'https://api.driftnet.io/v1/scan/protocols' \
  | jq . \
  | less -S

{
  "summary": {
    "other": 0,
    "values": {
      "1000": 10,
      "10000": 11,
      "10443": 64,
      ...
      "943": 21,
      "9443": 9,
      "993": 10
    }
  }
}

The values object in the return contains the extracted values, together with their counts.

You might want to restrict the summary to particular contexts, or to exclude particular contexts (e.g. to exclude summarize HTTP headers, or to exclude a particular HTTP header). You can use the summary_context= and summary_nocontext= parameters for this.

By default, the summary is limited to a maximum of 100 values; if there are more unique values than this, then the total count of non-summarized values is placed in other. You can increase the maximum number of values in the summary using the summary_limit= parameter, up to a ceiling of 10,000 values per call.

Enterprise users can increase the limit on the number of returned values up to a maximum of 1,000,000. For each block of 10,000 results returned, one unit of API quota will be consumed.

Enterprise users can include an overall cardinality count in the response by setting summary_cardinality=true.

You can request Driftnet to schedule protocol-level collection on a particular IP/port pair by using the scan/protocols/prioritize endpoint.

curl -s -H 'Authorization: Bearer <your-api-token>' \
     'https://api.driftnet.io/v1/scan/protocols/prioritize?ip=8.8.8.8&port=443' \
  | jq .