Support for Complex Queries

Summary

The Driftnet UI and API now support complex queries. This update allows users to combine search terms using AND, OR, and NOT operators, have greater control over grouping, exact and prefix matching, and context aware term values — making searches more flexible and precise.

Background

Before this update, Driftnet allowed multiple search terms to be combined using the logical AND operator. You could supply multiple terms by using query, field and keyword parameters in the API, or via search chips in the UI.

With the new expression parameter, both API and UI users can now construct complex queries in a single string, combining terms with AND, OR, and NOT operators.

Logical Operators

Logical operators allow you to define relationships between search terms — narrowing or broadening your results depending on your needs.

The basic search unit (or term) in Driftnet is a type and a value. Typically these are separated by a colon in a query. For instance to search for a specific SSH server banner, enter in the search input or via the API expression parameter:

server-banner:SSH-2.0-ROSSSH

You might wish to exclude from your results services running on the default SSH port. This can now be achieved using the NOT operator:

server-banner:SSH-2.0-ROSSSH and not port-tcp:22

Excluding terms in the UI using the NOT operator can be achieved quickly with the "Exclude From Query" menu option.

Exclude From Query menu option

Furthermore, you might be interested in only a selection of countries presenting this banner on a non-standard port. You can now add these using the OR operator.

server-banner:SSH-2.0-ROSSSH and not port-tcp:22 and (geo-country=CN or geo-country=RU)

Note in this example how parentheses are used to avoid ambiguity in the search, and geo-country has been added with an exact (or keyword) value indicated by replacing the colon : separator with an equals sign =.

Results of a search showing the use of logical operators.

Tips and Syntax Notes

Quoting Values

When a value contains spaces, enclose it in double quotes ("..."). This ensures Driftnet treats it as a single value rather than multiple implicitly AND'ed terms.

As an example, to search for the entity type Tencent Cloud Computing you should quote the value:

entity:"Tencent Cloud Computing"

If you want to search within a quoted value for the literal quote, this should be backslash escaped as \". Parentheses should be escaped as \( and \).

Value Modifier Operators

As we saw in the earlier example, the two type-value separators available are the colon : and the equals sign =.

For textual fields <type>:<value> terms will match words inside the text whereas <type>=<value> will return exact complete matches only.

Ending a value in an asterisk * will use the value as a prefix. If you have a quoted value the asterisk follows the closing double quote.

Advanced Scenarios

Context Restrictions

Driftnet's data model presents values with both a type (what the thing is) and a context (where we saw it). For example, we report observed HTTP headers under the type http-header with context as the HTTP header (e.g. Server, Content-Length, Location etc).

Report section showing the type, context and value data model

Using the new syntax <type>;<context>=<value>, where a semi-colon separates the type from the context, you are now able to specify all three aspects of a report item in your search term.

Consider the following example, where we are looking for a Havoc C2 Server indicator:

http-header;X-Havoc=true

Observations of a search showing the use of including context in a search term.

Multisearch

Enterprise customers can now take advantage of the expression parameter in the /multi/search API endpoint. This update makes it possible to combine multiple service characteristics into one sophisticated query, enabling deeper and more precise searches.

As an example, we can combine the JARM and Visible Services data to find Mythic C2 server candidate IP addresses. For illustration we will use expressions to limit the results to TCP port 443 (as opposed to the default TCP port of 7443). We search using a well known jarm-fuzzyhash and HTML title.

A snippet of Python showing example parameters in the API call:

# We want to look in the protocol-scan results, AND the jarm results
"source": ["protocols", "jarm"],

# We are looking for services on port-tcp 443, with a well known JARM hash and the well known HTML title "Mythic"
"expression": [
    'title="Mythic" and port-tcp:443',
    "jarm-fuzzyhash=1dd40d40d00040d00042d43d000000831b6af40378e2dd35eeac4e9311926e and port-tcp:443",
],

# We want to know the IP addresses which match
"target": ["ip;"],

Example reporting based on results from a multi/search query for Mythic C2.

Resources

Further documentation is available in the Internet Scan section of our API Documentation.

For identification of available search term types and context you can review the Data Dictionary.

For a quick reminder of the expression syntax within the UI click on the help icon below the search input for a reminder.

— Driftnet Engineering