# Querying data

> Querying data via the Qualy API.

Most of our endpoints allows you to retrieve specific data through query requests using various parameters.

Our API also lets you select what properties to return (like an SQL `SELECT` command), limit the quantity, sort and skip results.

---

## The response envelope

Every successful response from the Qualy API is wrapped in a `data` envelope. Your object — or array of objects — is always under the `data` key:

```json
{
  "data": { "_id": "653fc651d14vbfe63d4fd49c", "email": "email@email.com" }
}
```

This means you access fields as `response.data._id`, **not** `response._id`. In JavaScript, destructure it as you parse:

```javascript
const { data: contact } = await response.json();
console.log(contact._id);
```

List endpoints (`GET` requests that return an array) add two fields **next to** `data` for pagination:

* `count` — the number of items in `data` for this page.
* `hasMore` — `true` when more pages are available (the returned count equals your `limit`).

```json
{
  "count": 14,
  "hasMore": true,
  "data": [ /* … */ ]
}
```

> **Note — Two exceptions**
>
> The health check (`/health`) and public keys (`/public-keys`) endpoints return their body unwrapped. Every other endpoint uses the envelope.

---

## Query paramaters

| Paramater | Example | Description |
| --- | --- | --- |
| `filter` | `status[0]=due&status[1]=paid-partial` | Defines a query object to filter the data based on the specified criteria. |
| `projection` | `profile.firstName email` | Selects which properties to return, like an SQL `SELECT`. Space-separated field names; supports dot notation for nested fields. Prefix a field with `-` to exclude it (e.g. `-_id`). |
| `skip` | `skip=10` | Specifies the number of documents to skip in the query. This parameter is useful for pagination, allowing you to navigate through large sets of data. |
| `limit` | `limit=10` | Defines the number of documents to include in the API response. It enables users to limit the quantity of data retrieved. |
| `sort` | `createdAt` | Let's you sort the response based on different properties. Ascending or descending order, etc. To sort on a descending order, a hyphen character before the property name should be added (e.g. `-createdAt`). |
| `population` | `0[path]=contact&0[select]=profile` | Allows the expansion of specific properties of the response by specifying which fields to expand. Read more on on [Expanding Responses](#expanding-responses). |

---

## Example query request

The following request and response payload is an example of a query made to the [Contacts API](https://v1-spec.qualyhq.com/#get-/v1/contacts).

```javascript
try {
  const response = await fetch('https://api.qualyhq.com/v1/contacts?projection=number%20profile.firstName%20profile.lastName%20profile.picture%20email%20profile.phone%20tags%20owners&sort=number&limit=14&skip=0', {
    method: 'GET',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': 'ApiKey your-api-key-here',
      'X-TENANT-ID': 'your-tenant-id-here',
    },
  });

  if (response.ok) {
    const data = await response.json();

    console.log(data);
  } else {
    throw new Error(`Request failed with status: ${response.status}`);
  }
} catch (error) {
  console.error(error);
}
```

### Query response

Note that the `_id` is automatically included, unless specified otherwise in the `projection` paramater (e.g. `-_id`).

```json
{
    "count": 1,
    "hasMore": false,
    "data": [{
        "_id": "653fc651d14vbfe63d4fd49c",
        "email": "email@email.com",
        "number": "123",
        "profile": {
            "firstName": "First name",
            "lastName": "Test",
            "phone": "+61 415 374 585"
        },
        "owners": [
            "64e382f1c79fe62f6fd1546c"
        ],
        "tags": [],
    }]
}
```

## Enconding query paramaters

### Objects

You need to encode JavaScript objects by converting key-value pairs into a query string format.

```javascript
// this object
{ key1: 'value1', key2: 'value2' }

// becomes
'key1=value1&key2=value2'
```

### Arrays

Arrays are handled by encoding their indices or keys.

```javascript
// this array
[ 'value1', 'value2' ]

// becomes
'0=value1&1=value2'

// or using the brackets notation
'arr[]=value1&arr[]=value2'
```

### Nested objects and arrays

Nested objects or arrays are encoded using dot notation.

```javascript
// this object
{ key: { nestedKey: 'value' } }

// becomes
'key.nestedKey=value'
```

### Special characters and URI component

#### Encoding special characters

You need to encode special characters, to allower their inclusion in query strings without causing parsing issues.

For example, spaces are replaced with `%20`.

#### URI component encoding

Use URI component encoding for preserving the integrity of URL query parameters. This encoding helps represent characters that have special meanings in URLs by converting them to a valid format.

> **Note — NPM packages can help you**
>
> The NPM package `qs` can help you encode all query paramaters automatically. It's the same package internally used by Qualy to stringify and parse query paramaters.
>
> [Go to the NPM package page](https://www.npmjs.com/package/qs)

## Expanding responses

Many objects allow you to request additional information as an expanded response by using the `population` paramater. This parameter is available on most API query requests, and applies to the response of that request only.

To expand responses, add an extra paramater to the URL called `population`, and using th same enconding as explained above, choose which properties you want to expand.

```javascript
// this object
[{ path: 'contact', select: 'profile email' }]

// becomes
'0[path]=contact&0[select]=profile email'
```

## Filtering results

You can filter results of your query. Qualy is compatible with a subset of the MongoDB query syntax. [You can learn more about how to create MongoDB-compatible object queries here.](https://www.mongodb.com/docs/manual/tutorial/query-documents/)

All filter inputs are sanitised before execution. The following sections describe what is supported and what is not.

### Supported filter operators

These comparison and logical operators are available for use in filters:

| Category | Operators | Example |
| --- | --- | --- |
| Comparison | `$eq`, `$gt`, `$gte`, `$lt`, `$lte`, `$ne`, `$in`, `$nin` | `age[$gte]=18` |
| Logical | `$or`, `$and`, `$not`, `$nor` | `$or[0][status]=active&$or[1][status]=pending` |
| Element | `$exists`, `$type` | `email[$exists]=true` |
| Array | `$elemMatch`, `$size`, `$all` | `tags[$elemMatch][$eq]=important` |

### Disallowed operations

The following operations are explicitly **not supported** and will be silently stripped from your query:

* **`$regex` and `$options`** — Regular expression matching is not available through query filters. Use the [Search API](#search) for free-text and pattern-based searching instead.
* **`$where`** — Server-side JavaScript execution is not permitted.
* **`$expr`** — Aggregation expressions within queries are not supported.
* **`$lookup`, `$unionWith`, `$merge`, `$out`** — Aggregation pipeline stages and cross-collection operations are not supported.
* **`$function`, `$accumulator`** — Custom server-side functions are not permitted.
* **`$comment`** — Query comments are stripped.

> **Warning — Filtering on sensitive fields**
>
> Queries that attempt to filter on sensitive fields (such as `password`, `apiKey`, `secret`, `accessToken`, `refreshToken`, or `privateKey`) will have those fields removed from the filter. This applies at any nesting depth.

### Query limits

* The `limit` parameter accepts values between `0` and `1000`.
* The `sort` and `projection` parameters only accept space-separated field names (e.g. `name email createdAt`). Dot notation for nested fields (e.g. `profile.firstName`) and a leading hyphen for descending order (e.g. `-createdAt`) are supported. JSON objects, special characters, and expressions are not accepted.
* Filter nesting depth is limited. Deeply nested filter structures beyond a reasonable depth will be truncated.

## Search

Some top-level API resources have support for retrieval via our [Search API](https://v1-spec.qualyhq.com/#post-/v1/search/-index-) methods. It differs from a "query", as it supports typo-tolerance, free-text search, and more. For example, you can search **contacts**, and search **partnerships**.

```javascript
try {
  const response = await fetch('https://api.qualyhq.com/v1/search/contacts', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': 'ApiKey your-api-key-here',
      'X-TENANT-ID': 'your-tenant-id-here',
    },
    body: JSON.stringify({
      "query": "test",
    }),
  });

  if (response.ok) {
    const data = await response.json();

    console.log(data);
  } else {
    throw new Error(`Request failed with status: ${response.status}`);
  }
} catch (error) {
  console.error(error);
}
```

### Example search results payload

The following payload shows the result of a query request on the **contacts** entity.

```json
{
    "page": 0,
    "nHits": 1,
    "nPages": 1,
    "hitsPerPage": 14,
    "processingTimeMs": 1,
    "hits": [{
        "_id": "653fc651d14vbfe63d4fd49c",
        "email": "email@email.com",
        "number": "123",
        "profile": {
            "firstName": "First name",
            "lastName": "Test",
        },
        "partnersReference": {},
        "owners": [
            "64e382f1c79fe62f6fd1546c"
        ],
        "primaryTeams": [
            "64e38320c79fe62f6fd154b8"
        ],
        "tags": [],
        "tenantId": "eu1-xpdqxiiphqsdtkpazqdqnraqi",
        "_highlightResult": {
            "email": {
                "value": "email@email.com",
                "matchLevel": "none",
                "matchedWords": []
            },
            "number": {
                "value": "123",
                "matchLevel": "none",
                "matchedWords": []
            },
            "profile": {
                "firstName": {
                    "value": "First name",
                    "matchLevel": "none",
                    "matchedWords": []
                },
                "lastName": {
                    "value": "<em>Test</em>",
                    "matchLevel": "full",
                    "fullyHighlighted": true,
                    "matchedWords": [
                        "test"
                    ]
                }
            }
        },
    }]
}
```

For the full API reference for the Search endpoint, [click here](https://v1-spec.qualyhq.com/#post-/v1/search/-index-).
