Indexing Your Metadata

Introduction

You can index your metadata for each data item. This metadata gets indexed and is available for searching. You can use it in other search parameters such as sorting, filtering, faceting and more.

Metadata Format

Your metadata needs to be a flat JSON file. It means it cannot have nested objects. Note that arrays cannot have nested objects either.

Example Metadata

{
    "age": 25,
    "scores": [80, 85, 90],
    "temperature": 25.5,
    "daily_temperatures": [22.3, 23.1, 21.8],
    "name": "John Conner",
    "favorite_fruits": ["apple", "banana", "grapes"],
    "is_active": true,
    "attendance": [true, false, true]
}

Important

Since we allow for empty metadata, and infer the types from your metadata, the very first data item need to have all metadata fields.

Metadata Consistency

For a particular collection, metadata has to be consistent. Meaning, if you have a metadata key scores for a particular data item, it has to be present for all other data items. The values of the metadata key scores can be empty. Please see Empty Metadata Values.

Consistent Arrays

Arrays need to have the same data type. For example arrays like [1, 2, "string"] is not allowed.

Allowed Data Types

Your data can be of the any of the following type:

int, float, string, bool int[], float[], string[], bool[]

Reserved Metadata Keys

Since we extract a lot of signals from your data, following metadata keys are not allowed.

tags_keywords chapter_start chatper_end chapter_title chapter_transcript chapter_ocr chapter_segments.

Empty Metadata Values

If you do not have values for a particular field, you can set it to be empty in your json. If the type of the field is an array, you should just set it to []. Setting it to [null] will yeild an error.

Example of empty metadata

{
    "age": null,
    "scores": [],
    "temperature": null,
    "daily_temperatures": null,
    "name": null,
    "favorite_fruits": [],
    "is_active": null,
    "attendance": []
}

Date in Metadata

Date type is not supported. You need to convert your date into a unix timestamp and then pass it in the metadata.