Target Schema

Target schemas follow JSON Schema format. Other important notes:

Provide a description for each target property in the description property.
Designate required fields in the required property.

Type: Indicates the property type or types

  {
    "type": ["string", "null"]
  }

Description: Provides a semantic description of the property. These are mapping instructions, but rather a high-level definition.

 "business_registration_status": {
   "description": "The status of the business’s registration.",
 }

Enum (classification): Enums allow you to use AI to classify data into your internal enum definitions. For example, if your internal models work with states in abbreviations, such as CA, NY, etc, then add these abbreviations to the enum. Any source data provided then will be classified into this enum, if the AI finds state source data. Thus, source data with California will be mapped to CA.

    "state": {
        "description": "The state where the user resides.",
        "type": "string",
        "enum": ["CA", "NY", "TX"]
    }

Filter: This is joint property with ‘enum’. If set to true, it will trigger Lume to run its Filtering AI Model, rather than its default Classification AI model. Use this for use cases where the goal is to classify proper nouns, such as account names, etc. This will also trigger confidence generation if your organization has confirmed so with the Lume team.

 {
   "filter": "true"
 }

Regex: Use a regex to enforce output data formats. If a generated mapping logic does not output data that the regex validates, Lume flags the job for you. See Status to learn more.

 {
   "pattern": "^(\\([0-9]{3}\\))?[0-9]{3}-[0-9]{4}$"
 }

isPrimaryKey: This property can only be set to true if the format is set to “uuid”. When true, it will generate a unique ID in this column.

{
  "format": "uuid",
  "isPrimaryKey": true
}

cleaning_instructions: Provides instructions to clean the data after it is pulled from a certain field, using GenAI. This allows for custom data cleaning operations.

{
  "cleaning_instructions": "Remove any special characters and convert to lowercase."
}

Periods (e.g. .) are a reserved character for property names in Lume’s API. Thus, only property names that do not contain the . character will be accepted.

Full Target Schema Example

{
  "type": "object",
  "properties": {
    "full_name": {
      "type": ["string"],
      "description": "The full name of the customer, including first name and last name."
    },
    "email_address": {
      "type": ["string"],
      "description": "The customer's primary email address used for communication.",
      "format": "email"
    },
    "customer_id": {
      "type": ["string"],
      "description": "Unique identifier for the customer in our system.",
      "isPrimaryKey": true,
      "format": "uuid"
    },
    "phone_number": {
      "type": ["string"],
      "description": "The customer's primary phone number.",
      "pattern": "^\\+?[1-9]\\d{1,14}$"
    },
    "subscription_tier": {
      "type": ["string"],
      "description": "The customer's current subscription tier.",
      "enum": ["free", "basic", "premium", "enterprise"],
    },
    "registration_date": {
      "type": ["string"],
      "description": "The date when the customer registered for the service.",
      "format": "date"
    },
      "receipt_number": {
      "type": ["string"],
      "description": "The numeric part of the receipt ID.",
      "cleaning_instructions": "Extract only the numeric portion from the receipt ID. For example, from 'REC-12345', extract '12345'."
    }
  },
  "required": [
    "full_name",
    "email_address",
    "customer_id",
    "phone_number",
    "subscription_tier",
    "registration_date",
    "receipt_number"
  ]
}

To learn how to edit target schemas in existing pipelines, see Editing Target Schema.

Questions

Important Concepts

Permissions