Edit Mapper

Mappers store the mapping logic to map data from source data to the pipeline’s target schema.

Lume provides multiple levers to edit your mapping logic. All levers are outlined further on step 2 below.

Manually Edit Mapper
Edit Target Schema
Suggest Data Samples

Editing Mapper

Initiate a workshop

You can initiate a workshop by calling Create Workshop with either a job_id or a pipeline_id. This returns a Workshop object.

Edit the mapper

There are three ways to edit the mapper. Indicate these edits in your payload to Run Workshop, and Lume will regenerate the Mapper based off of your edits.

Manual Edit allows you to manually make changes to the Mapper’s logic.

Use the mapper payload property in Run Workshop with Mapper to provide the mapping logic edits to Lume.

Below is an example:

Upon review, mapping is incorrect

In mappings, you discern that the Mapper mapped identifier from the source data to internal_id in your target schema. But, you desire that id from the source data is instead mapped to internal_id.

Submit edits to mapper

Update the one-to-one mapping to pull from id instead of identifier from the source data. You indicate this via the transformation structure. To learn more, see Building Mapper Logic.

Mapper edit payload

[
  {
      "targetField": "internal_id", // target field transformation is being applied to
      "transformation": {
          "extract": "id", // source property to extract one-to-one mapping to
      }
  },

  // ... more edits
]

Learn more about manual edits here.

Sample Suggestion allows you to edit mapping logic by communicating to the AI what the correct mapped values are, and Lume will regenerate the mapping logic based off of your suggestion.

Use the sample payload property in Run Workshop with Sample to provide the correctly mapped record to Lume.

For example, the mapped data below actually mapped key1 incorrectly to incorrect_X.

Output mapped data from original generation

{
    "key1": "incorrect_X",
    "key2": "correct_value"
}

After view, based on the source data of this job, you discern it should instead have mapped to correct_Y. Thus, pass in sample with the correct values for key1 and Lume will use this correction to regenerate the Mapper logic. Make sure to indicate the index of the source record being mapped from.

Updated Sample with corrected mapped data

{
  "sample": {
    "index": 0,
    "mapped_record": {
      "key1": "correct__Y",
      "key2": "correct_value"
    },
  }
}

Deploy the edits

After making edits, save them through either

Deploy Workshop.
Setting auto_deploy = true in a workshop edit flow, in the prior step.

This will update the Mapper in the corresponding pipeline for future jobs.

Welcome

Getting Started

Guides

Cookbooks

Security

Editing Mapper

Welcome

Getting Started

Guides

Cookbooks

Security

​Editing Mapper

Editing Mapper