Confidence Scores is in Alpha and executes only for enums smaller than size 1000. Reach out with feedback or questions regarding your confidence scoring.

If your enum size is greater than 1000, you may leverage a Filtering task (see below) where the limit is larger.

Lume generates confidence scores for lookup tables, allowing users to more easily review lookup tables. Confidence values are separated in buckets of Confident, Very High, High, Medium, Low, Very Low, and Incorrect.

Trigger confidence score generation

Users must manually trigger confidence score generation for a finished job. To do so, click Generate Confidence Scores in the App Workshop, or call the Generate Confidence Score Endpoint via the API. Confidence scores are automatically generated for Filtering tasks.

Classification vs Filtering

Confidence scores are executed for both lookup tasks: classification and filtering. See below to understand the differences in relation to confidence score generation for each.

Limit of 1000 enums.

This will generate classification confidence for the lookup. Meaning, it will focus on the semantic similarity of the key value pair over character similarity.

Confidence scores are accessible via the API through the Spec of result object. Example below.

Accessing Confidence Scores

Confidence scores are accessible via the API through the Spec of result object. Example below.


/*
* For this example, "loan_plan" has an enum ["P1", "P2", "P3"], meaning all
* source data from "plan" has to be classified into one of the three enum values, if possible.
*/

"loan_plan": {
    "@sources": [
        "plan"
    ],
    "@default_values": [],
    "confidence_score": {
        "lookup_confidence": {
            "plan_1": "Confident",
            "plan_2": "Very High",
            "plan_2_extra": "Medium",
            "plan_3": "Very High"
        }
    },
    "@lookup": {
        "plan_1": "P1",
        "plan_2": "P2",
        "plan_2_extra": "P2",
        "plan_3": "P3"
    },
}