Guidance for building robust, production-grade data pipelines with Lume
LUME_API_KEY
environment variable. This is the recommended approach for most environments.
POST
request to your URL with a payload summarizing the outcome.
run_id
to call lume.run_status()
to retrieve the full, detailed metadata object if needed.
X-Lume-Signature-256
header. This signature is an HMAC-SHA256 hash of the request body, created using your secret key.source_path
is unique and deterministic for each batch of data.
source_path
. Example: daily_invoices/2024-07-30
.source_path
that has already been successfully processed for a given Flow Version, the Lume platform will reject the request with an InvalidRequestError
, preventing duplicate pipeline executions.
To bypass this for a legitimate reprocessing, use the force_rerun=True
parameter in your lume.run()
call.
status
(SUCCEEDED
, PARTIAL_FAILED
, FAILED
).on_failure_callback
) or monitoring system to trigger when a run enters a FAILED
or CRASHED
state.run.id
, flow_version
, and source_path
. This makes debugging much easier.LUME_API_KEY
environment variable.
lume.init()
for details.