Common errors and solutions when using the Lume Python SDK.
This guide covers common issues you might encounter while using the Python SDK and how to resolve them. For detailed information on the specific exceptions the SDK can raise, see the Exceptions API Reference.
LUME_API_KEY
environment variable is not set or is not accessible to the Python script’s process.print(os.getenv("LUME_API_KEY"))
in your Python script to see what value is being read. It will print None
if the variable is not found..bashrc
, .zshrc
). Ensure the variable is set in the correct context for your execution environment.If the SDK cannot authenticate with the Lume platform, it will raise an AuthenticationError
.
LUME_API_KEY
environment variable is set correctly.lume.init()
, ensure the api_key
argument is correct.If you successfully authenticate but the request itself is invalid, the SDK will raise an InvalidRequestError
.
flow_version
does not exist or your source_path
is malformed. It can also be triggered by Lume’s idempotency check."my-flow:v1"
).source_path
that has already completed successfully, the API will raise an InvalidRequestError
to prevent duplicate runs. To override this, use the force_rerun=True
parameter in your lume.run()
call.These errors occur during a pipeline’s execution on the Lume platform.
FAILED
A FAILED
status indicates a non-recoverable error occurred in one of the pipeline stages. When using run.wait()
, this will raise a RunFailedError
.
SourceConnectorError
This is one of the most common errors and occurs during the SYNCING_SOURCE
stage.
read
permissions for the specific source_path
you are trying to access.source_path
is correctly formatted. For S3, it should be s3://bucket/key
. For databases, it’s a string identifier for the data to be processed.TargetConnectorError
This is similar to SourceConnectorError
but occurs during the SYNCING_TARGET
phase.
write
permissions to the target location (e.g., s3:PutObject
for S3, INSERT
for databases).PARTIAL_FAILED
This is not a true failure. It means the pipeline completed, but some rows failed transformation.
run.metadata['results']
object, specifically the rejected_rows
count and the target_locations['rejects']
path.AttributeError
or unexpected None
values when trying to access run.metadata
.run.metadata
attribute is only populated after the run has reached a terminal state (SUCCEEDED
, FAILED
, etc.). If you access it immediately after calling lume.run()
, it will be empty.run.wait()
or run.refresh()
before accessing run.metadata
to ensure the object has been updated with the final results from the Lume platform.source_path
points to an empty file or a database query that returns no results.source_path
is not empty.input_rows
field in run.metadata['results']
to confirm how many rows Lume ingested from your source.run.wait()
takes a very long time or times outIf run.wait()
exceeds its timeout, it will raise a RunTimeoutError
. This can happen for several reasons:
Large Data Volume: Syncing or transforming a very large amount of data can take a long time.
Complex Transformations: A highly complex Flow Version can increase processing time.
Source/Target Bottlenecks: Performance issues in your own data stores can slow down the sync stages.
Troubleshooting Steps:
status
to see which stage is taking the longest (SYNCING_SOURCE
, TRANSFORMING
, or SYNCING_TARGET
). You can do this by calling run.refresh()
in a separate thread, or by checking the run status in the Lume UI.run.metadata['pipeline']
for a detailed timing breakdown of each stage.timeout
parameter in your run.wait(timeout=...)
call.ApiError
or other connection errorsIf the SDK raises a generic ApiError
during a run.wait()
poll or a lume.run()
call, it typically indicates a transient network issue between your application and the Lume API.
When encountering an issue that you cannot resolve, gathering the right information is key to a quick resolution. Before reaching out for support, please have the following details ready:
run_...
). This is the most important piece of information.invoice_processor:v4
).source_path
that triggered the run.