Follow this guide to install the Lume Python SDK and run your first data transformation pipeline.
This guide will walk you through installing the Lume Python SDK, configuring your credentials, and running a complete data transformation pipeline.
Before you begin, ensure you have:
⚠️ Private SDK Access - Contact your Lume representative for access and installation instructions for the SDK.
There are two primary ways to configure your credentials:
Set your Lume API key as an environment variable. The SDK will automatically detect and use it.
Alternatively, you can initialize the client directly in your code. This is useful for environments where you can’t set environment variables, like some serverless functions.
This explicit initialization overrides any environment variables.
With the SDK installed and configured, you can trigger a pipeline with just a few lines of code.
For this example, we’ll assume:
customer_ingest:v1
.source_path
points to a specific file you want to process: s3://my-customer-data/new_records.csv
.Pro Tip: Use Webhooks for Production
While run.wait()
is great for simple scripts and getting started, we strongly recommend using Webhooks for production applications. They are more scalable and efficient than continuous polling.
When you call lume.run()
, you are not running the transformation locally. Instead, you are sending a request to the Lume platform to execute the following steps:
SYNCING_SOURCE
: Lume’s secure agent uses the pre-configured Connector to ingest the data from your source_path
into a temporary, isolated staging area.TRANSFORMING
: The Lume engine applies the logic from your customer_ingest:v1
Flow Version to the staged data.SYNCING_TARGET
: Lume’s agent writes the transformed data to your destination system, as defined in the Flow Version’s Target Connector.This “Sync-Transform-Sync” model ensures that your environment only needs to hold credentials to Lume, not to your source and target data systems. You can read more about this secure architecture in our Production Guide.
You’ve successfully run your first pipeline! Now you can explore more advanced topics: