Managing Pipelines
How to organize, manage, and leverage existing pipelines
Learn more about Pipelines here.
With Lume, you will likely build and manage multiple (if not hundreds and thousands) of pipelines. As an API, you can embed Lume anywhere in your code, fitting to your pipeline management architecture. This is where pipeline management comes in, and the key lever in pipeline management is pipeline names.
Pipeline names allow you to track and reuse deployed pipelines. Use these names to route certain data to their corresponding pipelines.
Example: If you are normalizing data from multiple data feeds, Feed A, Feed B, and Feed C, into one internal schema, you could create three pipelines with the below names:
FeedA_to_internalFeedB_to_internalFeedC_to_internal
When data comes in from Feed A, pipe it to the FeedA_to_internal pipeline.
Refer to Use Pipeline to learn more.