Configuring Running Pipeline Limit

Overview

Configure the maximum number of pipelines that can run simultaneously in your Sight Machine environment. This helps manage infrastructure resources, control costs, and ensure critical pipelines always have capacity.

• Who can configure this? Only users with the Administrator role or higher can change this setting.
• Who is affected? All users with access to Factory Build across all workspaces.

What is Running Pipeline Limit?

The Running Pipeline Limit is a system-wide setting that controls how many pipelines can run concurrently across all workspaces in your environment.

Why use it?

• Resource Management: Prevent environment overload by limiting concurrent data processing
• Cost Control: Manage infrastructure costs by capping pipeline execution
• Performance: Ensure critical pipelines get adequate resources
• Stability: Prevent system slowdowns from too many simultaneous operations

What happens when the limit is reached?

• Users cannot start or deploy additional pipelines
• A clear error message explains the limit and how to proceed
• Users must stop or remove running pipelines before starting new ones

Understanding the Feature

Default Behavior (No Limit Set)

• Value: Unset (no limit)
• Effect: Any number of pipelines can run simultaneously
• Use case: New environments, ample infrastructure, or few concurrent users

With Limit Set

• Value: Integer (e.g., 10, 25, 50, 100)
• Effect: Maximum this many pipelines can run at once across all workspaces
• Use case: Shared environments, cost-sensitive deployments, resource-constrained infrastructure

Important Limitation

⚠️ Once a limit is set, it cannot be returned to "Unset"

If you need "unlimited" pipelines after setting a limit:

• Set the value to a very high number (e.g., 100 or 200)
• This effectively removes the constraint while keeping a technical limit in place
• Example: If your infrastructure supports 30 pipelines max, setting limit to 100 is effectively unlimited

Scope

• Environment-wide: Applies to all workspaces and all users
• Across all states: Includes Production, Development, and Pre-Production workspaces
• Real-time enforcement: Checked immediately when users attempt to start pipelines

Configuring the Running Pipeline Limit

⚠️ Only Administrators can configure this setting

Step 1: Navigate to Feature Configuration

1. Log into Sight Machine
2. Select Settings from the main navigation menu
3. Select Feature Configuration from the settings list
4. Select Running Pipeline Limit from the dropdown

✅ Verification: You should see a section titled "Running Pipeline Limit" with a description "Set the maximum number of running Pipelines allowed."

Step 2: Review Current Configuration

Before making changes, check the current settings:

1. Current value: Shows current limit (or "Unset" if no limit)
2. Last updated: Shows date/time and user who made the last change
3. Running pipelines: Check how many pipelines are currently running

Step 3: Set the Pipeline Limit

1. In the Maximum Allowed Running Pipelines field, enter your desired limit

   • Example values: 10, 20, 30, 50
   • For "effectively unlimited": Set to 100 or 200

2. ⚠️ Important: You can increase or decrease this value later, but you cannot unset it once configured. If you need "unlimited" in the future, set it to a very high value like 100.

Step 4: Confirm Limit (If Lowering)

If you're setting a limit lower than the current value, you'll see a confirmation modal:

1. Modal title: "Confirm lower limit"
2. Message: "This value is lower than the current limit. Please confirm to set the new limit."
3. Select "Set Limit" to proceed or "Cancel" to abort

⚠️ Lowering the limit does not stop currently running pipelines. It only prevents new pipelines from starting once the new limit is reached.

Step 5: Apply Changes

1. Select "Apply" at the bottom of the Feature Configuration page
2. The new limit takes effect immediately, indicating value, and date and user who made the change.

✅ Verification: The "Current value" field now shows your new limit.

What Users Experience

When Attempting to Start a Pipeline

If a user tries to start a pipeline when the limit is reached:

1. Error modal appears with title: "Pipeline Deployment Limit Reached"

2. Message:

   "You've reached the maximum number of running Pipelines allowed in this environment ([limit]). To run or deploy a new Pipeline, pause or remove one or more of the existing running Pipelines. If you require additional capacity, please adjust the number of Pipelines in the Running Pipeline Limit section of the Feature Configuration section of the Settings."

3. What users can do:

   • Stop or remove existing pipelines (see options below)
   • Contact an Administrator to increase the limit (non-admin users cannot change the limit themselves)

How Users Can Free Up Capacity

ℹ️ Note: Non-admin users (Developer, Process Expert roles) cannot change the running pipeline limit. They can only manage existing pipelines or contact an Administrator.

If you are NOT an Administrator, you have these options:

Option 1: Stop Pipelines

1. Navigate to Factory Build → Workspaces
2. Find running pipelines
3. Select Pause button on pipelines not actively needed
4. Wait for pipeline to stop
5. Retry starting the desired pipeline

Option 2: Delete Unused Workspaces

1. Identify development workspaces no longer needed
2. Navigate to Workspaces list
3. Select workspace Options → Archive
4. All pipelines in archived workspace stop automatically

Option 3: Request Limit Increase (Non-Admins)
If none of the above options work:

1. Contact an Administrator in your organization
2. Explain why you need additional pipeline capacity
3. Administrator can increase the limit in Settings → Feature Configuration

ℹ️ Note: Only Administrators can change the running pipeline limit. If you're a Developer or Process Expert, you'll need admin assistance to increase capacity.

Monitoring Pipeline Usage

1. Navigate to Factory Build → Workspaces
2. Locate Running Pipelines count tab in the top left corner (below Workspaces header)

💡 Tip: Use the filter to find all workspaces with running pipelines quickly.

Security and Permissions

Who Can Configure the Limit?

⚠️ Administrator role ONLY

• Only users with the Administrator role can access Feature Configuration
• Only Administrators can view, create, update, or delete the running pipeline limit
• Sight Machine role: Full access (internal Sight Machine staff only)

Cannot configure:

• ❌ Developer role: Cannot access Feature Configuration settings
• ❌ Process Expert role: Cannot access Feature Configuration settings
• ❌ Basic User role: Cannot access Feature Configuration settings
• ❌ Customer role: Cannot access Feature Configuration settings

What Can Non-Admins Do?

If you are NOT an Administrator and need the limit changed:

1. Identify an Administrator in your organization or check with your Sight Machine account administrator
2. Request a change with justification:
   • Current limit and why it's insufficient
   • Number of pipelines you need to run
   • How long you need the increased capacity
3. Administrator makes the change in Settings → Feature Configuration

Who Can View the Limit?

• Administrators: Can view the configured value in Settings → Feature Configuration
• All users: See the limit value only when the error modal appears (when hitting the limit)
• Non-admins: Cannot view the setting in advance, only see it when blocked

Who Is Affected by the Limit?

The limit applies to everyone:

• All roles: Administrator, Developer, Process Expert, Basic User, Customer
• All workspaces: Production, Development, Pre-Production, Archived
• All operations: Start pipeline, Deploy pipeline, Resume pipeline
• No exceptions: Even Administrators must follow the limit when starting pipelines (but they can change the limit)