FlowScale allows you to deploy your ComfyUI workflows as scalable APIs or shareable AI apps using Gradio UIs. This guide will cover how to prepare, deploy, and monitor your workflows, as well as the scalability options and post-deployment processes.

​

Table of Contents

1. Preparing Workflows for Deployment
2. API Deployment
3. Gradio UI Deployment
4. Scaling and Performance
5. Post-deployment Monitoring

​

1. Preparing Workflows for Deployment

Before deploying a workflow as an API or Gradio UI, you need to configure the workflow settings, particularly input/output (I/O) settings, and test it on the ComfyUI instance.

​

Configurations Needed Before Deployment

1. Open the Workflow:
   If you are deploying an existing workflow, you must first import it and run it successfully on the ComfyUI instance. Ensure that all models and custom nodes are set up correctly. If you are building a workflow from scratch, make sure you’ve saved it successfully.

   Workflow Ready for Deployment

2. Saving the Workflow:
   Once the workflow has been run successfully, save the changes. An API JSON file is created along with the workflow file, which is required for deployment. See the Building Workflows document for more details.

3. Setting Up I/O Configurations:

   • I/O Configuration is crucial for defining the workflow’s inputs and outputs, which will be used in API and Gradio UI deployments.
   • Hover over the API JSON file in the workflows list, click the triple-dot menu, and select “I/O Configuration”. A dialog box will appear showing two columns: Inputs and Outputs.
   API JSON -> I/O Configuration
   • Inputs:
     Select the input nodes from the dropdown. These inputs are derived from the nodes used in the workflow. Label each input, as this label will appear in the Gradio UI or API documentation.
   • Outputs:
     Similarly, choose the output nodes and label them. Save the I/O configuration after completing this step.
   Select input and output workflow nodes
   Select Input Parameters & Save

​

Testing the Workflow Before Deployment

• Before deploying the workflow, ensure it runs smoothly on the ComfyUI instance. Running it verifies that all nodes and models are properly configured and ready for deployment.

  Tip: If the workflow runs successfully, it confirms that all necessary components (models, nodes) are installed and functional.

​

2. API Deployment

Deploying a workflow as an API on FlowScale enables scalable, serverless API endpoints for external use.

​

Steps for Deploying a Workflow as an API

1. Deploy the Workflow:

   • Once the I/O configuration is complete, go to the top-right corner of the editor page and click Deploy.
   Deploy button
   • A dialog box will appear, showing a list of workflows in your project. Workflows with complete I/O configurations will have a green checkmark, while those without configurations will have a red cross.
   Deploy Workflows

2. Confirm Deployment:

   • Select the workflows you wish to deploy and click Deploy. The system will take you to the Deployment Tab in the project settings, where you can track the status of the deployment.
   • Loader Status: This will display the current deployment status, such as preparing, deploying, and completed.
   API Deployments Tab
   Deploying Workflows

​

API Documentation (Swagger)

• After deployment, FlowScale generates API documentation using Swagger. This documentation includes details on how to call the API, endpoints, request/response formats, and authentication information.

• You can access the API Docs by clicking the API Docs button in the deployment section.

  API Docs Link
  API Docs

​

Accessing API Endpoints and Usage Limits

• Once deployed, the API endpoints can be accessed via the API Docs section.

• Authentication: API requests are authenticated using an API Key, which can be found in the API Settings under Project Settings. You can regenerate the key or disable authentication to make the API publicly accessible.

  API Key

​

3. Gradio UI Deployment

Deploying a workflow as a Gradio UI creates a shareable interface that can be accessed via a URL.

​

Steps for Deploying Workflows as Gradio UIs

1. Deploy the Workflow:
   Follow the same steps as API deployment (explained above). Once the workflow is deployed, a Gradio UI link will be generated alongside the API endpoints.

2. Accessing and Sharing Gradio UIs:
   Once deployed, the Gradio UI link will appear in the Gradio UI section of the deployment page. This link can be shared with others, and they do not need to run ComfyUI on their machine to access the workflow.

   • Public Links: The generated Gradio UIs are publicly accessible, meaning you can share the URL with anyone to use the AI app.
   Gradio URLs
   AI App (Gradio UI)

​

Customizability of Gradio UIs

• Once a Gradio UI is deployed, its design and interface are determined by the I/O configuration. The interface is not customizable after deployment.

​

4. Scaling and Performance

FlowScale workflows are designed to be scalable and handle multiple API requests efficiently.

​

Handling Scalability

• Queue-based System: FlowScale uses ComfyUI’s queue-based system to handle multiple concurrent requests. When several users access the API at the same time, their requests are queued to ensure smooth execution without overwhelming the system.

​

Scaling Up or Down Based on Demand (Coming Soon)

• Currently, there is no manual functionality to scale workflows up or down based on demand. However, in the near future, FlowScale plans to introduce auto-scaling features. (This feature does not need to be included in the documentation for now.)

​

Optimizing Workflows for Better Performance

• Minimize Node Complexity: Keep the node list lightweight. The more nodes a workflow has, the longer the cold start time will be.
• Avoid Heavy Nodes: Certain nodes (e.g., MixLab nodes) can significantly slow down the cold start time, potentially increasing it from 15 seconds to over two minutes. Use lightweight nodes whenever possible for faster workflow execution.

​

5. Post-deployment Monitoring

Once a workflow is deployed, it’s essential to monitor its performance and make updates as needed.

​

Monitoring API Requests and Errors

• Deployment Logs: You can view the deployment logs to check for errors, API usage statistics, and other relevant details.

• To access the logs, go to the Deployment Tab and click See Logs next to the deployed workflow.

  See Deployment Logs

​

Redeploying Workflows After Updates

• If you make changes to a workflow after it has been deployed, you can redeploy it:
  1. Stop the Previous Deployment: Go to the deployment section and stop the current deployment.
  See Current Deployment
  2. Reconfigure the Workflow: Update the input/output settings of the new workflow version in the editor.
  3. Deploy Again: Click the Deploy button and follow the same steps to redeploy the workflow with the updated configuration.

​

Conclusion

Deploying workflows in FlowScale is a seamless process that enables scalable API endpoints and shareable Gradio UIs. By following this guide, you can ensure that your workflows are properly configured, optimized for performance, and monitored effectively after deployment.