<- Back to MindStudioPricingAbout

Running Sub-Workflows to Iterate and Process Data

Learn how to run sub-workflows within a parent workflow in MindStudio to process structured data iteratively.

Using sub-workflows in MindStudio enables you to break out a specific task (like scraping a URL or generating summaries) and apply it repeatedly across a list of items. This approach is highly effective when dealing with variable-length structured data such as JSON arrays.

Why Use Sub-Workflows?

When dealing with dynamic lists (like search results or multiple URLs), manually duplicating logic is inefficient and error-prone. Instead, sub-workflows allow you to:

  • Reuse logic

  • Keep workflows modular and organized

  • Handle variable-length lists

  • Execute iterations in parallel or sequentially

Parent Workflow Overview

The parent workflow in this example:

  1. Accepts a topic as input

  2. Runs a Google Search block

  3. Iterates over the returned list of URLs

  4. For each URL, runs a sub-workflow to scrape and summarize content

  5. Aggregates the results and uses them to generate a long-form article

Creating the Sub-Workflow

The sub-workflow (scrape URL) should:

  • Accept a launch variable: URL

  • Use a Scrape URL block to get page content

  • Generate a summary, key takeaways, and quotes

  • Return a structured JSON object with these outputs

Example JSON output from the sub-workflow:

{
  "url": "...",
  "summary": "...",
  "takeaways": ["..."],
  "quotes": ["..."]
}

Setting Up the Run Workflow Block

In the parent workflow:

  1. Add a Run Workflow block

  2. Select the sub-workflow you created

  3. Switch mode to Run Multiple Times

  4. Under Input Data, pass the output from the Google Search block (e.g., search)

  5. Use Auto Extract or JSON Array Input:

Option 1: Auto Extract

  • Use a prompt like extract all URLs

  • Reference the extracted value as item

Option 2: JSON Array Input (Recommended for Structured Data)

  • Choose JSON Array Input

  • Use dot notation to reference: item.url

  1. Configure Execution Mode:

    • Parallel: Recommended for speed if iterations are independent

    • Sequential: Use when order matters or there's shared state

  2. Set Error Behavior:

    • Choose whether to fail on errors or ignore failed runs

    • Set retry attempts if needed

  3. Define Output Variable:

    • For example: sources (an array of all scraped summaries)

Using Sub-Workflow Output

Once all sub-workflows complete:

  • The output (sources) can be passed into a Generate Text or Generate Asset block

  • You can format this data into:

    • An HTML page

    • A long-form article with footnotes

    • A structured JSON document

Example Use Case

  • Input topic: "Future of Space Travel"

  • Google Search returns ~27 results

  • Each URL is processed in the scrape URL sub-workflow

  • Resulting summaries are aggregated and used to generate a detailed article with citations and source list

Benefits of Sub-Workflow Design

  • Scalable: Works with any number of inputs

  • Modular: Easier to maintain or reuse scrape logic

  • Flexible: You can switch sources, change formats, or reuse logic across different agents

Summary

Running sub-workflows in MindStudio allows you to:

  • Iterate over dynamic lists

  • Process and transform structured data

  • Improve workflow performance using parallel execution

  • Simplify complex builds with modular design

Use sub-workflows whenever you need to apply the same logic repeatedly to parts of a list—especially when dealing with external data, scraping, or transforming structured JSON.

Last updated