How to run a sub-workflow asynchronously

#1

Hello everyone,

How does Corteza enable us to execute a workflow asynchronously?

#2

Hi!

Would this answer your questions – Parallelism :: Corteza Docs?

In short, you’d wanna use a fork/join gateway in order to execute both branches at the same time.
Keep in mind, if you wish to parallelise sections and resume normal exexution, you need to connect all the parallel branches to the fork join gateway like so:

image

#3

Hello @tjerman

Thanks for your help.
This is the feature I was looking for.
I applied it and it works.

1 Like
#4

Hello @tjerman

What about queue message send ?

1 Like
#5

I’m not sure I understand the question.

Are you asking about message queues to run things in parallel? You oculd take a peek at this resource

So in short, you’d define a message queue in the Admin webapp, define a workflow that consumes messages on the given message queue and update the main workflow that initializes processing.

The downside is that we currently don’t provide an elegant way to synchronise the different workflows to notify once work has been done (so you need to do some magic in order to block the main workflow until all the parallel bits were completed).

What I’d suggest you do is:

  1. Define a sub-workflow which encapsulates all the computational logic that will be done in parallel
  2. Use a fork gateway to invoke the sub workflow for processing; use as many branches as you want (this would denote the amount of parallel workers)
  3. Use a join gateway to resume the main workflow

Now, in case this scenario is too optimistic, I suggest the following (considering you probably want to process an arbitrary amount of work where you’d generally use a loop):

  1. In the Admin webapp, define a message queue
  2. In your namespace, create a new module called process_sync (or similar) – we’ll use this module to keep track of parallel processing and use it as a mux of some sort.
  3. Define a workflow that processes the task you want to run in parallel. This workflow needs to create an entry in the process_sync module, and delete/update the given record after it’s completed.
  4. In your main workflow, in your iterator, run the message queue send to invoke the processor workflow
  5. In your main workflow, after the iterator is complete, run another iterator with a delay which looks at the process_sync module if all the processes are finished

Here are a few more options I’ve seen people do/I’d consider:

  • If you have a processing-heavy bit of work, consider setting up Corredor and processing it with an automation script
  • Define a function on some cloud provider, invoke it from the workflow

I hope this is what you were asking about; if not, let me know and I’ll send over another wall of text :sweat_smile: