Serverless for Science: Running Long-Lived BioXAI Workloads Without Long-Lived Infrastructure

The experience we want: start a job, keep working, get the result.

Most web requests finish in seconds. You open a page, ask for something, and get a response while the connection is still open.

Bio AI work has a different sense of time. A model may need minutes or hours to process a set of molecules, proteins, or cells. One experiment may split into many pieces of work, some on CPUs and others on GPUs. Those pieces may finish at different times. A few may fail even when the rest succeed.

The scientist should not have to manage any of that.

The experience should still be simple: start one job, continue working, and receive understandable results when they are ready. There should be no machine to reserve, no cluster to watch, and no need to translate a scientific question into an infrastructure project.

Serverless compute is only part of the answer. Compute that starts when needed and disappears afterward is useful, but a long job has to stay understandable while the machines doing the work come and go.

One job is rarely one request

The shape of a real pipeline. Each compute stage fans out across every sequence, and independent methods run in parallel rather than one after another.

Imagine a scientist submits a collection of candidates for analysis. To the user, that is one experiment. Underneath, it may become dozens or hundreds of tasks.

Some tasks are independent and can run side by side. Others depend on data prepared by an earlier step. One candidate may finish quickly while another needs much longer. The work may also need different kinds of hardware at different stages.

Our own pipelines look like this. A shared stage prepares something every later step needs, and then several independent methods evaluate the same inputs. Because those methods do not depend on each other, they run as parallel branches of the same workflow, and the pipeline continues only once both report back. Within each branch, every sequence is dispatched independently rather than one after another. Very little of that shape is a straight line.

The compute is also expensive. Idle GPUs waste money, a single fixed setup wastes capacity on the stages that don't need it, and scientists can't be expected to size infrastructure before the experiment starts.

A task produces prepared inputs, intermediate files, status updates, execution details, and final outputs. If those records live only inside the machine doing the calculation, they disappear when it stops.

If 98 pieces finish and two don't, a single red light doesn't describe the job. We need to know what completed, what failed, and whether the failed pieces can be tried again without repeating the successful work.

So the job and the compute tasks are separate things: the job is durable, the workers are replaceable.

Split the job from the machines running it

Our architecture separates three responsibilities: orchestration, compute, and persistence. Each has one job.

First, the job needs a durable coordinator. AWS Step Functions / AWS Durable Lambdas fill that role: it records where the job is in its journey and decides what should happen next. It can dispatch work, wait for it to progress, evaluate the outcome, collect results, and move into a clear success or failure path.

The model computation needs CPU or GPU workers that can come and go. We run that work in Modal Functions / AWS SageMaker, which packages it as containerized workloads. The workflow can hand work to those functions without holding one web request open for the entire calculation, then continue when an outcome is available.

Modal manages a pool of containers for each function. It can add containers when demand rises and remove idle capacity when demand falls, within the limits we configure. That makes the compute layer elastic: a small job does not need the infrastructure of a large one, and a larger batch can use parallel capacity without someone preparing a permanent fleet first.

The job needs memory that outlives any worker. DB / Blob provides that durable state, holding the information the workflow needs to understand the job along with inputs, outputs, and execution metadata. Because that state is outside the compute containers, a worker can finish, fail, or disappear without taking the job's memory with it.

Some workloads also need a normal shared filesystem while several compute tasks are working. For those cases, we use Modal Volumes / S3 as temporary shared working data. A Volume can make filesystem data available beyond one container invocation and across functions. In our architecture, it is a workspace, not the source of truth for the whole job. Durable job state and results remain in DB / Blob, and temporary data is cleaned up when it is no longer needed.

Serverless scales much faster after you expose parallelism

The work still has to be broken into pieces that can run at the same time, and nothing in a serverless platform will do that for you. If a pipeline processes a hundred sequences by looping over them, moving it onto elastic infrastructure just means it loops over them somewhere else.

Most of the work went into input preparation. Before any model runs, the pipeline turns one submitted file into individually addressable units of work. Each sequence gets its own inputs, its own output location, and its own entry in the job record. From that point on, every compute stage can dispatch all of them at once and let the compute layer decide how many containers to run.

The workflow does not hold one execution per sequence. It writes one status record, dispatches the whole set, and polls that single record until the set is done. Where the science allows it, whole stages run side by side too, so independent methods become parallel branches rather than consecutive steps.

The effect shows up in wall-clock time.

On ConvergeAB, 1,000 sequences and 10,000 sequences both run in about 8.5 hours end to end. Ten times the input, the same wall clock, and the supported ceiling moved from 1,000 to 10,000 in one refactor. The models did not get faster, not one of them changed. The pipeline stopped serializing work that never needed to be serialized.

The supported ceiling moved from 1,000 sequences to 10,000 in a single refactor while compute time hasn’t changed.

Writing in Forbes, Dara-Abasi Ita reported on an improved version of cetuximab produced "in eight hours from a single prompt”. (More information in our internal Case Study).

The models did not get faster. Not one of them changed. What changed is that the pipeline stopped serializing work that never needed to be serial. Once every unit is independent and dispatched together, total time is governed by the slowest unit plus scheduling overhead.

Flat runtime isn't free or infinite. It holds as long as there's capacity to absorb the fan-out, and it costs whatever that capacity costs. A bigger experiment becomes a budget decision instead of a queue.

The result is boring, in the best possible way

From the scientist's desk, it's uneventful. Start a job, work on something else, get the results.

Behind that simple contract, the workflow may coordinate long-running CPU and GPU tasks, preserve intermediate progress, recover selected work, clean up temporary resources, and assemble outputs. Those are platform responsibilities, not prerequisites for asking a biological question.