About five years ago, I wrote a blog post explaining how fibers work at a low level^[1] and how they give programmers greater control over scheduling. Since then, I have used fibers as the foundation for cooperative multithreading in both SORT, my offline renderer, and its asset build pipeline. In this post, I want to share some of the design decisions behind this system and the flexibility it provides.

If you are unfamiliar with fibers, I recommend first reading my previous blog post for an introduction to the basics. Without that background, some parts of this article may be difficult to follow.

One important disclaimer: although I worked at Naughty Dog for a few years, this article does not describe how Naughty Dog designed its job system. I did not work on that system during my time there. Even if I had, I would not be legally permitted to share any details about it. This article focuses entirely on the job system I developed for my offline renderer by myself. Any similarities between my design and Naughty Dog’s are purely coincidental.

Why Build a Job System?

Before explaining how my job system works under the hood, it is important to establish reasonable expectations for what we can gain by implementing such a system. A thorough understanding of the benefits helps programmers determine whether the development effort is justified. This type of system will not benefit every kind of software. Fiber-based job systems have been adopted in games^{[2, 3]} and simulation software^[4]. They offer the following benefits and flexibility over traditional job systems based on operating system threads.

• Cooperative multitasking
  Unlike job systems based on threads, which use preemptive multitasking, a job system based on fibers uses largely cooperative multitasking. Developers are responsible for ensuring that all threads work together correctly. This can reduce unnecessary context switches and potentially improve performance.
• Application controlled scheduling
  Unlike context switching managed by the operating system, a job system built on fibers gives developers control over the scheduling algorithm because the scheduler can observe every job transition. It can also record queue time, execution time, dependency waits, fiber suspensions, and worker utilization. This adds complexity, but the extra control and visibility are valuable in software with strict performance requirements.
• Avoiding operating system context switches
  As mentioned above, operating system context switches can be significantly more expensive. On some platforms, such as PlayStation, a job system built on fibers allows developers to pin threads to hardware cores, reducing the frequency of these context switches. They may still occur because of IO threads that perform I/O, which I will discuss later in this article. Even so, pinning worker threads can greatly reduce both the overall frequency of context switches and the number of unnecessary ones.
• Low-cost suspension and resumption
  When a job must wait for a dependency, it can suspend its fiber while the worker thread executes another ready fiber. Switching between fibers is generally much cheaper than blocking a worker thread and relying on the operating system to schedule another thread. Once the dependency is satisfied, the job can resume from where it paused, with its call stack intact.
• Explicit memory control
  A custom fiber implementation can use application-allocated stack memory on Windows, Linux, and macOS. Platform-provided APIs offer different levels of control: Windows’ built-in thread and fiber APIs let applications specify stack sizes but allocate the memory themselves, while POSIX threads can use application-allocated stacks. Control over stack allocation is therefore a benefit of particular implementations and APIs, rather than an inherent advantage of fibers over threads.
• More transparent synchronization primitives
  Jobs should use synchronization primitives that suspend the waiting fiber and let the worker thread execute other ready jobs. Blocking operating system primitives would instead block the worker thread, preventing it from doing useful work while the job waits. Even worse, native synchronization primitives can also rely on thread ownership or thread-local state, making them unsafe to hold across a fiber suspension if the fiber later resumes on a different thread. Implementing fiber-aware synchronization primitives adds complexity, but gives developers control over their behavior and allows them to tailor synchronization to the job system.

It is worth noting that a job system built on fibers also comes with several costs. These benefits are not free.

• Limited debugging support
  Visual Studio displays paused threads in its Parallel Stacks debugger view, but it does not display suspended fibers. Developers can build a debugger extension to expose them, as discussed later in this article.
• Subtle low level bugs
  Some low level bugs can be particularly difficult to diagnose, and developers may occasionally need to inspect assembly code. Recent advances in agentic coding have reduced this burden considerably.
• Custom synchronization primitives
  Operating system synchronization primitives may no longer be usable, requiring developers to implement the primitives they need. This creates additional work, but the extra control was also listed as a benefit above.
• Blocking calls require vigilance
  A blocking operating system call made from a fiber blocks the entire worker thread. With no spare worker threads available, this can leave processing capacity unused. Developers must therefore be careful to avoid such calls.
• Limited thread local storage
  It is not strictly correct to say that thread local storage is unsupported, as I will discuss in detail later. However, to improve load balancing, a job system may move fibers between threads and therefore discourage or prohibit its use. Even without fiber over thread migration, TLS may still be unsafe. This poses a major challenge when migrating a large engine built around a traditional job system that relies on threads. Adopting a job system built on fibers is therefore much easier when planned from the beginning.
• Portability
  Fiber implementations depend heavily on the platform ABI. Different CPU architectures use different ABIs, so each one may require a separate fiber implementation. Supporting a wide range of CPU architectures therefore adds the burden of implementing and maintaining fiber support for every relevant ABI.
• Potential memory leaks
  If your system relies heavily on smart pointers to prevent memory leaks, a fiber based job system can still introduce a legitimate risk, even if all heap allocations are managed by smart pointers. Simply deallocating a fiber’s stack does not run the destructors of objects that are still alive on it. If the fiber is not unwound correctly before it is destroyed, smart pointers and other objects that own resources may never release them.

Now that we have covered most of the major benefits and drawbacks, it should be easier to decide whether a job system built on fibers is suitable for your software. I find these benefits particularly attractive for several features that I want to implement in my offline renderer. Here are a few examples:

• In my texture cache system, when a texture cache miss occurs, the job system can switch to another fiber. Because the renderer knows which texture each suspended fiber is waiting for, it can resume only fibers whose textures are ready, avoiding another immediate yield. Operating system level multithreading cannot offer the application the same degree of scheduling control.
• Context switches are lightweight. Because switching between fibers is inexpensive, I can use context switches more freely. This makes nested parallelism much easier. As we will see later, my job system allows a fiber to yield at any point in the call stack, including from recursive calls.
• I can adjust the amount of memory allocated to fiber stacks based on the capabilities of the target machine. For example, on a machine with limited memory, I can reduce the number of fibers to lower peak memory demand. Using a uniform stack size also allows me to preallocate a large memory pool without fragmentation between fiber stacks.
• Because all low level components are implemented consistently and remain entirely under my control, I can profile the renderer more effectively than before. By contrast, multithreading behavior at the operating system level is relatively opaque and may vary across platforms, which is legit concern.

In addition to all the benefits, building the system myself also gives me a valuable opportunity to learn how a job system based on fibers works. With these benefits in mind, I ultimately decided against using an existing job system library, such as Intel TBB^[5], to drive my renderer and asset build system.

If any of the points above seem confusing, don’t feel discouraged. I will walk through the details of my fiber based job system implementation to make each of these points clear.

The Programming Model

Before we discuss the implementation in detail, let’s look at the programming model of SORT’s fiber based job system. My goal is to keep the interface simple enough that high level user code can take advantage of the system with minimal interaction. The high level code explicitly submits work and waits for dependencies, while the scheduler handles fiber suspension and resumption behind those operations. This allows a function to retain an ordinary sequence of calls even when some of those calls need to wait for work elsewhere in the system.

The following example shows how to schedule a job to run asynchronously.

1float data[16];
2sort_job::sort_sync::WaitGroup jobs_done(16);
3for(int i = 0; i < 16; ++i) {
4    sort_job::schedule_parallel([&data, &jobs_done, i]() {
5        data[i] = (float)i;
6        jobs_done.Signal();
7    });
8}
9jobs_done.Wait();

Listing 1 simply fills an array of floats, setting each element to its index. This is a rather silly use case given how little work each job does, but it provides a simple example of how the job system works in SORT. This seemingly simple code triggers quite a bit of logic inside the scheduler, which I will discuss in detail later. For now, from a high level perspective, the code spawns sixteen independent jobs that can run in parallel with one another.

What is nice about this is that high level user code can schedule a job without worrying about classes such as Job or Fiber, which are hidden under the hood. This greatly reduces the complexity of the calling code and improves readability.

When designing the job system, I try to consider as many use cases as I can. This naturally leads to another use case: spawning a job inside another job. SORT’s job system makes this straightforward:

 1LoadScene();
 2
 3schedule_parallel([]() {
 4    PrepareScene();
 5
 6    // Once the scene is ready, submit another job to render it.
 7    schedule_parallel([]() {
 8        RenderScene();
 9    });
10});

Again, this example does not represent an actual scene rendering workflow, but it illustrates the flexibility of the job system. Recursive scheduling is essential in SORT to avoid restricting parallelism in the renderer.

Now that we have seen two use cases for scheduling asynchronous jobs, let’s look at how a job can yield, which is the most important part of a fiber based job system.

1PrepareScene();
2
3sort_job::yield();
4
5RenderScene();

In this case, the job yields after preparing the scene and can resume later. This yield also serves no practical purpose because it does not wait for anything, but it illustrates the idea of yielding. In practice, SORT does not expose this yield() interface, though it could easily be implemented. All yielding within a job happens through synchronization primitives. Below is one example of many.

 1sort_job::sort_sync::WaitGroup scene_ready(1);
 2
 3sort_job::schedule_parallel([&scene_ready]() {
 4    PrepareScene();
 5    scene_ready.Signal();
 6});
 7
 8// This cost is overlapped with the work in `PrepareScene`.
 9somthing_expensive();
10
11// Yield this fiber if scene preparation is still in progress.
12scene_ready.Wait();
13
14// Scene preparation is complete when Wait() returns.
15RenderScene();

This is a more practical example of how the job system can help. The code first submits an asynchronous job to prepare the scene, then proceeds with another expensive operation as soon as the job has been submitted. If something_expensive finishes before scene preparation, the WaitGroup synchronization primitive prevents the job from proceeding to render the scene until preparation is complete.

Of course, in practice, SORT needs other synchronization primitives as well. But we’ve covered enough of the programming model for now. It is time to discuss how the system is implemented to support all of these use cases.

Implementing the Job System

Windows provides convenient APIs for creating and switching between fibers. However, neither macOS nor Linux offers an equivalent dedicated fiber API. About five years ago, I built a small open source library called tiny fiber, which lets high level code work with fibers without handling the low level details. I initially tested it on every combination of x86 and ARM64 with Windows, Linux, and macOS, except Windows on ARM64, as I never had a device to test it on. Though, given that Windows has dedicated fiber interfaces, it should be easy to port it to Windows on ARM64. I’ve been using this fiber library in SORT ever since. The core fiber implementation has remained largely unchanged since then. This library will be our starting point for discussing the job system. For anyone who needs to understand details in it, please read my previous blog post.

Even with a working fiber implementation, building a fiber based job system still requires a considerable amount of work. To explain my job system clearly, I will first introduce some of its key concepts:

• Fiber
  Fibers are the fundamental concept behind this blog post. A fiber is a lightweight execution context with its own stack that can be suspended and resumed by the application. Fibers can be thought of as cooperatively scheduled user threads. My job system uses several types of fibers, including thread fibers, background fibers, and worker fibers, each with different responsibilities. Thread fibers represent the original execution contexts of worker threads. Background fibers run the scheduling loop, while worker fibers execute jobs.
  For the record, the definition of a fiber used here is the same as the one used in most other articles^{[6, 7, 4, 8]}. However, I have also come across slightly different definitions^[9]. Throughout this article, I use “fiber” in the sense defined above.
• Thread
  A thread is the familiar virtual execution unit provided by the operating system. Threads in my job system are no different from those used elsewhere. There are two broad categories: worker threads and I/O threads. Worker threads execute jobs through fibers. I/O threads primarily execute jobs directly, handling potentially blocking operations such as I/O.
• Job
  A job is a unit of work to be processed. It usually involves enough work to justify running it asynchronously, but it can be as small as a single instruction, as in Listing 1. A job can pause during execution while waiting for other jobs it depends on. It can be resumed when its dependency is available.
• Job Context
  A job context owns a worker fiber and can be assigned a job to execute on it. It wraps the fiber with additional information, such as whether the fiber is idle or busy. This abstraction keeps the fiber implementation focused on fiber operations, with the job context holding information specific to the job system. For example, a job context tracks the fiber local storage associated with its fiber.
• Scheduler
  The scheduler coordinates job execution. It assigns new jobs to available job contexts and resumes suspended jobs when they become ready to continue.
• Incoming Job Pool
  A pool that holds newly submitted jobs until a worker thread can pick them up and execute them using an available job context. This allows the system to accept more jobs than it can execute immediately, without blocking the submitting job.
• Idle Job Context Pool
  A pool of reusable job contexts that are not currently assigned to any job. Each context retains its fiber, ready to be reused for a new job.
• Ready Job Context Pool
  A pool of job contexts whose suspended jobs are ready to resume. Each context retains its assigned job and worker fiber. Unlike the incoming job pool, this pool holds contexts for jobs that have already started. Jobs still waiting for dependencies remain with their synchronization primitives until they are ready.
• Synchronization Primitives
  Fibers need a way to communicate with one another to synchronize their execution. Any data structure that serves this purpose is considered a synchronization primitive.

These are the most important concepts used in my job system. This overview briefly introduces the purpose of each one. Later in this post, we will explore each of them in detail.

Architecture Overview

The diagram below provides a high level overview of the job system’s structure.

A regular job can be pushed to the incoming job pool at any time. Once a job is queued, a background fiber picks it up along with an idle job context containing a worker fiber. A pool of background fibers is available to run. Normally, the number of background fibers matches the number of physical CPU cores. With two-way simultaneous multithreading (SMT) enabled, each physical core supports two hardware threads^{[10, 11]}. (For the rest of this article, we count each physical core with two-way SMT enabled as two cores, without repeating this distinction.) Each worker thread, including the main thread, is converted into a thread fiber, then creates and switches to a separate background fiber. The worker fiber provides the stack memory on which the job executes. If the job waits on a synchronization primitive, its fiber yields and the primitive keeps track of its job context. Once the wait condition is satisfied, the primitive places that context in the ready job context pool. Each job context in this pool holds a paused job that a background fiber can pick up and resume.

The workflow for an I/O job is much simpler. The I/O thread has a single queue and keeps pulling jobs from it whenever they are available. Fiber aware synchronization primitives should not be used here, as all I/O jobs run on the I/O thread rather than on a worker fiber.

This is just a high level overview of the job system. Next, I will describe each part in more detail.

From Main Thread to Running Background Fibers

Every C++ program starts on the main thread, not in a fiber. Early in the program, the main thread performs the setup needed to initialize the job system. It first spawns N - 1 worker threads, where N is the number of physical cores. It also spawns an I/O thread to handle I/O jobs. The I/O thread starts pulling jobs as soon as it is spawned, while each worker thread performs a few setup steps:

• Set thread affinity. On platforms such as Windows and Linux, this allows us to restrict a thread to the physical core it is currently running on. This is ideal because we take responsibility for context switching in the job system and do not need the OS to switch threads for us under the hood, which is a waste of resources. Thread affinity only restricts where a worker thread can run. The OS can still preempt the worker thread and schedule another thread on the same core, including one from another application. Unfortunately, this feature is unavailable on some platforms, such as macOS.

• Track data specific to the thread using a global variable named g_worker_context. This variable uses thread local storage, so each thread has its own copy. It keeps track of the relevant fibers, including the background fiber, the thread fiber, and the currently running fiber. It also stores a pointer to the current job context, which is needed from time to time during job execution. For example, this tracker lets us access fiber local storage in O(1) time.

• Convert the current thread into a fiber. On Linux and macOS, this only requires creating a fiber data structure. Windows provides an opaque interface (ConvertThreadToFiberEx) for this, but I found that the RBP register remains unchanged after the call, which suggests that Windows does something similar. This is sufficient as long as fiber switching can keep track of the current stack context. Further details are explained in my previous post^[1].

• Create a background fiber once the worker thread has been converted into a thread fiber. The application can explicitly configure the background fiber’s stack size. Then immediately switch from the thread fiber to the background fiber. Separating the thread fiber from the background fiber is not strictly necessary, but it is useful for a few reasons:

  • As mentioned earlier, converting a thread into a fiber on Linux and macOS only creates a fiber data structure, leaving the current stack unchanged. Although my experiment on Windows indicates similar behavior, there is no guarantee that it will remain unchanged in the future. Using a separate background fiber guarantees that our job pulling loop runs on that fiber’s own stack.

  • Separating the thread fiber from the background fiber draws a clear distinction between thread initialization and the job pulling loop. This makes each fiber’s responsibility clearer.

  • With a dedicated background fiber, all threads, including the main thread, have identical call stacks when debugging a job. The screenshot below shows the parallel call stacks of a running SORT renderer session in Visual Studio. Apart from the I/O thread and a job system debug thread, all threads originate from a fiber call.

    

  • A dedicated background fiber lets us explicitly control its stack size.

  None of these benefits is essential, but I found the symmetric call stacks hard to resist. This design also adds little overhead to the program.
  There is one obvious drawback to this approach. Without a dedicated background fiber, we can inspect the call stack back to the start of each thread, provided the thread has not actually been converted to a fiber. With a dedicated background fiber, the original thread call stack is no longer visible once execution switches to that fiber. In Figure 2, the main function is absent from the displayed call stacks, which is unusual. Since little happens before execution switches to the background fiber, I find this limitation acceptable. Later, we’ll introduce a debugging tool that reveals these fibers’ call stacks.

• The main thread allows for more flexibility than the worker threads. In SORT’s job system, however, the main thread is converted into a thread fiber immediately after spawning all other threads and follows the same steps described above. This is a deliberate design choice to make all threads behave the same way.

Hopefully, the explanation above is clear. If not, the diagram below illustrates what happens to the call stacks of the main thread and worker threads.

With this setup complete, all physical CPU cores execute the same logic: the scheduling loop in a background fiber.

Job Queues and Scheduling

Job queuing is relatively simple. Regular jobs and I/O jobs follow a similar queuing process but use separate pools. Each pool is essentially a linked list under the hood that keeps track of incoming jobs. Jobs are stored as lambda functions, which means I need to make sure any memory that the captured variables point to or reference remains valid until the job finishes. This is generally a requirement for other job systems as well.

A mutex ensures that no consumer pulls jobs from the pool while an incoming job is being pushed to it. Using a mutex within a fiber requires care because it does not work in all cases, which will be explained later. However, using a mutex here is safe because we can guarantee that no fiber switch occurs between locking and unlocking it. A major drawback of using a mutex to protect the pool is that it may put a thread to sleep. Because we size the worker thread pool based on the number of CPU cores, this could leave CPU resources idle. In theory, though, the I/O thread could use that opportunity to run. One possible future improvement is a lock-free pool^[12] that allows producers and consumers to operate concurrently.

With job queuing implemented, the next piece of the puzzle is the logic for pulling jobs from the pools, which is handled by the job scheduling algorithm. Similarly, regular jobs and I/O jobs each have their own dedicated scheduling algorithm. Scheduling I/O jobs is relatively simple because they run on a regular thread rather than a fiber, leaving the OS responsible for scheduling the thread. For regular jobs, things are a bit more complicated. The job system itself must decide which job takes over when another finishes or is suspended. Candidates for execution include both incoming jobs and previously suspended jobs waiting to resume. The contexts of suspended jobs that are ready to resume are kept in the ready job context pool. Unlike jobs in the incoming job pool, each suspended job is associated with a job context.

SORT’s fiber based job system exposes the scheduling algorithm to the application, allowing each application to implement an algorithm tailored to its needs. A default implementation is provided for applications that do not need custom scheduling. In fact, the two major systems that currently use the fiber job system, the renderer itself and the asset build pipeline, both use the default implementation. Further customization of the job scheduling algorithm remains possible and is an area for future improvement.

The specific scheduling algorithm is less important than the ability to customize it to suit any application. The default implementation is just one of many possible approaches. Below is a graph of how the default job scheduling algorithm works.

The background fiber, where the job system applies its scheduling algorithm, does nothing more than run a scheduling loop, after which it gives control back to the thread fiber. It first tries to pull contexts from the ready job context pool. For each context that is ready to resume, it transfers control to the worker fiber stored in that context. The running fiber may finish its work or pause again while waiting for something else to become ready. In either case, control returns to the background fiber exactly where it left off. The scheduling algorithm keeps pulling contexts from the ready job context pool until it is empty.

There are three main reasons why I chose to favor paused jobs over incoming jobs. This is not guaranteed to improve performance, but in general, it makes sense to me.

• It helps reduce delays in job execution. Finishing more jobs is likely to unblock other jobs, allowing the whole dependency chain to move forward. For example, during rendering, a tile is more likely to appear sooner in the interactive preview.
• Unlike an incoming job, a paused job is more than just a lambda function. It retains a job context, including its worker fiber and stack. Favoring paused jobs makes it more likely that these contexts can be released earlier, potentially reducing peak job context usage. Each context generally occupies a fixed amount of memory, largely determined by the worker fiber stack size configured for the application. In addition to stack memory, paused jobs may have allocated fiber local storage, which incoming jobs do not yet have.
• SORT uses a texture cache. When a shader accesses a texture, the required texture block may not be in memory. In that case, an I/O job is submitted to an I/O thread to read the data from disk, and the requesting fiber is then suspended. Once the request is fulfilled, the paused job context is returned to the ready job context pool so that execution can resume. However, if the scheduling algorithm keeps favoring incoming jobs, the newly loaded texture block may be evicted before the paused job can use it, leading to cache thrashing. Texture cache thrashing can severely hurt rendering performance. Favoring paused jobs greatly reduces this risk.

After exhausting the ready job context pool, the scheduler attempts to start a new job from the incoming job pool. Each item in this queue is just a lambda function, with no execution resources assigned to it yet. To run the job, the scheduler must first acquire a job context. If an idle context is available, it reuses that context. Otherwise, it tries to create a new one. The program may limit the number of job contexts to keep peak memory usage under control. If the scheduler cannot acquire a context, it leaves the job in the incoming job pool and moves on to the next step in the scheduling loop. The hope is that more paused jobs will finish, freeing their job contexts for reuse by incoming jobs.

The rest of the loop is simple and straightforward. It first checks for queued work, including incoming jobs and paused jobs that may be ready to resume. Work may be queued even if the loop has just drained the pools, because new jobs can arrive while the background fiber continues running. If no jobs are queued or running, the scheduling loop exits. The job system can also be configured to keep the loop running indefinitely, even when there are no jobs. This could be useful for a rendering server in the future, if the project ever gets that far. When no jobs are queued and the loop needs to keep running, the scheduler waits on a condition variable, putting the current thread to sleep and allowing other threads, such as the I/O thread, to run.

Scheduling Algorithm Limitation

This is only a very barebone and simple scheduling algorithm. Even though it works fairly well for my renderer so far, it may not be sufficient for all other applications. Specifically, in certain cases, it may end up in a dead lock.

The following pseudocode demonstrates how we can use the job system to build a BVH in parallel. To be clear, this example illustrates a practical use of the job system in a renderer. It is not part of SORT’s actual BVH construction process, which is significantly more complicated because of SIMD.

 1void BuildBVH(BuildWork work) {
 2    auto [left, right] = Split(work);
 3    sort_job::sort_sync::WaitGroup children_done(2);
 4
 5    // Each child can recursively schedule more jobs.
 6    sort_job::schedule_parallel([left, &children_done]() {
 7        BuildBVH(left);
 8        children_done.Signal();
 9    });
10
11    sort_job::schedule_parallel([right, &children_done]() {
12        BuildBVH(right);
13        children_done.Signal();
14    });
15
16    // Suspend this fiber until both children finish.
17    // The worker thread can run other jobs while this fiber waits.
18    children_done.Wait();
19}

Lines 11 and 16 schedule asynchronous jobs to build the left and right subtrees, respectively. Each scheduling call returns immediately, allowing the caller to continue. We use a WaitGroup synchronization primitive to wait until both child builds have finished. WaitGroup is a custom synchronization primitive designed for the job system that supports waiting within a fiber. In this example, its counter starts at 2. Each child job calls Signal() when it finishes, and Wait() returns once both jobs have signaled. It works similarly to std::latch^[13], which was introduced in C++20, while SORT uses its own implementation with C++17.

Although this approach appears to work well, it carries a risk of deadlock. Each scheduled subtree build needs a job context with a worker fiber before it can execute. A parent job waiting for its children retains its job context until all of those children have finished. If the BVH is large enough that waiting parent jobs occupy all available job contexts, the scheduler cannot acquire a context to run the queued child jobs. Those children cannot finish, so their parents cannot finish and release their contexts either. The result is a deadlock that leaves the renderer waiting indefinitely.

In practice, SORT doesn’t limit number of job context by default. Even if it does so, a reasonable job context limitation almost never hit this corner case in practice. Leaving such a limitation in the fiber based job system in SORT is completely fine to me as it is not a generic job system built for arbitary applications, and only a few subsystems rely on it. As long as it works reasonably well, there is good value in it.

Job scheduling is a complicated topic^{[14, 15]}. The default implementation in this fiber based job system is only a toy example. However, the fiber based job system scheduler in SORT does not need to be as complex as an OS scheduler. An OS scheduler provides solutions to all kinds of applications. It needs to distribute CPU time fairly among threads, allowing more threads than physical cores to appear to run simultaneously. Fairness among jobs is not a major concern for the renderer. What matters is keeping all CPU cores busy with useful work. The goal is to choose an algorithm that meets the needs of the systems in SORT that rely on it, such as the texture cache and asset build pipeline. Although we may not match the sophistication of a mature OS scheduler, we have an advantage: access to information about the renderer’s workload that the OS does not have. This knowledge can help us achieve efficiencies that would be difficult for a general scheduling algorithm to match. For example, the job system guarantees that a fiber waiting for a texture I/O request will not resume until that request completes. The next section explains how this works.

Fiber Migration Between Threads

One important design decision is whether to maintain a ready job context pool for each thread. More specifically, do we allow fiber migration between threads, meaning that a suspended fiber can resume on a different worker thread? This choice has several important implications for the system:

• Load Balancing
  Imagine that one thread has ten suspended fibers while all other threads in the renderer are idle. If the job system allows fiber migration, those idle threads can run the jobs once they are ready to resume. Otherwise, all ten jobs must execute on their original thread, preventing them from running in parallel even when other workers are idle.
  Examples of these two approaches include Google’s Marl, which requires same-thread resumption^[16], and Naughty Dog’s job system, which allows fiber migration, as described in its GDC 2015 presentation^[2].
• Cache Locality
  Same-thread resumption may improve a suspended fiber’s cache hit rate. Resuming on a different core will likely lose some benefits of the original core’s private caches, potentially slowing execution.
  This is not a strong argument, though. If a fiber resumes only after several other jobs have finished, much of its cached data may already have been evicted. For briefly paused jobs, it may be helpful.
• Thread Local Storage
  Requiring same-thread resumption makes TLS slightly easier to use. With fiber migration, TLS accesses across a suspension can refer to different instances, potentially causing problems.

  1thread_local ObjectPool pool;
  2void job() {
  3    Object* object = pool.allocate();
  4    dependency.Wait();
  5    pool.release(object);
  6}

  In Listing 6, if the fiber migrates to a different thread, the pool used to release object on line 5 differs from the one that allocated it. This can introduce subtle bugs. One possible solution might seem to be saving the pool’s address on the fiber’s stack, as shown below:

  1thread_local ObjectPool pool;
  2void job() {
  3    ObjectPool* thread_pool = &pool
  4    Object* object = thread_pool->allocate();
  5    dependency.Wait();
  6    thread_pool->release(object);
  7}

  Without fiber migration, Listing 6 and Listing 7 are equivalent. This is what I mean by TLS being slightly easier to use. Same-thread resumption guarantees access to the same TLS instance. However, this guarantee is limited. Other fibers may access the memory managed by that instance before the suspended fiber resumes, making the data unreliable when the fiber resumes. Same-thread resumption can therefore still lead to logical or reentrancy problems, although it does not itself introduce concurrent access. In Listing 7, migration creates an additional risk: the resumed fiber may access the original thread’s pool while that thread is also using it. If the pool does not support concurrent access, this can cause a data race. It is important to be aware of this restriction.
  Many thread synchronization primitives rely on TLS internally, making them unsuitable for protecting critical sections that span a fiber suspension. This is why we need to implement our own synchronization primitives for fibers.
• Compatibility with an Existing Codebase
  Because same-thread resumption makes TLS slightly easier to use, this restriction may simplify migrating an existing system from a job system built on threads to one built on fibers. Allowing fiber migration generally introduces more corner cases to handle. The example above illustrates a typical corner case we may encounter. Either way, it is not a trivial task to do.

SORT has a relatively small codebase of roughly 60,000 lines. Since I am its only contributor, there is no communication overhead. I found little reason not to support fiber migration, even though the job system was introduced relatively late in the renderer’s development. Overall, I have found this approach well suited to many problems in the renderer, such as the texture cache system.

Suspending and Resuming Fibers

So far, we’ve discussed how we use fibers to structure the job system and how the scheduler selects jobs to run. However, we have yet to cover one key feature. We need to be able to pause a fiber during execution and switch to another fiber. This section explains how that works.

Jobs still waiting for dependencies have no shared pool, unlike newly submitted jobs in the incoming job pool and resumable jobs in the ready job context pool. This is intentional: when a synchronization primitive signals, looking up the corresponding paused fibers in a shared pool would add overhead. Even with a data structure such as a hash table that supports O(1) lookups on average, avoiding the lookup altogether is more efficient. Instead, each synchronization primitive keeps track of the fibers it pauses. For example, in Listing 5, the WaitGroup instance holds the paused job context.

The figure above walks through Listing 5 step by step. I recommend exploring each step to see how the state changes. The key details are summarized below:

1. A WaitGroup is created with its internal counter set to 2.
2. Two asynchronous jobs are scheduled, one for each child. Each schedule_parallel call simply pushes a lambda function into the incoming job pool.
3. The original BVH build job must wait for both children to finish. Waiting pauses the current fiber and returns control to the thread’s background fiber to schedule other work. Meanwhile, the WaitGroup stores the waiting job context so it can move it to the ready job context pool once the wait condition is satisfied.
4. The background fiber acquires an idle context with Worker Fiber B and switches to it to execute the left subtree job. Only the right subtree job remains in the incoming job pool.
5. Another background fiber may pick up the right subtree job and execute it on Worker Fiber C.
6. The right subtree job signals the WaitGroup just before completing, reducing its internal counter by 1.
7. The left subtree job also signals the WaitGroup just before completing, reducing the counter by 1 again. The counter now reaches zero, so the WaitGroup moves the waiting job context to the ready job context pool, where the scheduler can pick it up.
8. The scheduler picks up the previously paused job and resumes it on Worker Fiber A.
9. The original BVH build job completes and returns control to the background fiber. The job pointer in Worker Fiber A’s job context is cleared immediately after the background fiber takes control.

The flow above is just one possible execution sequence. The exact order varies between runs and is not deterministic, but several key design principles remain the same:

• Each synchronization primitive tracks paused fibers in its own pool, rather than placing them in a shared pool used by all thread schedulers. This local pool is hidden from all schedulers in the job system. This design ensures that no scheduler attempts to resume waiting fibers until they are marked as ready.
• Whenever a job pauses, its fiber always yields control to the thread’s background fiber. The background fiber acts as a gateway between jobs. Although control may appear to pass directly from one worker fiber to another, it always goes through the background fiber.
• The final signal satisfies the wait condition and automatically moves the waiting job contexts to the ready job context pool.
• The scheduler owns the job context’s memory, while the synchronization primitive holds only a pointer to it. In this example, the synchronization primitive lives on the fiber’s stack and remains alive while the fiber is paused.
• In SORT’s job system, fibers are never deleted. When a job completes, its job context, including the fiber and its stack memory, returns to the idle job context pool for reuse by incoming jobs, as described earlier.

Fiber Local Storage

Although each fiber has its own stack, the renderer still needs to allocate small blocks of heap memory from time to time. Most of these allocations are needed only while a job processes an item, such as a pixel. For example, SORT’s Shading Language (SSL)^[17] may need to allocate memory for closures during shader execution. These dynamically created structures are not a natural fit for stack allocation. A straightforward solution is to use the global heap allocator and free the memory once the item has been processed, but this can be expensive. SORT already uses an arena allocator^[18] for each item evaluation, so providing access to that arena is an important part of the design.

Thread local storage can be used as job specific memory for thread based job system. In our case, we need its equilant item, fiber local storage (FLS). Because fiber is a programmer level concept, compiler won’t be able to identify it with ease, making it difficult to support FLS at language level. However, libraries such as Boost.Fiber provide fiber-local storage^[19]. Since I have my own fiber library implemented by myself, this is not an option for me.

I eventually chose to implement my own interface for FLS. There are two major reasons of it:

• Windows has its own dedicated interface to support FLS as well^[20]. The same doesn’t apply where I had to implement my own fiber implementation with assembly language. To make sure FLS allocation behavior is consistent across different Operating Systems, it is necessary to come up with custom implementation.
• Even if all platforms support FLS, this may not be our real demand either. As mentioned earlier, a fiber can be and will be reused by other jobs in the future. With FLS, these memory from one job may leak to another job, which is mostly useless data, leading to waste of resources. If we think carefully about it, what we really need is job local storage, rather than FLS.

To support this, I attached an arena allocator to each job context. The job system lets each running job access its current job context through the previously mentioned TLS variable g_worker_context, giving it access to an allocator dedicated to that job. The arena is reset when the job finishes, before the job context returns to the idle job context pool. The following function lets each job allocate memory whose lifetime is limited to that job. (The function name is slightly misleading: it allocates job-local storage rather than fiber-local storage.)

1template<typename StructType>
2static StructType* AllocateFiberLocalStorage(uint32 item_cnt = 1) {
3    if (auto fc = g_worker_context.current_job_context) {
4        auto memory = fc->allocator.Allocate<StructType>(item_cnt);
5        return (StructType*) new ((char*)memory) char[sizeof(StructType)];
6    }
7    return nullptr;
8}

Ending the Background Fiber

The background scheduler exits its scheduling loop only when no jobs remain, including running and paused jobs. Running jobs must finish because they may spawn additional jobs. Only then can the background fiber switch back to the thread fiber that originally entered it. This is intentionally the only point at which control can return to the thread fiber.

1worker.background_fiber = createFiber(
2    m_config.worker_fiber_stack_size, [this]() {
3        SchedulingPolicy::Run(*this);
4        GetThreadWorkerContext().current_job_context = nullptr;
5        switchToFiber(GetThreadWorkerContext().thread_fiber);
6    });

Smart pointers help prevent memory leaks by releasing owned resources when their destructors run. However, freeing a suspended fiber’s stack does not automatically run the destructors of objects on that stack. The purpose of making sure the last thing we do in the back ground fiber is simply to make sure all local memory will deallocate themselves properly. In Listing 9, since the switch is the fiber’s final operation and there are no local variables to destroy, it is guranteed that smart pointers will do their job as long as we properly shutdown all worker thread in our job system.

Synchronization Primitives

As mentioned earlier, the limitations of TLS when used with fibers mean that native synchronization primitives require care in a fiber based job system.

• Using these primitives across fiber suspension and resumption is risky unless we fully understand their behavior on every target platform. Even then, future implementation changes could break compatibility. As a rule of thumb, prefer synchronization primitives designed for the fiber job system over native ones.
• Even when their use does not span a suspension and resumption boundary, native primitives still require care. Some block the current OS thread, which can be appropriate in a thread based job system. In our fiber based system, however, there are few additional threads available to run, mainly the I/O thread. Relying on these blocking primitives shifts more scheduling back to the OS and undermines the benefits of cooperative fiber scheduling.

In practice, worker fibers primarily use custom synchronization primitives designed for fibers, with a few exceptions left for future optimization. I consider native primitives more acceptable outside worker fibers, and they are used in several places on background fibers. These uses still need further review to determine whether they are justified.

SORT is a relatively simple renderer with limited synchronization needs, so I have only needed to implement four types of synchronization primitives so far. The following sections explain each one. Their implementations are straightforward. I will focus on optimizing them only if they become a performance bottleneck. Adding more as the system grows should be also straightforward.

Spinlock

Spinlock is the simplest sychrnoiztion primitive. It doesn’t put current on-going fiber on pause at all. It keeps pulling until the lock is cleared. There is no job context pool needed for this type of primitive because of it.

 1class SpinLock {
 2public:
 3    SORT_FORCEINLINE SpinLock() = default;
 4    SpinLock(const SpinLock&) = delete;
 5    SpinLock& operator=(const SpinLock&) = delete;
 6
 7    SORT_FORCEINLINE void Lock() {
 8        while (m_locked.test_and_set(std::memory_order_acquire)) {
 9            #if defined(SORT_X64_TARGET)
10                _mm_pause();
11            #elif defined(SORT_ARM64_TARGET)
12                __yield();
13            #endif
14        }
15    }
16
17    SORT_FORCEINLINE void Unlock() {
18        m_locked.clear(std::memory_order_release);
19    }
20
21private:
22    std::atomic_flag m_locked = ATOMIC_FLAG_INIT;
23};

Above is its complete implementation in SORT. The code is pretty self-explanatory. A few details worth mentioning

• If memory_order_acquire and memory_order_release are unfamiliar, they specify memory ordering guarantees. In this spinlock, acquire ordering keeps subsequent memory accesses ordered after successful lock acquisition, while release ordering keeps preceding accesses ordered before unlocking. Compilers and CPUs may reorder operations to improve performance while preserving the required behavior within a single thread. Communication between threads, however, requires explicit synchronization. When a successful acquire observes the flag cleared by a release, the previous owner’s writes are ordered before the new owner’s subsequent reads. These are guarantees, not merely compiler hints, and they provide the necessary synchronization without disabling out-of-order execution entirely.
• Although _mm_pause and __yield may sound like they yield control to the OS, they provide processor hints for busy-wait loops. These hints can reduce the cost of spinning between attempts. They neither request an OS thread switch nor switch to another fiber. The current fiber retains control, and the processor determines how to handle the hint.

Mutex

A mutex is one of the simplest synchronization primitives in SORT. Like a conventional mutex, it provides exclusive ownership, but here the owner is a fiber rather than a thread. Using this data structure outside a worker fiber will likely cause a crash. If a fiber attempts to acquire a mutex that is already owned, its context is added to the mutex’s waiting queue. The fiber then pauses and yields control to the background fiber. The following code shows the lock operation.

 1void Mutex::Lock() {
 2    auto tc = SchedulerBase::GetExecutingJobContext();
 3    sAssertMsg(IS_PTR_VALID(tc), SCHEDULER, "Mutex can only be locked by a fiber job.");
 4
 5    m_lock.Lock();
 6    if (IS_PTR_INVALID(m_owner)) {
 7        m_owner = tc;
 8        m_lock.Unlock();
 9        return;
10    }
11
12    sAssertMsg(m_owner != tc, SCHEDULER, "Mutex is not recursive.");
13    sAssertMsg(!tc->is_io_context, SCHEDULER,
14               "I/O jobs cannot wait for a contended mutex.");
15    SORT_PERF_EVENT("Mutex::Lock");
16    RegisterWaiterLocked(tc);
17    m_lock.Unlock();
18
19    Park();
20}

The core logic is simple. Each mutex stores a pointer to the job context that currently owns it. A null pointer means the mutex is available. A spinlock protects the ownership check and update. If the mutex has no owner, the current fiber acquires it. Otherwise, the current fiber’s context is added to the mutex’s waiting queue before the fiber pauses.

The corresponding unlock operation is shown below.

 1void Mutex::Unlock() {
 2    [[maybe_unused]] auto tc = SchedulerBase::GetExecutingJobContext();
 3    sAssertMsg(IS_PTR_VALID(tc), SCHEDULER, "Mutex can only be unlocked by a fiber job.");
 4
 5    JobContext* next_tc = nullptr;
 6    m_lock.Lock();
 7    sAssertMsg(m_owner == tc, SCHEDULER, "Mutex can only be unlocked by its owner.");
 8    next_tc = TakeWaiterLocked();
 9    m_owner = next_tc;
10    m_lock.Unlock();
11
12    if (next_tc) {
13        SORT_PERF_EVENT("Mutex::Unlock");
14        Resume(next_tc);
15    }
16}

The unlock operation verifies that it is running in a fiber job, then acquires the spinlock and checks that the current fiber owns the mutex. Attempting to unlock a mutex without owning it triggers an assertion. I find this restriction reasonable for the renderer. std::mutex imposes a similar ownership requirement^[21]. While holding the spinlock, it takes a job context from the waiting queue and transfers ownership to it. If no fibers are waiting, the owner pointer becomes null. After releasing the spinlock, it places the selected context, if any, in the global ready job context pool so the scheduler can resume it.

Conditional Variable

The fiber aware conditional variable in SORT’s job system is different than the native conditional variable. It doesn’t always need a lock for the wait behavior. The internal spinlock serializes predicate evaluation and protects registration of waiting fibers. However, it does not automatically protect the data read by the predicate: other threads must synchronize their writes to that data as well. Without an external lock, this requires another mechanism, such as atomic variables or a lock acquired inside the predicate and by writers. The current renderer and asset-builder call sites follow these rules, so this particular race is not a concern in those usages. While native conditional lock uses the lock user provide to guard the critical section. The wait behavior without a lock is similar to std::atomic::wait in C++20^[22].

Below is the implementation of wait in the condition variable in SORT’s job system.

 1template <typename Predicate, typename Lock>
 2SORT_FORCEINLINE void ConditionVariable::Wait(Predicate&& pred, Lock& lock) {
 3    sAssert(IS_PTR_VALID(SchedulerBase::GetBound()), SCHEDULER);
 4
 5    while (true) {
 6        m_lock.Lock();
 7        if (pred()) {
 8            m_lock.Unlock();
 9            return;
10        }
11
12        auto tc = SchedulerBase::GetExecutingJobContext();
13        sAssertMsg(IS_PTR_VALID(tc) && !tc->is_io_context, SCHEDULER,
14                   "I/O jobs cannot wait on a condition variable.");
15        RegisterWaiterLocked(tc);
16        m_lock.Unlock();
17
18        lock.unlock();
19        Park();
20        lock.lock();
21    }
22}

The behavior is similar to that of a native condition variable. The caller must hold the input lock when calling this function. If the predicate is true, the function returns without pausing the fiber, and the lock remains held. Otherwise, it adds the current job context to its waiting list, releases the input lock, and yields control to the background fiber.

When another part of the program determines that the condition is met, it can notify one or all waiting fibers, making them eligible to resume. A notified fiber resumes at line 21 and attempts to reacquire the lock, which can pause the fiber again.

One useful feature of this implementation is that an external lock is optional. The caller can supply an empty lock whose lock and unlock methods do nothing. The condition variable otherwise follows the same waiting and notification logic, without acquiring or releasing an external lock.

WaitGroup

WaitGroup is useful in several parts of SORT, including the BVH build example in Listing 5. As introduced earlier, the caller initializes it with a nonnegative integer counter. A fiber that waits on the WaitGroup pauses if the counter is greater than zero. Each call to Signal() decrements the counter by one. The implementation reuses the condition variable shown in the previous section.

 1class WaitGroup : private ConditionVariable {
 2public:
 3    WaitGroup(int cnt = 0) : m_counter(cnt) {}
 4    SORT_FORCEINLINE void Signal();
 5    SORT_FORCEINLINE void Wait();
 6
 7private:
 8    std::atomic<int>   m_counter = 0;
 9};
10
11static_assert(!std::is_polymorphic_v<WaitGroup>);
12
13SORT_FORCEINLINE void WaitGroup::Signal() {
14    if (--m_counter == 0)
15        NotifyAll();
16}
17
18SORT_FORCEINLINE void WaitGroup::Wait() {
19    ConditionVariable::Wait([this] { return m_counter == 0; });
20}

The implementation is straightforward. WaitGroup hides the condition variable’s general predicate interface and specializes it for waiting until a counter reaches zero. If the counter is not zero, the condition variable pauses the calling fiber. Each signal call reduces the counter by one, the signal that brings the counter to zero notifies all waiting fibers.

Debugging and Profiling

Implementing a fiber based job system is one challenge, and debugging it is another. Fibers manipulate CPU registers directly, which can make debugging tricky. Recent advances in agentic coding have made debugging much easier, but good debugging tools remain essential for tracking down bugs.

Detect Fiber Stack Overflow

As mentioned earlier, the application determines each fiber’s stack size. However, we cannot know the renderer’s maximum stack requirements in advance. SORT executes shader logic through its own shading language, SSL, and those shaders are written during material creation, rather than renderer compile time. An unusually complex shader kernel could exceed a worker fiber’s fixed stack budget and crash the renderer.

In practice, this would require an extremely complex shader graph. Still, rare does not mean impossible. Although we cannot choose an ideal worker fiber stack size without knowing the shader kernels when compiling the renderer, we should at least report a diagnostic when a stack overflow occurs.

The Windows fiber API already provides stack overflow detection. However, our custom implementation for Linux and macOS must handle this itself. The idea is inspired by Windows’ solution^[23]: when allocating a fiber’s stack, we round the requested size up to a multiple of the virtual memory page size reported by the OS, then add one guard page to detect stack overflow, as shown in Figure 6. The guard page is inaccessible, so any attempt to read from or write to it triggers a memory protection fault. When a fiber begins executing, its stack pointer starts near the high end of the usable stack and moves toward lower addresses as the stack grows. If the stack grows into the guard page, the resulting fault gives the application a chance to capture diagnostic information and report the stack overflow.

Inspecting Suspended Fibers

Debuggers commonly provide a way to inspect each thread’s call stack, including those of blocked threads. This helps us understand what each thread is doing and is particularly useful when diagnosing deadlocks. As mentioned earlier, suspended fibers are not automatically exposed to the debugger in the same way as threads, so most tools do not display their call stacks. Without access to these call stacks, debugging a fiber based job system can be much more difficult.

To address this, I added a small debug registry to the job system and built extensions for Visual Studio and VS Code to display it. Figure 7 shows the extension running in VS Code on macOS. When fiber debugging is enabled, each fiber has a record containing its ID, state, job name, and saved call stack. This gives the debugger a way to find fibers that are otherwise invisible to it.

The key is to capture the call stack immediately before switching away from a fiber, while it is still executing. At this point, we can walk its stack and save the frame addresses for later inspection. The debugger also uses a saved register context to inspect individual stack frames. On Linux and macOS, it reuses the context already saved for fiber switching. On Windows, the debugging code captures an additional context before the switch.

When execution stops at a breakpoint, the extension reads these records from the process and resolves the saved addresses into function names and source locations. Fibers with common call paths are grouped into a graph similar to the Parallel Stacks view in Visual Studio. This is a separate view provided by the extension.

These are snapshots from the last suspension point. Once a fiber resumes, it is no longer shown in this view. A fresh stack snapshot is captured the next time it yields. Finished jobs are also removed from the stack view. Capturing a call stack on each switch adds overhead, so this feature is enabled explicitly with --debug-fibers when needed for debugging. It is compiled out of release builds by default.

Profiling Jobs

Understanding the cost of each job is important in any job system that distributes work across multiple threads. Many profiling tools used in graphics programming, such as easy profiler, can help with this. However, tools designed around thread based execution may not correctly handle jobs that suspend and resume on fibers. Although profiling tools with fiber support, such as Tracy, exist, maintaining their integration across all platforms supported by SORT would still require additional work.

For these reasons, I implemented a custom profiler that displays job execution on a timeline. Figure 8 shows it profiling SORT’s asset builder, a computationally intensive tool that converts raw assets into formats the renderer can consume. The asset builder distributes its work across multiple threads using the job system described in this article. The profiling view below shows how long each job took, making it easier to identify performance bottlenecks.

The implementation details are probably of little interest these days. It mostly comes down to recording timestamps at the right points during job execution, with nothing particularly fancy involved. With the help of agentic coding, tools like this becomes much easier to develop.

Conclusion

In this blog post, we walked through the design and implementation of SORT’s fiber based job system, from scheduling and synchronization to memory management and debugging tools.

Several high level systems in SORT rely on it, including BVH construction, shader compilation, path tracing, the texture cache, and the asset build pipeline. These systems can schedule work and wait for dependencies without managing the underlying fibers themselves.

Compared with mature libraries such as TBB, this is a relatively simple, toy design tailored to SORT’s needs. There is still room to improve scheduling and synchronization, but I am happy with what the system provides today. The extra work at a low level has made parallel code throughout SORT much easier to write. Hopefully, the examples and tradeoffs discussed here help you decide whether a similar approach would be useful in your own projects.

References