SIGN IN SIGN UP

Max Executor Threads (#566)

# Overview

For one of my services, we use `async DBOS.scheduled workflows` with a
high frequency of every 30s for about 10 different jobs, and gradually
the Python process keeps accumulating memory (even if there are no
spikes in concurrent workflows being run).

# Details

After forking the repository, I noticed that the `max_workers` was set
to `sys.maxsize`, which on 64-bit arch platforms is
`9223372036854775807`. I'm not sure if this is intentional, but
`ThreadPoolExecutors` default `min(32, (os.process_cpu_count() or 1) +
4)` is much safer to prevent large memory consumption over time (since
`ThreadPoolExecutor` doesn't free idle threads until the executor is
destroyed).

# Solution

I removed the fixed `sys.maxsize` for the number of workers and added a
configurable parameter called `max_executor_threads` which, if
specified, is used; else the default value for `ThreadPoolExecutor` is
used.

## Note 
I haven't had much time to dig into the repo, but I believe the root
cause of the gradual increase in threads is that some references are
lingering, preventing idle threads from being utilized by the
`ThreadPoolExecutor`, particularly when using async scheduled workflows.
This leads the executor to spawn new threads periodically.
M
Mahit Mehta committed
c68119a18eedb2a252a4d0044fec130ed26845c5
Parent: c1ba8e3
Committed by GitHub <noreply@github.com> on 1/26/2026, 5:52:40 PM