Skip to content

forkserver requires a writeable temp file system #155717

Description

@jankatins

Bug report

Bug description:

We run containers in our k8s without any writeable filesystem (readOnlyRootFilesystem=True), including no writeable temp file system.

The following code therefore raises:

>>> tempfile._get_default_tempdir()
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    tempfile._get_default_tempdir()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.14/tempfile.py", line 222, in _get_default_tempdir
    raise FileNotFoundError(_errno.ENOENT,
                            "No usable temporary directory found in %s" %
                            dirlist)
FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/']

Unfortunately, multiprocessing defaults to forkserver (in this case started by granian, see emmett-framework/granian#905):

_concrete_contexts = {
'fork': ForkContext(),
'spawn': SpawnContext(),
}
if reduction.HAVE_SEND_HANDLE:
_concrete_contexts['forkserver'] = ForkServerContext()
# bpo-33725: running arbitrary code after fork() is no longer reliable
# on macOS since macOS 10.14 (Mojave). Use spawn by default instead.
# gh-84559: We changed everyones default to a thread safeish one in 3.14.
if reduction.HAVE_SEND_HANDLE and sys.platform != 'darwin':
_default_context = DefaultContext(_concrete_contexts['forkserver'])
else:
_default_context = DefaultContext(_concrete_contexts['spawn'])

And then fails if it started on such a read only filesystem.

[INFO] Starting granian (main PID: 1)
[INFO] Listening at: http://0.0.0.0:8009
Traceback (most recent call last):
  File "/opt/app/venv/bin/granian", line 10, in <module>
    sys.exit(cli.entrypoint())
             ~~~~~~~~~~~~~~^^
  File "/opt/app/venv/lib/python3.14/site-packages/granian/cli.py", line 606, in entrypoint
    cli(auto_envvar_prefix='GRANIAN')
    ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app/venv/lib/python3.14/site-packages/click/core.py", line 1524, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/opt/app/venv/lib/python3.14/site-packages/click/core.py", line 1445, in main
    rv = self.invoke(ctx)
  File "/opt/app/venv/lib/python3.14/site-packages/click/core.py", line 1308, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app/venv/lib/python3.14/site-packages/click/core.py", line 877, in invoke
    return callback(*args, **kwargs)
  File "/opt/app/venv/lib/python3.14/site-packages/granian/cli.py", line 600, in cli
    server.serve()
    ~~~~~~~~~~~~^^
  File "/opt/app/venv/lib/python3.14/site-packages/granian/server/mp.py", line 513, in serve
    super().serve(spawn_target, target_loader, wrap_loader)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app/venv/lib/python3.14/site-packages/granian/server/common.py", line 760, in serve
    serve_method(spawn_target, target_loader)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app/venv/lib/python3.14/site-packages/granian/server/common.py", line 588, in _serve
    self.startup(spawn_target, target_loader)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app/venv/lib/python3.14/site-packages/granian/server/common.py", line 494, in startup
    self._spawn_workers(spawn_target, target_loader)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app/venv/lib/python3.14/site-packages/granian/server/common.py", line 346, in _spawn_workers
    wrk.start()
    ~~~~~~~~~^^
  File "/opt/app/venv/lib/python3.14/site-packages/granian/server/common.py", line 72, in start
    self.inner.start()
    ~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.14/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
                  ~~~~~~~~~~~^^^^^^
  File "/usr/lib/python3.14/multiprocessing/context.py", line 306, in _Popen
    return Popen(process_obj)
  File "/usr/lib/python3.14/multiprocessing/popen_forkserver.py", line 35, in __init__
    super().__init__(process_obj)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/usr/lib/python3.14/multiprocessing/popen_fork.py", line 20, in __init__
    self._launch(process_obj)
    ~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/usr/lib/python3.14/multiprocessing/popen_forkserver.py", line 51, in _launch
    self.sentinel, w = forkserver.connect_to_new_process(self._fds)
                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/usr/lib/python3.14/multiprocessing/forkserver.py", line 89, in connect_to_new_process
    self.ensure_running()
    ~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.14/multiprocessing/forkserver.py", line 166, in ensure_running
    address = connection.arbitrary_address('AF_UNIX')
  File "/usr/lib/python3.14/multiprocessing/connection.py", line 82, in arbitrary_address
    return os.path.join(util.get_temp_dir(),
                        ~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.14/multiprocessing/util.py", line 216, in get_temp_dir
    base_tempdir = _get_base_temp_dir(tempfile)
  File "/usr/lib/python3.14/multiprocessing/util.py", line 171, in _get_base_temp_dir
    base_tempdir = tempfile.gettempdir()
  File "/usr/lib/python3.14/tempfile.py", line 312, in gettempdir
    return _os.fsdecode(_gettempdir())
                        ~~~~~~~~~~~^^
  File "/usr/lib/python3.14/tempfile.py", line 305, in _gettempdir
    tempdir = _get_default_tempdir()
  File "/usr/lib/python3.14/tempfile.py", line 222, in _get_default_tempdir
    raise FileNotFoundError(_errno.ENOENT,
                            "No usable temporary directory found in %s" %
                            dirlist)
FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/workdir']

Given that it already checks for some other problems (the SEND_HANDLE) and defaults to spawn if there are problems, it would be nice to also check for read only tempfiles and default to spawn in that case.

e.g. something long the lines of

try:
     tempfile.gettempdir()
     CAN_RUN_FORKERSERVER = True
except FileNotFoundError: # read only filesystem
    CAN_RUN_FORKERSERVER = False

And then use that similarly to HAVE_SEND_HANDLE.

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Metadata

Metadata

Assignees

No one assigned

    Labels

    stdlibStandard Library Python modules in the Lib/ directorytopic-multiprocessingtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions