Bug report
Bug description:
We run containers in our k8s without any writeable filesystem (readOnlyRootFilesystem=True), including no writeable temp file system.
The following code therefore raises:
>>> tempfile._get_default_tempdir()
Traceback (most recent call last):
File "<python-input-1>", line 1, in <module>
tempfile._get_default_tempdir()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.14/tempfile.py", line 222, in _get_default_tempdir
raise FileNotFoundError(_errno.ENOENT,
"No usable temporary directory found in %s" %
dirlist)
FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/']
Unfortunately, multiprocessing defaults to forkserver (in this case started by granian, see emmett-framework/granian#905):
|
_concrete_contexts = { |
|
'fork': ForkContext(), |
|
'spawn': SpawnContext(), |
|
} |
|
if reduction.HAVE_SEND_HANDLE: |
|
_concrete_contexts['forkserver'] = ForkServerContext() |
|
|
|
# bpo-33725: running arbitrary code after fork() is no longer reliable |
|
# on macOS since macOS 10.14 (Mojave). Use spawn by default instead. |
|
# gh-84559: We changed everyones default to a thread safeish one in 3.14. |
|
if reduction.HAVE_SEND_HANDLE and sys.platform != 'darwin': |
|
_default_context = DefaultContext(_concrete_contexts['forkserver']) |
|
else: |
|
_default_context = DefaultContext(_concrete_contexts['spawn']) |
And then fails if it started on such a read only filesystem.
[INFO] Starting granian (main PID: 1)
[INFO] Listening at: http://0.0.0.0:8009
Traceback (most recent call last):
File "/opt/app/venv/bin/granian", line 10, in <module>
sys.exit(cli.entrypoint())
~~~~~~~~~~~~~~^^
File "/opt/app/venv/lib/python3.14/site-packages/granian/cli.py", line 606, in entrypoint
cli(auto_envvar_prefix='GRANIAN')
~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/app/venv/lib/python3.14/site-packages/click/core.py", line 1524, in __call__
return self.main(*args, **kwargs)
~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/opt/app/venv/lib/python3.14/site-packages/click/core.py", line 1445, in main
rv = self.invoke(ctx)
File "/opt/app/venv/lib/python3.14/site-packages/click/core.py", line 1308, in invoke
return ctx.invoke(self.callback, **ctx.params)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/app/venv/lib/python3.14/site-packages/click/core.py", line 877, in invoke
return callback(*args, **kwargs)
File "/opt/app/venv/lib/python3.14/site-packages/granian/cli.py", line 600, in cli
server.serve()
~~~~~~~~~~~~^^
File "/opt/app/venv/lib/python3.14/site-packages/granian/server/mp.py", line 513, in serve
super().serve(spawn_target, target_loader, wrap_loader)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/app/venv/lib/python3.14/site-packages/granian/server/common.py", line 760, in serve
serve_method(spawn_target, target_loader)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/app/venv/lib/python3.14/site-packages/granian/server/common.py", line 588, in _serve
self.startup(spawn_target, target_loader)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/app/venv/lib/python3.14/site-packages/granian/server/common.py", line 494, in startup
self._spawn_workers(spawn_target, target_loader)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/app/venv/lib/python3.14/site-packages/granian/server/common.py", line 346, in _spawn_workers
wrk.start()
~~~~~~~~~^^
File "/opt/app/venv/lib/python3.14/site-packages/granian/server/common.py", line 72, in start
self.inner.start()
~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.14/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
~~~~~~~~~~~^^^^^^
File "/usr/lib/python3.14/multiprocessing/context.py", line 306, in _Popen
return Popen(process_obj)
File "/usr/lib/python3.14/multiprocessing/popen_forkserver.py", line 35, in __init__
super().__init__(process_obj)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/usr/lib/python3.14/multiprocessing/popen_fork.py", line 20, in __init__
self._launch(process_obj)
~~~~~~~~~~~~^^^^^^^^^^^^^
File "/usr/lib/python3.14/multiprocessing/popen_forkserver.py", line 51, in _launch
self.sentinel, w = forkserver.connect_to_new_process(self._fds)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
File "/usr/lib/python3.14/multiprocessing/forkserver.py", line 89, in connect_to_new_process
self.ensure_running()
~~~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.14/multiprocessing/forkserver.py", line 166, in ensure_running
address = connection.arbitrary_address('AF_UNIX')
File "/usr/lib/python3.14/multiprocessing/connection.py", line 82, in arbitrary_address
return os.path.join(util.get_temp_dir(),
~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.14/multiprocessing/util.py", line 216, in get_temp_dir
base_tempdir = _get_base_temp_dir(tempfile)
File "/usr/lib/python3.14/multiprocessing/util.py", line 171, in _get_base_temp_dir
base_tempdir = tempfile.gettempdir()
File "/usr/lib/python3.14/tempfile.py", line 312, in gettempdir
return _os.fsdecode(_gettempdir())
~~~~~~~~~~~^^
File "/usr/lib/python3.14/tempfile.py", line 305, in _gettempdir
tempdir = _get_default_tempdir()
File "/usr/lib/python3.14/tempfile.py", line 222, in _get_default_tempdir
raise FileNotFoundError(_errno.ENOENT,
"No usable temporary directory found in %s" %
dirlist)
FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/workdir']
Given that it already checks for some other problems (the SEND_HANDLE) and defaults to spawn if there are problems, it would be nice to also check for read only tempfiles and default to spawn in that case.
e.g. something long the lines of
try:
tempfile.gettempdir()
CAN_RUN_FORKERSERVER = True
except FileNotFoundError: # read only filesystem
CAN_RUN_FORKERSERVER = False
And then use that similarly to HAVE_SEND_HANDLE.
CPython versions tested on:
3.14
Operating systems tested on:
Linux
Bug report
Bug description:
We run containers in our k8s without any writeable filesystem (readOnlyRootFilesystem=True), including no writeable temp file system.
The following code therefore raises:
Unfortunately, multiprocessing defaults to forkserver (in this case started by granian, see emmett-framework/granian#905):
cpython/Lib/multiprocessing/context.py
Lines 326 to 339 in 5d3f02a
And then fails if it started on such a read only filesystem.
Given that it already checks for some other problems (the SEND_HANDLE) and defaults to spawn if there are problems, it would be nice to also check for read only tempfiles and default to spawn in that case.
e.g. something long the lines of
And then use that similarly to
HAVE_SEND_HANDLE.CPython versions tested on:
3.14
Operating systems tested on:
Linux