migrate entrypoints handling to standard library importlib.metadata

[eli-schwartz]

We wrap a couple different versions of the stdlib interface, and fall back to pkg_resources for compatibility with existing use cases on very old versions of python.

For python 3.8 and on, we make sure to completely avoid external dependencies, even backports.

Bug: #577

[eli-schwartz]

@althonos I have resurrected #553 in spirit with what I hope is a universally satisfying and backwards-compatible change. What do you think? Please also see some of my other (recent) PRs which improve the testsuite. It would be great to have some of these merged so that we could have a reliable python 3.12 experience.

[MorningLightMountain713]

Can we please get these fixes merged...

[eli-schwartz]

I haven't given up on this PR by the way -- I am not aware of any issues with it though, so I'm just waiting for someone with commit access to review...

[feanil]

@althonos would it be possible to get a review of this and #590 ?

[eli-schwartz]

I remain committed to seeing this and other PRs go through. As a Linux distro maintainer I consider it my duty as a downstream repackager to do what I can to ensure upstream is healthy as well.

I do lack any ability to merge, release, or otherwise do anything to fix issues other than propose PRs, but I am happy to do that for whatever it's worth...

[eli-schwartz]

@lurch @althonos @willmcgugan,

Today's setuptools release has deleted pkg_resources entirely, and pyfilesystem2 is (as a result) completely and utterly broken as it cannot be imported.

What can I do to help you merge my PRs? :)

[borisuu]

I was about to get working on a PR for the replacement of pkg_resources. I was pleasantly surprised someone else has already done the work, so let's get this merged and keep this project up-to-date.

[shayneoneill]

Any updates on this. Is there a verson of setuptools I can downgrade to in the interim as its blocking me pretty hard at work :(

[shayneoneill]

Ok. For anyone looking for this, there is a working version here: https://github.com/milahu/pyfilesystem2/tree/pkg-resources-part-1-and-2

You'll need to mangle your requirements.txt. / Pipfile / etc to point to that specifically.

Hopefully the maintainer of the official version will merge these fixes in.

[edgarrmondragon]

You'll need to mangle your requirements.txt. / Pipfile / etc to point to that specifically.

You could also just install it from this PR:

uv add git+https://github.com/PyFilesystem/pyfilesystem2@refs/pull/589/head

[yuukigogo000-ai]

Thanks for these — I hit the pkg_resources breakage and worked through #589 and #590 together. They do fix the import. One thing I ran into that seems worth catching before this lands:

Registry.protocols stops listing third-party openers on Python 3.10+.

The 3.10+ branch passes name straight through:

def entrypoints(group, name=None):
    ep = importlib.metadata.entry_points(group=group, name=name)
    return tuple(n for n in ep)

Registry.protocols calls entrypoints("fs.opener") without a name, so name=None reaches the stdlib. EntryPoint.matches compares every keyword with operator.eq, so name=None matches nothing:

>>> import importlib.metadata as im
>>> len(list(im.entry_points(group="console_scripts")))
18
>>> len(list(im.entry_points(group="console_scripts", name=None)))
0

The 3.8/3.9 branch is unaffected — if name: guards it there. Only the 3.10+ branch.

Reproduction

master (77a8562) + #590 + #589, built into a wheel and installed into a clean venv, with one third-party opener distribution on the path:

master, pkg_resources path (setuptools<81):
  "probe589" in registry.protocols  ->  True    (15 protocols)

+ #590 + #589:
  "probe589" in registry.protocols  ->  False   (14 protocols)
  registry.get_opener("probe589")   ->  ProbeOpener     # still works
  fs.open_fs("probe589://")         ->  works           # still works

That is what makes it easy to miss: opening a third-party URL is completely unaffected, so nothing looks broken in normal use. Only enumeration is wrong — including repr(registry), which formats self.protocols. Code that does if proto in registry.protocols as an input guard will reject a protocol that would have opened fine.

fs-s3fs reproduces it the same way if you prefer a real plugin.

Why the existing test does not catch it

test_registry_protocols was written for exactly this behaviour, but it patches importlib.metadata.entry_points with a MagicMock that returns the extensions regardless of the arguments it is called with, so the name=None filtering never happens inside the test.

Suggested fix

Select on the group only and filter by name in Python:

def entrypoints(group, name=None):
    eps = importlib.metadata.entry_points(group=group)
    return tuple(ep for ep in eps if name is None or ep.name == name)

That leaves get_opener behaviour identical and restores protocols.

What I actually tested

CPython 3.12.10 on Windows, against #589 at 136e7e2 and #590 at c009996, cherry-picked onto master 77a8562 (both applied cleanly), built as a wheel and installed into a fresh venv, then exercised from outside the source tree.

I did not test 3.10 or 3.11 — I only have 3.12 here. It is the same code path, and importlib_metadata 4.8.3 / 6.0.0 / 7.0.0 / 8.0.0 all return nothing for name=None, so I expect them to behave the same, but I have not run them.

One unrelated note for whoever merges this: #589 removes import pkg_resources from tests/test_opener.py but test_repr still calls it, so tests/test_opener.py needs #588 applied alongside #589 or it fails with NameError.