Python is pretty good at calling compiled code :)

What use cases would you like to see? fsspec is also used by xarray, Intake, DVC (see other comments) and others, and supported by, e.g., pyarrow. Please share your performance concerns as issues.

You may want to make issues out of this comment. fsspec tries to provide sensible defaults and lots of options for how you cache bytes, file listings and connection objects.

I don't know if it was a question to fsspec; we have MemoryFileSystem, addressed with "memory://" URLs.

Note that this is a pip issue specifically - updating boto3 might update botocore to be incompatible with aiobotocore. Users of conda are always fine. Or you can pin your version requirements and be fine however you install.

First: hi, and thanks for all the dask :waves:

Conda - yes, because it has less strict versioning for setuptools based packages. pin-ing versions, no - because you end up with fsspec[s3fs] requiring a version incompatible with any other recent package requiring boto3.