What use cases would you like to see? fsspec is also used by xarray, Intake, DVC (see other comments) and others, and supported by, e.g., pyarrow. Please share your performance concerns as issues.
You may want to make issues out of this comment. fsspec tries to provide sensible defaults and lots of options for how you cache bytes, file listings and connection objects.
Note that this is a pip issue specifically - updating boto3 might update botocore to be incompatible with aiobotocore. Users of conda are always fine. Or you can pin your version requirements and be fine however you install.