Files

file utility functions

pado.io.files.find_files(urlpath, *, glob='**/*', storage_options=None)

iterate over the files with matching at all paths

pado.io.files.is_fsspec_open_file_like(obj)

test if an object is like a fsspec.core.OpenFile instance

Return type:

TypeGuard[OpenFileLike]

pado.io.files.update_fs_storage_options(fs, *, storage_options)

update the storage_options of an existing filesystem

pado.io.files.urlpathlike_local_via_fs(obj, fs)

take an urlpath and access it via another fs

Return type:

OpenFileLike

pado.io.files.urlpathlike_get_fs_cls(obj)

get the urlpathlike filesystem class

pado.io.files.urlpathlike_get_path(obj, *, fs_cls=None)

get the path from the urlpath

pado.io.files.urlpathlike_get_storage_args_options(obj)

get the path from the urlpath

pado.io.files.urlpathlike_is_localfile(urlpath, must_exist=True)

Check whether an urlpath corresponds to a file on a local filesystem.

Parameters:

• urlpath (UrlpathLike) – URL to be checked

• must_exist (bool, optional) – If True, the file must exist on the local machine’s filesystem. Default is True.

Returns:

True if urlpath corresponds to a local file, False otherwise

Return type:

bool

pado.io.files.urlpathlike_to_string(urlpath, *, ignore_options=())

convert an urlpath-like object and stringify it

Return type:

str

pado.io.files.urlpathlike_to_fsspec(obj, *, mode='rb', storage_options=None)

use an urlpath-like object and return an fsspec.core.OpenFile

pado.io.files.urlpathlike_to_fs_and_path(obj, *, storage_options=None)

use an urlpath-like object and return an fsspec.AbstractFileSystem and a path

pado.io.files.urlpathlike_to_path_parts(obj)

take an urlpathlike object and return the path parts

this does not instantiate the fsspec.AbstractFilesystem class. (does not open connections, etc on instantiation)

Return type:

Tuple[str, ...]

pado.io.files.urlpathlike_to_localpath(obj, *, mode='rb', storage_options=None)

take an urlpathlike object and return a local path

pado.io.files.urlpathlike_to_uri(urlpath, *, repr_fallback=False, ignore_options=())

convert an urlpath-like object to an fsspec URI

Parameters:

• urlpath (UrlpathLike) – an urlpath-like object

• repr_fallback (bool) – allow falling back to a repr like representation if encoding to a uri is impossible due to storage_args or storage_options

• ignore_options (Collection[str] | Literal[True]) – a set of options that should be omitted from the uri representation

Returns:

a URI string representation of the urlpath-like

Return type:

uri

pado.io.files.fsopen(fs, path, *, mode='rb')

small helper to support mode ‘x’ for fsspec filesystems

pado.io.files.uncompressed(file)

contextmanager for reading nested compressed files

supported formats: GZIP, LZMA, ZIP, TAR

Return type:

Iterator[IO[bytes]]