sup3r.preprocessing.loaders.Loader

Contents

sup3r.preprocessing.loaders.Loader#

class Loader(file_paths, *, features='all', res_kwargs=None, chunks='auto', BaseLoader=None)[source]#

Bases: object

Parameters:

  • file_paths (str | pathlib.Path | list) – Location(s) of files to load

  • features (list | str) – Features to return in loaded dataset. If ‘all’ then all available features will be returned.

  • res_kwargs (dict) – Additional keyword arguments passed through to the BaseLoader. BaseLoader is usually xr.open_mfdataset for NETCDF files and MultiFileResourceX for H5 files.

  • chunks (dict | str | None) – Dictionary of chunk sizes to pass through to dask.array.from_array() or xr.Dataset().chunk(). Will be converted to a tuple when used in from_array(). These are the methods for H5 and NETCDF data, respectively. This argument can be “auto” in additional to a dictionary. If this is None then the data will not be chunked and instead loaded directly into memory.

  • BaseLoader (Callable) – Optional base loader update. The default for H5 files is MultiFileResourceX and for NETCDF is xarray.open_mfdataset

Override parent class to return type specific class based on source_file

Methods

Attributes

TypeSpecificClasses

Contents