cloud_fs.cloud_fs.FileSystem

class FileSystem(path, anon=False, profile=None, **kwargs)

Bases: object

Class to abstract file location and allow file-system commands that are

Parameters

• path (str) – S3 object path or file path

• anon (bool, optional) – Whether to use anonymous credentials, by default False

• profile (str, optional) – AWS credentials profile, by default None

Methods

copy	Copy file(s) from src_path to dst_path.
cp	Copy file to given destination
exists	Check if file path exists
glob	Find all file paths matching the given pattern
isdir	Check if path is a directory
isfile	Check if path is a file
ls	List everyting under given path
mkdirs	Make desired directory and any intermediate directories
mv	Move file or all files in directory to given destination
open	Open S3 object and return a file-like object
rm	Delete file or files in given directory
size	Get file size in bytes
walk	Recursively search directory and all sub-directories

Attributes

path	File path to perform filesystem operation on

property path

File path to perform filesystem operation on

Returns

str

cp(dst, **kwargs)

Copy file to given destination

Parameters

• dst (str) – Destination path

• kwargs (dict) – kwargs for s3fs.S3FileSystem.copy

exists()

Check if file path exists

Returns

bool

isfile()

Check if path is a file

Returns

bool

isdir()

Check if path is a directory

Returns

bool

glob(**kwargs)

Find all file paths matching the given pattern

Parameters

kwargs (dict) – kwargs for s3fs.S3FileSystem.glob

Returns

list

ls()

List everyting under given path

Returns

list

mkdirs(**kwargs)

Make desired directory and any intermediate directories

Parameters

kwargs (dict) – kwargs for s3fs.S3FileSystem.mkdirs

mv(dst, **kwargs)

Move file or all files in directory to given destination

Parameters

• dst (str) – Destination path

• kwargs (dict) – kwargs for s3fs.S3FileSystem.mv

open(mode='rb', **kwargs)

Open S3 object and return a file-like object

Parameters

• mode (str) – Mode with which to open the s3 object

• kwargs (dict) – kwargs for s3fs.S3FileSystem.open

Returns

Return a file-like object from the filesystem

rm(**kwargs)

Delete file or files in given directory

Parameters

kwargs (dict) – kwargs for s3fs.S3FileSystem.rm

size()

Get file size in bytes

Returns

float

walk()

Recursively search directory and all sub-directories

Returns

• path (str) – Root path

• directory (list) – All directories in path

• file (list) – All files in path

classmethod copy(src_path, dst_path, anon=False, profile=None, **kwargs)

Copy file(s) from src_path to dst_path. Either can be local or in the cloud.

Parameters

• src_path (str) – Source path to copy file(s) from, can be local or in the cloud

• dst_path (str) – Destination path to copy file(s) to, can be local or in the cloud

• anon (bool, optional) – Whether to use anonymous credentials, by default False

• profile (str, optional) – AWS credentials profile, by default None