nsrdb.gap_fill.cloud_fill.CloudGapFill

class CloudGapFill

Bases: object

Framework to fill gaps in cloud properties.

Methods

fill_cloud_cat(category, cloud_prop, cloud_type)	Fill cloud properties of a given category (water, ice).
fill_cloud_prop(prop_name, cloud_prop, ...)	Perform full cloud property fill.
fill_cloud_type(cloud_type[, fill_flag, missing])	Fill the cloud type data.
fill_file(f_cloud, f_ancillary[, rows, ...])	Gap fill cloud properties in an h5 file.
flag_missing_properties(prop_name, ...)	Look for missing cloud properties and set fill_flag accordingly.
handle_persistent_nan(dset, cloud_prop, ...)	Handle any remaining NaN property values and warn.
log_fill_results(fill_flag)	Log fill flag results.
make_zeros(cloud_prop, cloud_type, sza[, ...])	Set clear and night cloud properties to zero

Attributes

CATS
FILL

static fill_cloud_cat(category, cloud_prop, cloud_type)

Fill cloud properties of a given category (water, ice).

Parameters:

• category (str) – ‘water’ or ‘ice’

• cloud_prop (pd.DataFrame) – DataFrame of cloud properties (single-property) with shape (time x sites). Missing values must be NaN.

• cloud_type (pd.DataFrame) – Integer cloud type data with no missing values.

Returns:

cloud_prop (pd.DataFrame) – DataFrame of cloud properties with missing values filled for cloud types in the specified category.

static make_zeros(cloud_prop, cloud_type, sza, sza_lim=89.0)

Set clear and night cloud properties to zero

Parameters:

• cloud_prop (pd.DataFrame) – DataFrame of cloud properties (single-property) with shape (time x sites).

• cloud_type (pd.DataFrame) – Integer cloud type data with no missing values.

• sza (pd.DataFrame) – DataFrame of solar zenith angle values to determine nighttime.

• sza_lim (int | float) – Value above which sza indicates nighttime (sun below horizon).

Returns:

cloud_prop (pd.DataFrame) – DataFrame of cloud properties with properties during clear and night time timesteps set to zero.

static handle_persistent_nan(dset, cloud_prop, cloud_type)

Handle any remaining NaN property values and warn.

Parameters:

• dset (str) – Name of the cloud property being filled.

• cloud_prop (pd.DataFrame) – DataFrame of cloud property values. Fill should have been attempted, this method catches any remaining NaN values.

• cloud_type (pd.DataFrame) – Integer cloud type data with no missing values.

Returns:

cloud_prop (pd.DataFrame) – DataFrame of cloud property values with no remaining NaN’s.

static log_fill_results(fill_flag)

Log fill flag results.

Parameters:

fill_flag (np.ndarray) – Integer array of flags showing what data was filled and why.

static fill_cloud_type(cloud_type, fill_flag=None, missing=-15)

Fill the cloud type data.

Parameters:

• cloud_type (pd.DataFrame) – Integer cloud type data with missing flags.

• missing (int) – Flag for missing cloud types.

• fill_flag (np.ndarray) – Integer array of flags showing what data was filled and why.

Returns:

• cloud_type (pd.DataFrame) – Integer cloud type data with missing values filled using the temporal nearest neighbor.

• fill_flag (np.ndarray) – Integer array of flags showing what data was filled and why.

static flag_missing_properties(prop_name, cloud_prop, cloud_type, sza, fill_flag)

Look for missing cloud properties and set fill_flag accordingly.

Parameters:

• prop_name (str) – Name of the cloud property being filled.

• cloud_prop (pd.DataFrame) – DataFrame of cloud property values.

• cloud_type (pd.DataFrame) – Integer cloud type data with no missing values.

• sza (pd.DataFrame) – DataFrame of solar zenith angle values to determine nighttime.

• fill_flag (np.ndarray) – Integer array of flags showing what data was filled and why.

Returns:

fill_flag (np.ndarray) – Integer array of flags with missing cloud property flags set.

classmethod fill_cloud_prop(prop_name, cloud_prop, cloud_type, sza, fill_flag=None, cloud_type_is_clean=False)

Perform full cloud property fill.

Parameters:

• prop_name (str) – Name of the cloud property being filled.

• cloud_prop (pd.DataFrame) – DataFrame of cloud property values.

• cloud_type (pd.DataFrame) – Integer cloud type data with no missing values.

• sza (pd.DataFrame) – DataFrame of solar zenith angle values to determine nighttime.

• fill_flag (np.ndarray) – Integer array of flags showing what data was filled and why.

• cloud_type_is_clean (bool) – Flag to show if cloud_type input has already been cleaned. default is False so cloud_type will be cleaned in this method

Returns:

• cloud_prop (pd.DataFrame) – DataFrame of cloud property values with no remaining NaN’s.

• fill_flag (np.ndarray) – Integer array of flags showing what data was filled and why.

classmethod fill_file(f_cloud, f_ancillary, rows=slice(None, None, None), cols=slice(None, None, None), col_chunk=None)

Gap fill cloud properties in an h5 file.

Parameters:

• f_cloud (str) – File path to a cloud file with datasets ‘cloud_type’, ‘cloud_fill_flag’, and some cloud property dataset(s) with prefix ‘cld_’.

• f_ancillary (str) – File path containing solar_zenith_angle

• rows (slice) – Subset of rows to gap fill.

• cols (slice) – Subset of columns to gap fill.

• col_chunk (None | int) – Optional chunking method to gap fill one column chunk at a time to reduce memory requirements. If provided, this should be an integer specifying how many columns to work on at one time.