nsrdb.data_model.merra.MerraVar

class MerraVar(name, var_meta, date, **kwargs)[source]

Bases: AncillaryVarHandler

Framework for MERRA source data extraction.

Parameters:

name (str) – NSRDB var name.
var_meta (str | pd.DataFrame | None) – CSV file or dataframe containing meta data for all NSRDB variables. Defaults to the NSRDB var meta csv in git repo.
date (datetime.date) – Single day to extract data for.

Methods

`pre_flight`()	Perform pre-flight checks - source file check.
`scale_data`(array)	Perform a safe data scaling operation on a source data array.
`unscale_data`(array)	Perform a safe data unscaling operation on a source data array.

Attributes

`DEFAULT_DIR`
`MERRA_ELEV`
`NN_METHOD`
`attrs`	Return a dictionary of dataset attributes for HDF5 dataset attrs.
`cache_file`	Get the nearest neighbor result cache csv file for this var.
`chunks`	Get the variable's intended storage chunk shape.
`data_source`	Get the data source.
`date`	Get the date for this handler
`date_stamp`	Get the MERRA datestamp corresponding to the current datetime date
`description`	Long variable description.
`doy`	Get the day of year for this handler
`dset_name`	Get the source dataset name for the NSRDB variable.
`dtype`	Get the data type attribute.
`elevation_correct`	Get the elevation correction preference.
`file`	Get the file path for the target NSRDB variable name based on the glob self.pattern.
`file_set`	Get the source file set name for the NSRDB variable.
`final_dtype`	Get the variable's intended storage datatype.
`grid`	Return the MERRA source coordinates with elevation.
`mask`	Get a boolean mask to locate the current variable in the meta data.
`name`	Get the NSRDB variable name.
`next_date`	Get the date after the date for this handler.
`next_date_stamp`	Get the MERRA datestamp corresponding to the next datetime date.
`next_file`	Get the file path for the date for the target NSRDB variable name based on the glob self.next_pattern.
`next_file_exists`	Check if file for next date exists
`next_pattern`	Get the source file pattern for the next date, which is sent to glob().
`pattern`	Get the source file pattern which is sent to glob().
`physical_max`	Get the variable's physical maximum value.
`physical_min`	Get the variable's physical minimum value.
`scale_factor`	Get the variable's intended storage scale factor.
`source_data`	Get single variable data from the MERRA source file for the current date and the next day if available.
`source_dir`	Get the source directory containing the variable data files.
`spatial_method`	Get the spatial interpolation method.
`temporal_method`	Get the temporal interpolation method.
`time_index`	Get the MERRA native time index.
`units`	Get the units attribute.
`var_meta`	Return the meta data for NSRDB variables.

property pattern

Get the source file pattern which is sent to glob().

Returns:: str

property next_pattern

Get the source file pattern for the next date, which is sent to glob().

Returns:: str

property date_stamp

Get the MERRA datestamp corresponding to the current datetime date

Returns:: date (str) – Date stamp that should be in the MERRA file, format is YYYYMMDD

property next_date_stamp

Get the MERRA datestamp corresponding to the next datetime date. This is used to get the file for the next date for temporal interpolation

Returns:: date (str) – Date stamp that should be in the MERRA file, format is YYYYMMDD

pre_flight()[source]

Perform pre-flight checks - source file check.

Returns:: missing (str) – Look for the source file and return the string if not found. If nothing is missing, return an empty string.

property time_index

Get the MERRA native time index.

Returns:: MERRA_time_index (pd.DatetimeIndex) – Pandas datetime index for the current day at the MERRA2 resolution (1-hour).

property source_data

Get single variable data from the MERRA source file for the current date and the next day if available.

Returns:: data (np.ndarray) – 2D numpy array (time X space) of MERRA data for the specified var.

property grid

Return the MERRA source coordinates with elevation.

It seems that all MERRA files DO NOT have the same grid.

Returns:: self._grid (pd.DataFrame) – MERRA source coordinates with elevation

property attrs

Return a dictionary of dataset attributes for HDF5 dataset attrs.

Returns:: attrs (dict) – Namespace of attributes to define the dataset.

property cache_file

Get the nearest neighbor result cache csv file for this var.

Returns:: _cache_file (False | str) – False for no caching, or a string filename (no path).

property chunks

Get the variable’s intended storage chunk shape.

Returns:: chunks (tuple) – Data storage chunk shape (row_chunk, col_chunk).

property data_source

Get the data source.

Returns:: data_source (str) – Data source.

property date

Get the date for this handler

Returns:: datetime.date

property description

Long variable description.

Returns:: description (str) – Description of the variable to provide more info than the sometimes opaque dset names.

property doy

Get the day of year for this handler

Returns:: int

property dset_name

Get the source dataset name for the NSRDB variable. This is typically the netcdf or h5 source dataset name for the variable such as T2M or TOTANGSTR (for MERRA temp and alpha)

Returns:: str

property dtype

Get the data type attribute.

Returns:: dtype (str) – Intended NSRDB disk data type.

property elevation_correct

Get the elevation correction preference.

Returns:: elevation_correct (bool) – Whether or not to use elevation correction for the current var.

property file

Get the file path for the target NSRDB variable name based on the glob self.pattern.

Returns:: str

property file_set

Get the source file set name for the NSRDB variable. This is typically used for MERRA source filesets such as tavg1_2d_aer_Nx or tavg1_2d_slv_Nx (for MERRA)

Returns:: str

property final_dtype

Get the variable’s intended storage datatype.

Returns:: dtype (str) – Data type for the current variable.

property mask: Get a boolean mask to locate the current variable in the meta data.

property name: Get the NSRDB variable name.

property next_date

Get the date after the date for this handler. This is used to get the data for the next date for temporal interpolation

Returns:: datetime.date

property next_file

Get the file path for the date for the target NSRDB variable name based on the glob self.next_pattern. The file is used to get the data for the next date for temporal interpolation

Returns:: str

property next_file_exists: Check if file for next date exists

property physical_max

Get the variable’s physical maximum value.

Returns:: physical_max (float) – Physical maximum value for the variable. Variable range can be truncated at this value. Must be consistent with the final dtype and scale factor.

property physical_min

Get the variable’s physical minimum value.

Returns:: physical_min (float) – Physical minimum value for the variable. Variable range can be truncated at this value. Must be consistent with the final dtype and scale factor.

scale_data(array)

Perform a safe data scaling operation on a source data array.

Steps:

Enforce physical range limits
Apply scale factor (mulitply)
Round if integer
Enforce dtype bit range limits
Perform dtype conversion
Return manipulated array

Parameters:: array (np.ndarray) – Source data array with full precision (likely float32).
Returns:: array (np.ndarray) – Source data array with final datatype.

property scale_factor

Get the variable’s intended storage scale factor.

Returns:: scale_factor (float) – Scale factor for the current variable. Data is multiplied by this scale factor before being stored.

property source_dir

Get the source directory containing the variable data files.

Returns:: source_dir (str) – Directory containing source data files (with possible sub folders).

property spatial_method

Get the spatial interpolation method.

Returns:: spatial_method (str) – NN or IDW

property temporal_method

Get the temporal interpolation method.

Returns:: temporal_method (str) – linear or nearest

property units

Get the units attribute.

Returns:: units (str) – NSRDB variable units.

unscale_data(array)

Perform a safe data unscaling operation on a source data array.

Parameters:: array (np.ndarray) – Scaled source data array with integer precision.
Returns:: array (np.ndarray) – Unscaled source data array with float32 precision.

property var_meta

Return the meta data for NSRDB variables.

Returns:: _var_meta (pd.DataFrame) – Meta data for NSRDB variables.