nsrdb.blend.blend.Blender

class Blender(meta_out, out_fpath, east_fpath, west_fpath, map_col='gid_full', lon_seam=-105.0)[source]

Bases: object

Class to blend east and west satellite extents

Parameters:

  • meta_out (str | pd.DataFrame) – Final output blended meta data (filepath or extracted df).

  • out_fpath (str) – Filepath to save output file to.

  • east_fpath (str) – NSRDB file for eastern extent.

  • west_fpath (str) – NSRDB file for western extent.

  • map_col (str, optional) – Column in the east and west meta data that map sites to the full meta_out gids.

  • lon_seam (int, optional) – Vertical longitude seam at which data transitions from the western source to eastern, by default -105 (historical closest to nadir). 5min conus data (2019 onward) is typically blended at -113.0 because the conus west satellite extent doesnt go that far east.

Methods

blend_dir(meta_out, out_dir, east_dir, ...)

Initialize and run the blender on two source directories with a file tag to search for.

blend_file(meta_out, out_fpath, east_fpath, ...)

Initialize and run the blender using explicit source and output filepaths.

run_blend(source_fpath, source_meta[, ...])

Run blending from one source file to the initialized output file.

run_full(meta_file, out_dir, east_dir, west_dir)

NSRDB East-West Blend.
run_blend(source_fpath, source_meta, chunk_size=100000)[source]

Run blending from one source file to the initialized output file.

Parameters:

  • source_fpath (str) – Source filepath (h5) to blend.

  • source_meta (pd.DataFrame) – Source meta data to be blended - must be reduced to only data that is going to be written to final output file from source. Site gids must be sequential in source and destination.

  • chunk_size (int) – Number of sites to read/write at a time.

classmethod blend_file(meta_out, out_fpath, east_fpath, west_fpath, map_col='gid_full', lon_seam=-105.0, chunk_size=100000)[source]

Initialize and run the blender using explicit source and output filepaths.

Parameters:

  • meta_out (str | pd.DataFrame) – Final output blended meta data (filepath or extracted df).

  • out_fpath (str) – Filepath to save output file to.

  • east_fpath (str) – NSRDB file for eastern extent.

  • west_fpath (str) – NSRDB file for western extent.

  • map_col (str, optional) – Column in the east and west meta data that map sites to the full meta_out gids.

  • lon_seam (int, optional) – Vertical longitude seam at which data transitions from the western source to eastern, by default -105 (historical closest to nadir). 5min conus data (2019 onward) is typically blended at -113.0 because the conus west satellite extent doesnt go that far east.

  • chunk_size (int) – Number of sites to read/write at a time.

classmethod blend_dir(meta_out, out_dir, east_dir, west_dir, file_tag, out_fn=None, map_col='gid_full', lon_seam=-105.0, chunk_size=100000)[source]

Initialize and run the blender on two source directories with a file tag to search for. This can only blend one file.

Parameters:

  • meta_out (str | pd.DataFrame) – Final output blended meta data (filepath or extracted df).

  • out_dir (str) – Directory to save output file to.

  • east_dir (str) – NSRDB output directory for eastern extent.

  • west_dir (str) – NSRDB output directory for western extent.

  • file_tag (str) – String to look for in files in east_dir and west_dir to find source files.

  • out_fn (str) – Optional output filename. Will be inferred from the east file (without ‘_east’) if not input.

  • map_col (str, optional) – Column in the east and west meta data that map sites to the full meta_out gids.

  • lon_seam (int, optional) – Vertical longitude seam at which data transitions from the western source to eastern, by default -105 (historical closest to nadir). 5min conus data (2019 onward) is typically blended at -113.0 because the conus west satellite extent doesnt go that far east.

  • chunk_size (int) – Number of sites to read/write at a time.

classmethod run_full(meta_file, out_dir, east_dir, west_dir, out_fn=None, east_fn=None, west_fn=None, file_tag=None, map_col='gid_full', lon_seam=-105.0, chunk_size=100000, log_file='blend.log', log_level='DEBUG')[source]

NSRDB East-West Blend.

Parameters:

  • meta_file (str) – Filepath to final output blended meta data csv file.

  • out_dir (str) – Directory to save blended output.

  • east_dir (str) – Source east directory.

  • west_dir (str) – Source west directory.

  • out_fn (str) – Optional output filename

  • east_fn (str) – Optional east filename (found in east_dir)

  • west_fn (str) – Optional west filename (found in west_dir)

  • file_tag (str) – File tag found in files in east and west source dirs.

  • map_col (str) – Column in the east and west meta data that map sites to the full meta_out gids.

  • lon_seam (float) – Vertical longitude seam at which data transitions from the western source to eastern, by default -105.0 (historical closest to nadir). 5min conus data (2019 onwards) is typically blended at -113.0 because the conus west satellite extent doesnt go that far east.

  • chunk_size (int) – Number of sites to read/write at a time.

  • log_file (str) – File to use for logging

  • log_level (str) – Level to use for logging.