jade.jobs.results_aggregator.ResultsAggregator

class jade.jobs.results_aggregator.ResultsAggregator(filename, timeout=300, delimiter=',')

Bases: object

Synchronizes updates to the results file.

One instance is used to aggregate results from all compute nodes. One instance is used for each compute node.

Constructs ResultsAggregator.

Parameters:

• filename (Path) – Results file.

• timeout (int) – Lock acquistion timeout in seconds.

• delimiter (str) – Delimiter to use for CSV formatting.

Methods

append(output_dir, result[, batch_id])	Append a result to the file.
append_result(result)	Append a result to the file.
clear_results_for_resubmission(jobs_to_resubmit)	Remove jobs that will be resubmitted from the results file.
clear_unsuccessful_results()	Remove failed and canceled results from the results file.
create(output_dir, **kwargs)	Create a new instance.
create_files()	Initialize the results file.
get_results()	Return the current results.
get_results_unsafe()	Return the results.
list_results(output_dir, **kwargs)	Return the current results.
load(output_dir, **kwargs)	Load an instance from an output directory.
load_node_results(output_dir, batch_id, **kwargs)	Load a per-node instance from an output directory.
load_node_results_file(path, **kwargs)	Load a per-node instance from an output directory.
move_results(func)	Move the results to a new location and delete the file.
process_results()	Move all temp results into the consolidated file, then clear the file.

classmethod create(output_dir, **kwargs)

Create a new instance.

Parameters:

output_dir (str)

Return type:

ResultsAggregator

classmethod load(output_dir, **kwargs)

Load an instance from an output directory.

Parameters:

output_dir (str)

Return type:

ResultsAggregator

classmethod load_node_results(output_dir, batch_id, **kwargs)

Load a per-node instance from an output directory.

Parameters:

• output_dir (str)

• batch_id (int)

Return type:

ResultsAggregator

classmethod load_node_results_file(path, **kwargs)

Load a per-node instance from an output directory.

Parameters:

path (Path)

Return type:

ResultsAggregator

create_files()

Initialize the results file. Should only be called by the parent process.

classmethod append(output_dir, result, batch_id=None)

Append a result to the file.

output_dir : str result : Result batch_id : int

append_result(result)

Append a result to the file.

result : Result

clear_results_for_resubmission(jobs_to_resubmit)

Remove jobs that will be resubmitted from the results file.

Parameters:

jobs_to_resubmit (set) – Job names that will be resubmitted.

clear_unsuccessful_results()

Remove failed and canceled results from the results file.

get_results()

Return the current results.

Returns:

list of Result objects

Return type:

list

get_results_unsafe()

Return the results. It is up to the caller to ensure that a lock is not needed.

Returns:

list of Result objects

Return type:

list

move_results(func)

Move the results to a new location and delete the file.

Parameters:

func (function)

Returns:

list of Result

Return type:

list

classmethod list_results(output_dir, **kwargs)

Return the current results.

Parameters:

output_dir (str)

Returns:

list of Result objects

Return type:

list

process_results()

Move all temp results into the consolidated file, then clear the file.

Returns:

list of Result objects that are newly completed

Return type:

list