cntk.eval.evaluator module

An evaluator provides functionality to evaluate minibatches against the specified evaluation function.

class Evaluator(eval_function, progress_writers=None)

Bases: cntk.cntk_py.Evaluator

Class for evaluation of minibatches against the specified evaluation function.

Parameters:

• eval_function (Function) – evaluation function.
• progress_writers (list) – optionally, list of progress writers from cntk.utils to track training progress.

evaluation_function

The evaluation function that the evaluator is using.

print_node_timing()

Prints per-node average timing per-minibatch for each primitive function statistics would reset after print

summarize_test_progress()

Updates the progress writers with the summary of test progress since start and resets the internal accumulators.

test_minibatch(arguments, device=None, distributed=False)

Test the evaluation function on the specified batch of samples.

Parameters:

• arguments –

  maps variables to their input data. The interpretation depends on the input type:

  • dict: keys are input variable or names, and values are the input data. See forward() for details on passing input data.
  • any other type: if node has a unique input, arguments is mapped to this input. For nodes with more than one input, only dict is allowed.

  In both cases, every sample in the data will be interpreted as a new sequence. To mark samples as continuations of the previous sequence, specify arguments as tuple: the first element will be used as arguments, and the second one will be used as a list of bools, denoting whether a sequence is a new one (True) or a continuation of the previous one (False). Data should be either NumPy arrays or a MinibatchData instance.

• device (DeviceDescriptor) – the device descriptor that contains the type and id of the device on which the computation is to be performed.
• distributed (bool, optional) – flag indicating if evaluation results should be aggregated across workers.

Note

See forward() for examples on passing input data.

Returns:the average evaluation criterion value per sample for the tested minibatch. Return type:float