cntk.ops.functions module

CNTK function constructs. This is the core abstraction of all primitive operators in the CNTK computational graph.

BlockFunction(op_name, name)

Decorator for defining a @Function as a BlockFunction. Same as @Function, but wrap the content into an as_block().

class CloneMethod

Bases: enum.Enum

Describes different ways how clone() works.

clone = 'clone'

New learnable parameters are created and initialized with the current values of the corresponding parameters of the Function being cloned

freeze = 'freeze'

Parameters are cloned and made immutable; i.e. Constants in the new clone (e.g. for use as a fixed feature extractor)

share = 'share'

Parameters are shared between the Function being cloned and the new clone

class Function(*args, **kwargs)

Bases: cntk.cntk_py.Function

Base class of all primitive tensor operators.

If it has only one output, one can invoke Variable methods on it, which it will relay to its only output.

Function objects can also be constructed directly from a Python lambda, by means of the @Function decorator. The Function‘s input signature is defined by the lambda.

Example

>>> @Function
... def f(x):
...     return x * x
>>> print(f)    # inspect the Function's type
ElementTimes(x: Sequence[tensor]) -> Sequence[tensor]

The above form creates a CNTK Function whose arguments are placeholder variables. Such a function can only be combined with other symbolic functions.

To train a Function or pass data to it, you need to declare the types of the arguments. In this case, the @Function decorator creates a CNTK Function whose arguments are input variables.

If you use Python 3, Functions with types are declared using Python annotation syntax, e.g.:

@Function
def f(x:Tensor[13]):
    return x * x

If you are working with Python 2.7, use CNTK’s @Signature decorator instead:

>>> from cntk.layers.typing import *
>>> @Function
... @Signature(Tensor[13])
... def f(x):
...     return x * x
>>> print(f)
ElementTimes(x: Tensor[13]) -> Tensor[13]

make_block=True is an internal parameter used to implement @BlockFunction. If BlockFunction() passes True, then the result will be wrapped in as_block(), using the supplied op_name and name parameters, which are otherwise ignored.

argument_map(*args, **kwargs)

Determines the {placeholder: variable} map for use with various call operations Returns a dictionary from this function’s placeholders to whatever arguments are passed. Accepted are both positional and keyword arguments. This mimics Python’s argument interpretation, except that keyword arguments are not optional (there is no concept of default value). This does not require the arguments to be Variables or Functions. It is also called by train_minibatch().

arguments

List of all input variables of the Function that are not of type Parameter or Constant.

Note that due to the different matrix storage format in C++(column major) and Python(row major), the order of arguments for some ops(Times, TransposeTimes, and Gemm) in C++ and Python are not the same. In previous CNTK versions, the default for this api was to return arguments in C++ order. Now the default for this api is set to python order. This way it will return arguments in the same order as they are fed into ops. If you wish to still get arguments in C++ order, you can simply override the global option.

Example

>>> import cntk as C
>>> a = C.input_variable((3,4), name='a')
>>> b = C.input_variable((4,5), name='b')
>>> c = C.times(a, b)
>>> c.arguments    # python order
    (Input('a', [#], [3 x 4]), Input('b', [#], [4 x 5]))

>>> from cntk.default_options import set_global_option
>>> set_global_option('python_operand_order', False)
>>> c.arguments    # C++ order
    (Input('b', [#], [4 x 5]), Input('a', [#], [3 x 4]))

attributes

List of the attributes of the function

backward(state, root_gradients, variables, as_numpy=True)

Backpropagates supplied root_gradients for one or more of the output variables of the Function, to calculate gradients with respect to variables. Formally, multiplies the values of root_gradients by the Jacobian of the Function and returns the subset of the output that corresponds to variables.

Example

>>> # compute the value and the derivative of the sigmoid at 0
>>> v = C.input_variable(shape=(1,), needs_gradient=True)
>>> f = C.sigmoid(v)
>>> df, fv = f.forward({v:[[0]]}, [f.output], set([f.output]))
>>> value = list(fv.values())[0]
>>> grad = f.backward(df, {f.output: np.ones_like(value)}, set([v]))
>>> value
array([[ 0.5]], dtype=float32)
>>> list(grad.values())[0]
array([[ 0.25]], dtype=float32)

Parameters:

• state (BackPropState) – state obtained from a previous call to the func:cntk.ops.Function.forward method on this Function for the computation that this gradient backpropagation corresponds to.
• root_gradients (dict) – the gradients that will be backpropagated
• variables (set) – a list of input variables with respect to which the gradients have to be computed.
• as_numpy (bool) – whether to return the gradients as a NumPy array. Default True. Specifying this as False returns a CNTK Value which avoids a costly conversion but returns a somewhat opaque object. Also, the Value objects are temporary and only guaranteed to be valid until the next forward/eval/backward/grad call. You must explicitly clone the temporay Value objects if they need to be accessed later.

Note

See forward() for more examples on passing input data.

Returns:mapping of variables to NumPy arrays Return type:dict

block_arguments_mapping

Returns the mapping from the arguments of the composite underlying this block function to the Variables that they are bound to in the outer graph of Functions that this block Function is part of.

block_root

Returns the root of the Function graph underlying this block Function. Throws an exception if this is not a block Function.

clone(method, substitutions=None)

Clones the function. The parameters of the Function are either cloned, shared or frozen as specified by the method argument and any variable substitutions requested are applied in the cloned Function instance.

Parameters:

• method (CloneMethod) –

  one of

  • ‘clone’: the returned function gets its own copy of parameters (default)
  • ‘share’: the returned function shares its parameters with this function
  • ‘freeze’: parameters are cloned and made immutable (constant).
• substitutions (dict) – a dictionary mapping variables in this function to variables in the cloned function

Returns:

the cloned Function

Return type:

Function

constants

List of all Constant variables of this Function

custom_attributes

Get function custom attributes in cntk_py.Dictionary for both read and write.

declare_args(*arg_types)

Back-compat wrapper for update_signature() (beta12 and before).

eval(arguments=None, outputs=None, device=None, as_numpy=True)

Evaluate the Function’s outputs using the specified arguments as input.

Parameters:

• arguments –

  maps variables to their input data. The interpretation depends on the input type:

  • dict: keys are input variable or names, and values are the input data. See forward() for details on passing input data.
  • any other type: if node has a unique input, arguments is mapped to this input.

  For nodes with more than one input, only dict is allowed.

  In both cases, every sample in the data will be interpreted as a new sequence.

  Sequences can be marked as continuations of the same sequence in the previous minibatch (that is the sequence in the same slot). There are two possibilities for this:

  • specifying arguments as a tuple where the first element is used as arguments and the second one will be used as a list of bools, denoting whether a sequence is a new one (True) or a continuation of the sequence in the same slot of the previous minibatch (False). This will be applied to all batches.
  • specifying arguments as a dictionary of variables to tuples where the first element is used as arguments and the second one will be used as a list of bools, denoting whether a sequence is a new one (True) or a continuation of the sequence in the same slot of the previous minibatch (False). This will be applied to all batches.

  Data should be either NumPy arrays or a MinibatchData instance.

• outputs (iterable, optional) – outputs to fetch values for. If not set, all outputs of the function will be fetched.
• device (DeviceDescriptor) – the device descriptor that contains the type and id of the device on which the computation is to be performed.
• as_numpy (bool) – whether to return the result as a NumPy array. Default True. Specifying this as False returns a CNTK Value which avoids a costly conversion but returns a somewhat opaque object. Also, the Value objects are temporary and only guaranteed to be valid until the next forward/eval/backward/grad call. You must explicitly clone the temporay Value objects if they need to be accessed later.

Note

See forward() for examples on passing input data.

Returns:Dict with keys of output variable names and values of output variable. A single NumPy array if there is only one output value. Return type:dict or NumPy Array

find_all_with_name(name, depth=0)

Returns a list of primitive function with name in the graph starting from this node. Throws an exception if name occurs multiple times. If you expect only one function to be returned, use find_by_name().

Example

>>> a = C.input_variable(shape=1, name='i')
>>> b = C.input_variable(shape=1, name='i')
>>> c = C.plus(a, b, name='c')
>>> len(c.find_all_with_name('i'))
2
>>> c.find_all_with_name('z')
[]

Parameters:

• name (str) – names to look for
• depth (int, default 0) – how deep into the block hierarchy the DFS algorithm should go into. Set to -1 for infinite depth.

Returns:

list of Function objects matching name

See also

find_by_name()

find_by_name(name, depth=0)

Returns a primitive function with name in the graph starting from this node. Throws an exception if name occurs multiple times. If you expect multiple functions to be returned, use find_all_with_name().

Example

>>> a = C.input_variable(shape=1, name='a')
>>> b = C.input_variable(shape=1, name='b')
>>> c = C.plus(a, b, name='c')
>>> print(c.find_by_name('b').name)
b
>>> c.find_by_name('z') is None
True

If you need a full function out of it that can be evaluated, you need to upcast it (currently done via combine):

>>> d = c * 5
>>> C.combine([d.find_by_name('c')]).eval({a:[[1]], b:[[2]]})
array([[ 3.]], dtype=float32)

Parameters:

• name (str) – names to look for
• depth (int, default 0) – how deep into the block hierarchy the DFS algorithm should go into. Set to -1 for infinite depth.

Returns:

Function object matching name

See also

find_all_with_name()

forward(arguments, outputs=None, keep_for_backward=None, device=None, as_numpy=True)

Computes the values of speficied variables in outputs, using values provided in arguments that correspond to each input Variable of the function (i.e. those that have is_input = True).

Example

>>> # Example of passing dense data
>>> v = C.input_variable(shape=(3,))
>>> f = C.reciprocal(v)
>>> _, fv = f.forward({v:[[1, 2, 4]]})
>>> list(fv.values())[0]
array([[ 1.  ,  0.5 ,  0.25]], dtype=float32)

Example

>>> # Passing sparse values as one-hot with a vocabulary size of 5
>>> vocab_size = 5
>>> v = C.sequence.input_variable(shape=(vocab_size,), is_sparse=True)
>>> f = C.times(v, np.eye(vocab_size))
>>> # Passing a batch of two sequences:
>>> # 1st sequence: word 1
>>> # 2nd sequence: words 2 and 4
>>> batch = [[1],[2,4]]
>>> sparse_batch = C.Value.one_hot(batch, vocab_size)
>>> _, fv = f.forward({v:sparse_batch})
>>> list(fv.values())[0]
[array([[ 0.,  1.,  0.,  0.,  0.]], dtype=float32),
 array([[ 0.,  0.,  1.,  0.,  0.], [ 0.,  0.,  0.,  0.,  1.]], dtype=float32)]

Example

>>> # Doing the same, but with a CSR matrix from scipy.sparse
>>> vocab_size = 5
>>> from scipy.sparse import csr_matrix
>>> v = C.sequence.input_variable(shape=(vocab_size,), is_sparse=True)
>>> f = C.times(v, np.eye(vocab_size))
>>> # Note that csr_matrix automatically uses a sparse representation underneath.
>>> sparse_batch = [csr_matrix([[0,1,0,0,0]]), csr_matrix([[0,0,1,0,0], [0,0,0,0,1]])]
>>> _, fv = f.forward({v:sparse_batch})
>>> list(fv.values())[0]
[array([[ 0.,  1.,  0.,  0.,  0.]], dtype=float32),
 array([[ 0.,  0.,  1.,  0.,  0.], [ 0.,  0.,  0.,  0.,  1.]], dtype=float32)]

>>> # Much more efficient, however, is to incrementally create CSR arrays.
>>> # See https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.csr_matrix.html
>>> # for more information.
>>> def seq_to_csr_matrix(seq, vocab_size):
...     indptr = [0]
...     indices = []
...     data = []
...     for term_idx in seq:
...         indices.append(term_idx)
...         data.append(1)
...         indptr.append(len(indices))
...     return csr_matrix((data, indices, indptr), shape=(len(seq), vocab_size))
>>> sparse_batch = [seq_to_csr_matrix(seq, vocab_size) for seq in batch]
>>> _, fv = f.forward({v:sparse_batch})
>>> list(fv.values())[0]
[array([[ 0.,  1.,  0.,  0.,  0.]], dtype=float32),
 array([[ 0.,  0.,  1.,  0.,  0.], [ 0.,  0.,  0.,  0.,  1.]], dtype=float32)]

Parameters:

• arguments –

  maps variables to their input data. The interpretation depends on the input type:

  • dict: keys are input variable or names, and values are the input data. To specify a minibatch, provide a list of arrays. The shape of each array must be compatible with the shape of the dictionary key. If the array denotes a sequence then the elements of the sequence are grouped along axis 0.
  • any other type: if node has a unique input, arguments is mapped to this input.

  For nodes with more than one input, only dict is allowed.

  In both cases, every sample in the data will be interpreted as a new sequence.

  Sequences can be marked as continuations of the same sequence in the previous minibatch (that is the sequence in the same slot). There are two possibilities for this:

  • specifying arguments as a tuple where the first element is used as arguments and the second one will be used as a list of bools, denoting whether a sequence is a new one (True) or a continuation of the sequence in the same slot of the previous minibatch (False). This will be applied to all batches.
  • specifying arguments as a dictionary of variables to tuples where the first element is used as arguments and the second one will be used as a list of bools, denoting whether a sequence is a new one (True) or a continuation of the sequence in the same slot of the previous minibatch (False). This will be applied to all batches.

  Data should be either NumPy arrays or a MinibatchData instance.

• outputs (iterable, optional) – outputs to fetch values for. If not set, all outputs of the function will be fetched.
• keep_for_backward (set, default None) – the subset of the Function’s output variables for which gradients shall be calculated in a subsequent backward call. If None, the returned state will be None and a subsequent call to backward() will not be possible.
• device (DeviceDescriptor, default None) – the device descriptor that contains the type and id of the device on which the computation is. If None, the default device is used.
• as_numpy (bool) – whether to return the result as a NumPy array. Default True. Specifying this as False returns a CNTK Value which avoids a costly conversion but returns a somewhat opaque object. Also, the Value objects are temporary and only guaranteed to be valid until the next forward/eval/backward/grad call. You must explicitly clone the temporay Value objects if they need to be accessed later.

Returns:

A tuple (BackPropState, map of outputs to NumPy arrays). The BackPropState is a handle taken by backward().

grad(at, wrt=None, outputs=None, device=None, as_numpy=True, grad_root=None)

Computes the gradient of this Function at location at with respect to wrt. The Function must have a single output.

Example

>>> x = C.input_variable(shape=(1,), needs_gradient=True)
>>> y = C.sqrt(x)
>>> a = np.asarray([1,4,16],dtype=np.float32).reshape(3,1)
>>> y.grad({x:a})
array([[ 0.5  ],

       [ 0.25 ],

       [ 0.125]], dtype=float32)

Parameters:

• at (dict) – mapping of the Function’s arguments to values
• wrt (list, default None) – list of Variables with respect to which the gradient will be computed. If omitted, the gradients with respect to all arguments of this Function that need gradient will be computed.
• outputs (iterable, optional) – outputs (including intermediate outputs in the graph) to fetch values for. If not specified, values for none of the outputs are fetched.
• device (DeviceDescriptor, default None) – the device descriptor that contains the type and id of the device on which the computation is performed. If None, the default device is used.
• as_numpy (bool, default True) – whether to return the gradients as a NumPy array. Default True. Specifying this as False returns a CNTK Value which avoids a costly conversion but returns a somewhat opaque object. Also, the Value objects are temporary and only guaranteed to be valid until the next forward/eval/backward/grad call. You must explicitly clone the temporay Value objects if they need to be accessed later.
• grad_root (Variable, optional) – specify the root of gradients calculation. If not specified, the output of this function will be used as gradient root.

Returns:

Dict with keys of wrt variables and gradient values of wrt variables. A single NumPy array if there is only one gradient value. If outputs were specified (to fetch values for), this method returns a tuple where the 2nd element of the tuple is the outputs values; a dict with keys of specified outputs variables and values of computed outputs, or a single NumPy array if there is only one output value. Each element has the same shape as the wrt or outputs variables including dynamic axes (such as the batch axis).

Return type:

dict or NumPy Array or a tuple of these

inputs

List of variables that are inputs of this function. Note that ‘inputs’ here denotes all Variables that feed into this Function including any Parameter/Constant Variables that are children of this Function.

is_block

Returns a boolean indicating if this Function is a block function which is basically a composite encapsulated as an opaque block which appears as a primitive during traversing the graph of Functions that this block is part of.

is_composite

Returns a boolean indicating if this Function is a composite Function. A composite Function is a Function that is composed of primitive Functions.

is_primitive

Returns a boolean indicating if this Function is a primitive Function. A primitive Function is the lowest level building block for composite Function graphs and is either a CNTK built-in operator, a composite Function encapsulated as a Block or a user-defined Function

static load(model, device=None, format=<ModelFormat.CNTKv2: 0>)

Load the model, that has been saved using save().

Parameters:

• model (str, bytes or bytearray) – either a file path of a model file or a byte buffer containing the binary representation of a model.
• device (DeviceDescriptor, defaults to the current globally default device) – specifies the device to allocate the model on.
• format (ModelFormat, defaults to CNTKv2 format) – specifies the format of the file to load. if the specified format is ONNX, then model must be a filename.

Returns:

root node

name

Name of this function

Parameters:

• getter (str) – returns the name of the function.
• setter (str) – sets the name of the function. Setting the name of a Function is only allowed if the Function does not already have a name. Calling this method, when this Function already has a name, results in an exception.

op_name

Name of the operation that this Function performs

output

The single output variable if there is only one, or raises an exception.

outputs

List consisting of all output variables of this function.

parameters

List of all parameter variables of this function.

placeholders

List of all placeholders variables of this function.

print_node_timing()

Prints per-node average timing per-minibatch for each primitive function. statistics would reset after print

static register_udf_deserialize_callback(op_name, callback)

Register a callback function to be invoked when deserializing a user- defined function with the corresponding op name.

When loading a model, CNTK will try to automatically reconstruct any (non-native) user-defined functions by invoking a static deserialize() method of the corresponding UserFunction sub-class. This method allows to override default UDF deserialization behavior by specifying a user- defined function op name and the corresponding callback that should be invoked instead of the deserialize method.

Parameters:

• op_name (str) – unique op name of the user-defined function.
• callback (function) – a function taking three arguments (a list of inputs to the UserFunction, a string name, and a state dictionary generated by the corresponding serialize() method) and returns an instance of the user-defined function.

replace_placeholder(substitution)

In-place replace the only placeholder in the function graph with the specified substitution.

Parameters:substitution (Variable) – the variable that will replace the placeholder Returns:itself Return type:Function Raises:Exception – when the function has multiple placeholders.

replace_placeholders(substitutions)

In-place replace specified placeholders in the Function graph with the specified replacements in the map.

Parameters:substitutions (dict) – map from placeholder to variables Returns:itself Return type:Function

restore(filename)

Restore the models parameters (in-place) from a saved model file

Parameters:filename (str) – saved model path Returns:this method only has the side-effect of loading the model parameters from the file Return type:None

root_function

The primitive function at the root of the graph of functions underlying this function.

save(filename, format=<ModelFormat.CNTKv2: 0>)

Save this function graph into a model file using the specified format.

Use distributed.Communicator.is_main() to gate your call to save() in distributed environment.

Parameters:filename (str) – model path

set_attribute(name, value)

Allows to change a function attribute.

Parameters:

• name (string) –

  one of

  • ‘dropoutRate’: modifies the dropout rate of a dropout function (can only be invoked on a function instance returned either from dropout() or find_by_name()).
  • ‘rngSeed’: modifies the seed of a stateful function (can only be invoked on function instance returned from dropout(), random_sample(), random_sample_inclusion_frequency() or find_by_name())
• value (float in case of 'dropoutRate', int for 'rngSeed') – the new value of the corresponding attribute.

signature

Returns the signature of a Function. This is the .arguments[] list without placeholders that belong to an outer, not yet completed @Function def.

test(minibatch_source, minibatch_size=32, streams=None, model_inputs_to_streams=None, callbacks=None)

Measures the performance of a model, given by its criterion function, in the form of average metric value (or loss if model has only one output) on a set of data.

This is a convenience wrapper around cntk.eval.evaluator.Evaluator.

Parameters:

• minibatch_source (MinibatchSource) – minibatch source for the test data
• minibatch_size (minibatch_size_schedule or int) – minibatch size for evaluation
• streams (tuple) – the streams of the minibatch_source in argument order
• model_inputs_to_streams (dict) – mapping between input variables and input streams
• callbacks (progress writer or list of them) – optionally, list of progress writers from cntk.logging to automatically track training progress.

Returns:

An object test_summary with test_summary.metric being the average metric, and test_summary.samples the number of labels in the test set.

train(minibatch_source, minibatch_size=32, streams=None, model_inputs_to_streams=None, parameter_learners=[], callbacks=[], progress_frequency=None, max_epochs=None, epoch_size=None, max_samples=None)

Trains a model, given by its criterion function, using the specified training parameters and configs. Different aspects of training such as data sources, checkpointing, cross validation, progress printing can be configured using the corresponding config classes.

The input data can be specified as a data reader (MinibatchSource) for large corpora; or directly as numpy/scipy arrays if the data is so small that it is feasible to keep it all in RAM.

Data is processed in minibatches. The minibatch size defaults to 32, which is a choice that commonly works well. However, for maximum efficiency, we recommend to experiment with minibatch sizes and choose the largest that converges well and does not exceed the GPU RAM. This is particularly important for distributed training, where often, the minibatch size can be increased throughout the training, which reduces data bandwidth and thus speeds up parallel training.

If input data is given through a data reader (as opposed to directly as a numpy/scipy array), the user must also specify the epoch size. This is because data readers are used for large corpora, and the traditional definition of epoch size as number of samples in the corpus is not very relevant. Instead, CNTK really means the number of samples between summary actions, such as printing training progress, adjusting the learning rate, and/or checkpointing the model.

The function returns an object that contains these members: epoch_summaries is a list that contains the progression of epoch loss (.loss) and metric (.metric) values and the corresponding number of labels (.samples) that they were averaged over. This is the same value that a progress printer would print as epoch summaries. updates is a similar list with the more fine-grained minibatch updates. If a TestConfig was specified, then test_summary is the metric and sample count on the specified test set for the final model.

A number of callback mechanisms can optionally be specified as a list as callbacks. CNTK has a fixed set of callback types, and only those types are allowed in the callbacks list: An object of type ProgressWriter from cntk.logging is used for progress logging; a CheckpointConfig configures the checkpointing mechanism, which keeps copies of models at regular intervals and allows to seamlessly restart from a last checkpoint; a TestConfig allows to specify a test set that is evaluated at the end of the training; and a CrossValidationConfig specifies a user callback that can be used to adjust learning hyper-parameters or to denote to stop training, optionally based on a separate cross-validation data set.

This is a convenience wrapper around cntk.train.trainer.Trainer cntk.train.training_session.TrainingSession.

Parameters:

• self – the criterion function of a model to be trained. This is either a single-valued function (the loss) or a tuple-valued function (loss and metric).
• minibatch_source (MinibatchSource or tuple of numpy/scripy arrays) – data source used for training. For large data, use a MinibatchSource. For small data, pass a tuple of numpy/scipy arrays. The number of streams/arrays must match the number of arguments of self.
• streams (tuple) – (only if minibatch_source is a data reader) the streams of the minibatch_source in argument order. Not to be given if minibatch_source is specified as numpy/scipy arrays rather than a data reader.
• minibatch_size (int or minibatch_size_schedule, defaults to 32) – minibatch size (or schedule) for training
• epoch_size (int) – in CNTK, epoch size means the number of samples between outputting summary information and/or checkpointing. This must be specified unless the user directly passes numpy/scipy arrays for the minibatch_source.
• max_epochs (int, defaults to 1) – maximum number of samples used for training; requires epoch_size
• parameter_learners (list) – list of learners from cntk.learners
• callbacks (list) – list of callback objects, which can be of type ProgressWriter from cntk.logging (for logging), CheckpointConfig (for check-pointing), TestConfig (for automatic final evaluation on a test set), and CrossValidationConfig (for cross-validation based training control). Except for progress writers, at most one of each is allowed.
• model_inputs_to_streams (dict) – alternative to streams, specifying the mapping as a map from input variables to streams
• max_samples (int) – maximum number of samples used for training; mutually exclusive with max_epochs
• progress_frequency (int) – frequency in samples for aggregated progress printing. Defaults to epoch_size if given, or None otherwise

Example

>>> # a simple logistic-regression model
>>> N = 250
>>> np.random.seed(0)
>>> Y = np.random.randint(size=N, low=0, high=2)  # labels
>>> X = (np.random.randn(N, 2)+3) * (Y[:,None]+1)   # data
>>> # Our model expects float32 features, and cross-entropy expects one-hot encoded labels.
>>> import scipy.sparse
>>> Y = scipy.sparse.csr_matrix((np.ones(N,np.float32), (range(N), Y)), shape=(N, 2))
>>> X = X.astype(np.float32)
>>> model = cntk.layers.Dense(2, activation=None) # model function
>>> import cntk.layers
>>> @cntk.Function.with_signature(cntk.layers.Tensor[2], cntk.layers.SparseTensor[2]) # criterion function
... def criterion(data, label_one_hot):
...     z = model(data)  # apply model. Computes a non-normalized log probability for every output class.
...     return cntk.cross_entropy_with_softmax(z, label_one_hot)
>>> learner = cntk.sgd(model.parameters, 0.1)
>>> progress = criterion.train((X, Y), minibatch_size=25, max_epochs=2, epoch_size=125, parameter_learners=[learner])
>>> print("%.2f" % progress.epoch_summaries[-1].loss) # get the final epoch's loss value
0.68

Returns:

An object progress with progress.epoch_summaries and progress.updates being the progressions of av loss, av metric, and number of labels

for epochs and updates (groups of minibatches), respectively. If a TestConfig was given, then progress.test_summary includes the result (.metric and .samples)

type

Get type of a Function’s output.

uid

The internally generated unique name of the function.

update_signature(*arg_types, **kwarg_types)

Defines input shapes, in-place e.g. model.update_signature(42) pass a list of objects that define the dimensions etc. of the placeholders Currently you can pass an int, a tuple, an Input, or a dict created with Type()

static with_signature(*args, **kwargs)

Decorator for defining a @Function with a given signature. Same as @Function followed by @Signature.

Example

>>> from cntk.layers.typing import *
>>> @Function.with_signature(Tensor[13])
... def f(x):
...     return x * x
>>> print(f)
ElementTimes(x: Tensor[13]) -> Tensor[13]
>>> # which is equivalent to this:
>>> @Function
... @Signature(Tensor[13])
... def f(x):
...     return x * x
>>> print(f)
ElementTimes(x: Tensor[13]) -> Tensor[13]

class ModelFormat

Bases: enum.Enum

Describes the supported disk format for CNTK model.

CNTKv2 = 0

Default CNTK version 2 format, it supports all CNTK functionalities.

ONNX = 1

Open Neural Network Exchange format from https – //github.com/onnx/onnx, ONNX currently support subset of CNTK functionalities.

class UserFunction(inputs, as_numpy=True, attributes=None, name='')

Bases: cntk.ops.functions.Function

Base class of all user extension functions.

If it has only one output, one can invoke Variable methods on it, which it will relay to its only output.

Parameters:

• inputs (list) – inputs to this function
• as_numpy (bool, optional) – whether the data should be automatically converted from and to NumPy. Defaults to True. Specifying this as False passes the data as CNTK Value objects.
• name (str) – name of this function

clone(cloned_inputs)

Creates a clone of this user-defined function.

It assumes that the constructor signature of the user’s implementation of the user function takes the inputs as individual arguments followed by the operator name. If the signature is different, then this method needs to be overriden.

Parameters:cloned_inputs – list of cloned inputs to the new user-defined Function clone to be created. Returns:A cloned instance of this user-defined function.

static deserialize(inputs, name, state)

A stub deserialize method for illustration purposes. User-defined functions need to provide their own implementation in order for CNTK to be able to reconstruct them when loading a model.

Parameters:

• inputs (list) – a list of inputs to the function
• name (str) – name of this function
• state (dict) – a state dictionary generated by the corresponding serialize() method.

Returns:

An instance of the user-defined function.

infer_outputs()

Returns a list of all output variables this user-defined function outputs.

Output variables are created by output_variable().

op_name

Unique operation name of this user-defined function. This property defaults to ‘<module>.<class>’, but can be overridden.

serialize()

Generates a dictionary that captures the state of this user-defined function.

This method must be overridden, if a user function has any state that needs to be preserved in the model dictionary.

load_model(model, device=None, format=<ModelFormat.CNTKv2: 0>)

Alias for load().

native_user_function(op_id, operands, attributes=None, user_function_instance_name='')

Creates an instance of a user-defined Function previously registered using the ‘register_native_user_function’ method.

Parameters:

• op_id (str) – Id of the native user-defined Function to instantiate. This must be the id that was used when registering the native user-function with the ‘register_native_user_function’ method.
• operands (list) – input operands of the new instance of the native user-defined Function.
• user_function_instance_name (str) – Name of the instance of the created native user-defined Function.

Returns:

Function

register_native_user_function(op_id, module_name, factory_method_name)

Registers a native user-defined Function that can be subsequently instantiated using the ‘native_user_function’ method.

Parameters:

• op_id (str) – Unique id of the native user-defined Function to register. This id must be unique and an error will be reported if it matches the ‘op_id’ specified for any other registered native user-defined Function.
• module_name (str) – Name of the module containing the factory method for creating instances of the native user-defined Function being registered. This is typically the name of a DLL/so which exports a factory method for creating instances of the native user-defined Function.
• factory_method_name (str) – Name of the factory method for creating instances of the native user-defined Function being registered. This method must be an exported method of the specified module.