Skip to content

Loading

Rretrieve objects from a registry

.ml.registry.get. model retrieve a model from a registry modelStore retrieve the modelStore table for a registry metric retrieve metric information for a model parameters retrieve parameter information for a model predict retrieve a model from the registry, wrapped in a common interface update retrieve the update method for models within the registry, wrapped in a common interface

Once saved to the ML Registry, entities that have been persisted should be accessible to any user with access to the registry’s location.

.ml.registry.get.metric

Retrieve metric information associated with a model

.ml.registry.get.metric[folderPath;experimentName;modelName;version;param]

Where

argument type description
folderPath string, :: folder path of the registry; if null, the current directory
experimentName string, :: name of an experiment from which to retrieve a model; if null, the newest model within this experiment will be used; if modelName and experimentName both null, the newest model in the unnamedExperiments section
modelName string, :: name of the model to be retrieved; if null, the newest model associated with the experiment
version long[], :: version of a named model to retrieve, major and minor versions as a long pair; if null, the newest model
param dictionary, symbol, :: search parameters for the retrieval of metrics

returns the metric table for a specific model, which can be filtered.

The .ml.registry.[new/set/log] namespaces let you persist or create items within a registry.

Log a number of metrics associated with a model.

q).ml.registry.set.model[::;{x};"mymodel";"q";::]
q).ml.registry.log.metric[::;::;::;::;`metric1;2.0]
q).ml.registry.log.metric[::;::;::;::;`metric1;2.1]
q).ml.registry.log.metric[::;::;::;::;`metric2;1.0]
q).ml.registry.log.metric[::;::;::;::;`metric2;1.0]
q).ml.registry.log.metric[::;::;::;::;`metric3;3.0]

Retrieve all metrics associated with the model.

q).ml.registry.get.metric[::;::;::;::;::]
timestamp                     metricName metricValue
----------------------------------------------------
2021.06.01D09:51:35.638489000 metric1    2
2021.06.01D09:51:35.652863000 metric1    2.1
2021.06.01D09:51:35.666593000 metric2    1
2021.06.01D09:51:35.679152000 metric2    1
2021.06.01D09:51:35.694630000 metric3    3

Retrieve all metrics named metric1.

q).ml.registry.get.metric[::;::;::;::;`metric1]
timestamp                     metricName metricValue
----------------------------------------------------
2021.06.01D09:51:35.638489000 metric1    2
2021.06.01D09:51:35.652863000 metric1    2.1

Retrieve multiple metrics.

q).ml.registry.get.metric[::;::;::;::;`metric2`metric3]
timestamp                     metricName metricValue
----------------------------------------------------
2021.06.01D09:51:35.666593000 metric2    1
2021.06.01D09:51:35.679152000 metric2    1
2021.06.01D09:51:35.694630000 metric3    3

Equivalently this can be done using a dictionary input.

q).ml.registry.get.metric[::;::;::;::;enlist[`metricName]!enlist `metric2`metric3]
timestamp                     metricName metricValue
----------------------------------------------------
2021.06.01D09:51:35.666593000 metric2    1
2021.06.01D09:51:35.679152000 metric2    1
2021.06.01D09:51:35.694630000 metric3    3

.ml.registry.get.model

Retrieve a model from an ML Registry

.ml.registry.get.model[folderPath;experimentName;modelName;version]

Where

argument type description
folderPath string, :: folder path of the registry; if null, current directory
experimentName string, :: name of an experiment from which to retrieve a model; if null, the newest model within this experiment will be used; if modelName and experimentName both null, the newest model within the unnamedExperiments section
modelName string, :: name of the model to be retrieved; if null, the newest model associated with the experiment
version long[], :: version of a named model to retrieve, major and minor versions as a long pair; if null, the newest model

returns a dictionary; the model and information related to the generation of the model.

You can use .ml.registry.set.model to include code files to be loaded on model retrieval, with .q, .p, .py or .k extensions. When you call .ml.registry.get.model these files are loaded prior to model retrieval.

Set a number of models within a new registry.

q).ml.registry.set.model[::;{x};"model";"q";::]
q).ml.registry.set.model[::;{x+1};"model";"q";::]
q).ml.registry.set.model[::;{x+2};"model1";"q";::]

Get the latest addition to the Registry.

q).ml.registry.get.model[::;::;::;::]
modelInfo| `registry`model`monitoring!(`description`modelInformation`experime..
model    | {x+2}

Get the latest version of model.

q).ml.registry.get.model[::;::;"model";::]
modelInfo| `registry`model`monitoring!(`description`modelInformation`experime..
model    | {x+1}

Get version 1.0 of model.

q).ml.registry.get.model[::;::;"model";1 0]
modelInfo| `registry`model`monitoring!(`description`modelInformation`experime..
model    | {x}

.ml.registry.get.modelStore

Retrieve the modelStore table associated with an ML Registry

.ml.registry.get.modelStore[folderPath;config]

Where

argument type description
folderPath string, :: folder path, where the registry is to retrieved from; if null, the current directory
config :: currently unused, must be ::

retrieves the modelStore and sets as the modelStore variable; returns null.

q).ml.registry.get.modelStore[::;::]
q)modelStore
registrationTime              experimentName modelName uniqueID              ..
-----------------------------------------------------------------------------..
2021.06.01D08:51:28.593730000 "undefined"    "mymodel" 7a214d0a-d9d2-890e-014..

.ml.registry.get.parameters

Retrieve parameter information associated with a model

.ml.registry.get.parameters[folderPath;experimentName;modelName;version;paramName]

Where

argument type description
folderPath string, :: folder path of the registry; if null, the current directory
experimentName string, :: name of an experiment from which to retrieve a model; if null, the newest model within this experiment will be used; if both modelName and experimentName null, the newest model in the unnamedExperiments section
modelName string, :: name of the model to be retrieved; if null, the newest model associated with the experiment
version long[], :: specific version of a named model to retrieve, major and minor versions as a long pair; if null, the newest model
paramName symbol, string name of the parameter to retrieve

returns the value (string, dictionary, table, or float) of the named parameter saved for the model.

Set a number of parameters associated with a model.

.ml.registry.set.parameters[::;::;"mymodel";1 0;"paramFile1";`param1`param2!1 2]
.ml.registry.set.parameters[::;::;"mymodel";1 0;"paramFile2";("value1";"value2")]

Retrieve the set parameters.

q).ml.registry.get.parameters[::;::;::;::;`paramFile1]
param1| 1
param2| 2

q).ml.registry.get.parameters[::;::;::;::;`paramFile2]
"value1"
"value2"

.ml.registry.get.predict

Retrieve a model from the ML Registry wrapping in a common interface

.ml.registry.get.predict[folderPath;experimentName;modelName;version]

Where

argument type description
folderPath string, :: folder path of the registry; if null, the current directory
experimentName string, :: name of an experiment from which to retrieve a model; if null, the newest model in this experiment; if both modelName and experimentName null, the newest model in the unnamedExperiments section
modelName string, :: name of the model to be retrieved; if null, the newest model associated with the experiment
version long[], :: version of a named model, major and minor versions as a long pair; if null, the newest model

returns as a function a wrapped version of the model, providing a common callable interface for all models within the ML Registry. The model can accept vector, matrix, table, or dictionary input and returns predictions generated by the model.

Models within the ML Registry can be of many forms: q, Python, Sklearn, Keras, etc. So this function provides a common entry point to allow the models to be retrieved such that they are all callable using the same function call.

You can use .ml.registry.set.model to include code files to be loaded on model retrieval, with .q, .p, .py or .k extensions. When you call .ml.registry.get.predict these files are loaded prior to model retrieval.

Set a number of models within a new registry.

q).ml.registry.set.model[::;{x};"model";"q";::]
q).ml.registry.set.model[::;{x+1};"model";"q";::]
q).ml.registry.set.model[::;{x+2};"model1";"q";::]

Get the latest addition to the Registry.

q).ml.registry.get.predict[::;::;::;::]
{x+2}{[data;bool]
  dataType:type data;
  if[dataType<=20;:data];
  data:$[98h=dat..

Get the latest version of model.

q).ml.registry.get.predict[::;::;"model";::]
{x+1}{[data;bool]
  dataType:type data;
  if[dataType<=20;:data];
  data:$[98h=dat..

Get version 1.0 of model.

q).ml.registry.get.predict[::;::;"model";1 0]
{x}{[data;bool]
  dataType:type data;
  if[dataType<=20;:data];
  data:$[98h=dat..

.ml.registry.get.update

Retrieve the update method for models within the ML Registry wrapping in a common interface

.ml.registry.get.update[folderPath;experimentName;modelName;version;supervised]

Where

argument type description
folderPath string, :: folder path of the registry; if null, the current directory
experimentName string, :: name of an experiment from which to retrieve a model; if null, the newest model in this experiment; if both modelName and experimentName null, the newest model in the unnamedExperiments section
modelName string, :: name of the model to be retrieved; if null, the newest model associated with the experiment
version long[], :: version of a named model, major and minor versions as a long pair; if null, the newest model
supervised boolean whether the model being retrieved is supervised (affects the rank of the returned function)

returns as a function a wrapped version of the model, providing a common callable interface for all models within the ML Registry. The model takes a vector, matrix, table, or dictionary argument and returns an updated version of the originally persisted model.

Models stored within ML Registry can be of many forms: q, Sklearn, Keras, etc. Many of these have an update capability to allow the models to be updated as new data becomes available. So this function provides a common entry point to allow the models’ update functionality to be retrieved in a common format.

To be retrieved from the registry the model must contain the following characteristics

model type supported requirements
q Yes Model originally saved to registry must contain an update key.
Sklearn Yes Model originally saved to registry must support the partial_fit method.
Keras No
Pytorch No
Theano No

You can use .ml.registry.set.model to include code files to be loaded on model retrieval, with .q, .p, .py or .k extensions. When you call .ml.registry.get.update these files are loaded prior to model retrieval.

Fit models to be persisted to the registry.

X:100 2#200?1f
yReg:100?1f
yClass:100?0b
online1:.ml.online.clust.sequentialKMeans.fit[flip X;`e2dist;3;::;::]
online2:.ml.online.sgd.linearRegression.fit[X;yReg;1b;::]
sgdClass:.p.import[`sklearn.linear_model][`:SGDClassifier]
sgdModel:sgdClass[pykwargs `max_iter`tol!(1000;0.003)][`:fit] . (X;yClass)

Set a number of models within a new registry.

.ml.registry.set.model[::;online1;"onlineCluster";"q";::]
.ml.registry.set.model[::;online2;"onlineRegression";"q";::]
.ml.registry.set.model[::;sgdModel;"SklearnSGD";"sklearn";::]

Get the latest Sklearn updatable model from the registry.

q).ml.registry.get.update[::;::;"SklearnSGD";::;1b]
.[{[f;x]embedPy[f;x]}[foreign]enlist]{(x y;z)}[locked[;0b]]

Get a q updatable supervised model from the registry.

q).ml.registry.get.update[::;::;"onlineRegression";::;1b]
.[{[config;secure;features;target]
  modelInfo:config`modelInfo;
  theta:mode..{(x y;z)}[locked[;0b]]

Get a q updatable unsupervised model from the registry.

q).ml.registry.get.update[::;::;"onlineCluster";::;0b]
{[returnInfo;data]
  modelInfo:returnInfo`modelInfo;
  inputs:modelInfo`input..locked[;0b]