This section provides the API documentation for the
.ml.* q library provided by KX for use by customers of kdb Insights, both for interactions with the platform, microservices and for independent use.
Presently this library provides users with the ability to run analytic workflows and interact with the KX ML Registry, this can be broken down as follows:
- Data preprocessing (data encoding, feature hashing, infinity replacement).
- ML models (clustering, regression, classification).
- Store and version ML models on-prem or within a cloud storage solution.
- A common model retrieval API for deployment of models from various Python/q libraries.
- Storage of information relating to model training/monitoring allowing sysadmins to control the promotion of models to production.
- Enhanced team collaboration oportunities and management oversight by centralising an ML teams work to a common storage location.
There are many ways users can make use of the machine learning functionality provided here:
- Train models on historical data for deployment to real-time systems:
- Anomaly detection within manufacturing.
- Trade surveillance.
- Preprocessing of streaming data prior to storage:
- Modification of data in flight to allow storage of feature sets for future training.
- Machine Learning Operations development cycle:
- Storage of frequently updated models for future deployment (trading models/recommendation engines).
- Deployment of stored models to production settings.
- Use of versioning to maintain control over model accountability.