Skip to content

Previous KDB.AI Release Notes

This page documents previous updates to KDB.AI Cloud UI and KDB.AI Server. Bookmark it to set yourself up for continued success while using KDB.AI.

KDB.AI Cloud UI

v1.4.0

Release date 2024.10.21

New

The KDB.AI Cloud UI 1.4.0 release contains the latest capabilities and improvements introduced in the KDB.AI Server 1.4.0 release.

v1.3.0

Release date 2024.09.19

New

The KDB.AI Cloud UI 1.2.4 release contains all the latest capabilities and enhancements introduced in the KDB.AI Server 1.3.0 release.

v1.2.4

Release date 2024.08.15

Fixes and improvements

  • This update includes several bug fixes and enhancements to improve KDB.AI performance and stability.
  • v1.2.2 and v1.2.3 introduced a series of internal changes that significantly enhance KDB.AI reliability and performance.

New

  • Request new database button is now available to new users who haven't created a table within 7 days of creating their KDB.AI account.

v1.2.1

Release date 2024.07.02

Fixes and improvements

  • KDB.AI Cloud UI 1.2.1 includes bug fixes and internal improvements.

v1.2.0

Release date 2024.06.27

The KDB.AI Cloud UI 1.2.0 release contains all the new functionalities and performance improvements introduced in the KDB.AI Server 1.2.0 release.

v1.1.2

Release date 2024.05.23

Fixes and improvements

  • KDB.AI sample notebooks can be run from the UI directly in Google Colab. No need to switch windows or install the required libraries on local laptop/environment anymore.

v1.1.1

Release date 2024.05.09

Fixes and improvements

  • During the sign-up process, personal details are now entered after login, rather than on the signup page.
  • You can use Google SSO to sign up and login.

v1.1.0

Release date 2024.03.27

New

  • The KDB.AI Cloud UI 1.1.0 release contains all the latest search capabilities and integrations introduced in the KDB.AI Server 1.1.0 release.

v1.0.3

Release date 2024.03.13

Fixes and improvements

  • KDB.AI Cloud UI 1.0.3 includes internal improvements.

v1.0.2

Release date 2024.01.30

New

KDB.AI Cloud UI now provides three sections:

  • Overview: view Database Capacity and a Quickstart Guide with embedded cope snippets.
  • Connection Details: view instance endpoint URL and manage API keys for your environment. In addition, a Connection Guide is provided with embedded code snippets.
  • Tables: view list of Tables and schemas in your database and delete tables from the UI directly. The delete functionality only allows for deletion of complete tables, not row level data. See Delete table for details. In addition, a Tables Creation Guide is provided with embedded code snippets.

v1.0.0

Release date 2023.12.04

New

  • Introducing REST API support, including insert and train functions.

v0.2.0

Release date 2023.11.03

Fixes and improvements

  • Minor fixes and performance improvements

v0.1.1

Release date 2023.09.27

Fixes and improvements

  • Fixed issues with training functionality surrounding CPU instructions on certain machines.

  • Improved behavior surrounding synchronous API calls, including create and drop.

  • Improved responses from similarity search API by removing unneeded empty rows.

v0.1.0

Release date 2023.09.12

New

Introducing the first public release of KDB.AI Cloud UI.

  • Supported by Python clients with version less than or equal to 0.1.1

KDB.AI Server

v1.4.0

Release date 2024.10.21

Welcome to the KDB.AI Server 1.4.0 release, crafted to make your vector search experience faster, more consistent and powerful! Key highlights:

  1. macOS support
  2. Database layer
  3. Multiple indexes
  4. q API
  5. Version information
  6. Enhanced REST API
  7. Optimized kdb+ integration
  8. Enhanced symbol management

1. macOS support (New)

We're excited to announce that KDB.AI is now fully supported on macOS via Docker, bringing you the following benefits:

  • Enhanced Performance: Optimized for macOS to ensure smooth and efficient operation.
  • Seamless Integration: Easily integrate KDB.AI with your existing macOS workflows and applications.

Tip: macOS users can set up and leverage KDB.AI’s advanced features directly on their devices, boosting productivity and streamlining data management tasks.

2. Database layer (New)

We're introducing a new database layer above tables for better data management. Also, you can reference external database tables, such as kdb+ HDB tables.

  • Scalability: Reduces redundancy and simplifies handling large datasets, making it more scalable for enterprise use.
  • Automatic Setup: A default database is created automatically to ease the initial setup process.

Tip: Use the database layer to organize multiple tables and indexes efficiently, reducing complexity in large-scale environments.

3. Multiple indexes (New)

We're thrilled to support multiple indexes that can share the same embedding column. This means there’s no need to duplicate the embeddings. Get ready for:

  • Flexible Index Management: Create and manage multiple indexes on a single table for diverse querying needs.
  • Simultaneous Searches: Execute searches across different indexes at the same time, ideal for multimodal datasets.
  • Dimensional Experimentation: Support for indexes with different dimensions to refine search accuracy.

Tip: Leverage multiple indexes to perform hybrid searches, combining dense and sparse indexes for comprehensive results.

4. q API (New)

A fully documented public q API is here, allowing q developers to use KDB.AI’s features within their q environment, empowering you with:

  • Reduced Friction: Provides a consistent toolset for developing advanced applications.
  • Enhanced Capabilities: Leverage KDB.AI’s power directly in q programming.

Tip: Use the q API to integrate KDB.AI’s advanced search functionalities into your existing q-based applications effortlessly.

5. Version information (New)

Quickly access KDB.AI server version information for compatibility checks in q API, Python API, and REST API.

  • Deployment Management: Helps ensure consistency across different environments.
  • Simplified Troubleshooting: Facilitates easier management of deployments.

Tip: Regularly check version information to maintain compatibility and streamline troubleshooting.

6. Enhanced REST API (Improvement)

1.4.0 brings improved adherence to RESTful conventions for a more user-friendly experience:

  • Consistent Error Handling: Better error management and troubleshooting tips for reliable application development.
  • Developer-Friendly: Enhances the overall developer experience with more intuitive API interactions.

Tip: Use the enhanced REST API to build robust applications with consistent error handling and improved debuggability.

7. Optimized kdb+ integration (Improvement)

  • Direct Access: Query kdb+ tables directly from KDB.AI, maintaining data integrity while utilizing advanced search features.
  • Time Series Analysis: Run Time Series Similarity (TSS) searches on kdb+ tables for insightful decision-making.
  • Optimized Indexing: Create indexes on kdb+ data within KDB.AI without altering original tables.

Tip: Use direct table access to seamlessly integrate kdb+ data with KDB.AI’s search capabilities, streamlining your data analysis workflows.

8. Enhanced symbol management (Improvement)

Symbol atoms are stored in symfiles, which are used to enumerate symbols in splayed or partitioned databases. Using symbols instead of strings can cause inefficiencies, especially when symfiles become large due to numerous non-distinct values. Using Python bytes objects to avoid too many symbols helps, providing:

  • Optimized Storage: Helps in reducing symfile size and improving performance.
  • Performance Management: Prevents degradation from large, non-distinct symbol values.
  • Effective Data Handling: Ensures efficient data processing and retrieval.

Tip: Use strings instead of symbols for large numbers of distinct values to keep symfile sizes manageable. Symbols are effective when you have many repeated values. If each value occurs only once, use strings instead. For more details, refer to the Kx documentation.

Summary

We hope you enjoy the v1.4.0 enhancements and new capabilities. Feel free to explore these updates. If you need assistance, reach out to our Support Team at support@kdb.ai or join our Slack channel.

For a complete list of past features and improvements, visit the previous KDB.AI Server release notes.

Thanks for reading. Stay tuned for our next release!

The KDB.AI Team

v1.3.0

Release date 2024.09.19

Requires Python client >= 1.3.0

Welcome to the KDB.AI Server 1.3.0 release! Key highlights:

  1. Fuzzy filtering on metadata
  2. On-disk index: qHNSW
  3. Early Access program
  4. Data migration
  5. Integration with Unstructured.io

1. Fuzzy filtering on metadata (New)

We’ve added support for fuzzy filtering on metadata, allowing for more flexible and approximate matching in queries. This enhancement significantly improves search relevance and result accuracy, making it easier to find what you’re looking for even with imperfect data.

Fuzzy filters are the ultimate game changers when dealing with data that might include errors, typos, or variations. For instance, applying fuzzy filters to searches allows you to locate documents with terms similar to your query, even if they aren’t an exact match.

Tip

Fuzzy filters have a wide range of applications across various domains. Key use cases include spell checking, data cleaning, autocomplete and suggestions, information retrieval, product matching in e-commerce, name matching in databases, geographic search, and code search.

Discover the concepts behind fuzzy filtering and head to our How to use fuzzy filters section to learn how to apply this method to your searches.

2. On-disk index: qHNSW (New)

Introducing the qHNSW index, a high-performance Hierarchical Navigable Small Worlds (HNSW) index that you can store and access directly on disk. This means you can use large indexes without the need for equivalent amounts of RAM, optimizing performance and resource utilization.

When stored on disk, the qHNSW index maintains its hierarchical graph structure, where vertices are connected based on their distances. This setup is crucial for enabling quick and efficient traversal through the graph during searches.

Tip

qHNSW can be extremely handy in several use cases such as web searches, e-commerce, Content-Based Image Retrieval (CBIR), genomic data search, protein structure search, geospatial applications, and financial fraud detection.

Read more about the differences between in-memory vs. on-disk indexes, then learn how to use the qHNSW index.

3. Early Acesss for KDB.AI integration with kdb+ (New)

We're thrilled to unveil an Early Access program for KDB.AI integration with kdb+. Participants will be provided with code samples to facilitate an understanding of how to set up this integration. If you're keen on joining, please send us an e-mail to obtain early insights and contribute to the development journey.

More about kdb+/q

kdb+ is a powerful column-based relational time series database (TSDB) with in-memory (IMDB) abilities. Built on top of the q programming language, kdb+ allows to operate directly on data. kdb+ is widely used in High-Frequency Trading (HFT) for storing, analyzing, processing, and retrieving large datasets at high speed.

4. Data migration (Improvement)

For users of previous versions such as 1.1.0 or 1.2.0 who wish to retain their existing data, please reach out to KX for detailed migration instructions. Our team is ready to assist you in ensuring a smooth transition to the latest version. Feel free to reach out if you have any questions or need further assistance!

5. Integration with Unstructured.io (New)

KDB.AI is now a destination connector on Unstructured.io, a popular ETL (Extract, Transform, Load) platform for unstructured data. This makes it much easier to ingest various complex documents into your KDB.AI vector database.

The benefits of this integration are:

  • Supports formats like plain text (TXT, XML), documents (CSV, HTML, PDF, and PPTX) and images (PNG, JPG).
  • Simplifies and speeds up the efficient transformation of unstructured data into structured formats.
  • More accurate LLM-generated RAG responses based on the ingestion of semantically relevant data into your vector database.

Tip

Rely on this integration to process and analyze unstructured data effectively and achieve effortless document extraction. Get ready to improve recommendation systems and chatbots by seamlessly handling both structured and unstructured data.

To see the integration at work, go to the Unstructured integration page and follow the steps.

Summary

We hope you enjoy the v1.3.0 enhancements, new index and integrations. Feel free to explore these updates. If you need assistance, reach out to our Support Team at support@kdb.ai or join our Slack channel.

For a complete list of past features and improvements, visit the previous KDB.AI Server release notes.

Download the latest KDB.AI Server version!

Thanks for reading. Stay tuned for our next release!

The KDB.AI Team

v1.2.4 release notes

Release date 2024.08.15

Fixes and improvements

  • This update includes several bug fixes and enhancements to improve KDB.AI performance and stability.
  • v1.2.2 and v1.2.3 introduced a series of internal changes that significantly enhance KDB.AI reliability and performance.

v1.2.0

Release Date 2024.06.27

Welcome to the KDB.AI Server 1.2.0 release! Let's look at the highlights:

  1. Higher performance – launching a series of significant performance enhancements.
  2. On disk indexes: qFlat – a Flat index that's not memory bound, leading to major infrastructure savings.

Now check out what's in store for each:

1. Higher performance (Improvement)

KDB.AI Server 1.2.0 brings considerable speed-ups, demonstrating KDB.AI’s ability to help you build fast, efficient, accurate, cost-predictable, and scalable solutions for your organization’s projects.

2. On-disk indexes: qFlat (New)

Managing indexes on-disk, as opposed to in-memory, provides a low-memory solution for working with large datasets of vector embeddings. That's why the 1.2.0 release introduces an on-disk alternative to the in-memory Flat index.

Labelled qFlat, this index supports all the API create and search capabilities. The only difference is that you need to specify qFlat instead of flat if you prefer the on-disk index to the in-memory one.

Tip

Pick qFlat over Flat if you prioritize capacity over speed. Keep in mind that on-disk is slower but cost-effective, while in-memory is faster but more expensive.

Read more about the differences between in-memory vs. on-disk indexes, then learn how to use the qFlat index.

Note

Please ensure that your data explicitly matches the data type required by your table schema. Not doing this may result in a type error. The example below ensures the correct data type for the StockPrice column.

import numpy as np

table_schema = {
    "columns": [
        {"name": "StockPrice", "pytype": "float32"},
        # rest of schema
    ]}

df = pd.DataFrame()
df = ReadStockData('stock_price_data.csv')
# Ensure correct stock price type
df['StockPrice'] = [np.float32(x) for x in df.StockPrice]

Summary

We hope you'll make the best of the v1.2.0 performance improvements and the new qFlat index. Now feel free to try them out. If you need any help, email our Support Team at support@kdb.ai or join our Slack channel.

For a full list of features and improvements we rolled out in the past, check out the previous KDB.AI Server release notes.

Download the latest KDB.AI Server version!

Thanks for reading. Stay tuned for the next release!

The KDB.AI team

v1.1.0

Release date 2024.03.13

The KDB.AI Server 1.1.0 release is here! This time we focused on enhancing your KDB.AI toolset with three new searches and an integration. Here are the highlights:

1. Hybrid search – combines a sparse vector search with a dense vector search.

2. Transformed TSS – temporal similarity search for transformed data.

3. Non-transformed TSS – temporal similarity search for non-transformed data.

4. LlamaIndex integration – you can now find KDB.AI in the LlamaIndex library.

5. Azure ML integration - includes the latest OpenAI release 1.17.0 and more notebooks to run tests on.

Now let's see what's in store for each new addition:

1. Hybrid search (NEW)

KDB.AI Server 1.1.0 brings a specialized vector search that increases the result relevancy across your applications. It's called a hybrid search and it’s a mix of two powerful search methods:

  • The precision of keyword-based sparse vector search.

  • The contextual understanding of semantic dense vector search.

This means that with KDB.AI, you can now combine and re-rank the results based on a user-set alpha value that weighs the importance of each search type.

Tip

Hybrid search is particularly useful in content recommendation systems, customer support automation, legal document search or human resources and recruitment.

Discover more uses cases on our Learning hub, head to our docs to understand the concepts behind this search method, or even better, learn how to conduct a hybrid search.

2. Transformed temporal similarity search (NEW)

Looking for a compression model that dimensionally reduces time series windows by over 99%, while faithfully preserving the original data’s shape? Using the Transformed TSS method, KDB.AI can store and perform lightning-fast vector searches on time-series data.

This breakthrough enables rapid and profound analytical insights, based on the following:

  • 1% of embedding size, 10x increase in search performance.

  • Memory and disk storage can be 1/100th of the original size.

  • Embeddings are auto generated at ingestion; no need for an external model/LLM.

  • When low latency results are more important than low latency data availability.

Leveraging windowed time series data compressed with Transformed TSS on KDB.AI enables the deciphering of emerging trends.

Tip

By providing fast comparison between current data and vast historical datasets, Transformed TSS is a game changer for financial and market predictions; online advertisement optimization, or retail customer understanding.

Head to the KDB.AI Learning hub for detailed use cases, read the docs to understand the main concepts behind this search method, then learn how to perform a Transformed TSS search.

3. Non-transformed temporal similarity search (NEW)

KDB.AI Non-Transformed TSS is a groundbreaking algorithm for near real-time similarity searches across fast-moving time series data. This new feature provides:

  • A precise and efficient method to analyse patterns and trends in time series data.

  • No need to embed, extract, or store time series vectors in the database.

  • The ability to re-configure search without requiring the overhead of re-indexing.

Tip

By enabling pattern matching, outlier detection, and near real-time analysis of datasets, you can count on Non-Transformed TSS for financial market analysis, sensor monitoring, cybersecurity threat detection and prevention, and healthcare monitoring.

Explore more use cases on the KDB.AI Learning hub, read our docs to learn the key terms, and learn how to execute a Non-Transformed TSS search.

4. LlamaIndex integration (NEW)

We're thrilled to introduce the integration of KDB.AI on LlamaIndex, a platform that simplifies the storage and retrieval of public and private datasets for RAG-enabled applications. With LlamaIndex and KDB.AI, developers can easily build popular RAG-enabled applications quickly and at scale, without worrying about the technical details and complexities.

The KDB.AI and LlamaIndex integration:

  • Enables LLMs to return dynamic and timely information.

  • Helps reduce memory requirements.

  • Enhances accessibility when contrasted with processing searches via raw embeddings.

  • Augments the LLM’s native output, resulting in a more precise and contextually relevant response to the user’s question.

Tip

You can combine LlamaIndex and KDB.AI to improve variety of applications, such as: document Q&A, data-augmented chatbots, knowledge base, FAQs, workflows, procedures, structured analytics, and content generation (blogs, articles, books, etc.).

Read more details on our LlamaIndex integration page and try our Advanced RAG with temporal filters using LlamaIndex and KDB.AI vector store notebook.

5. Azure ML integration (Improvement)

The Azure Marketplace package allows you to deploy an Azure ML studio, with an integrated KDB.AI Server 1.1.0. This includes the following third party dependencies:

  • KDB.AI client 1.1.0

  • LangChain OpenAI 0.1.1

  • OpenAI 1.17.0

Discover 6 new example notebooks, together with a much richer and extended set of data you can leverage to run your notebook tests on.

Read more about the Azure setup.

Summary

We hope you’re as excited as we are about the new KDB.AI Server searches and the LlamaIndex integration. Try them out and email our Support Team at support@kdb.ai or join our Slack channel if you need any help.

Download the latest KDB.AI Server version!

Thanks for reading. Stay tuned for the next release!

The KDB.AI team

v1.0.0

Release date 2023.11.27

New

  • Introducing REST API support, including insert and train functions.

v0.2.0

Release date 2023.11.03

Fixes and improvements

  • Minor fixes and performance improvements

v0.1.2

Release date 2023.10.17

Fixes and improvements

  • For KDB.AI Server instances it is no longer required to specify a hostname with an ordinal parameter (-h). If a hostname does have an ordinal parameter then the environment variable RT_NO_ORDINAL must be set to 0.

v0.1.1

Release date 2023.09.27

Fixes and improvements

  • Fixed issues with training functionality surrounding CPU instructions on certain machines.

  • Improved behavior surrounding synchronous API calls, including create and drop.

  • Improved responses from similarity search API by removing unneeded empty rows.

v0.1.0

Release date 2023.09.12

New

Introducing the first public release of KDB.AI.

  • Supported by Python clients with version less than or equal to 0.1.1