Protobuf/Protocol Buffers interface example
It is assumed in the example that you are executing logic in the root of the protobufkdb
repository. The file structure for the required components from this location is:
./
├── proto/
│ ├── google
│ ├── kdb_type_specifier.proto
│ └── sample.proto
└── q/
└── protobufkdb.q
Here sample.proto
is defined as follows:
syntax = "proto3";
option cc_enable_arenas = true;
import "kdb_type_specifier.proto";
message Office {
repeated string name = 1;
double longtitude = 2;
double latitude = 3;
int32 accessed = 4 [(kdb_type) = DATE];
}
message Region {
string region = 1;
map<string, Office> office = 2;
}
Load Protobufkdb library
From the root of the repository load the protobufkdb
library
q)\l q/protobufkdb.q
Import schema file
When using dynamic import functionality, specify the path of your Proto (schema) files first. As such we tell protobufkdb
the location of schema files of interest, specifying its path ("./proto"
) and importing the relevant file ("sample.proto"
).
This step is not necessary when using a library built from source, where schema files are linked appropriately.
q).protobufkdb.addProtoImportPath["proto"]
q).protobufkdb.importProtoFile["sample.proto"]
Check schema file
Ensure the schema file has been imported appropriately. Display the schemas in it.
q).protobufkdb.displayMessageSchema[`Office]
message Office {
repeated string name = 1;
double longtitude = 2;
double latitude = 3;
int32 accessed = 4 [(.kdb_type) = DATE];
}
q).protobufkdb.displayMessageSchema[`Region]
message Region {
string region = 1;
map<string, .Office> office = 2;
}
Serialize data
A Protobuf message is expressed in a mixed list in q. This data can be serialized to Protobuf provided the message content matches the format of a named target schema.
// Serialize the details of an office to a Protobuf array
q)HQ:(("Head_Office";"Brian_Conlon_House"); 54.179127; -6.337371; 2020.03.23)
q)encodedHQ:.protobufkdb.serializeArrayFromList[`Office; HQ]
q)encodedHQ
"\n\013Head_Office\n\022Brian_Conlon_House\021Qj/\242\355\026K@\031\253y\216\..
// Serialize the details of all offices in a region to a Protobuf array using Arenas
q)newry:(("Newry_Office";"The_Conlon_Building"); 54.1751772; -6.3378739; 2020.08.14)
q)belfast:(enlist "Belfast"; 54.592595; -5.927475; 2020.08.14)
q)EMEA:`HQ`Newry`Belfast!(HQ; newry; belfast)
q)encodedEMEA:.protobufkdb.serializeArrayArenaFromList[`Region; ("EMEA"; EMEA)]
q)encodedEMEA
"\n\004EMEA\022<\n\002HQ\0226\n\013Head_Office\n\022Brian_Conlon_House\021Qj/..
Deserialize data
Retrieve the contents of a Protobuf message serialized in the above example.
// Retrieve encoded information about an office from a Protobuf message
q).protobufkdb.parseArrayToList[`Office; encodedHQ]
("Head_Office";"Brian_Conlon_House")
54.17913
-6.337371
2020.03.23
// Retrieve encoded information about a region from a Protobuf message using Arenas
q).protobufkdb.parseArrayArenaToList[`Region; encodedEMEA]
"EMEA"
`HQ`Newry`Belfast!((("Head_Office";"Brian_Conlon_House");54.17913;-6.337371;2..
Save serialized data
You can serialize and save data to a file by specifying a target schema and the target file name.
q).protobufkdb.saveMessageFromList[`Office; "proto/record_HQ"; HQ]
Load serialized data
Retrieve data serialized and saved to a file based on a specified schema.
q).protobufkdb.loadMessageToList[`Office; "proto/record_HQ"]
("Head_Office";"Brian_Conlon_House")
54.17913
-6.337371
2020.03.23
More examples: protobufkdb/examples