KURL reference

This page provides detailed technical reference for the Kurl module APIs.

Registration

By registering with Kurl, requests made against a given domain wildcard will use cached authentication information returned from the registration call.

There are a number of different ways that the module allows you to register. The .kurl.init function listed below will cycle through these options for you.

If you want to explicitly call the underlying APIs, these are also documented below.

Registration credentials

Pre-requisite

When registering with cloud vendors you must first obtain an access key, secret and token for your cloud provider. These can be defined in a config file or as environment variables. See examples for each below:

Amazon Web ServicesMicrosoft AzureGoogle Cloud Platform

export AWS_ACCESS_KEY_ID=".."
export AWS_SECRET_ACCESS_KEY=".."
export AWS_SESSION_TOKEN=".."

These values could also be stored in a credentials file, typically this is ~/.aws/credentials.

cat ~/.aws/credentials
[default]
aws_access_key_id=".."
aws_secret_access_key=".."
aws_session_token=".."

export AZURE_STORAGE_ACCOUNT="..."
export AZURE_STORAGE_SHARED_KEY="..."

gcloud init
export GCP_TOKEN=$(gcloud auth print-access-token)

.kurl.init

This API accepts a list of vendors or none to register against.

Input Parameters

Name	Type	Description	Required	Default
vendors	list/symbols	The cloud vendor to connect to, options include - aws, azr, gcp, none or leave empty to register with 3 cloud vendors	No	all

Example

.kurl:use`kx.kurl
.kurl.init`aws // register with AWS
.kurl.init[]  // register with AWS, AZR, GCP

.kurl.register

This API accepts a 4 item list. The elements in this list as defined below:

Input Parameters

Name	Type	Description	Required	Default
type	symbol	one of aws_cred, aws_sts, oauth2, oauth2_jwt, azure, basic	Yes	None
domain	str	wildcard for domain	Yes	None
tenant	str/symbol	session name, username, ID, etc., meaningful to your application	Yes	""
authInfo	list/dict	authentication info specific to the type	Yes	None

Example

Registering using environment variables:

info:`AccessKeyId`SecretAccessKey`Token!(getenv`AWS_ACCESS_KEY_ID;getenv`AWS_SECRET_ACCESS_KEY;getenv`AWS_SESSION_TOKEN)
.kurl.register (`aws_cred;"*amazonaws.com";"";info)

AWS Registration

.kurl.aws.registerByCredentialsFile

This API accepts a credentials file.

Input Parameters

Name	Type	Description	Required	Default
file	symbol	credentials file location	Yes	None

Example

.kurl.aws.registerByCredentialsFile hsym`$getenv[`HOME],"/.aws/credentials"

setenv[`AWS_SHARED_CREDENTIALS_FILE] "/tmp/credentials"
.kurl.aws.registerByCredentialsFile hsym`$getenv`AWS_SHARED_CREDENTIALS_FILE

.kurl.deregister

This API accepts a 2 item list. The elements in this list as defined below:

Input Parameters

Name	Type	Description	Required	Default
domain	str	wildcard for domain	Yes	None
tenant	str/symbol	session name, username, ID, etc., meaningful to your application	Yes	""

Example

.kurl.deregister("*amazonaws.com";"")

Requests

Request examples

In the examples below it assumes that a CSV file is present in a S3 bucket which you can access.

.kurl.sync

Input Parameters

Name	Type	Description	Required	Default
url	str	URL to target	Yes	None
method	str	GET or POST request	Yes	None
options	dict	Can contain the keys:	Yes	""

Example

.kurl.sync ("https://kx-bucket.s3.us-east-2.amazonaws.com/test_data/data.csv";`GET;::)
200i
"sym,price,size\nFDP,1.2,100\n"

// example with no registration required
.kurl.sync ("https://api.census.gov/data/2020/dec/pl?get=NAME,P1_001N&for=state:*"; `GET; ::)

.kurl.async

Input Parameters

Example

.kurl.async ("https://kx-bucket.s3.us-east-2.amazonaws.com/test_data/data.csv";`GET;``callback!(`;{.debug.x:x}))
.debug.x
200i
"sym,price,size\nFDP,1.2,100\n"