Bloomberg Market Data Feed Handler

This page provides an overview of the KX Bloomberg Market Data Feed Handler, also known as the KX B-PIPE Feed Handler.

The Bloomberg B-PIPE Feed Handler allows you to stream data into a kdb+, kdb Insights SDK or kdb Insights Enterprise installation. It is made up of two components - the B-PIPE feed handler itself, and a second feed which streams EMRS entitlements data suitable for row-level access control. Both feeds should be used together for a compliant streaming data solution.

Get started

Get started with one of the following:

• Standalone Deployment
• kdb Insights SDK Deployment
• kdb Insights Enterprise Deployment

Settings

This section lists the settings for:

• bpipe
• rt
• persistence
• resources
• configMap

bpipe

The table below describes the settings for the KX BPIPE Feed Handler:

Setting	Command line	Environment variable	Type	Description
loggingFile	--logging.file	KXFEED_BPIPE_LOGGING_FILE	string	The application log file path for general log messages from the KX B-PIPE feed handler.
loggingLevel	--logging.level	KXFEED_BPIPE_LOGGING_LEVEL	string	Level of logging from the KX B-PIPE feed handler. Options are trace, debug, info, warning, error, and fatal (from lowest to highest severity).
bpipeHost	--bpipe.host	KXFEED_BPIPE_HOST	string	Host to connect to.
bpipePort	--bpipe.port	KXFEED_BPIPE_PORT	string	Socket port to use in the connection.
bpipeAuthType	--bpipe.auth.type	KXFEED_BPIPE_AUTH_TYPE	string	Authentication option: none, user, app, userapp, dir.
bpipeAppName	--bpipe.app.name	KXFEED_BPIPE_APP_NAME	string	Application name used for authentication.
bpipeSubscribeSyms	--bpipe.subscribe.syms	KXFEED_BPIPE_SUBSCRIBE_SYMS	string	Subscription symbols that are comma or semicolon separated.
bpipeMappingFile	--bpipe.mapping.file	KXFEED_BPIPE_MAPPING_FILE	string	Message type to table mapping file.
dbSchemaFile	--db.schema.file	KXFEED_BPIPE_DB_SCHEMA_FILE	string	Path to the database schema file.
dbConfigFile	--db.config.file	KXFEED_BPIPE_DB_CONFIG_FILE	string	Path to the database connection configuration file.
restServer	--rest.server	REST_SERVER	string	Start REST server to accept BPIPE subscribe and unsubscribe commands. Default port is 8080. Example: curl -X POST localhost:8080 -d "subscribe:sym1,sym2" or unsubscribe:sym1,sym2.
restServerPort	--rest.server.port	REST_SERVER_PORT	string	REST server port; default is 8080.
bpipeTableField1–10	--bpipe.table.fields1 … --bpipe.table.fields10	KXFEED_BPIPE_TABLE_FIELDS1 … KXFEED_BPIPE_TABLE_FIELDS10	string	Format: tableName:FIELD1,FIELD2,.... Example: --bpipe.table.fields1=summary:BLOOMBERG_EVENT_TIME_RT,SECURITY_NAME_REALTIME.

rt

Info

The following settings apply only to kdb Insights Enterprise deployments.

The table below describes the settings for configuring the RT publisher within the KX BPIPE Feed Handler.

Field	Type	Description
sinkName	string	The name of the RT stream to publish to.
logLevel	string	The logging verbosity of the RT publisher (INFO, WARN, ERROR, OFF).
logPath	string	The mount path of the Persistent Volume Claim (PVC) where the RT publisher stores local log files.
volCapacity	string	The size of the Persistent Volume Claim (PVC) that the RT publisher uses to store local files.

persistence

Info

The following settings apply only to kdb Insights Enterprise deployments.

The table below describes the settings for configuring persistence within the KX BPIPE Feed Handler.

Field	Type	Description
enabled	boolean	Enable or disable persistent storage, claimed through PVCs.
useLocalValues	boolean	Override global accessMode and storageClass.
storageClass	string	The storage class to apply to the PVC.
storageSize	string	The requested size of the volume.
accessModes	string	A list of desired access modes for the PVC.

resources

Info

The following settings apply only to kdb Insights Enterprise deployments.

Field	Description
limits	The maximum amount of resources that can be requested.
requests	The minimum amount of resources that must be requested.

For more information, refer to the RT publisher documentation.

configMap

Info

These settings are only relevant for kdb Insights Enterprise deployments.

Field	Description
create	Create a config map to mount configs in the config directory of the KX FIX Feed Handler helm chart. If using an external config map set this to false.
mount	Mount the named config map on the path /opt/kx/kxfeed_bpipe/config.
annotations	Annotations to add to the config map.
name	The name of the config map to use. If not set, a name is generated using the helm chart full name template.
files	List of files in the config directory that the config map should mount.

Schema consideration

The dbSchemaFile, in conjunction with the bpipeTableFields, defines the structure of the tables published downstream by the KX BPIPE Feed Handler.

Here's an example of a table definition in the dbSchemaFile:

    <table>
    <name>trade</name>
    <columns>
        <column>
            <name>sym</name>
            <type>SymbolAtom</type>
        </column>
        <column>
          <name>received</name>
          <type>TimestampAtom</type>
      </column>
        <column>
            <name>eID</name>
            <type>IntAtom</type>
            <id>EID</id>
        </column>
        <column>
            <name>messageFragment</name>
            <type>IntAtom</type>
        </column>
        <column>
            <name>messageType</name>
            <type>SymbolAtom</type>
            <id>MKTDATA_EVENT_TYPE</id>
        </column>
        <column>
            <name>messageSubType</name>
            <type>SymbolAtom</type>
            <id>MKTDATA_EVENT_SUBTYPE</id>
        </column>
        <column>
            <name>timeOfTrade</name>
            <type>TimeAtom</type>
            <id>RT_TIME_OF_TRADE</id>
        </column>
        </columns>
    </table>

Each table is defined by a list of columns. Each column has the following attributes:

• name: The name of the column.

• type: The kdb+ type of the column. The KX BPIPE Feed Handler supports the following types:

  • SymbolAtom: A kdb+ symbol. Also known as an interned string.
  • FloatAtom: A kdb+ float.
  • IntAtom: A kdb+ int.
  • CharAtom: A kdb+ character.
  • DateAtom: A kdb+ date.
  • TimestampAtom: A kdb+ timestamp
  • CharList: A kdb+ string.

• id: The BPIPE mnemonic.

The received column is a special column in a table definition. It does not require an id to be specified. Instead, when the KX BPIPE Feed Handler sees that a table definition has a column with the name received, it fills it with the current timestamp.

Mapping configuration

Here's an example mapping configuration for the bpipeMappingFile setting:

{
    "summary":"SUMMARY",
    "reference":"REFERENCE",
    "trade":"TRADE;MARKETDEPTH:TABLE",
    "quote":"QUOTE;MARKETDEPTH:TABLE"
}

In this file, each table is mapped to a list of Bloomberg message types, separated by ;.

Next steps

• Deploy the BPIPE Feed Handler with kdb Insights Enterprise
• Deploy the BPIPE Feed Handler with kdb Insights SDK
• Deploy the BPIPE Feed Handler standalone