Working with files¶
This page explains how q names files and resolves paths. The formats themselves — text, binary, native KDB-X, and those reached through interfaces — each have their own page.
Every file operation in q starts from a file handle: a symbol that names a file or directory. The same handle serves read and write, and the shape of the data you pass decides which format you get.
File handles¶
A file handle is a symbol whose name begins with a colon. For example, this is the handle for the file new.txt in the current directory:
`:new.txt
The name after the colon is an optional relative or absolute path followed by the file name. Handles are a special form of symbol, distinguished only by that leading colon. Use hsym to convert an existing symbol into a handle rather than assembling the name by hand:
q)hsym `$"/tmp/test.txt"
`:/tmp/test.txt
Applying hsym twice gives the same result, so it is safe to apply to a name that might already be a handle — useful when the path is assembled at run time:
q)hsym hsym `$"/tmp/test.txt"
`:/tmp/test.txt
q always displays / as the path separator, even on Windows, where you can type either / or \.
Resolving relative paths¶
q reads files from relative or absolute paths. When it resolves a relative path, it searches these locations in order:
- The current directory
- The directory named by the
QHOMEenvironment variable - The directory named by the
QLICenvironment variable
q finds a file new.txt that exists only under QHOME from any working directory:
q)getenv `QHOME / location where new.txt is stored
"/Users/myuser/Development/q"
q)\pwd / current directory, which has no new.txt
"/tmp"
q)read0 `:new.txt
"I'm stored in QHOME!!"
Because q searches the current directory first, a file with the same name there takes precedence:
q)\pwd
"/tmp"
q)read0 `:new.txt
"I'm stored in the current directory!!!"
Absolute paths for anything long-lived
Search-order resolution is convenient when you use q interactively but also means a script result depends on where it runs from. Use hsym on an absolute path in code you intend to keep.
The formats¶
Each format has its own page:
- Text files — delimited and fixed-width records, key-value text, JSON, and streaming an input too large to hold in memory.
- Binary files — raw bytes and fixed-width binary records, with no type information of their own.
- q formatted files — q's own format, which records type and attributes so you can map a file straight back into memory.
- External file formats — Parquet through the built-in module, and the interfaces that reach Python, Arrow, ODBC, and HDF5.
- File utilities — size, directory listing, existence, deletion, and shell commands.
Next steps¶
- Build an on-disk database from an input too large for memory by streaming into a database.
- Read the operator references in full:
0:File Text and1:File Binary. - Persist data in the native format with q files.
- See how tables live on disk as splayed, partitioned, and segmented databases, and the shape of a whole database.
- Tune file compression and set up encryption at rest.
- Browse the file system keywords for the complete set of one-shot and handle-based operations.