# Logging, recovery and replication¶

Software or hardware problems can cause a q server process to fail, possibly resulting in loss of data not saved to disk at the time of the failure. A q server can use logging of updates to avoid data loss when failures occur; note that the message is logged only if it changes the state of the process’ data.

Logging is enabled by using the -l or -L command-line arguments.

$q logTest -l q)\p 5001 q)\l trade.q Now update messages from clients are logged. For instance: q)/ this is a client q)h: hopen :localhost:5001 q)h "insert[trade](10:30:01; intel; 88.5; 1625)" q)h "last trade" time | 10:30:01.000 sym | intel price| 88.5 size | 1625 Assume that the q server process dies. If we now restart it with logging on, the updates logged to disk are not lost: $ q logTest -l
q)last trade
time | 10:30:01.000
sym  | intel
price| 88.5
size | 1625

Warning

Updates done locally in the server process are logged to disk only if they are sent as messages to self. The syntax for this uses 0 as the handle:

q) // in server
q)0 ("insert";trade; (10:30:01.000; intel; 88.5; 1625))

## Checkpointing¶

A logging server uses a .log file and a .qdb data file. The command

q)\l
checkpoints the .qdb file and empties the log file.

However, the checkpoint is path-dependent. Consider the following:

/tmp/qtest$q qtest -l q) A listing of the directory gives: /tmp/qtest$ ls
qtest.log
Back in q:
q)\l
yields
/tmp/qtestls
qtest.log
qtest.qdb
However, if there is a change of directory within the q session then the *.qdb checkpoint file is placed in the latter directory. For instance:
/tmp/qtest$q qtest -l q)\cd ../newqdir q)\l results in /tmp/qtest$ ls
qtest.log
/tmp/qtest$cd ../newqdir /tmp/newqdir$ ls
qtest.qdb
The simplest solution is to provide a full path to the log file at invocation.
$/tmp/qtest$ q /tmp/testlog -l
q).z.f
/tmp/testlog
q)\cd ../newqdir
q)\l
results in
/tmp/qtest$ls . testlog.log testlog.qdb /tmp/qtest$ ls ../newqdir
.

When you type

$q logTest -l this reads the data file (.qdb), log file, and the q script file logTest.q, if present. If any of the three files exists (.q, .qdb, and .log), they should all be in the same directory. ## Logging options¶ The -l option is recommended if you trust (or duplicate) the machine where the server is running. The -L option involves an actual disk write (assuming hardware write-cache is disabled). Another option is to use no logging. This is used with test, read-only, read-mostly, trusted, duplicated or cache databases. ## Errors and rollbacks¶ If either message handler (.z.pg or .z.ps) throws any error, and the state was changed during that message processing, this will cause a rollback. ## Replication¶ Given a logging q process listening on port 5000, e.g. started with $ q test -l -p 5000
a single q process can replicate that logging process via
$q -r :localhost:5000:username:password if starting these processes from different directories, be sure to specify the absolute path for the logging process, e.g. $ q /mylogs/test -l -p 5000`