# Working with binary files

_This page explains how to read and write raw binary data in q: whole files and fixed-width records, appending over several writes, and compressing as you write._

## Reading bytes

[`read1`](../../../ref/read1.md) reads a file as a byte vector. The following example writes and then reads back a 40-byte file of 10 4-byte integers:
```q
q)`:integers.dat 1: raze reverse each 0x0 vs/:`int$til 10
`:integers.dat
q)read1 `:integers.dat
0x00000000010000000200000003000000040000000500000006000000070000000800000009000000
```

You can use an offset and a length to read a subsection — here from byte 4 for 36 bytes:
```q
q)read1 (`:integers.dat;4;36)
0x010000000200000003000000040000000500000006000000070000000800000009000000
```

### Reading text as bytes

Casting the bytes from `read1` to characters gives the file as one string with its line breaks preserved, which is what you want when the breaks themselves matter. For the `test.txt` created in [text files](text-files.md):
```q
q)"c"$read1 `:test.txt
"hello\nworld\n"
```

Compare with [`read0`](text-files.md#reading-text), which splits on the line breaks and discards them.

## Fixed-width binary records

[`1:`](../../../ref/file-binary.md#read-binary) reads binary data as fixed-width records, given the field types, widths, and endianness. For more detail on options, see [column types and widths](../../../ref/file-binary.md#column-types-and-widths). It repeats the record definition until it reaches the end of the file.

If you read the 10-integer file above as records of two 4-byte integers, you get two columns of five:
```q
q)("ii";4 4)1:`:integers.dat
0 2 4 6 8
1 3 5 7 9
```

An offset and a length apply here too. Start at byte 4 and read 8 bytes to get a single record:
```q
q)("ii";4 4)1:(`:integers.dat;4;8)
1
2
```

This is how you skip a header or footer, where the layout differs from the records that follow.

> **Tip — Binary load is multithreaded**
>
>
> Like the text loaders, `1:` reads in parallel when q runs with [secondary threads](../../../ref/syscmds.md#s-number-of-secondary-threads).
>
## Writing bytes

[`1:`](../../../ref/file-binary.md#save-binary) with a file handle on the left writes raw bytes. The following example writes the five ASCII bytes that spell `hello` — `0x68` is `h`, `0x65` is `e`, and so on:
```q
q)`:file.txt 1: 0x68656c6c6f
`:file.txt
```

`1:` writes any vector as its binary representation. This creates a 400-byte file of 100 4-byte integers:
```q
q)`:ints.dat 1:"i"$til 100
`:ints.dat
```

> **Warning — `1:` writes values only, not metadata**
>
>
> A raw binary write records the bytes of the values and nothing else, so it discards attributes. The [q format](kdb-files.md) preserves them.
>
> Writing a plain list and a [sorted](../../../ref/set-attribute.md#sorted) list produces byte-identical files, as [`md5`](../../../ref/md5.md) hashes of the results confirm:
>
> ```q
> q)a: til 10
> q)b: `s#til 10
> q)`:a 1: a
> `:a
> q)`:b 1: b
> `:b
> q)md5 "c"$read1 `:a
> 0xcdff2051b710c66f3373e26c901ee505
> q)md5 "c"$read1 `:b
> 0xcdff2051b710c66f3373e26c901ee505
> ```
>
> Contrast with [`save`](kdb-files.md#save-and-load), which keeps the attribute — the 16 bytes of [file header](kdb-files.md#the-file-header) are the difference.
>
## Appending bytes with `hopen`

[`hopen`](../../../ref/hopen.md#files) opens a binary file for appending in the same way it opens a text file. Apply the handle to bytes or characters, and [`hclose`](../../../ref/hopen.md#hclose) it when done:
```q
q)h:hopen `:test.dat
q)h "a"
6i
q)h "bc"
6i
q)h 0x21
6i
q)hclose h
```

Each write echoes the handle itself, not a byte count. `test.dat` holds `abc!` with no line breaks, `0x21` being the ASCII code for `!`. The file carries no type information, so [`get`](../../../ref/get.md#get) cannot read it as q data:
```q
q)get `:test.dat
'test.dat
  [0]  get `:test.dat
       ^
```

Opening a file that _does_ carry a q header behaves differently: the same handle then appends typed values and the file stays readable — see [appending with `hopen`](kdb-files.md#appending-with-hopen). Which behavior you get depends on the file, not on how you open it.

## Compressing a binary file

Extend the left argument of `1:` with [compression parameters](../../interact_with_databases/file-compression.md#compression-parameters) to compress as you write. [`-21!`](../../../ref/internal.md#-21x-compressionencryption-stats) reports the compression achieved:
```q
q)(`:file;17;2;9) 1:"i"$til 1000000
`:file
q)-21!`:file
compressedLength  | 1383683
uncompressedLength| 4000000
algorithm         | 2i
logicalBlockSize  | 17i
zipLevel          | 9i
```

Set [`.z.zd`](../../../ref/dotz.md#zzd-compressionencryption-defaults) to apply the same parameters to every write, so you don't need to repeat them:
```q
q).z.zd:(17;2;9)
q)`:file 1:"i"$til 1000000
`:file
q)-21!`:file
compressedLength  | 1383683
uncompressedLength| 4000000
algorithm         | 2i
logicalBlockSize  | 17i
zipLevel          | 9i
q)`:another_file 1:til 10000
`:another_file
q)-21!`:another_file
compressedLength  | 15194
uncompressedLength| 80000
algorithm         | 2i
logicalBlockSize  | 17i
zipLevel          | 9i
```

You don't need parameters to read a compressed file, because the file records the algorithm it used.

## Next steps

- Parse the same bytes as text instead: [text files](text-files.md).
- Keep types and attributes by writing q's own format: [q formatted files](kdb-files.md).
- Tune compression properly for a database in [file compression](../../interact_with_databases/file-compression.md).
- Read the operator reference in full: [`1:` File Binary](../../../ref/file-binary.md).
