March 31, 2014
New Compression Options Save You Space
Note: c-treeACE became FairCom DB in November 2020.
New compression options in c‑treeACE V10.3 help reduce your most demanding space requirements. As low-level data records are written to and read from disk, c‑treeACE intervenes just before they are passed to the operating system’s file system, and will “compress” before writing and “un-compress” after reading each data record.
Remember, c‑treeACE supports files up to 16 exabytes (18 million terabytes)! When data is sparsely populated, compression can substantially reduce your space needs.
Run Length Encoding (RLE) Compression
A new Run Length Encoding (RLE) compression option has been provided. This option can outperform zlib compression in many simple cases. For example, c‑treeACE data is frequently space or zero padded. RLE performs very well with this type of data.
Configure the RLE compression algorithm in ctsrvr.cfg, similar to the keywords for ZLIB or USER supplied compression as described below.
User-Defined Compression
In addition to Run Length Encoding (RLE) and zlib compression, c‑treeACE data record compression supports user-definable compression modules. The ctSETCOMPRESS function accepts a compression type of ctCMPRECuser, which, accompanied by a user defined DLL name, performs custom compression.
Configuring c‑treeACE Data Compression
The CMPREC_TYPE configuration keyword now supports the following options:
CMPREC_TYPE <"ZLIB" | "USER" | "RLE">
To enable data compression routines:
CMPREC_TYPE <"ZLIB" | "USER" | "RLE">
CMPREC_VERSION <a number >= 1>
CMPREC_DLL <name of DLL>
These keywords should be entered in the configuration file in the order shown. A DLL name is required for a CMPREC_TYPE of“USER”.
Dynamically load zlib
In V10.3.0 and later, c‑treeACE dynamically loads zlib instead of statically linking with it.
If the zlib library cannot be loaded and compression is attempted to be used, c‑treeACE returns error CMPR_ERR (946), sets sysiocod to 5 (indicating a DLL load error), and the first time c‑treeACE returns this error the following message is logged to CTSTATUS.FCS:
zlib compression support is not available: <error message>
Compressing/un-compressing existing data
It is now possible to enable/disable compression on existing data files programmatically by using the CMPIFIL() function or through the use of the ctcmpcif utility.
To enable or disable compression when calling CMPIFIL(), set the chgcompressIFILoption bit in the tfilno member of the IFIL structure whose address you pass to CMPIFIL(). When this bit is set, CMPIFIL() switches the compression state of the file: compressed files become un-compressed, un-compressed files become compressed.
To enable or disable compression when using the ctcmpcif utility, command-line options are provided:
- -compress – Create the compacted data file with compression
- -nocompress – Create the compacted data file without compression
For more information, see ctSetCompress in the c‑treeACE Programmer’s Reference Guide.
Discuss your unique compression needs with the FairCom engineering team and see what solutions can be evolved.