Lightning Fast Memory Files

The c-tree Server offers sophisticated caching of data and index files for the fastest retrieval times possible. These caches ensure your data safely persists to disk, simultaneously providing direct memory access to data. It would be extremely advantageous at times to have this data available directly with c-tree data handling API calls.

Consider the case of transient data in flight. While it is not critical at all times to ensure the actual transaction data is written to disk, it is important to quickly retrieve this item back when it is appropriate to process it. Fast indexed based retrieval is particularly important when there are thousands of these transactions in flight at any given moment. Financial authentication systems are a frequent example of this.

Memory files are very similar to c-tree data caches. The c-tree Server offers highly advanced tunable caches for your data to remain in memory for quick client retrieval. The c-tree application developer or c-tree Server administrator can control what information is maintained in cache. Add an entire file, a percentage of a file, or even omit certain files from cache. c-tree Server caches can be very, very large. 16GB memory caches have been reported in an actual c-tree Server commercial production environment.

Memory files are a unique feature of the c-tree Server. Memory files are just like any other c-tree data and index file. You open and close them, add, delete and update records. They can even persist after a file close should another client need to access the data later. Just be sure the c-tree Server doesn't shut down! Memory files remain volatile and do not persist to disk.

Applications of memory files

• Temporary files: temporary files can be created as memory data or index files. The file contents are always memory resident, and when the memory file is finally closed, it ceases to exist. Index files, in particular, are particularly good candidates as they can always be rebuilt if necessary. Create your c-tree data indexes in memory and leave your data on disk for the best of all worlds.
• Store read-only file contents: at server or application startup, the contents of a disk file can be read into a memory file and subsequent read requests can be satisfied quickly from the memory file. The reduced I/O to the disk drive hardware subsystem can provide substantial increased application throughput with many applications.
• In-memory list management: applications frequently need to maintain in-memory lists; a memory index provides easy creation and manipulation of items in a B+-tree-indexed list using the c-tree API, rather than requiring the developer to implement custom list management routines.

Create Memory Files with a c-tree Server Configuration Keyword

The simplest way to begin using memory files is through the use of a server configuration keyword. This feature allows developers to create memory files using their existing application code, provided that the file is created using an Xtd8 create function such as CreateIFileXtd8(). To create a memory file using the server configuration keyword, specify one or more entries of the form:

MEMORY_FILE <file name>#<max size>

where the file name may include wild card characters, and the maximum size is optional. If no maximum size is specified, then 4 GB is used. If a file is being created and matches one of the MEMORY_FILE file name entries, then it will be created as a memory file unless it is a superfile host, superfile member, mirrored, segmented or partitioned file.

To cause all possible files to be created as memory files, add the following configuration entry:

MEMORY_FILE *

The MEMORY_FILE keyword is useful to quickly test how a file or set of files will behave as memory files.

How do I Begin to Use Memory Files in My Application?

Memory files can also be created programmatically and monitored using our performance monitoring utilities. Grab all the details in Chapter 5 of the V8.14 Update Guide found on our support documentation web site.