December 31, 2014
Unix Shared Memory for Faster Localhost Connections
Note: c-treeACE became FairCom DB in November 2020.
c‑treeACE V10.0 introduced shared memory connections for localhost Unix/Linux connections allowing much faster local client/server communication. Windows users had enjoyed this support with prior versions. Shared-memory communication between clients and servers residing on the same machine generally provides much better performance for locally running applications. Local shared memory connections are supported across the board including ISAM and SQL connections, and this includes JDBC and Windows ADO.NET Data providers.
Note: Shared memory support is not extended to Linux Kernels 2.4 or MacOS X systems as those platforms do not support the necessary interprocess mutexes.
As shared memory for Unix connections is still new to many folks, it is important they understand all of the parameters around this system level support.
Configuration
Include the following server configuration in ctsrvr.cfg to enable this support:
COMM_PROTOCOL FSHAREMM
c‑treeACE client libraries are compiled with this featured enabled by default.
System Files, Permissions and Ownership
The c‑treeACE shared memory communication protocol creates a file used by clients to find the shared memory identifier for its shared memory logon region, and creates a Unix domain socket as a file for initial communication between a client and server.
c‑treeACE creates the directory /tmp/ctreedbs and the file /tmp/ctreedbs/<servername>.logon. This file name is determined by the value specified with the SERVER_NAME configuration option (but see important note below). This file contains an identifier of a shared-memory region used for clients to connect. The following configuration option allows this directory to be directly specified:
SHMEM_DIRECTORY <directory_name>
IMPORTANT: SERVER_PORT applies to the TCP/IP protocol and overrides SERVER_NAME if both are used together.
If your server combines shared memory and TCP/IP usage, here are a few tips:
- If you are content with the TCP/IP port resulting from the SERVER_NAME option, then use that option and you can connect using the name with either protocol.
- If you wish to explicitly set the TCP/IP port, use SERVER_PORT to set that port (then connect with #port to use TCP/IP on that port) and SERVER_NAME to set the name used by the shared memory protocol. Note that this approach means that a connection attempt will not be able to ‘fall back’ to using TCP/IP if the shared memory connection fails, unless you choose your server name so that it matches your SERVER_PORT setting. For example, consider the following set of options:
SERVER_PORT 7000
SERVER_NAME #7000
Then connect with a server name of #7000. The client will attempt to connect using shared memory first and if that fails it will connect with TCP/IP.
c‑treeACE must have sufficient read, write, create, and delete permissions with this directory. The following server keyword sets the shared memory resource permissions:
SHMEM_PERMISSIONS <permissions>
The default is 660. 666 will allow access to c‑treeACE by any user account.
Note: Use caution when increasing the access permissions to the shared memory resources. For example, shared memory permission of 666 allows any user to attach to a shared memory segment and read or write to it. This means that any process could make a request to c‑tree Server or could read the request data of another process through such a shared memory region.
By default, a client application must belong to the server owner’s primary group to use shared memory. This is configurable with the SHMEM_GROUP keyword.
SHMEM_GROUP <group>
Shared Memory Keys
When more than one c‑treeACE process is run on a Unix system, the shared memory key used by the servers might hash to the same value, causing problems connecting to the servers. This happens as the ftok() system call is used by default to generate the shared memory keys, and ftok() is not guaranteed to return unique values. Another possibility is that another unrelated process might happen to use the same shared memory key as generated by c‑treeACE.
As of V10.4, an administrator can specify a specific shared memory key for ISAM and SQL shared memory communication protocols to ensure that keys do not match keys already in use on the system. This is specified with the following c‑treeACE configuration options:
SHMEM_KEY_ISAM <isam_shared_memory_key>
SHMEM_KEY_SQL <sql_shared_memory_key>
The shared memory key values can be specified in either decimal or hexadecimal format. For example:
; Set shared memory key for ISAM connections to the specified decimal value:
SHMEM_KEY_ISAM 12345
; Set shared memory key for ISAM connections to the specified hexadecimal value:
SHMEM_KEY_ISAM 0xabcd
Client Configuration
From the client side, either set the global variable ctshmemdir to the directory name before connecting, or set the CTREE_SHMEM_DIRECTORY environment variable. The environment variable takes precedence over the ctshmemdirsetting. This allows the directory to be dynamically overridden without having to recompile client code.
Errors with Shared Memory Protocol
c‑treeACE logs error messages to CTSTATUS.FCS when a shared-memory connection attempt fails. The message is of the form
FSHAREMM: <error message>
Adjusting System Limits
Many Unix/Linux implementations have a default limit of 128 shared-memory connections via system semaphores. This is typically a kernel configuration. When running c‑treeACE with more than 128 shared-memory connections, you may encounter the following error:
FSHAREMM: Connect named pipe failure: 13
To increase this limit, add the following to your local /etc/sysctl.conf file.
kernel.msgmni = 1024
kernel.sem = 250 256000 32 1024
Run this command to then enable support:
/sbin/sysctl -p
Note: In general, you will require root superuser access to make these changes. (Consult your specific Unix/Linux documentation for the actual file location and parameters of this configuration.)
The ipcs command displays current limits:
#ipcs -l
------ Semaphore Limits --------
max number of arrays = 128
Once enabled, shared memory for local Unix connections allows significant performance gains for those applications. For many legacy applications making the jump to c‑treeACE client/server architecture, this is a substantial feature to consider.