Configure Linux-based systems
The default operating system settings on Unix-style operating systems, such as Linux, Raspbian, Android IoT, AIX, and Solaris, are adequate for running a typical FairCom server. To customize the server to meet larger (or smaller) needs, you can change the operating system limits listed below.
Configure shared memory communications on Linux
FairCom's ISAM, CTDB, and SQL APIs to communicate with the FairCom server using shared memory and TCP/IP. FairCom's client libraries for these APIs automatically pick the best protocol.
Applications and FairCom servers can reside on the same computer. When they do, ISAM, CTDB, and SQL APIs (including JDBC and ADO.NET) automatically use shared memory communications because shared memory is faster than TCP/IP. When applications and FairCom servers run on different computers, communications automatically occur over TCP/IP because shared memory does not work across computers.
FairCom's jsonAction APIs communicate using TCP/IP protocols, including HTTP and WebSocket.
Configuration
By default, the following server configuration in ctsrvr.cfg enables shared memory communications:
COMM_PROTOCOL FSHAREMM
You do not need to modify the shared memory settings listed in the following sections unless you have specific needs.
Note
The COMM_PROTOCOL option specifies the protocol used for ISAM connections. By default, local SQL connections use shared memory unless the SQL_OPTION NO_SHARED_MEMORY keyword is specified. See the FairCom DB SQL server operations and utilities guide for more information about the communication protocol for SQL connections.
System files, permissions, and ownership
The FairCom shared memory communication protocol creates a file used by clients to find the shared memory identifier for its shared memory log on region, and creates a Unix domain socket as a file for initial communication between a client and server.
The FairCom server creates the directory /tmp/ctreedbs and the file /tmp/ctreedbs/<servername>.logon. This file name is determined by the value specified with the SERVER_NAME configuration option (but see the important note below). This file contains an identifier of a shared memory region used for clients to connect. The following configuration option allows this directory to be directly specified:
SHMEM_DIRECTORY <directory_name>
Important
SERVER_PORT applies to the TCP/IP protocol and overrides SERVER_NAME if both are used together.
If you are content with the TCP/IP port resulting from the
SERVER_NAMEoption, then use that option and you can connect using the name with either protocol.If you wish to explicitly set the TCP/IP port, use
SERVER_PORTto set that port (then connect with #port to use TCP/IP on that port) andSERVER_NAMEto set the name used by the shared memory protocol. Note that this approach means that a connection attempt will not be able to 'fall back' to using TCP/IP if the shared memory connection fails, unless you choose your server name so that it matches yourSERVER_PORTsetting. For example, consider the following set of options:SERVER_PORT 7000 SERVER_NAME #7000
Then connect with a server name of #7000. The client will attempt to connect using shared memory first and if that fails it will connect with TCP/IP on port 7000.
The FairCom server must have sufficient read, write, create, and delete permissions with this directory.
The following server keyword sets the shared memory resource permissions:
SHMEM_PERMISSIONS <permissions>
The default is 660. 666 will allow access to FairCom EDGE by any user account.
Note
Use caution when increasing access permissions to shared memory resources. For example, shared memory permission of 666 allows any user to attach to a shared memory segment and read or write to it. This means that any process can make a request to a FairCom Server or could read the request data of another process through such a shared memory region.
By default, a client application must belong to the server owner’s primary group to use shared memory. This is configurable with the SHMEM_GROUP keyword.
SHMEM_GROUP <group> Possible errors indicating problems: FSHAREMM: Could not get group ID for group <group> for shared memory FSHAREMM: Failed to set group for LQMSG shared memory region: X
Shared memory keys
When more than one FairCom server process is run on a Unix system, the shared memory key used by the servers might hash to the same value, causing problems connecting to the servers. This happens as the ftok() system call is used by default to generate the shared memory keys, and ftok() is not guaranteed to return unique values. Another possibility is that another unrelated process might happen to use the same shared memory key as generated by the FairCom server.
An administrator can specify a specific shared memory key for ISAM and SQL shared memory communication protocols to ensure that keys do not match keys already in use on the system. This is specified with the following FairCom configuration options:
SHMEM_KEY_ISAM <isam_shared_memory_key> SHMEM_KEY_SQL <sql_shared_memory_key>
The shared memory key values can be specified in either decimal or hexadecimal format, for example:
; Set shared memory key for ISAM connections to the specified decimal value: SHMEM_KEY_ISAM 12345 ; Set shared memory key for ISAM connections to the specified hexadecimal value: SHMEM_KEY_ISAM 0xabcd
Client configuration
From the client side, either set the global variable ctshmemdir to the directory name before connecting, or set the CTREE_SHMEM_DIRECTORY environment variable. The environment variable takes precedence over the ctshmemdir setting. This allows the directory to be dynamically overridden without having to recompile client code.
Errors with shared memory protocol
The FairCom server logs error messages to CTSTATUS.FCS when a shared memory connection attempt fails. The message is of the form:
FSHAREMM: <error message>
Adjust system limits
When running the FairCom server with more than 128 shared memory connections, you may encounter one of the following errors:
FSHAREMM: Connect named pipe failure: 13 FSHAREMM: Connect named pipe failure: 28
Many Unix/Linux implementations have a default limit of 128 system semaphores, which are used by FairCom shared memory connections. However, this value applies system wide among all processes.
FSHAREMM: Failed to create system semaphore: check system semaphore limits such as SEMMNI
The following error can be reported as well:
FSHAREMM: Failed to create shared memory segment: check shared memory limits such as SHMMNI
These are typically kernel configurations. The FairCom server requires (2 + # shared memory CONNECTIONS) shared memory segments (SHMMNI) and semaphores (SEMMNI).
The ipcs command displays current limits:
#ipcs l
Semaphore Limits
max number of arrays = 128
Shared Memory Limits
max number of segments = 128To increase limits to allow up to 1024 shared memory segments and semaphores, consider adding the following to your local /etc/sysctl.conf file.
kernel.shmmni = 1024 kernel.sem = 250 256000 32 1024 Run this command to then enable support: /sbin/sysctl p
Note
In general, you will require root superuser access to make these changes. Consult your specific Unix/Linux documentation for the actual file location and parameters of this configuration.
Enable Shared Memory
Shared memory communications are enabled by default in FairCom Server V12 and later.
Shut down your FairCom server and add the following keyword to your
ctsrvr.cfgfile:COMM_PROTOCOL FSHAREMM
Restart the FairCom server.
Execute any FairCom utility you've linked with V9.5 or later of the FairCom server you have on the same machine as the FairCom server process. Even if you are linked with a FairCom TCP/IP library, it will automatically detect if you are running on the same machine and try to connect via shared memory. This way you don't need multiple versions of your application and utilities.
You can monitor your connections by listing the clients from the ctadmn command-line utility on Linux, or by using the FairCom Monitor browser-based user interface.
Shared memory usage notes
FairCom's client libraries for ISAM, CTDB, and SQL APIs support shared memory by default.
When the FairCom Server detects a request to connect from the same machine as the client, it first attempts to connect using shared memory. If that succeeds, the connection uses the shared memory protocol. Otherwise, the connection is made using TCP/IP.
32-bit clients can connect to 64-bit servers (and vice versa).
By default, a client application must belong to the server owner’s primary group to use shared memory. This is configurable with the
SHMEM_GROUPkeyword.Shared memory uses a Unix domain (file system) socket for transferring data between the client and server. The Unix domain socket exists as a file named
/tmp/ctreedbs/<servername>.logon.Note
The COMPATIBILITY SHMEM_PIPE keyword has been deprecated and no longer has any effect.
A Unix/Linux server using shared memory communications will create a directory
/tmp/ctreedbs. If this directory already exists (for example, if a different user had started the server, even from a previous run) and the server does not have write permission to this directory, startup will fail, most likely reporting a DSRV_ERR error (509, duplicate server), even if no other server is currently running.pthread mutex shared memory support is expected for Unix systems. However, if a client and the server are compiled with incompatible options (for example, the client uses System V semaphores but the server uses pthread mutexes) the connection attempt will fail and the FairCom server will log one of the following messages to CTSTATUS.FCS:
If client is using System V semaphore and server is using pthread mutex:
A shared memory connection attempt by an incompatible client failed: pthread mutex required
If server is using System V semaphore and client is using pthread mutex:
A shared memory connection attempt by an incompatible client failed: SYSV sema required
System tools
The Unix/Linux ipcs utility is useful for listing the shared memory regions and semaphore sets that currently exist on a system. ipcs can also be used to remove shared memory regions and semaphore sets. Under normal circumstances, the FairCom server removes shared memory regions and semaphore sets for connections that have been closed. However, if the FairCom process terminates abnormally, it may be necessary to manually remove the shared memory regions and semaphore sets belonging to this process.
The Unix version logs message when shared memory cannot create semaphore or segment.
On Unix systems, the FairCom server logs a message to CTSTATUS.FCS when it fails to create a system semaphore or shared memory segment due to a resource limit setting. This helps the server administrator understand the cause of the error. Example message:
Fri Aug 22 15:14:44 2014 User# 00013 FSHAREMM: Failed to create system semaphore: check system semaphore limits such as SEMMNI Fri Aug 22 15:14:44 2014 User# 00013 NewUser: Unable to create an instance of a named pipe
Other possible shared memory messages
FairCom ISAM and SQL ports are independent of each other. In general, there is a shared memory connection protocol enabled for each, in addition to TCP/IP ports. Keep in mind, that's a total of four connection protocols, with configuration options for each.
Possible SQL shared memory connection errors can appear such as the following. Analyze and correct these as you would for the ISAM errors previously mentioned as the same parameters should be examined.
User# 00012 sqlshmlisten: shared memory protocol initialization failed: 1 User# 00012 sqlshmlisten: shared memory protocol accept failed: 2 User# 00012 sqlshmlisten: Failed to get shared memory environment: XX Named pipe creation failed with error ERROR_PIPE_BUSY. The operation will be retried User# 00012 sqlshmlisten: shared memory protocol listen failed: 4 Failed permissions on the temporary directory needed. User# 00012 SQLSHAREMM: Failed to open /tmp/ctreedbs/CTSQL_6597 for shared memory: 13
Specify shared memory keys on Unix
When more than one FairCom server was run on a Unix system, the shared memory keys used by different servers could have the same value, which prevented connections to the servers. In addition, it was possible for unrelated applications to collide with default keys generated by FairCom servers.
To address this key collision, it is possible for an administrator to specify specific shared memory keys for ISAM and SQL shared memory communication protocols ensuring the keys do not match existing keys already in use on the system.
FairCom server configuration options are available to directly specify a shared memory key. SQL and ISAM each require separate shared memory support:
SHMEM_KEY_ISAM <isam_shared_memory_key> SHMEM_KEY_SQL <sql_shared_memory_key>
Shared memory key values can be specified in either decimal or hexadecimal format. For example:
; Set shared memory key for ISAM connections to the specified decimal value:SHMEM_KEY_ISAM 12345 ; Set shared memory key for ISAM connections to the specified hexadecimal value:SHMEM_KEY_ISAM 0xabcd
Use of domain sockets for faster Unix/Linux shared memory connections
FairCom servers on Unix and Linux use a Unix domain socket instead of named pipes for the initial shared memory protocol communication between the client and server.
The FairCom server uses a Unix domain socket instead of a pair of named pipes for the initial communication when a client connects to the server. The FairCom Server creates the named pipes, and when a client connects, the server waits for the client to write to either the socket or the named pipe. In this way, the server is able to support both clients that use the new method and those that use the original method.
The FairCom server configuration option
COMPATIBILITY SHMEM_PIPEcan be used to restore the behavior of earlier releases that only used named pipes. We expect this keyword to be used only in an unexpected situation in which the new option is not working as well as the original option.FairCom clients (both ISAM and SQL) use the Unix domain socket method when connecting using the shared memory protocol if the server indicates that it supports it. If not, the clients use the original method.
A FairCom client library can be compiled with
#define NO_ctFeatUNIX_SHMEMsocketto force the client to use the original method only.
System group assignment of Unix/Linux shared memory resources
On Unix/Linux systems, a user can belong to more than one group of which one group is the primary group, and all other groups are secondary groups. When the SHMEM_PERMISSIONS option is used to only enable user and group permissions on shared memory resources, the resources created for shared memory connections (files, semaphores, shared memory regions) are assigned with the user's current primary group by default.
To address this situation, the configuration option, SHMEM_GROUP, prevents a user account that shares a secondary group with the user account under which the FairCom server process is running failing to connect using shared memory.
This option causes the FairCom server to assign group membership to the specified group. This option applies to the resources for both the ISAM and the SQL shared memory protocol.
user1belongs to groups group1 and group2user2belongs to group group2
If the user1 account runs the FairCom server with SHMEM_PERMISSIONS 660 in ctsrvr.cfg, a client program run by the user2 account will fail to connect using shared memory.
To allow the client program run by user2 to connect, add the following configuration option to ctsrvr.cfg and restart the FairCom server:
SHMEM_GROUP group2
This causes the shared memory resources to be assigned to group2, which allows the user2 client program to connect.