FairCom RTG V3 Update Guide

Up to 15 Percent Faster with Increased Default Index Page Size

Background

Index node page size is an important performance attribute to consider. Page size is equal to a node size within the index binary tree. Each index node is a single I/O to and from persisted storage into a single allocated buffer cache page. There is a direct one to one mapping of cache pages to index nodes.

Ideally, the page size should be configured as close to the I/O read/write block size used by the operating system. In many OS environments this is 4K. Selected storage systems allow tuning this system parameter for optimum database performance. However, a larger c-tree page size can still take advantage of multiples of system block I/O. Modern storage systems generally have enough redundant capacity that I/O operations are completed even when power is lost. It is still recommended that uninterruptible power supplies (UPS; battery backup) are used to ensure absolute data integrity at all times.

The default page size has been 8192 bytes since V8.14 for FairCom servers. (This is configurable with the sect parameter of standalone applications; each sect = 128 bytes.)

Larger Page Size Benefits

Numerous performance tests have shown increasing the default index page size (standalone sect parameter) can bring big gains to overall performance. Increasing the page size brings several beneficial properties to an index:

• Larger nodes contain more keys per node which can be read per I/O reducing overall I/O.
• Larger nodes allow more keys per tree level reducing overall tree depth. Shallower tree depth results in faster index searching.
• Larger nodes allow more efficient storage of larger keys. (c-tree enforces a three key per node minimum.)

32K Page Size Default

FairCom has increased the default server page size to 32768 bytes. Internal FairCom testing consistently shows up to 15% gains in overall throughput with a 32K page size. This confirms much evidence from the field with high performance applications using larger page sizes.

Page size is set with the PAGE_SIZE parameter in ctsrvr.cfg.

PAGE_SIZE 32768

Be sure to review your current c-tree page size usage compared with this new default. FairCom strongly encourages thorough performance testing using the new page size with your specific application based on the compatibility information below.

Warning: Changing the PAGE_SIZE is a maintenance task that should be done carefully and with a full reliable backup. Practice on a copy of your data and server folder until you are confident with the results. For procedures, see Adjusting PAGE_SIZE in the FairCom Installation Guide.

Note: A file created with a larger PAGE_SIZE cannot be opened by a server with a smaller PAGE_SIZE.

Compatibility

The page (node) size is a permanent attribute of an index when it is created. Index nodes remains that size until rebuilt or compacted with a different size. There are limitations with existing files using a new larger page size.

• c-tree can open existing indexes with a smaller page size than currently configured. There is a minor memory use trade off in doing so. As each index node maps to a single server cache page, and the server cache page is allocated as a page size, the unused space in the cache page is lost. This can be significant. If a server configured for a 32K page size index cache is at 100% capacity of 8K index nodes, up to 75% of cache memory is unused. For very large caches this is significant.

  It will be extremely beneficial to rebuild existing indexes and take full advantage of both increased index page size benefits as well as full cache memory usage.

  Opening indexes created with page sizes larger than currently configured results in error 40 (KSIZ_ERR).

• Superfiles can ONLY be opened with the same configured page size as they were originally created. This has important implications with critical c-tree housekeeping files:
  • FAIRCOM.FCS - This file maintains all user, group and password hash information.
  • ctdbdict.fsd - This file maintains the catalog of available databases (SQL and c-treeDB).
  • <database_name>.fdd - This file maintains the SQL database system tables (catalog or dictionary).

  Using these existing superfiles with a different page size will result in a server startup failure. Opening a superfile with a different page size also results in error 40 (KSIZ_ERR). You will find this message logged in CTSTATUS.FCS should the server make this failed attempt.

Rebuilding Existing Indexes

The following options are available for rebuilding indexes to take advantage of increased page size.

ALWAYS MAKE CLEAN BACKUP COPIES OF THESE FILES BEFORE YOU BEGIN THESE PROCEDURES

The index rebuild switch -rebuild for the ctutil utility is the quickest, easiest option for most indexes. Simply be sure you have set the new PAGE_SIZE setting in ctsrvr.cfg and then perform a rebuild using ctutil -rebuild pass the new sect size.

ctutil -rebuild myfile

Remember not to include the file extension on your file name.

For superfiles, use the superfile compact utility, ctscmp. This utility has an option for passing the sect size. Remember, each sect = 128 bytes. For 32768 your sect size will be 256, as shown in the following example:

> ctscmp ctdbdict.fsd Y 256