|
|
|
A number of changes were made to the index rebuild logic to improve performance. In addition to the rebuild performance increase mentioned earlier, several new features allow you to fine-tune rebuild performance:
Option to skip initial data file scan
Normally the rebuild functions RBLIFIL(), RBLIFILX(), and RBLIFILX8() start by scanning the data file, checking for valid record marks if it is a variable-length data file, and checking that the logical and physical end of file values are correct. If the data file is known to be in a good state, it can be beneficial to skip this scan.
The rebuild functions now support an option to skip the initial scan of the data file. To use this option, run the ctrbldif utility with the -skipdatascan command-line option. If you are calling the rebuild functions, OR the skipdatascanIFILoption bit into the tfilno field of the IFIL structure that you pass to the rebuild function. When using this and other options, remember to negate the tfilno value after you OR in the options. For example:
myifil.tfilno = -(redosrlIFILoption | skipdatascanIFILoption);
RBLIFIL(&myifil);
ctrbldif progress notifications
The ctrbldif utility now supports an option so that it can receive progress notifications from FairCom Server during the rebuild. Use the -callback command-line option to enable this feature.
For example, here is a typical rebuild, without the -callback parameter:
D:\FairCom\v11.2.0\winX64\tools\cmdline\admin\client>ctrbldif ord.dat admin ADMIN FAIRCOMS
c-treeACE(tm) Version 11.2.0.7875(Build-160501) Index rebuild utility
Copyright (C) 1992 - 2016 FairCom Corporation
ALL RIGHTS RESERVED.
Successfully retrieved IFIL.
Scanning data file and rebuilding indexes...
Successfully rebuilt file.
Here is the same rebuild with the -callback parameter:
D:\FairCom\v11.2.0\winX64\tools\cmdline\admin\client>ctrbldif ord.dat -callback admin ADMIN FAIRCOMS
c-treeACE(tm) Version 11.2.0.7875(Build-160501) Index rebuild utility
Copyright (C) 1992 - 2016 FairCom Corporation
ALL RIGHTS RESERVED.
Successfully retrieved IFIL.
Scanning data file and rebuilding indexes...
Rebuilding data file...
Rebuilding index file.
Writing index file...
Rebuilding additional index #1.
Writing index file...
Rebuilding additional index #2.
Writing index file...
Rebuilding additional index #3.
Writing index file...
Rebuilding additional index #4.
Writing index file...
180000
Successfully rebuilt file.
Rebuild queue optimization
FairCom Server supports an option to use a worker thread to sort the keys read from the data file. The rebuild thread reads records from the data file and constructs key values, which it passes to the sort thread using an inter-thread queue. After all the key values have been passed to the sort thread, the sort thread sorts the key values and passes them back to the rebuild thread in a second queue.
This option is enabled by specifying MAX_REBUILD_QUEUE <max_queue_size> in ctsrvr.cfg with a positive value. The value is a number in bytes indicating the maximum size of the queue used to pass the keys to the worker thread. MAX_REBUILD_QUEUE defaults to 100 MB. A 64-bit server has a limit of 4 TB - 1 for this setting, and a 32-bit server has a limit of 4 GB - 1 for this setting.
Rebuild performance
The performance of the original and optimized rebuild have been compared using a 17 GB copy of the mark.dat and mark.idx files created by the ctmtap test program. Using MAX_K_TO_USE 100 and MAX_REBUILD_QUEUE 100 MB, the original rebuild completed in an average of 1816 seconds, and the optimized rebuild completed in an average of 623 seconds, which is 66% faster (about 1/3 the time of the original rebuild).
