It is possible to monitor memory use and allocation call stacks for each suballocator list.
Support has been added for monitoring FairCom Server memory use and collecting allocation call stacks for each suballocator list. The ability to monitor FairCom Server's memory use has been enhanced in the following ways:
It now tracks the number and byte count of allocations that do not go through FairCom DB's memory suballocator.
FairCom Server now makes its memory suballocator usage figures available to monitoring tools. The ctstat utility's ‑ml option can be used to display current memory allocation figures. Example:
# ctstat -ml -t -i 2 -h 1 -s FAIRCOMS
Thu May 01 14:31:22 2014
name allocated in use pct
KLNTYP 0 0 0.00%
PI1TYP 32768 1704 0.01%
PI2TYP 32768 6800 0.01%
PI4TYP 1376256 1224360 0.31%
PI8TYP 1245184 1155592 0.28%
PIwTYP 32768 19008 0.01%
PIxTYP 32768 16120 0.01%
PIyTYP 49152 25800 0.01%
PIzTYP 32768 0 0.01%
PIaTYP 82560 69768 0.02%
PIbTYP 296064 270600 0.07%
SHDTYP 0 0 0.00%
BATTYP 0 0 0.00%
ILKTYP 16384 0 0.00%
FNMTYP 0 0 0.00%
COMTYP 16384 0 0.00%
ABTTYP 0 0 0.00%
LOKTYP 16384 0 0.00%
DCMTYP 0 0 0.00%
IXCTYP 32129024 29018360 7.24%
DTCTYP 6782976 6547464 1.53%
CTCTYP 81920 70000 0.02%
IXBTYP 180854968 180854968 40.74%
DTBTYP 203169792 203169792 45.76%
MBATYP 17690815 17690815 3.98%
TOTAL: 443971703 440141151
Note: If memory allocation call stack is enabled for a suballocator list, the name of the suballocator list is followed by an asterisk in this output (for example, as MBATYP*).
Just as the ctstat utility does, a FairCom Server client can read the memory use figures by calling the ctSNAPSHOT() API function with the new mode ctPSSmemAlloc. This mode returns the memory use figures in a new structure named ctGMMS. See ctstat.c for an example of this ctSNAPSHOT() call.
On Windows and Linux systems, FairCom Server supports collection of call stacks for memory allocations. This support, which existed prior to this revision, is enabled using the configuration option DIAGNOSTICS MEMTRACK. Now FairCom Server allows allocation call stack collection to be dynamically enabled or disabled for specific memory suballocator lists, provided that DIAGNOSTICS MEMTRACK was specified in the configuration file. The ctMEMSTAT() API function is used to change these settings, and the ctstat utility's ‑mt option provides a convenient way to use this function.
Only a member of the ADMIN group is allowed to change memory allocation settings.
Memory Suballocator List Descriptions
KLNTYP - key buffer, size of the largest supported maximum key value (MAXLEN = 1024)
PI1TYP - allocations of size > 0 and <= PI_UNIT (PI_UNIT = 16)
PI2TYP - allocations of size > PI_UNIT and <= 2 * PI_UNIT
PI4TYP - allocations of size > 2 * PI_UNIT and <= 4 * PI_UNIT
PI8TYP - allocations of size > 4 * PI_UNIT and <= 8 * PI_UNIT
PIwTYP - allocations of size > 8 * PI_UNIT and <= 16 * PI_UNIT
PIxTYP - allocations of size > 16 * PI_UNIT and <= 32 * PI_UNIT
PIyTYP - allocations of size > 32 * PI_UNIT and <= 64 * PI_UNIT
PIzTYP - allocations of size > 64 * PI_UNIT and <= 128 * PI_UNIT
PIaTYP - allocations of size > 128 * PI_UNIT and <= 256 * PI_UNIT
PIbTYP - allocations of size > 256 * PI_UNIT and <= 512 * PI_UNIT
SHDTYP - preimage space list entry, size of SHADLST structure (the variable-length contents are allocated separately if larger than ctSHADOWlen)
BATTYP - batch retrieval list entry, size of BATLST structure
ILKTYP - user lock table list entry, size of LOKS structure
FNMTYP - file name, size of the largest supported file name (MAX_NAME = 255)
COMTYP - commit node list entry, size of COMLST structure
ABTTYP - abort node list entry, size of ABTLST structure
LOKTYP - system lock table list entry, size of RECLOK structure
DCMTYP - commit data list entry, size of COMLST structure
IXCTYP - index cache page control structure, size of TREEBUFF structure (allocated at startup and does not change after that)
DTCTYP - data cache page control structure, size of DATBUF structure (allocated at startup and does not change after that)
CTCTYP - system file control block entry, size of CTFILE structure
IXBTYP - index cache page buffer, size of PAGE_SIZE + MAXLEN + 2 * sizeof(ctRECPT) (allocated at startup and does not change after that)
DTBTYP - data cache page buffer, size of PAGE_SIZE (allocated at startup and does not change after that)
MBATYP - allocation size > 512 * PI_UNIT
Examples:
Enable memory allocation call stack collection for all suballocator lists:
ctstat -mt +ALL -u ADMIN -p ADMIN -s FAIRCOMS
Enable memory allocation call stack collection for only the MBATYP and LOKTYP suballocator lists:
The use of the existing and new memory monitoring options is now restricted to members of the ADMIN group.
When memory allocation call stack tracking is supported on a per-suballocator list basic, the DIAGNOSTICS MEMTRACK option must still be specified in ctsrvr.cfg to support collecting memory allocation call stacks, but collection of allocation call stacks is initially disabled for all suballocator lists. An administrator must use the ctstat utility's ‑mt option as shown above to enable collection of memory allocation call stacks for the desired suballocator lists.
If troubleshooting unexpected memory growth, first use ctstat‑ml to monitor memory use by suballocator list. Then enable memory allocation call stack collection just for the lists that show the unexpected growth. This approach can reduce the overhead of the memory allocation call stack collection and can simplify analysis of the unexpected memory growth.
