|
|
|
|
|
|
|
|
Empress Technical News – December 2013 Empress Expandable Mapped Files Efficient Shared Memory for Empress Application Developers IntroductionThe Empress Shared Memory feature provides users the ability to enhance the performance of the Empress RDBMS through the use of system shared memory. introduces additional flexibility in configuring Mapped File shared memory in the latest release of Empress 10.20 with Expandable Mapped Files. Normally, each database process has its own memory space allocated by the operating system and cannot access the memory space of another database process. Shared memory is an operating system feature that allows several processes to share a specified section of memory. By using shared memory, file I/O can be reduced, and database performance can be improved. Empress uses shared memory in three different ways to provide enhanced performance: lock managers, global buffers and mapped files Figure 1). All three concepts are great ways to place part of the Empress database located on disk into shared memory. In an extreme case, all Empress Database tables could be mapped into shared memory. Lock ManagersLock Managers allow placing of certain lock information in shared memory where processes can access it. Empress also allows the Lock Manager (shared memory vs. file) to be defined on a table-by-table basis. Some tables may be defined to use shared memory and some may continue to use files for managing locks.
Figure 1: Empress Shared Memory Global BuffersEvery time a process wishes to get data from a file, it must read that file. Buffers enable Empress to keep parts of a file or many files in shared memory. This speeds up access and therefore increases performance. When data is written to a file that has a Global Buffer, the data is simultaneously written to the Global Buffer and the file. Global Buffers are called write-through buffers. This also means that the data in the Global Buffer will represent the data in the file. Mapped FilesMapped Files are similar to Global Buffers in that they store data from files in shared memory. However, a Mapped File in shared memory has only one file associated with it. Also a Mapped File is not divided into blocks. Instead, it is one large block of memory directly associated with a given section of a file. This means that the Mapped File is essentially a window into a given file. The window can be the entire file, part of the file beginning at the start of the file, or a section of the file beginning and ending at user-defined points.
Expandable Mapped FilesIf a file associated with a Mapped File in a shared memory is a read-only file than the size is well known in advance and a Mapped File can be precisely configured. However, if a file grows, there is no good estimation on how much it could grow. Having an unnecessarily large shared memory partition allocated in memory is wasteful of memory and has a performance hit as well. To address these Mapped File constraints, introduced the Expandable Mapped File. When a file grows (e.g. new records are inserted), new Mapped File extensions are added to accommodate the new records (see Figure 2). Figure 2: Empress Expandable Shared Memory Partition
Defining Mapped File Shared Memory PartitionIn general, as shown in Figure 1, Shared Memory is divided into partitions. There are three types of partitions: Lock Manager, Global Buffer, and Mapped File. There can be any number of each type in shared memory but it is not necessary that all types be there. For example, shared memory could contain just two Mapped Files. The only limitation on the number and size of each partition is the amount of memory in the system available for shared memory. The shared memory partitions section has a specific layout which is set up by editing the Empress Database file to append the partition definition at the end of tabzero file: An example of the shared memory partitions section at the end of the Empress Database tabzero file that contains two Mapped Files is as follows: MSSHMKEY=123456 MSSHMPROTECTION=
MSSHMEND The contents of the file are discussed in the following sections. Shared Memory KeyThe first entry in the partition file is the Empress variable MSSHMKEY. The key is a user-defined number that the system uses to identify a given shared memory block. This number is arbitrary, but each site should have a procedure to assign keys. All databases that want to use the same shared memory block must use the same key to identify it. It looks like this: MSSHMKEY=123456 Parameter for Data Dictionary IndicatorNext comes the Empress variable If this partition is used by sys_dictionary this should be set to " on" to get special protection on the shared memory partition. If it is not used by sys_dictionary then this variable will be unset. It looks like this: MSSHMPROTECTION=on Partition NameThere can be any number of partitions in any order. All the partition definitions have the same first two lines. They are partition name and partition type, in that order. The partition name is any string of characters whose length is less than 32 (any more will be truncated). A partition name from the example looks like this: MSPARTNAME=Mapped1 Partition TypeThere are three partition types that can be chosen: LOCK, GBUF, MAPF for Lock Manager, Global Buffer, and Mapped File respectively. Partition type from the example looks like: MSPARTTYPE=MAPF Each partition type has unique parameters that must be defined. For the purpose of this text we will focus only on Mapped File parameters. Parameters for Mapped Files A Mapped File partition has four possible parameters: starting offset, size, flush/sync behavior and number of extents. The starting offset is the offset from the start of the file where the mapping will begin (i.e., the start of the file window). The entries for both Mapped1 and Mapped2 in the example look like this: MSPARTMAPFSTART=0 The size should be chosen to hold the most often used data, or if the file is small enough, the entire file. The entries for both Mapped1 and Mapped2 in the above example look like this: MSPARTMAPFSIZE=1048576 The MSPARTMAPFNUMEXT value represents the maximum number of extra segments that Empress will create if the data cannot be mapped in the existing segment. An example of using the MSPARTMAPFNUMEXT value is: MSPARTMAPFNUMEXT=16
If the above definition is used, the maximum shared memory allocation for the file to be mapped will be 1,048,576 * (16 + 1) = 17,825,792 bytes. The formula is: max_size = MSPARTMAPFSIZE *(MSPARTMAPFNUMEXT + 1) MSPARTMAPFNUMEXT cannot be greater than 16. The MSPARTMAPFSYNC value represents flush and sync Mapped File behavior. MSPARTMAPFSYNC= If MSPARTMAPFSYNC parameter is set to nothing or not set at all, on each database engine flush, local buffers will be flushed to the Mapped File and operating system buffers. This is a write-through concept. MSPARTMAPFSYNC=close With the close setting, on each database engine flush, local buffers will be flushed to the Mapped File. On close table operations (e.g. invoking an Empress C/C++ mr routine mrclose(), Mapped File will be flushed to the operating system buffers, if required. MSPARTMAPFSYNC=never With the never setting, on each database engine flush, local buffers will be flushed to the Mapped File. Mapped File will never be flushed to the operating system buffers or device. An example of using expandable Mapped file shared memory partition to demonstrate the example given in Figure 2 is as follows: MSSHMKEY=123456
Instead of SummaryIf a file associated with a Mapped File in a shared memory is a read-only file than the size is well known in advance and a Mapped File can be precisely configured. However, if a file grows and there is no good estimation on how much it could grow, than a new way of configuring Mapped File shared memory partition is desirable. To address these situations Empress introduced Expandable Mapped Files, an excellent tool for Empress Application developers to control memory usage and achieve the best possible performance.
Empress Software Inc.
|
|
Company |
Products |
Services |
Partners |
Media |
|
|
USA: 301-220-1919 Canada
& International: 905-513-8888 |
|||||
