PowerPath, AIX 5.3, and LTG: Solutions

by dave on April 1, 2009


Over at StorageMonkeys, Rclark posted the following question:

We’re trying to increase the LTG and max_transfer sizes on PowerPath on AIX 5.3. We’ve gone through the steps that worked in the past, but the LTG will not increase. I’ve looked through PowerLink, but haven’t found anything topical. Any links or pointers would be appreciated.

In talking with the smart people in EMC who deal with PowerPath and the various OS platforms that it supports, they came back with the following solution for this question.  Here’s their feedback:

“The max_transfer size in AIX is directly related to the logical track group size (LTG).  The LTG size is automatically set to the size of the lowest value of max_transfer of any disk in the VG so you can’t just “set it” any more.  If the volume group was created in AIX 5.3 it will be capable of dynamically determining the LTG size based on the disk topology. It will be listed as dynamic. This can, however, be disabled by the user with the varyonvg -M option.  If so, it will be listed as Static. If the volume group was created prior to AIX 5.3 then the VG will not be listed as Static or Dynamic.

One of the functions of AIX LVM is to manage disk blocks. At the highest level, this management is done by allocating physical partitions (PP) to logical volumes (LV). The size of the physical partition granule is defined when the physical volume is defined. The level of disk block management below the physical partition is the logical track group (LTG). The LTG provides an efficient mechanism for handling mirror write consistency, mirror resynchronization, and read/write consistency. Prior to AIX 5.2, the LVM enforced a rule that prohibited operations that overlapped disk blocks involving writes. Starting in AIX 5.2, the read/write consistency feature became optional with the –o option on the mklv command.

One of the characteristics of pre-AIX 5L volume management was that no disk I/Os were passed from the LVM to any device driver in sizes greater than the LTG size of 128k bytes. Large disk I/O requests made by applications were broken up by the LVM on LTG alignments. So, a one million byte (1MB) disk I/O request resulted in at least eight discrete requests to the disk device driver. For the case of a single reader or writer thread operating on a disk, this was not a performance limiting problem. However, if a number of threads were doing I/O to the disk, the breaking up of I/O requests could result in additional seeks, reducing aggregate throughput.

Starting in AIX 5.2, the maximum size of an I/O passed from an application to the underlying disk device driver is tunable. Throughput can be optimized by setting LVM’s LTG to match the disk’s maximum transfer size. This is now a configuration option on Symmetrix disk subsystems with minimum EMC ODM 5.2.0.0 package. The configurable maximum transfer size per hdisk setting allows the maximum size of an I/O to the disk to reach up to 1MB using an LVM LTG size of 1MB. This enables an application to issue an I/O the size of an LTG and have that I/O passed as a single request to the underlying disk, as long as it does not cross an LVM partition boundary. If the max transfer size it not changed then the default setting is 256K.

The max_transfer size can be changed on a per hdisk or hdiskpower device using the following commands. The following example is for a non-powerpath installation but the same rules apply using the hdiskpower device.

To view the allowable range use the following command:

lsattr -R -a max_transfer -l hdisk<x>

Which should return the following:

0x20000
0x40000
0x80000
0x100000
0x200000
0x400000
0x800000
0x1000000

1. The command following command will only give you the current maximum transfer setting

lquerypv -M hdisk<x>

Which should return something like:

256

2. To change the device max transfer size you can use the following series of commands

rmdev -l hdisk<x>
chdev -l hdisk<x> -a max_transfer=0x80000
mkdev -l hdisk<x>

3. Use the following command to view your changes.

lquerypv -M hdisk<x>

Which should return:
512

Hopefully, this will help any of you out who may be wanting to do the same!


Reblog this post [with Zemanta]

Share