[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[linux-lvm] Tracing IO requests?



Hello,

I have the following setup:

md_RAID6(10x2TB) -> LVM2 -> cryptsetup -> XFS

When copying data onto the target XFS, I notice a large number of READs occurring on the physical hard drives. Is there any way of monitoring what might be causing these read ops?

I have setup the system to minimize read-modify-write cycles as best I can, but I fear I've missed some possible options in LVM2 or cryptsetup. Here are the specifics:

11:43:54 sde 162.00 12040.00 34344.00 286.32 0.40 2.47 1.67 27.00 11:43:54 sdf 170.00 12008.00 36832.00 287.29 0.62 3.65 2.12 36.00 11:43:54 sdg 185.00 10552.00 37920.00 262.01 0.49 2.65 1.84 34.00 11:43:54 sdh 152.00 11824.00 37304.00 323.21 0.29 1.78 1.71 26.00 11:43:54 sdi 140.00 13016.00 35216.00 344.51 0.68 4.71 3.21 45.00 11:43:54 sdj 181.00 11784.00 36240.00 265.33 0.43 2.38 1.55 28.00 11:43:54 sds 162.00 11824.00 34040.00 283.11 0.46 2.84 1.67 27.00 11:43:54 sdt 157.00 11264.00 35192.00 295.90 0.65 4.14 2.29 36.00 11:43:54 sdu 154.00 12584.00 35424.00 311.74 0.46 2.79 1.69 26.00 11:43:54 sdv 131.00 12800.00 33264.00 351.63 0.39 2.75 1.98 26.00 11:43:54 md5 752.00 0.00 153688.00 204.37 0.00 0.00 0.00 0.00 11:43:54 DayTar-DayTar 752.00 0.00 153688.00 204.37 12.42 16.76 1.33 100.00 11:43:54 data 0.00 0.00 0.00 0.00 7238.71 0.00 0.00 100.00

Where md5 is the RAID6 holding the drives right above it, DayTar-DayTar are the VG and LV respectively, and data is the cryptsetup device derived from the LV. Hard drives are set to "blockdev --setra 1024", md5 is set for stripe_cache_size of 6553 and preread_bypass_threshold of 0. XFS is mounted with the following options:

/dev/mapper/data on /data type xfs (rw,noatime,nodiratime,allocsize=256m,nobarrier,noikeep,inode64,logbufs=8,logbsize=256k)

And here are the format options of XFS:

meta-data=/dev/mapper/data isize=256 agcount=15, agsize=268435455 blks
         =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=3906993152, imaxpct=5
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=521728, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=0
realtime =none                   extsz=4096   blocks=0, rtextents=0

I wasn't sure how to do any kind of stripe alignment with the md RAID6 given the layers in between. Here are the LVM2 properties:

  --- Physical volume ---
  PV Name               /dev/md5
  VG Name               DayTar
  PV Size               14.55 TiB / not usable 116.00 MiB
  Allocatable           yes (but full)
  PE Size               256.00 MiB
  Total PE              59616
  Free PE               0
  Allocated PE          59616
  PV UUID               jwcRz9-Yl0k-OHRQ-p5yR-AbAP-j09z-PCgSFo

  --- Volume group ---
  VG Name               DayTar
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               1
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               14.55 TiB
  PE Size               256.00 MiB
  Total PE              59616
  Alloc PE / Size       59616 / 14.55 TiB
  Free  PE / Size       0 / 0
  VG UUID               X8gbkZ-BOMq-D6x2-xx6y-r2wF-cePQ-JTKZQs

  --- Logical volume ---
  LV Name                /dev/DayTar/DayTar
  VG Name                DayTar
  LV UUID                cdebg4-EcCR-6QR7-sAhT-EN1h-20Lv-qIFSH8
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                14.55 TiB
  Current LE             59616
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     16384
  Block device           253:0

And finally the cryptsetup properties:

/dev/mapper/data is active:
  cipher:  aes-cbc-essiv:sha256
  keysize: 256 bits
  device:  /dev/mapper/DayTar-DayTar
  offset:  8192 sectors
  size:    31255945216 sectors
  mode:    read/write

Anyone have any suggestions on how to tune this to do better at pure writing by eliminating needless reading?

Thanks,

--Bart


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]