From Wiki-UX.info

Wiki-UX / How to use IOzone for File System Performance Measurements
Jump to: navigation, search

How to use IOzone for File System Performance Measurements

Abstract

This article explains how to use the IOzone open-source tool to perform file system performance assessments.

Contents


Getting and Compiling the Software

1. Obtain the latest version of the IOzone source code.

2. Compile IOzone from source.

HP-UX Bundle or ANSI C Compiler

Use the standard hpux-11.0w make definition to create a 64 bit binary.

# make hpux-11.0w
 
Building iozone for HP-UX (11.0w)
 
        cc -c +DD64 +O3 -Dunix -D_LARGEFILE64_SOURCE  -D_HPUX_SOURCE \
            -DNAME='"hpux-11.0w"' -DHAVE_ANSIC_C -DASYNC_IO -DVXFS  iozone.c \
             -o iozone_hpux-11.0w.o
cc: "iozone.c", line 19848: warning 604: Pointers are not assignment-compatible.
cc: "iozone.c", line 19848: warning 563: Argument #3 is not the correct type.
cc: "iozone.c", line 21966: warning 604: Pointers are not assignment-compatible.
cc: "iozone.c", line 21966: warning 563: Argument #3 is not the correct type.
cc: "iozone.c", line 22188: warning 604: Pointers are not assignment-compatible.
cc: "iozone.c", line 22188: warning 563: Argument #3 is not the correct type.
        cc -c +DD64 +O3 -Dunix -D_LARGEFILE64_SOURCE  -D_HPUX_SOURCE \
                -DHAVE_ANSIC_C -DASYNC_IO -DVXFS  libasync.c  -o libasyncw.o
        cc -c +DD64 +O3 -Dunix -D_LARGEFILE64_SOURCE  -D_HPUX_SOURCE \
                -DHAVE_ANSIC_C -DASYNC_IO -DVXFS  libbif.c  -o libbif.o
        cc +DD64 +O3  iozone_hpux-11.0w.o libasyncw.o \
                libbif.o -lpthread -lrt -o iozone
# file iozone
iozone:         ELF-64 executable object file - PA-RISC 2.0 (LP64)


DSPP GCC 4.4.3

Edit makefile text file and change the ghpux definition to optimize for the itanium 2 platform. Remove the -static flag to create a dynamic library file to support the unwind library. It will create a 32 binary.


# make ghpux
 
Building iozone for GCC HP-UX (9.05)
 
        gcc -c -O -Dunix -D_HPUX_SOURCE -DHAVE_ANSIC_C -DNO_THREADS  iozone.c \
                -DNAME='"h=ghpux"' -o iozone_ghpux.o
iozone.c: In function 'main':
iozone.c:1658: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c:1806: warning: incompatible implicit declaration of built-in function 'strlen'
iozone.c:1883: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c:2913: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'record_command_line':
iozone.c:2984: warning: incompatible implicit declaration of built-in function 'strlen'
iozone.c:2985: warning: incompatible implicit declaration of built-in function 'strcat'
iozone.c: In function 'show_help':
iozone.c:3151: warning: incompatible implicit declaration of built-in function 'strlen'
iozone.c: In function 'throughput_test':
iozone.c:3382: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'purge_buffer_cache':
iozone.c:11661: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c:11662: warning: incompatible implicit declaration of built-in function 'strcat'
iozone.c: In function 'fill_area':
iozone.c:18093: warning: incompatible implicit declaration of built-in function 'bcopy'
iozone.c: In function 'get_traj':
iozone.c:18641: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'r_traj_size':
iozone.c:18778: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'w_traj_size':
iozone.c:18873: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'start_master_listen':
iozone.c:19318: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'child_send':
iozone.c:19410: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c:19419: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'master_send':
iozone.c:19471: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c:19480: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'start_child_send':
iozone.c:19635: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'start_child_listen':
iozone.c:19776: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'child_listen':
iozone.c:19878: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'start_child_listen_async':
iozone.c:19942: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'start_master_send':
iozone.c:20081: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'start_master_send_async':
iozone.c:20166: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'pick_client':
iozone.c:20269: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c:20281: warning: incompatible implicit declaration of built-in function 'strcat'
iozone.c:20350: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'become_client':
iozone.c:20487: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c:20526: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'tell_master_stats':
iozone.c:20914: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'tell_master_ready':
iozone.c:20967: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'wait_for_master_go':
iozone.c:20996: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'start_child_listen_loop':
iozone.c:21143: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'tell_children_begin':
iozone.c:21206: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'terminate_child_async':
iozone.c:21377: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'distribute_stop':
iozone.c:21420: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'send_stop':
iozone.c:21446: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'cleanup_children':
iozone.c:21485: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'find_remote_shell':
iozone.c:21547: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c:21551: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'find_external_mon':
iozone.c:21572: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c:21577: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'speed_main':
iozone.c:21666: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c:21679: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'sp_start_master_send':
iozone.c:21845: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'sp_start_child_listen':
iozone.c:21930: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'sp_start_master_listen':
iozone.c:22152: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'sp_start_child_send':
iozone.c:22247: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'get_date':
iozone.c:22335: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'get_pattern':
iozone.c:22366: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c:22367: warning: incompatible implicit declaration of built-in function 'strlen'
iozone.c: In function 'check_filename':
iozone.c:22428: warning: incompatible implicit declaration of built-in function 'strlen'
iozone.c: In function 'start_monitor':
iozone.c:22454: warning: incompatible implicit declaration of built-in function 'strlen'
iozone.c: In function 'stop_monitor':
iozone.c:22472: warning: incompatible implicit declaration of built-in function 'strlen'
        gcc -c -O -Dunix -D_HPUX_SOURCE -DHAVE_ANSIC_C -DNO_THREADS \
                 libbif.c -o libbif.o
libbif.c: In function 'create_xls':
libbif.c:194: warning: incompatible implicit declaration of built-in function 'exit'
libbif.c: In function 'do_label':
libbif.c:361: warning: incompatible implicit declaration of built-in function 'strlen'
libbif.c:377: warning: incompatible implicit declaration of built-in function 'strcpy'
        gcc -O -mtune=itanium2  iozone_ghpux.o libbif.o -o iozone
# file iozone
iozone:         ELF-32 executable object file - IA64


3. Save a copy of the binary executable iozone on /usr/contrib/bin directory for day-to-day usage.

# cp iozone /usr/contrib/bin
 
# chown root:sys /usr/contrib/bin/iozone
 
# chmod 0755 /usr/contrib/bin/iozone
 
# ls -l /usr/contrib/bin/iozone
-rwxr-xr-x   1 root       sys         458000 Sep  9 18:54 /usr/contrib/bin/iozone

Using iozone

Full documentation on the usage of iozone command is available at Iozone Filesystem Benchmark manual. Here is an overview of the options used to collect data for this article.

# iozone -e -r 1 -r 2 -r 4 -r 8 -r 16 -r 32 -r 64 -r 128 -r 256 -r 512 -r 1m -r 2m -r 4m -r 8m \
-s 64m -f /perf/tfile \
-i 0 -i 1
-e include flush (fsync,fflsuh) in the timing calcultations
-c include close() in the timing calculations
-r used to specify record size in Kbytes, to test.
-s Used to specify the size, in Kbytes of the file to test.
-f Used to specify the filename of the temporary file under test.
-t Run IOzone in throughput mode. This option allows the user to specify how many threads or process to have active during the measurement.

Testing Unbuffered File Systems

The following chart present the results of performing I/O Read/Write transaction with incremental block size, 1k up to 4096k (4Mb) on a VxFS file system. Buffer Cache has been disable using the OnlineJFS options mincache=direct,convosync=direct. These options are commonly used on SAN devices that provided their own buffering mechanisms.

The backing storage is a direct attached 36 GB 10k Ultra320 parallel SCSI drive.

GnuplotBasic Plot

write rewrite read reread
24621 26366 118944 93725
27745 28133 188716 143879
29492 30490 288196 197632
29652 32169 365224 244488
31018 32308 530806 356689
31656 32185 609541 452181
32215 32002 685013 476802
31415 32774 968679 745218
34309 32330 72118 72279
46738 45923 69042 68967
56810 56571 71982 72107
54601 57228 71824 71649
56809 57552 71796 71676
57110 56946 73474 73356

Note that increasing the block size improves the overall transfer rate of the file system, up to 1024k (1 MB), when additional increase of the block size does not provides additional I/O throughput.

Testing Buffered File Systems

GnuplotBasic Plot

write rewrite read reread
24547 26197 118700 100159
28126 28825 184472 143337
29813 31306 282961 197717
28468 31310 356838 241249
30764 32238 530728 405289
31840 33020 586725 443542
31354 32381 632214 460154
30595 32134 914693 653908
34168 34363 71450 71027
46742 47112 73134 73217
56030 56181 68667 69070
51514 56859 71072 72920
56617 56369 71471 71687
57285 57407 72622 72052

Note that the read results for the Buffered File System are almost an order of magnitude higher than the Unbuffered File System when the block size reaches the 256k sweet spot. At the same time, write speed is on the same range of the previous test.

Once the block size is higher than 256k, the read speed heavily falls. The drop is explained by the discovered_direct_iosz VxFS tunable. These parameters controls the discovery of Direct I/O transactions.

Always keep in mind that no matter how large and fast the buffer cache may, in the long run, all I/O transaction will need to access the backing storage device. Always perform unbuffered tests to understand the physical capacities of your storage subsystem.

VxFS DirectIO

On VxFS file systems, when transactions larger that 256k are made the kernels stops using the buffer cache and performs it's I/O directly from the backing storage. This is done to keep a balance over the system performance because heavy Direct I/O transactions will thrash the buffer with data that, due to it's nature, will not likely be reused in upcoming transactions, reducing the opportunity for cache data re-usability.

A common scenario is a file system [backup | restore], where data is read once [to | from] the backup media. Knowing in advance the block size used by the backup application allow to set the boundaries for these transactions to optimize the task.

# vxtunefs -p /perf | grep direct
discovered_direct_iosz = 262144
max_direct_iosz = 1048576

For example, to set discovery of Direct I/O toward applications using 64k blocks.

# vxtunefs -s -o discovered_direct_iosz=65536 /perf
UX:vxfs vxtunefs: INFO: V-3-22525: Parameters successfully set for /perf

GnuplotBasic Plot

write rewrite read reread
22902 25791 109221 99689
28272 29991 190783 165679
29492 30841 256590 184018
30032 32689 373101 244276
31285 32507 547644 408800
31902 32970 587081 438127
12932 12971 67447 68627
21663 22275 69985 71320
34095 34229 73355 73576
46473 46725 68156 73282
53821 57395 73090 73071
57067 55699 69962 72602
52457 57009 74144 74210
56516 57207 73154 73010

Reference

Authors

This page was last modified on 26 March 2010, at 14:02. This page has been accessed 4,280 times.