host-nfs-6-zfs-3xraidz2-sas-3tb20240616

2 x Intel Xeon E5-2697A v4 testing with a Supermicro PIO-648R-E1CR36L+-ST031 X10DRi-T4+ v1.01 (3.4 BIOS) and ASPEED on Debian GNU/Linux 12 via the Phoronix Test Suite.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2406186-MBI-HOSTNFS688
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results
Show Result Confidence Charts

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs
Condense Test Profiles With Multiple Version Results Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
Newly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test
June 16
  7 Hours, 13 Minutes
12 x 3001GB HUS723030ALS64
June 18
  3 Minutes
Invert Hiding All Results Option
  3 Hours, 38 Minutes
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

Newly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test

Kernel Notes: Transparent Huge Pages: madvise
Disk Notes: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096
Processor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0xb00003e
Security Notes: gather_data_sampling: Not affected + itlb_multihit: KVM: Mitigation of VMX disabled + l1tf: Mitigation of PTE Inversion; VMX: conditional cache flushes SMT vulnerable + mds: Mitigation of Clear buffers; SMT vulnerable + meltdown: Mitigation of PTI + mmio_stale_data: Mitigation of Clear buffers; SMT vulnerable + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines; IBPB: conditional; IBRS_FW; STIBP: conditional; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Mitigation of Clear buffers; SMT vulnerable

12 x 3001GB HUS723030ALS64

Processor: 2 x Intel Xeon E5-2697A v4 @ 3.60GHz (32 Cores / 64 Threads), Motherboard: Supermicro PIO-648R-E1CR36L+-ST031 X10DRi-T4+ v1.01 (3.4 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 24 x 32 GB DDR4-1866MT/s Samsung M386A4G40DM0-CPB, Disk: 12 x 3001GB HUS723030ALS64 + 15 x 3001GB Seagate ST33000650SS + 9 x 3001GB Seagate ST3000NM0023 + 2048GB Lexar SSD NS100 + 32GB Cruzer Blade, Graphics: ASPEED, Monitor: DELL S2009W, Network: 4 x Intel 10-Gigabit X540-AT2

OS: Debian GNU/Linux 12, Kernel: 6.8.4-3-pve (x86_64), Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1600x900

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-bTRWOB/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-bTRWOB/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096
Processor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0xb00003e
Security Notes: gather_data_sampling: Not affected + itlb_multihit: KVM: Mitigation of VMX disabled + l1tf: Mitigation of PTE Inversion; VMX: conditional cache flushes SMT vulnerable + mds: Mitigation of Clear buffers; SMT vulnerable + meltdown: Mitigation of PTI + mmio_stale_data: Mitigation of Clear buffers; SMT vulnerable + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines; IBPB: conditional; IBRS_FW; STIBP: conditional; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Mitigation of Clear buffers; SMT vulnerable

host-nfs-6-zfs-3xraidz2-sas-3tb20240616OpenBenchmarking.orgPhoronix Test Suite2 x Intel Xeon E5-2697A v4 @ 3.60GHz (32 Cores / 64 Threads)Supermicro PIO-648R-E1CR36L+-ST031 X10DRi-T4+ v1.01 (3.4 BIOS)Intel Xeon E7 v4/Xeon24 x 32 GB DDR4-1866MT/s Samsung M386A4G40DM0-CPB12 x 3001GB HUS723030ALS64 + 15 x 3001GB Seagate ST33000650SS + 9 x 3001GB Seagate ST3000NM0023 + 2048GB Lexar SSD NS100 + 32GB Cruzer BladeASPEEDDELL S2009W4 x Intel 10-Gigabit X540-AT2Debian GNU/Linux 126.8.4-3-pve (x86_64)GCC 12.2.0ext41600x900ProcessorMotherboardChipsetMemoryDiskGraphicsMonitorNetworkOSKernelCompilerFile-SystemScreen ResolutionHost-nfs-6-zfs-3xraidz2-sas-3tb20240616 BenchmarksSystem Logs- Transparent Huge Pages: madvise- MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096- Scaling Governor: intel_cpufreq performance - CPU Microcode: 0xb00003e- gather_data_sampling: Not affected + itlb_multihit: KVM: Mitigation of VMX disabled + l1tf: Mitigation of PTE Inversion; VMX: conditional cache flushes SMT vulnerable + mds: Mitigation of Clear buffers; SMT vulnerable + meltdown: Mitigation of PTI + mmio_stale_data: Mitigation of Clear buffers; SMT vulnerable + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines; IBPB: conditional; IBRS_FW; STIBP: conditional; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Mitigation of Clear buffers; SMT vulnerable - 12 x 3001GB HUS723030ALS64: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-bTRWOB/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-bTRWOB/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v

host-nfs-6-zfs-3xraidz2-sas-3tb20240616fio: Rand Read - Sync - Yes - No - 4KB - /datafio: Rand Read - Sync - Yes - No - 4KB - /datafio: Rand Read - Sync - Yes - No - 8MB - /datafio: Rand Read - Sync - Yes - No - 16MB - /datafio: Rand Read - Sync - Yes - No - 64MB - /datafio: Rand Write - Sync - Yes - No - 4KB - /datafio: Rand Write - Sync - Yes - No - 4KB - /datafio: Rand Write - Sync - Yes - No - 8MB - /datafio: Rand Read - Sync - Yes - No - 128MB - /datafio: Rand Read - Sync - Yes - No - 512KB - /datafio: Rand Write - Sync - Yes - No - 16MB - /datafio: Rand Write - Sync - Yes - No - 64MB - /datafio: Rand Write - Sync - Yes - No - 128MB - /datafio: Rand Write - Sync - Yes - No - 512KB - /datafio: Seq Read - Sync - Yes - No - 4KB - /datafio: Seq Read - Sync - Yes - No - 4KB - /datafio: Seq Read - Sync - Yes - No - 8MB - /datafio: Rand Read - Linux AIO - Yes - No - 4KB - /datafio: Rand Read - Linux AIO - Yes - No - 4KB - /datafio: Rand Read - Linux AIO - Yes - No - 8MB - /datafio: Rand Read - POSIX AIO - Yes - No - 4KB - /datafio: Rand Read - POSIX AIO - Yes - No - 4KB - /datafio: Rand Read - POSIX AIO - Yes - No - 8MB - /datafio: Seq Read - Sync - Yes - No - 16MB - /datafio: Seq Read - Sync - Yes - No - 64MB - /datafio: Seq Write - Sync - Yes - No - 4KB - /datafio: Seq Write - Sync - Yes - No - 4KB - /datafio: Seq Write - Sync - Yes - No - 8MB - /datafio: Rand Read - Linux AIO - Yes - No - 16MB - /datafio: Rand Read - Linux AIO - Yes - No - 64MB - /datafio: Rand Read - POSIX AIO - Yes - No - 16MB - /datafio: Rand Read - POSIX AIO - Yes - No - 64MB - /datafio: Rand Write - Linux AIO - Yes - No - 4KB - /datafio: Rand Write - Linux AIO - Yes - No - 4KB - /datafio: Rand Write - Linux AIO - Yes - No - 8MB - /datafio: Rand Write - POSIX AIO - Yes - No - 4KB - /datafio: Rand Write - POSIX AIO - Yes - No - 4KB - /datafio: Rand Write - POSIX AIO - Yes - No - 8MB - /datafio: Seq Read - Sync - Yes - No - 128MB - /datafio: Seq Read - Sync - Yes - No - 512KB - /datafio: Seq Write - Sync - Yes - No - 16MB - /datafio: Seq Write - Sync - Yes - No - 64MB - /datafio: Rand Read - Linux AIO - Yes - No - 128MB - /datafio: Rand Read - Linux AIO - Yes - No - 512KB - /datafio: Rand Read - POSIX AIO - Yes - No - 128MB - /datafio: Rand Read - POSIX AIO - Yes - No - 512KB - /datafio: Rand Write - Linux AIO - Yes - No - 16MB - /datafio: Rand Write - Linux AIO - Yes - No - 64MB - /datafio: Rand Write - POSIX AIO - Yes - No - 16MB - /datafio: Rand Write - POSIX AIO - Yes - No - 64MB - /datafio: Seq Write - Sync - Yes - No - 128MB - /datafio: Seq Write - Sync - Yes - No - 512KB - /datafio: Rand Write - Linux AIO - Yes - No - 128MB - /datafio: Rand Write - Linux AIO - Yes - No - 512KB - /datafio: Rand Write - POSIX AIO - Yes - No - 128MB - /datafio: Rand Write - POSIX AIO - Yes - No - 512KB - /datafio: Seq Read - Linux AIO - Yes - No - 4KB - /datafio: Seq Read - Linux AIO - Yes - No - 4KB - /datafio: Seq Read - Linux AIO - Yes - No - 8MB - /datafio: Seq Read - POSIX AIO - Yes - No - 4KB - /datafio: Seq Read - POSIX AIO - Yes - No - 4KB - /datafio: Seq Read - POSIX AIO - Yes - No - 8MB - /datafio: Seq Read - Linux AIO - Yes - No - 16MB - /datafio: Seq Read - Linux AIO - Yes - No - 64MB - /datafio: Seq Read - POSIX AIO - Yes - No - 16MB - /datafio: Seq Read - POSIX AIO - Yes - No - 64MB - /datafio: Seq Write - Linux AIO - Yes - No - 4KB - /datafio: Seq Write - Linux AIO - Yes - No - 4KB - /datafio: Seq Write - Linux AIO - Yes - No - 8MB - /datafio: Seq Write - POSIX AIO - Yes - No - 4KB - /datafio: Seq Write - POSIX AIO - Yes - No - 4KB - /datafio: Seq Write - POSIX AIO - Yes - No - 8MB - /datafio: Seq Read - Linux AIO - Yes - No - 128MB - /datafio: Seq Read - Linux AIO - Yes - No - 512KB - /datafio: Seq Read - POSIX AIO - Yes - No - 128MB - /datafio: Seq Read - POSIX AIO - Yes - No - 512KB - /datafio: Seq Write - Linux AIO - Yes - No - 16MB - /datafio: Seq Write - Linux AIO - Yes - No - 64MB - /datafio: Seq Write - POSIX AIO - Yes - No - 16MB - /datafio: Seq Write - POSIX AIO - Yes - No - 64MB - /datafio: Seq Write - Linux AIO - Yes - No - 128MB - /datafio: Seq Write - Linux AIO - Yes - No - 512KB - /datafio: Seq Write - POSIX AIO - Yes - No - 128MB - /datafio: Seq Write - POSIX AIO - Yes - No - 512KB - /datapostmark: Disk Transaction PerformanceNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test12 x 3001GB HUS723030ALS6426868633304725652235404103367309022663083290921962345324111492941332703266682002315280717672307212420767101818003128238725152284201736392833268944611433328352019258428972202261621842064222325932366261724022190308826552440244426438832264002154105226926720492078224020341768527134846258364516500028272426191518451975247922932591240025852400241726324386OpenBenchmarking.org

Flexible IO Tester

Fio is an advanced disk benchmark that depends upon the kernel's AIO access library. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test60120180240300SE +/- 2.52, N = 3268

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO TesterType: Random Read - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test15K30K45K60K75KSE +/- 617.34, N = 368633

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test7001400210028003500SE +/- 36.59, N = 33047

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 16MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 29.01, N = 42565

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 64MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 16.80, N = 32235

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test90180270360450SE +/- 3.39, N = 15404

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO TesterType: Random Write - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test20K40K60K80K100KSE +/- 864.41, N = 15103367

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test7001400210028003500SE +/- 28.02, N = 33090

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 128MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 14.75, N = 32266

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test7001400210028003500SE +/- 24.02, N = 33083

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 16MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 38.67, N = 152909

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 64MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 27.26, N = 152196

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 128MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 9.82, N = 32345

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test7001400210028003500SE +/- 33.06, N = 153241

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test2004006008001000SE +/- 7.44, N = 151149

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO TesterType: Sequential Read - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test60K120K180K240K300KSE +/- 1886.96, N = 15294133

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 19.43, N = 32703

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test60120180240300SE +/- 0.88, N = 3266

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO TesterType: Random Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test15K30K45K60K75KSE +/- 200.00, N = 368200

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 22.91, N = 152315

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test60120180240300SE +/- 3.27, N = 15280

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO TesterType: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test15K30K45K60K75KSE +/- 845.47, N = 1571767

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 24.73, N = 152307

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 16MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 22.33, N = 152124

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 64MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test400800120016002000SE +/- 9.67, N = 32076

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test150300450600750SE +/- 8.39, N = 15710

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO TesterType: Sequential Write - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test40K80K120K160K200KSE +/- 2158.48, N = 15181800

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test7001400210028003500SE +/- 44.55, N = 33128

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 16MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 27.61, N = 42387

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 64MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 16.56, N = 32515

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 16MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 18.15, N = 32284

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 64MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test400800120016002000SE +/- 60.26, N = 152017

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test80160240320400SE +/- 4.10, N = 3363

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO TesterType: Random Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test20K40K60K80K100KSE +/- 1083.72, N = 392833

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 9.39, N = 32689

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test100200300400500SE +/- 4.36, N = 3446

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO TesterType: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test20K40K60K80K100KSE +/- 1201.85, N = 3114333

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 21.30, N = 32835

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 128MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test400800120016002000SE +/- 24.02, N = 152019

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 25.90, N = 152584

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 16MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 39.98, N = 32897

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 64MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 26.08, N = 152202

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 128MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 30.38, N = 32616

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 22.60, N = 152184

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 128MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test400800120016002000SE +/- 70.29, N = 152064

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 24.44, N = 152223

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 16MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 19.27, N = 152593

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 64MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 24.84, N = 152366

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 16MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 50.67, N = 152617

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 64MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 72.49, N = 152402

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 128MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 38.93, N = 152190

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: Sync - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test7001400210028003500SE +/- 59.06, N = 123088

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 128MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 29.96, N = 32655

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 27.18, N = 152440

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 128MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 93.27, N = 122444

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Random Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 47.06, N = 152643

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test2004006008001000SE +/- 9.04, N = 5883

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO TesterType: Sequential Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test50K100K150K200K250KSE +/- 2249.44, N = 5226400

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 23.69, N = 32154

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test2004006008001000SE +/- 11.70, N = 151052

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO TesterType: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test60K120K180K240K300KSE +/- 3008.66, N = 15269267

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test400800120016002000SE +/- 22.53, N = 52049

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 16MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test400800120016002000SE +/- 20.35, N = 152078

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 64MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 32.11, N = 32240

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 16MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test400800120016002000SE +/- 22.64, N = 152034

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 64MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test400800120016002000SE +/- 33.80, N = 121768

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test110220330440550SE +/- 3.59, N = 13527

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO TesterType: Sequential Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test30K60K90K120K150KSE +/- 918.79, N = 13134846

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 37.56, N = 152583

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test140280420560700SE +/- 4.63, N = 3645

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO TesterType: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test40K80K120K160K200KSE +/- 1154.70, N = 3165000

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 21.09, N = 32827

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 128MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 24.87, N = 52426

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test400800120016002000SE +/- 8.09, N = 31915

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 128MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test400800120016002000SE +/- 89.63, N = 121845

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Read - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test400800120016002000SE +/- 18.11, N = 151975

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 16MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 51.66, N = 122479

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 64MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 33.84, N = 152293

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 16MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 37.45, N = 152591

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 64MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 90.03, N = 152400

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 128MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 27.45, N = 152585

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 25.78, N = 152400

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 128MB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test5001000150020002500SE +/- 128.85, N = 152417

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.33Type: Sequential Write - IO Engine: POSIX AIO - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: /dataNewly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test6001200180024003000SE +/- 31.10, N = 152632

PostMark

This is a test of NetApp's PostMark benchmark designed to simulate small-file testing similar to the tasks endured by web and mail servers. This test profile will set PostMark to perform 25,000 transactions with 500 files simultaneously with the file sizes ranging between 5 and 512 kilobytes. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgTPS, More Is BetterPostMark 1.51Disk Transaction Performance12 x 3001GB HUS723030ALS649001800270036004500SE +/- 44.46, N = 343861. (CC) gcc options: -O3

85 Results Shown

Flexible IO Tester:
  Rand Read - Sync - Yes - No - 4KB - /data:
    MB/s
    IOPS
  Rand Read - Sync - Yes - No - 8MB - /data:
    MB/s
  Rand Read - Sync - Yes - No - 16MB - /data:
    MB/s
  Rand Read - Sync - Yes - No - 64MB - /data:
    MB/s
  Rand Write - Sync - Yes - No - 4KB - /data:
    MB/s
    IOPS
  Rand Write - Sync - Yes - No - 8MB - /data:
    MB/s
  Rand Read - Sync - Yes - No - 128MB - /data:
    MB/s
  Rand Read - Sync - Yes - No - 512KB - /data:
    MB/s
  Rand Write - Sync - Yes - No - 16MB - /data:
    MB/s
  Rand Write - Sync - Yes - No - 64MB - /data:
    MB/s
  Rand Write - Sync - Yes - No - 128MB - /data:
    MB/s
  Rand Write - Sync - Yes - No - 512KB - /data:
    MB/s
  Seq Read - Sync - Yes - No - 4KB - /data:
    MB/s
    IOPS
  Seq Read - Sync - Yes - No - 8MB - /data:
    MB/s
  Rand Read - Linux AIO - Yes - No - 4KB - /data:
    MB/s
    IOPS
  Rand Read - Linux AIO - Yes - No - 8MB - /data:
    MB/s
  Rand Read - POSIX AIO - Yes - No - 4KB - /data:
    MB/s
    IOPS
  Rand Read - POSIX AIO - Yes - No - 8MB - /data:
    MB/s
  Seq Read - Sync - Yes - No - 16MB - /data:
    MB/s
  Seq Read - Sync - Yes - No - 64MB - /data:
    MB/s
  Seq Write - Sync - Yes - No - 4KB - /data:
    MB/s
    IOPS
  Seq Write - Sync - Yes - No - 8MB - /data:
    MB/s
  Rand Read - Linux AIO - Yes - No - 16MB - /data:
    MB/s
  Rand Read - Linux AIO - Yes - No - 64MB - /data:
    MB/s
  Rand Read - POSIX AIO - Yes - No - 16MB - /data:
    MB/s
  Rand Read - POSIX AIO - Yes - No - 64MB - /data:
    MB/s
  Rand Write - Linux AIO - Yes - No - 4KB - /data:
    MB/s
    IOPS
  Rand Write - Linux AIO - Yes - No - 8MB - /data:
    MB/s
  Rand Write - POSIX AIO - Yes - No - 4KB - /data:
    MB/s
    IOPS
  Rand Write - POSIX AIO - Yes - No - 8MB - /data:
    MB/s
  Seq Read - Sync - Yes - No - 128MB - /data:
    MB/s
  Seq Read - Sync - Yes - No - 512KB - /data:
    MB/s
  Seq Write - Sync - Yes - No - 16MB - /data:
    MB/s
  Seq Write - Sync - Yes - No - 64MB - /data:
    MB/s
  Rand Read - Linux AIO - Yes - No - 128MB - /data:
    MB/s
  Rand Read - Linux AIO - Yes - No - 512KB - /data:
    MB/s
  Rand Read - POSIX AIO - Yes - No - 128MB - /data:
    MB/s
  Rand Read - POSIX AIO - Yes - No - 512KB - /data:
    MB/s
  Rand Write - Linux AIO - Yes - No - 16MB - /data:
    MB/s
  Rand Write - Linux AIO - Yes - No - 64MB - /data:
    MB/s
  Rand Write - POSIX AIO - Yes - No - 16MB - /data:
    MB/s
  Rand Write - POSIX AIO - Yes - No - 64MB - /data:
    MB/s
  Seq Write - Sync - Yes - No - 128MB - /data:
    MB/s
  Seq Write - Sync - Yes - No - 512KB - /data:
    MB/s
  Rand Write - Linux AIO - Yes - No - 128MB - /data:
    MB/s
  Rand Write - Linux AIO - Yes - No - 512KB - /data:
    MB/s
  Rand Write - POSIX AIO - Yes - No - 128MB - /data:
    MB/s
  Rand Write - POSIX AIO - Yes - No - 512KB - /data:
    MB/s
  Seq Read - Linux AIO - Yes - No - 4KB - /data:
    MB/s
    IOPS
  Seq Read - Linux AIO - Yes - No - 8MB - /data:
    MB/s
  Seq Read - POSIX AIO - Yes - No - 4KB - /data:
    MB/s
    IOPS
  Seq Read - POSIX AIO - Yes - No - 8MB - /data:
    MB/s
  Seq Read - Linux AIO - Yes - No - 16MB - /data:
    MB/s
  Seq Read - Linux AIO - Yes - No - 64MB - /data:
    MB/s
  Seq Read - POSIX AIO - Yes - No - 16MB - /data:
    MB/s
  Seq Read - POSIX AIO - Yes - No - 64MB - /data:
    MB/s
  Seq Write - Linux AIO - Yes - No - 4KB - /data:
    MB/s
    IOPS
  Seq Write - Linux AIO - Yes - No - 8MB - /data:
    MB/s
  Seq Write - POSIX AIO - Yes - No - 4KB - /data:
    MB/s
    IOPS
  Seq Write - POSIX AIO - Yes - No - 8MB - /data:
    MB/s
  Seq Read - Linux AIO - Yes - No - 128MB - /data:
    MB/s
  Seq Read - Linux AIO - Yes - No - 512KB - /data:
    MB/s
  Seq Read - POSIX AIO - Yes - No - 128MB - /data:
    MB/s
  Seq Read - POSIX AIO - Yes - No - 512KB - /data:
    MB/s
  Seq Write - Linux AIO - Yes - No - 16MB - /data:
    MB/s
  Seq Write - Linux AIO - Yes - No - 64MB - /data:
    MB/s
  Seq Write - POSIX AIO - Yes - No - 16MB - /data:
    MB/s
  Seq Write - POSIX AIO - Yes - No - 64MB - /data:
    MB/s
  Seq Write - Linux AIO - Yes - No - 128MB - /data:
    MB/s
  Seq Write - Linux AIO - Yes - No - 512KB - /data:
    MB/s
  Seq Write - POSIX AIO - Yes - No - 128MB - /data:
    MB/s
  Seq Write - POSIX AIO - Yes - No - 512KB - /data:
    MB/s
PostMark

Newly built, 3 VDAVs Of 12 SAS in RAIDZ2, Large Test

Kernel Notes: Transparent Huge Pages: madvise
Disk Notes: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096
Processor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0xb00003e
Security Notes: gather_data_sampling: Not affected + itlb_multihit: KVM: Mitigation of VMX disabled + l1tf: Mitigation of PTE Inversion; VMX: conditional cache flushes SMT vulnerable + mds: Mitigation of Clear buffers; SMT vulnerable + meltdown: Mitigation of PTI + mmio_stale_data: Mitigation of Clear buffers; SMT vulnerable + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines; IBPB: conditional; IBRS_FW; STIBP: conditional; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Mitigation of Clear buffers; SMT vulnerable

Testing initiated at 16 June 2024 09:15 by user root.

12 x 3001GB HUS723030ALS64

Processor: 2 x Intel Xeon E5-2697A v4 @ 3.60GHz (32 Cores / 64 Threads), Motherboard: Supermicro PIO-648R-E1CR36L+-ST031 X10DRi-T4+ v1.01 (3.4 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 24 x 32 GB DDR4-1866MT/s Samsung M386A4G40DM0-CPB, Disk: 12 x 3001GB HUS723030ALS64 + 15 x 3001GB Seagate ST33000650SS + 9 x 3001GB Seagate ST3000NM0023 + 2048GB Lexar SSD NS100 + 32GB Cruzer Blade, Graphics: ASPEED, Monitor: DELL S2009W, Network: 4 x Intel 10-Gigabit X540-AT2

OS: Debian GNU/Linux 12, Kernel: 6.8.4-3-pve (x86_64), Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1600x900

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-bTRWOB/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-bTRWOB/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: MQ-DEADLINE / errors=remount-ro,relatime,rw / Block Size: 4096
Processor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0xb00003e
Security Notes: gather_data_sampling: Not affected + itlb_multihit: KVM: Mitigation of VMX disabled + l1tf: Mitigation of PTE Inversion; VMX: conditional cache flushes SMT vulnerable + mds: Mitigation of Clear buffers; SMT vulnerable + meltdown: Mitigation of PTI + mmio_stale_data: Mitigation of Clear buffers; SMT vulnerable + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines; IBPB: conditional; IBRS_FW; STIBP: conditional; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Mitigation of Clear buffers; SMT vulnerable

Testing initiated at 18 June 2024 08:08 by user mbi.