vulkan-benchmarks

AMD Ryzen 9 7950X 16-Core testing with a ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS) and NVIDIA GeForce RTX 4090 24GB on Ubuntu 23.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2308069-PTS-VULKANBE16&grr&sor.

vulkan-benchmarks ProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLCompilerFile-SystemScreen ResolutionDisplay Driverabcdefghi40804080 rep4080 xxx4080 zzz30903090 rep3070RTX 3070 Ti40904090 repnv 4090AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads)ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS)AMD Device 14d832GBWestern Digital WD_BLACK SN850X 1000GB + 4001GBAMD Radeon RX 6700 XT (2855/1000MHz)AMD Navi 21/23ASUS MG28UIntel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411Ubuntu 23.046.4.6-060406-generic (x86_64)GNOME Shell 44.2X Server 1.21.1.7 + Wayland4.6 Mesa 23.3~git2307260600.87109c~oibaf~l (git-87109c3 2023-07-26 lunar-oibaf-ppa) (LLVM 15.0.7 DRM 3.52)GCC 12.2.0ext43840x2160MSI NVIDIA GeForce RTX 4060 8GBNVIDIA Device 22beX Server 1.21.1.7NVIDIA 535.86.054.6.0eVGA NVIDIA GeForce RTX 3060 12GBNVIDIA GA106 HD AudioNVIDIA GeForce RTX 3060 Ti 8GBNVIDIA GA104 HD Audio2560x1440NVIDIA GeForce RTX 4080 16GBNVIDIA Device 22bb3840x2160NVIDIA GeForce RTX 3090 24GBNVIDIA GA102 HD AudioNVIDIA GeForce RTX 3070 8GBNVIDIA GA104 HD Audio2560x1440NVIDIA GeForce RTX 3070 Ti 8GBNVIDIA GeForce RTX 4090 24GBNVIDIA AD102 HD Audio3840x2160OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- a: Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0xa601203- b: Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0xa601203- c: Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0xa601203- d: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- e: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- f: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- g: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- h: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- i: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- 4080: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- 4080 rep: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- 4080 xxx: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- 4080 zzz: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- 3090: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- 3090 rep: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- 3070: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- RTX 3070 Ti: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- 4090: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- 4090 rep: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203- nv 4090: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203Graphics Details- a: BAR1 / Visible vRAM Size: 12272 MB - vBIOS Version: 113-D5121100-101- b: BAR1 / Visible vRAM Size: 12272 MB - vBIOS Version: 113-D5121100-101- c: BAR1 / Visible vRAM Size: 12272 MB - vBIOS Version: 113-D5121100-101- d: BAR1 / Visible vRAM Size: 8192 MiB - vBIOS Version: 95.07.31.00.e3- e: BAR1 / Visible vRAM Size: 8192 MiB - vBIOS Version: 95.07.31.00.e3- f: BAR1 / Visible vRAM Size: 16384 MiB - vBIOS Version: 94.06.14.40.46- g: BAR1 / Visible vRAM Size: 16384 MiB - vBIOS Version: 94.06.14.40.46- h: BAR1 / Visible vRAM Size: 16384 MiB - vBIOS Version: 94.06.14.40.46- i: BAR1 / Visible vRAM Size: 8192 MiB - vBIOS Version: 94.04.25.00.2c- 4080: BAR1 / Visible vRAM Size: 16384 MiB - vBIOS Version: 95.03.0e.00.04- 4080 rep: BAR1 / Visible vRAM Size: 16384 MiB - vBIOS Version: 95.03.0e.00.04- 4080 xxx: BAR1 / Visible vRAM Size: 16384 MiB - vBIOS Version: 95.03.0e.00.04- 4080 zzz: BAR1 / Visible vRAM Size: 16384 MiB - vBIOS Version: 95.03.0e.00.04- 3090: BAR1 / Visible vRAM Size: 32768 MiB - vBIOS Version: 94.02.27.00.02- 3090 rep: BAR1 / Visible vRAM Size: 32768 MiB - vBIOS Version: 94.02.27.00.02- 3070: BAR1 / Visible vRAM Size: 8192 MiB - vBIOS Version: 94.04.25.00.2b- RTX 3070 Ti: BAR1 / Visible vRAM Size: 8192 MiB - vBIOS Version: 94.04.5b.00.02- 4090: BAR1 / Visible vRAM Size: 32768 MiB - vBIOS Version: 95.02.20.00.01- 4090 rep: BAR1 / Visible vRAM Size: 32768 MiB - vBIOS Version: 95.02.20.00.01- nv 4090: BAR1 / Visible vRAM Size: 32768 MiB - vBIOS Version: 95.02.20.00.01Security Details- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

vulkan-benchmarks vkpeak: fp16-vec4vkpeak: int32-scalarvkpeak: int16-vec4vkpeak: int32-vec4vkpeak: int16-scalarvkpeak: fp16-scalarvkpeak: fp32-vec4vkpeak: fp32-scalarvkpeak: fp64-scalarvkpeak: fp64-vec4ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenet-v3ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - FastestDetncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - vision_transformerncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - regnety_400mncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - squeezenet_ssdncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - yolov4-tinyncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - resnet50ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - alexnetncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - resnet18ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - vgg16ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - googlenetncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - blazefacencnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - efficientnet-b0ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mnasnetncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - shufflenet-v2ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - mobilenet-v2ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenetvkfft: FFT + iFFT C2C Bluestein benchmark in double precisionvkresample: 2x - Doublencnn: Vulkan GPU-v3-v3 - mobilenet-v3ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - FastestDetncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - vision_transformerncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - regnety_400mncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - squeezenet_ssdncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - yolov4-tinyncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - resnet50ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - alexnetncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - resnet18ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - vgg16ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - googlenetncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - blazefacencnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - efficientnet-b0ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mnasnetncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - shufflenet-v2ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenet-v3ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - mobilenet-v2ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenetvkfft: FFT + iFFT C2C 1D batched in double precisionncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenet-v3vkfft: FFT + iFFT C2C Bluestein in single precisionncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - yolov4-tinyncnn: CPU-v3-v3 - mobilenet-v3ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - FastestDetncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - vision_transformerncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - regnety_400mncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - squeezenet_ssdncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - resnet50ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - alexnetncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - resnet18ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - vgg16ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - googlenetncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - blazefacencnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - efficientnet-b0ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - mnasnetncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - shufflenet-v2ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v2-v2 - mobilenet-v2ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - mobilenetncnn: Vulkan GPU - FastestDetncnn: Vulkan GPU - vision_transformerncnn: Vulkan GPU - regnety_400mncnn: Vulkan GPU - squeezenet_ssdncnn: Vulkan GPU - yolov4-tinyncnn: Vulkan GPU - resnet50ncnn: Vulkan GPU - alexnetncnn: Vulkan GPU - resnet18ncnn: Vulkan GPU - vgg16ncnn: Vulkan GPU - googlenetncnn: Vulkan GPU - blazefacencnn: Vulkan GPU - efficientnet-b0ncnn: Vulkan GPU - mnasnetncnn: Vulkan GPU - shufflenet-v2ncnn: Vulkan GPU-v2-v2 - mobilenet-v2ncnn: Vulkan GPU - mobilenetncnn: CPU - FastestDetncnn: CPU - blazefacencnn: Vulkan GPU-v3-v3-v3-v3-v3 - mobilenet-v3ncnn: CPU-v3-v3-v3-v3-v3 - mobilenet-v3ncnn: CPU - vision_transformerncnn: CPU - regnety_400mncnn: CPU - squeezenet_ssdncnn: CPU - yolov4-tinyncnn: CPU - resnet50ncnn: CPU - alexnetncnn: CPU - resnet18ncnn: CPU - vgg16ncnn: CPU - googlenetncnn: CPU - efficientnet-b0ncnn: CPU - mnasnetncnn: CPU - shufflenet-v2ncnn: CPU-v2-v2 - mobilenet-v2ncnn: CPU - mobilenetncnn: CPU-v3-v3-v3 - FastestDetncnn: CPU-v3-v3-v3 - vision_transformerncnn: CPU-v3-v3-v3 - regnety_400mncnn: CPU-v3-v3-v3 - squeezenet_ssdncnn: CPU-v3-v3-v3 - yolov4-tinyncnn: CPU-v3-v3-v3 - resnet50ncnn: CPU-v3-v3-v3 - alexnetncnn: CPU-v3-v3-v3 - resnet18ncnn: CPU-v3-v3-v3 - vgg16ncnn: CPU-v3-v3-v3 - googlenetncnn: CPU-v3-v3-v3 - blazefacencnn: CPU-v3-v3-v3 - efficientnet-b0ncnn: CPU-v3-v3-v3 - mnasnetncnn: CPU-v3-v3-v3 - shufflenet-v2ncnn: CPU-v3-v3-v3-v2-v2 - mobilenet-v2ncnn: CPU-v3-v3-v3 - mobilenetncnn: Vulkan GPU-v3-v3-v3 - regnety_400mncnn: Vulkan GPU-v3-v3-v3 - FastestDetncnn: Vulkan GPU-v3-v3-v3 - vision_transformerncnn: Vulkan GPU-v3-v3-v3 - squeezenet_ssdncnn: Vulkan GPU-v3-v3-v3 - yolov4-tinyncnn: Vulkan GPU-v3-v3-v3 - resnet50ncnn: Vulkan GPU-v3-v3-v3 - alexnetncnn: Vulkan GPU-v3-v3-v3 - resnet18ncnn: Vulkan GPU-v3-v3-v3 - vgg16ncnn: Vulkan GPU-v3-v3-v3 - googlenetncnn: Vulkan GPU-v3-v3-v3 - blazefacencnn: Vulkan GPU-v3-v3-v3 - efficientnet-b0ncnn: Vulkan GPU-v3-v3-v3 - mnasnetncnn: Vulkan GPU-v3-v3-v3 - shufflenet-v2ncnn: Vulkan GPU-v3-v3-v3-v2-v2 - mobilenet-v2ncnn: Vulkan GPU-v3-v3-v3 - mobilenetvkfft: FFT + iFFT C2C 1D batched in single precisionncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - FastestDetncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - vision_transformerncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - regnety_400mncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - squeezenet_ssdncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - yolov4-tinyncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - resnet50ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - alexnetncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - resnet18ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - vgg16ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - googlenetncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - blazefacencnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - efficientnet-b0ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mnasnetncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - shufflenet-v2ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenet-v3ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - mobilenet-v2ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenetncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenet-v3vkfft: FFT + iFFT C2C 1D batched in single precision, no reshufflingncnn: CPU-v3-v3-v3-v3-v3-v3 - FastestDetncnn: CPU-v3-v3-v3-v3-v3-v3 - vision_transformerncnn: CPU-v3-v3-v3-v3-v3-v3 - regnety_400mncnn: CPU-v3-v3-v3-v3-v3-v3 - squeezenet_ssdncnn: CPU-v3-v3-v3-v3-v3-v3 - yolov4-tinyncnn: CPU-v3-v3-v3-v3-v3-v3 - resnet50ncnn: CPU-v3-v3-v3-v3-v3-v3 - alexnetncnn: CPU-v3-v3-v3-v3-v3-v3 - resnet18ncnn: CPU-v3-v3-v3-v3-v3-v3 - vgg16ncnn: CPU-v3-v3-v3-v3-v3-v3 - googlenetncnn: CPU-v3-v3-v3-v3-v3-v3 - blazefacencnn: CPU-v3-v3-v3-v3-v3-v3 - efficientnet-b0ncnn: CPU-v3-v3-v3-v3-v3-v3 - mnasnetncnn: CPU-v3-v3-v3-v3-v3-v3 - shufflenet-v2ncnn: CPU-v3-v3-v3-v3-v3-v3-v2-v2 - mobilenet-v2ncnn: CPU-v3-v3-v3-v3-v3-v3 - mobilenetvkfft: FFT + iFFT C2C 1D batched in half precisionvkfft: FFT + iFFT C2C multidimensional in single precisionvkfft: FFT + iFFT R2C / C2Rvkresample: 2x - Singleabcdefghi40804080 rep4080 xxx4080 zzz30903090 rep3070RTX 3070 Ti40904090 repnv 409023232.422272.6223123.772658.7313102.7513154.1512730.0813190.09841.40841.8047173.1720816113403.184.131.888.167.0912.8410.014.315.2823.517.901.383.862.983.353.178.053.621.3832.498.187.0712.9010.204.415.2923.757.943.902.973.343.168.05478875050491597330014210511.68623390.442269.2523396.592640.0813070.8113145.1912808.5912807.06839.2836.55469520822112734.0731.858.217.0712.87104.335.2323.567.851.373.822.953.333.148.044.051.383.163.1631.958.187.0612.7410.014.325.223.497.823.852.973.343.167.974.0731.658.057.0312.9810.014.295.2123.57.841.373.822.973.333.158.018.274.0631.717.1412.779.874.425.4223.427.971.373.852.963.333.158479485064391812327514216311.6923387.262269.0623385.442638.6913063.8613136.7912822.0112860.56839.01836.1646703.220847113113.174.0931.7987.0612.81104.335.2123.547.81.373.832.963.323.138.034.111.393.173.1631.778.277.112.8110.114.315.2423.457.933.882.993.353.188.024.0831.788.147.0412.8910.334.285.2623.997.881.383.892.973.343.1487.983.6931.667.0712.8610.034.35.2323.547.831.363.822.963.333.157.95479715059691744328124302111.68816864.478520.027352.858465.825676.028412.3311251.178531.96267.43267.742346500.0143.1712143107193.174.1132.128.177.0812.8510.104.315.2323.567.851.383.872.973.353.168.024.081.3832.438.237.0912.9510.004.305.2323.517.853.852.983.353.178.10426454336585181363283539932.85516865.298505.207336.258465.715675.998397.8011231.728515.58267.41267.252343500.0163.1812168105604.0831.938.107.0512.8710.104.315.2223.607.851.383.842.963.333.148.04426514336585191370903530432.85013440.976827.925959.756800.174480.596812.529006.576837.94214.17214.231814500.013.141056175714.2432.928.347.0813.1710.264.645.4824.558.151.383.862.973.43.138.274.221.373.153.1533.568.087.2313.3211.054.365.6924.197.923.872.973.553.158.453.8533.478.346.9713.0711.054.836.1324.458.071.434.043.123.43.168.568.54.233.367.0914.3410.254.355.324.127.941.373.852.963.333.168.655647657110104146262382659326.73813438.46824.215956.246795.394478.416810.559003.126832.74213.96213.951818500.011105483.16757413.143.143.9233.328.387.1410.334.355.2824.047.961.383.842.973.343.178.54.0732.428.367.113.6410.344.876.2224.28.961.413.862.983.353.188.172.571.383.153.1632.738.37.1317.2310.724.325.5523.787.983.9133.593.1622.743.9733.398.077.2613.0811.254.715.4824.929.151.374.143.053.383.158.987.994.0632.387.1912.8910.184.355.323.827.941.373.852.953.333.148.045645557094104171265412663826.76913490.246800.65978.386772.984495.986838.329036.176810.73213.37210.9610572762256431104298265242417500.0064.87147803.261006113.773.263.8338.018.467.2113.15.15.8827.4310.471.414.193.073.433.2910.025.1436.429.887.4615.1612.966.535.8227.838.751.45.883.23.493.2910.42.661.43.263.2937.89.948.9614.6512.095.35.630.9610.34.052.745.033.528.375.6936.558.218.1615.1114.055.015.8629.1210.171.284.683.393.523.310.087.994.4338.338.3315.4311.154.995.8529.0710.191.254.212.993.363.289.056973871163132270346863372720.935579288.2014.1935.68.677.7113.9311.484.985.9225.678.791.444.043.13.483.263.298.43349741712113.863.244.2834.918.617.6611.44.615.6125.488.41.434.063.063.463.288.844.235.568.397.5813.7910.814.625.67258.421.414.013.073.433.268.444.421.43.273.2835.078.247.7313.8511.164.755.6925.378.423.993.053.413.288.734.234.28.337.6413.8111.114.665.725.18.451.444.023.083.463.318.438.454.234.137.6613.7910.954.695.6525.048.491.424.053.093.443.298.43104556106210211076658696647313.1365583288.1664.1733.938.357.6213.7311.074.645.6725.568.421.424.013.063.443.313.278.38350383.241728713.553.264.1434.18.247.5510.84.725.6125.058.491.413.983.033.393.278.44.2135.078.567.6413.6710.844.655.6125.048.41.414.053.083.443.298.574.341.423.2735.288.677.8614.0311.764.675.6826.118.524.093.063.433.288.414.1834.278.577.5913.5510.794.655.6324.918.521.424.023.093.433.278.488.444.0934.297.6313.6810.844.695.6925.048.581.454.043.073.443.38.461044913.281062054.234.228.727.6713.7110.864.685.6425.018.521.434.073.093.473.38.45211058700686827913.1365587288.0393.7533.98.257.2713.5211.224.715.6525.338.261.313.972.983.343.053.148.31350713.261734313.623.274.1734.238.527.6710.914.675.6725.48.431.424.023.053.433.278.464.234.198.567.6213.6510.944.685.6225.018.421.424.043.073.53.268.374.171.423.333.3134.278.457.6213.610.824.685.5625.038.384.043.063.453.288.374.1934.378.757.6413.6910.914.655.66258.51.424.063.073.473.38.448.584.3135.47.713.9511.55.215.8926.088.991.424.223.133.513.48.881045283.081060993.834.148.387.2713.6311.264.695.7825.448.321.324.0133.43.28.34210713678876906813.1375584288.0283.284.6135.368.378.0615.2612.54.75.7426.098.551.414.053.083.443.263.289.19350583.241718513.613.244.234.18.497.6210.914.685.625.168.421.424.013.083.433.288.464.7934.328.587.6313.811.074.685.5925.828.411.424.043.063.463.298.474.161.43.23.2734.18.377.5513.6311.14.695.6325.48.43.993.043.423.258.384.0434.478.477.3513.8311.214.675.7125.458.551.414.033.063.433.288.48.14.1234.057.5113.6211.094.655.5925.268.371.393.953.013.373.238.381045433.061059263.8234.478.347.2513.4211.14.665.7725.268.291.313.952.963.363.168.25210991700406768913.12641149.120909.0216886.6620820.0913710.8820845.0927797.821269.72653.13653.153.164.0832.168.27.0512.8610.034.35.223.57.831.393.862.973.363.158.014282371.6993.193.8331.948.337.0512.9710.074.325.2123.517.841.383.882.943.323.133.128.03309453.161440613.13.184.0333.227.997.0410.384.315.1923.557.861.363.832.963.333.158.114.1131.948.387.1612.8810.14.355.2723.557.871.393.882.993.393.188.074.211.383.1533.018.257.5214.2610.34.35.2123.437.863.872.993.343.178.64.0431.897.957.0412.8710.054.315.1923.57.831.363.832.953.323.148.068.014.0431.867.0412.889.974.35.2323.57.821.363.852.973.343.1681413571439694.132.18.227.1212.8210.034.335.223.587.91.393.882.983.363.168.07255207510055534710.39940876.1220613.4116878.220517.4513606.7920640.6727393.220708.84648.714.132.138.197.0712.8910.044.35.223.477.821.383.852.983.363.178.054289371.4223.164.0731.978.077.0812.99.984.35.223.387.861.373.852.973.343.183.178.05311221444912.863.194.0832.098.347.0910.044.315.2223.527.891.393.872.993.373.198.044.0831.918.027.0612.8310.064.315.223.487.821.373.862.973.363.178.034.081.383.153.1531.88.247.1212.779.954.315.2923.437.93.862.973.353.178.014.1131.938.097.0912.8210.074.35.223.547.851.373.852.963.333.178.038.254.132.117.0812.8410.014.35.2423.437.851.383.852.973.363.178.011414374.0731.948.067.0712.9210.274.315.2723.727.861.383.842.963.323.193.158.033.171439564.0731.858.037.0912.8110.064.315.323.47.911.383.852.973.333.168.06265171548145443210.4285.974.4869.4818.2415.4626.3323.5410.6913.3451.2818.663.579.538.156.35.4918.5424.7456.66.7170.2917.6118.8328.7323.4411.8911.349.716.972.999.816.065.595.995.4617.096.5629.87.526.9371.0817.8815.423.5910.0812.1455.4818.61.778.414.5988.3518.398.4170.7616.2215.8228.5923.4810.8812.6848.2919.493.988.995.097.077.8121.118.652.988.065.3875.341813.227.6621.59.6214.0356.6418.259.236.886.829.6717.819.1881.7719.6617.7529.3424.071111.1449.75173.039.016.878.139.1917.8217.238.6373.5116.1529.4923.119.8613.3855.4220.723.187.816.024.897.2416.347.2370.5317.0215.3229.3822.1511.4312.1353.4818.82.696.638.555.897.345.9217.066.437.1265.4118.2514.2728.4122.1910.5912.6450.3219.22.539.196.077.817.2216.5222.0643.524.2538.329.108.3115.5612.525.416.6928.539.581.514.553.253.923.699.9824.8053.644.3238.279.058.6515.0012.425.346.4028.539.901.344.373.103.893.443.669.523.6215.213.614.4137.889.198.2812.735.536.5728.639.841.494.603.343.753.669.624.2638.039.078.4715.5412.605.256.2829.069.871.604.533.403.983.569.353.941.603.653.7637.918.838.1315.2012.735.496.0828.369.694.723.263.773.769.434.3337.869.028.3915.4212.115.556.1828.409.651.794.783.114.093.419.628.894.2638.298.2915.4412.355.676.2328.409.861.714.733.373.953.669.624.1438.509.147.4514.6413.156.255.9427.869.971.404.173.123.483.243.8310.023.704.1838.048.427.5714.5712.816.176.2227.989.682.484.743.244.023.9110.0327.1833.362.8238.8210.057.415.6912.984.947.7827.318.911.454.143.193.475.258.468039172.8833.284.4538.629.879.8115.9512.44.677.5227.449.051.424.154.933.523.623.489.16552143.342037315.553.534.1339.0610.19.2811.534.725.7829.0511.31.434.243.15.173.418.964.3938.768.647.8313.9714.134.645.6928.8210.621.394.343.183.453.310.555.481.273.123.2538.258.137.8613.6814.15.14627.7510.274.233.123.553.310.082.9338.798.139.3215.311.394.945.8128.559.971.174.093.195.183.468.969.63.9439.017.9315.4414.586.115.9728.218.871.334.1833.344.998.811538964.6239.3510.117.4316.05135.146.9627.328.551.354.633.233.563.364.7510.563.331526562.8538.7610.099.5115.8511.724.997.7430.168.381.34.475.193.483.369.0429034281406843519.2843.443.9138.698.459.3416.611.245.165.8429.3510.181.384.443.285.183.68.838119173.0433.34.1139.0310.699.4615.411.515.255.929.1710.391.414.414.993.483.343.369.02553833.332040415.453.313.9638.178.649.1612.175.336.0529.1210.381.464.043.135.273.348.374.5937.598.488.2215.7213.086.796.0127.0410.651.44.093.153.493.3110.235.271.454.93.5337.058.787.9816.7913.735.236.7729.2410.864.33.183.473.310.754.1639.1217.159.315.3413.825.277.7527.259.291.344.343.233.593.458.7410.344.1638.737.8116.3913.576.585.8127.598.91.416.283.13.43.319.541539394.5938.658.79.4415.4110.965.345.8729.8510.471.424.13.125.233.353.388.223.31559363.1238.7910.239.3813.8812.475.458.1430.748.971.424.355.113.423.4410.6128765180999813298.9624.814.0138.588.377.7215.6713.466.545.9729.48.852.915.883.13.373.2710.158132172.8873.362.6438.4610.037.0215.5513.134.697.3828.148.351.264.13.073.53.173.428.93549503.262060117.33.472.8139.1810.099.2113.635.148.1627.898.611.184.123.163.515.19.413.9339.049.819.1115.2612.455.27.4429.298.71.44.14.73.513.438.454.511.332.613.3538.910.179.1115.411.415.187.8229.548.934.374.773.453.68.154.0638.999.559.3715.6213.684.677.6127.049.021.164.044.613.463.398.917.735.9238.587.7216.6113.136.115.8427.7710.011.075.262.543.174.4510.541521705.8637.138.348.2616.313.256.325.5827.2510.141.45.823.13.434.963.2910.644.971551483.9338.958.257.4817.6713.296.626.0727.6110.751.425.943.123.323.2912.1229276882875848878.967OpenBenchmarking.org

vkpeak

fp16-vec4

OpenBenchmarking.orgGFLOPS, More Is Bettervkpeak 20230730fp16-vec43090 rep3090bcaedhfg9K18K27K36K45KSE +/- 5.96, N = 3SE +/- 0.36, N = 3SE +/- 0.37, N = 341188.0241149.1023390.4423387.2623232.4216865.2916864.4713490.2413440.9713438.47

vkpeak

int32-scalar

OpenBenchmarking.orgGIOPS, More Is Bettervkpeak 20230730int32-scalar30903090 repdefghabc4K8K12K16K20KSE +/- 15.02, N = 3SE +/- 0.03, N = 3SE +/- 0.34, N = 320909.0220767.648520.028505.206827.926824.296800.602272.622269.252269.06

vkpeak

int16-vec4

OpenBenchmarking.orgGIOPS, More Is Bettervkpeak 20230730int16-vec4bca30903090 repdehfg5K10K15K20K25KSE +/- 21.55, N = 3SE +/- 17.33, N = 3SE +/- 0.31, N = 323396.5923385.4423123.7716886.6616881.477352.857336.255978.385959.755956.38

vkpeak

int32-vec4

OpenBenchmarking.orgGIOPS, More Is Bettervkpeak 20230730int32-vec430903090 repdefghabc4K8K12K16K20KSE +/- 0.19, N = 3SE +/- 0.05, N = 3SE +/- 0.26, N = 320820.0920517.688465.828465.716800.176795.396772.982658.732640.082638.69

vkpeak

int16-scalar

OpenBenchmarking.orgGIOPS, More Is Bettervkpeak 20230730int16-scalar30903090 repabcdehfg3K6K9K12K15KSE +/- 1.30, N = 3SE +/- 0.09, N = 3SE +/- 0.02, N = 313710.8813608.5713102.7513070.8113063.865676.025675.994495.984480.594479.22

vkpeak

fp16-scalar

OpenBenchmarking.orgGFLOPS, More Is Bettervkpeak 20230730fp16-scalar3090 rep3090abcdehfg4K8K12K16K20KSE +/- 4.01, N = 3SE +/- 13.46, N = 3SE +/- 5.09, N = 320953.3020845.0913154.1513145.1913136.798412.338397.806838.326812.526811.35

vkpeak

fp32-vec4

OpenBenchmarking.orgGFLOPS, More Is Bettervkpeak 20230730fp32-vec43090 rep3090cbadehfg6K12K18K24K30KSE +/- 1.81, N = 3SE +/- 19.37, N = 3SE +/- 2.57, N = 327807.5827797.8012822.0112808.5912730.0811251.1711231.729036.179006.579003.12

vkpeak

fp32-scalar

OpenBenchmarking.orgGFLOPS, More Is Bettervkpeak 20230730fp32-scalar30903090 repacbdefgh5K10K15K20K25KSE +/- 4.18, N = 3SE +/- 16.18, N = 3SE +/- 0.30, N = 321269.7220925.3013190.0912860.5612807.068531.968515.586837.946832.746810.73

vkpeak

fp64-scalar

OpenBenchmarking.orgGFLOPS, More Is Bettervkpeak 20230730fp64-scalarabc3090 rep3090defgh2004006008001000SE +/- 0.22, N = 3SE +/- 0.01, N = 3SE +/- 0.00, N = 3841.40839.20839.01653.63653.13267.43267.41214.17213.96213.37

vkpeak

fp64-vec4

OpenBenchmarking.orgGFLOPS, More Is Bettervkpeak 20230730fp64-vec4abc3090defgh2004006008001000SE +/- 0.32, N = 3SE +/- 0.48, N = 3SE +/- 0.00, N = 3841.80836.55836.16653.15267.74267.25214.23213.95210.96

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3309040904090 repRTX 3070 Tinv 409030701.34332.68664.02995.37326.7165SE +/- 0.17, N = 153.163.363.443.524.815.97MIN: 3.12 / MAX: 3.67MIN: 3.21 / MAX: 4.83MIN: 3.3 / MAX: 4.34MIN: 2.95 / MAX: 536.1MIN: 3.13 / MAX: 149.75MIN: 2.84 / MAX: 111.81. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: FastestDet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: FastestDet40904090 repnv 409030903090 repRTX 3070 Ti30701.0082.0163.0244.0325.04SE +/- 0.29, N = 152.823.914.014.084.104.254.48MIN: 2.69 / MAX: 3.5MIN: 3.77 / MAX: 5.87MIN: 3.87 / MAX: 5.47MIN: 4.04 / MAX: 4.2MIN: 4.06 / MAX: 4.2MIN: 2.46 / MAX: 526.3MIN: 2.2 / MAX: 27.61. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vision_transformer

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vision_transformer3090 rep3090RTX 3070 Tinv 40904090 rep409030701530456075SE +/- 0.12, N = 1532.1332.1638.3238.5838.6938.8269.48MIN: 31.95 / MAX: 32.87MIN: 31.94 / MAX: 33.7MIN: 32.26 / MAX: 477.15MIN: 33.06 / MAX: 464.16MIN: 33.32 / MAX: 390.07MIN: 33.83 / MAX: 435.6MIN: 39.08 / MAX: 374.311. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: regnety_400m

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: regnety_400m3090 rep3090nv 40904090 repRTX 3070 Ti4090307048121620SE +/- 0.20, N = 158.198.208.378.459.1010.0518.24MIN: 8.12 / MAX: 8.98MIN: 8.14 / MAX: 8.74MIN: 8.08 / MAX: 10.1MIN: 8.05 / MAX: 12.64MIN: 7.61 / MAX: 454.62MIN: 8.13 / MAX: 173.18MIN: 7.5 / MAX: 201.091. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd30903090 rep4090nv 4090RTX 3070 Ti4090 rep307048121620SE +/- 0.22, N = 157.057.077.407.728.319.3415.46MIN: 6.98 / MAX: 7.81MIN: 6.99 / MAX: 7.81MIN: 6.81 / MAX: 8.46MIN: 7.13 / MAX: 8.97MIN: 6.35 / MAX: 364.95MIN: 6.88 / MAX: 268.7MIN: 7.08 / MAX: 147.311. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny30903090 repRTX 3070 Tinv 409040904090 rep3070612182430SE +/- 0.25, N = 1512.8612.8915.5615.6715.6916.6026.33MIN: 12.74 / MAX: 13.68MIN: 12.79 / MAX: 13.77MIN: 12.24 / MAX: 459.8MIN: 12.91 / MAX: 334.44MIN: 13.13 / MAX: 187.93MIN: 12.98 / MAX: 103.04MIN: 12.62 / MAX: 127.321. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet50

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet5030903090 rep4090 repRTX 3070 Ti4090nv 40903070612182430SE +/- 0.24, N = 1510.0310.0411.2412.5212.9813.4623.54MIN: 9.88 / MAX: 10.86MIN: 9.94 / MAX: 10.91MIN: 10.22 / MAX: 29.96MIN: 9.95 / MAX: 459.05MIN: 10.26 / MAX: 145.62MIN: 10.6 / MAX: 340.67MIN: 10.3 / MAX: 149.491. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: alexnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: alexnet30903090 rep40904090 repRTX 3070 Tinv 409030703691215SE +/- 0.21, N = 154.304.304.945.165.416.5410.69MIN: 4.25 / MAX: 4.63MIN: 4.24 / MAX: 4.85MIN: 4.52 / MAX: 6.23MIN: 4.73 / MAX: 6.38MIN: 4.23 / MAX: 364.66MIN: 4.56 / MAX: 110.58MIN: 4.32 / MAX: 148.921. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet18

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet1830903090 rep4090 repnv 4090RTX 3070 Ti409030703691215SE +/- 0.24, N = 155.205.205.845.976.697.7813.34MIN: 5.1 / MAX: 6.05MIN: 5.08 / MAX: 6.05MIN: 5.35 / MAX: 8.28MIN: 5.4 / MAX: 8.25MIN: 5.06 / MAX: 462.37MIN: 5.4 / MAX: 168.29MIN: 5.43 / MAX: 279.861. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vgg16

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vgg163090 rep30904090RTX 3070 Ti4090 repnv 409030701224364860SE +/- 0.26, N = 1523.4723.5027.3128.5329.3529.4051.28MIN: 23.25 / MAX: 24.24MIN: 23.26 / MAX: 24.34MIN: 24.27 / MAX: 230.86MIN: 24.21 / MAX: 515.3MIN: 24.55 / MAX: 485.35MIN: 26.17 / MAX: 411.51MIN: 24.83 / MAX: 242.121. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: googlenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: googlenet3090 rep3090nv 40904090RTX 3070 Ti4090 rep3070510152025SE +/- 0.22, N = 157.827.838.858.919.5810.1818.66MIN: 7.72 / MAX: 8.6MIN: 7.73 / MAX: 8.6MIN: 8.16 / MAX: 10.25MIN: 8.3 / MAX: 10.96MIN: 7.62 / MAX: 396.9MIN: 7.81 / MAX: 204.67MIN: 7.42 / MAX: 326.731. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: blazeface

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: blazeface3090 rep4090 rep30904090RTX 3070 Tinv 409030700.80331.60662.40993.21324.0165SE +/- 0.14, N = 151.381.381.391.451.512.913.57MIN: 1.35 / MAX: 1.88MIN: 1.33 / MAX: 1.98MIN: 1.36 / MAX: 3.12MIN: 1.38 / MAX: 2.98MIN: 1.11 / MAX: 380.46MIN: 1.29 / MAX: 113.97MIN: 1.08 / MAX: 141.041. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b03090 rep309040904090 repRTX 3070 Tinv 409030703691215SE +/- 0.16, N = 153.853.864.144.444.555.889.53MIN: 3.81 / MAX: 4.6MIN: 3.82 / MAX: 4.82MIN: 3.93 / MAX: 5.94MIN: 4.24 / MAX: 5.18MIN: 3.84 / MAX: 379.07MIN: 3.96 / MAX: 194.08MIN: 3.77 / MAX: 182.531. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mnasnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mnasnet30903090 repnv 40904090RTX 3070 Ti4090 rep3070246810SE +/- 0.11, N = 152.972.983.103.193.253.288.15MIN: 2.93 / MAX: 3.45MIN: 2.94 / MAX: 3.36MIN: 2.97 / MAX: 3.92MIN: 3.04 / MAX: 3.98MIN: 2.68 / MAX: 277.21MIN: 3.15 / MAX: 4.32MIN: 2.67 / MAX: 317.681. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v230903090 repnv 40904090RTX 3070 Ti4090 rep3070246810SE +/- 0.20, N = 153.363.363.373.473.925.186.30MIN: 3.32 / MAX: 3.82MIN: 3.33 / MAX: 3.83MIN: 3.25 / MAX: 5.26MIN: 3.33 / MAX: 5.01MIN: 3.12 / MAX: 496.78MIN: 3.45 / MAX: 200.36MIN: 3.28 / MAX: 147.571. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v230903090 repnv 40904090 repRTX 3070 Ti409030701.23532.47063.70594.94126.1765SE +/- 0.20, N = 153.153.173.273.603.695.255.49MIN: 3.11 / MAX: 3.78MIN: 3.12 / MAX: 3.78MIN: 3.11 / MAX: 4.1MIN: 3.44 / MAX: 4.27MIN: 3.07 / MAX: 544.13MIN: 3.11 / MAX: 367.53MIN: 2.97 / MAX: 152.081. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet30903090 rep40904090 repRTX 3070 Tinv 40903070510152025SE +/- 0.24, N = 158.018.058.468.839.9810.1518.54MIN: 7.96 / MAX: 8.47MIN: 7.98 / MAX: 8.94MIN: 8.12 / MAX: 10.14MIN: 8.29 / MAX: 10.15MIN: 7.79 / MAX: 434.9MIN: 8.08 / MAX: 193.04MIN: 8.01 / MAX: 164.451. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

VkFFT

Test: FFT + iFFT C2C Bluestein benchmark in double precision

OpenBenchmarking.orgBenchmark Score, More Is BetterVkFFT 1.2.31Test: FFT + iFFT C2C Bluestein benchmark in double precisionnv 40904090 rep40904080 xxx4080 zzz4080 rep4080abc3090 rep3090idegf2K4K6K8K10KSE +/- 0.33, N = 3SE +/- 4.37, N = 3SE +/- 11.20, N = 3813281198039558755845583557947174695467042894282241723462343181818141. (CXX) g++ options: -O3

VkResample

Upscale: 2x - Precision: Double

OpenBenchmarking.orgms, Fewer Is BetterVkResample 1.0Upscale: 2x - Precision: Double3070RTX 3070 Ti4090nv 40904090 rep4080 zzz4080 xxx4080 rep40803090 rep3090ifgde110220330440550SE +/- 0.01, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 324.7524.81172.88172.89173.04288.03288.04288.17288.20371.42371.70500.01500.01500.01500.01500.021. (CXX) g++ options: -O3

NCNN

Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3f3090 repade3090c4080 zzz40904090 repnv 4090RTX 3070 Tii3070246810SE +/- 0.02, N = 3SE +/- 0.00, N = 2SE +/- 0.02, N = 3SE +/- 0.20, N = 143.143.163.173.173.183.193.203.283.283.303.363.644.876.60MIN: 3.09 / MAX: 3.54MIN: 3.11 / MAX: 3.62MIN: 3.11 / MAX: 3.73MIN: 3.1 / MAX: 3.83MIN: 3.11 / MAX: 3.78MIN: 3.14 / MAX: 3.48MIN: 3.16 / MAX: 3.68MIN: 3.13 / MAX: 4.65MIN: 3.15 / MAX: 3.9MIN: 3.15 / MAX: 3.92MIN: 3.21 / MAX: 4.3MIN: 2.87 / MAX: 429.02MIN: 3.14 / MAX: 278.98MIN: 2.98 / MAX: 166.191. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: FastestDet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: FastestDetnv 40904080 xxx30903090 rep4090 rep4080 rep4080RTX 3070 Ti40904080 zzz3070246810SE +/- 0.27, N = 152.643.753.834.074.114.174.194.324.454.616.71MIN: 2.52 / MAX: 4.14MIN: 3.63 / MAX: 5.24MIN: 3.79 / MAX: 4.09MIN: 4.03 / MAX: 4.18MIN: 3.98 / MAX: 4.73MIN: 4.02 / MAX: 4.75MIN: 4.06 / MAX: 7.41MIN: 2.51 / MAX: 398.91MIN: 4.29 / MAX: 5.05MIN: 4.45 / MAX: 5.92MIN: 2.73 / MAX: 109.521. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vision_transformer

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vision_transformer30903090 rep4080 xxx4080 rep4080 zzz4080RTX 3070 Tinv 409040904090 rep30701632486480SE +/- 0.11, N = 1531.9431.9733.9033.9335.3635.6038.2738.4638.6239.0370.29MIN: 31.72 / MAX: 34.34MIN: 31.71 / MAX: 33.78MIN: 32.72 / MAX: 37.77MIN: 32.77 / MAX: 36.2MIN: 33.87 / MAX: 42.41MIN: 34.13 / MAX: 38.49MIN: 32.29 / MAX: 507.7MIN: 32.39 / MAX: 435.46MIN: 33.33 / MAX: 465MIN: 33.61 / MAX: 343.67MIN: 39.39 / MAX: 250.191. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: regnety_400m

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: regnety_400m3090 rep4080 xxx30904080 rep4080 zzz4080RTX 3070 Ti4090nv 40904090 rep307048121620SE +/- 0.24, N = 158.078.258.338.358.378.679.059.8710.0310.6917.61MIN: 7.99 / MAX: 8.88MIN: 7.93 / MAX: 9.88MIN: 8.25 / MAX: 9.32MIN: 8.05 / MAX: 9.76MIN: 8.04 / MAX: 10.13MIN: 8.3 / MAX: 14.66MIN: 7.52 / MAX: 417.33MIN: 7.81 / MAX: 243.06MIN: 7.81 / MAX: 171.2MIN: 8.17 / MAX: 339.6MIN: 7.85 / MAX: 165.341. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssdnv 409030903090 rep4080 xxx4080 rep40804080 zzzRTX 3070 Ti4090 rep40903070510152025SE +/- 0.24, N = 157.027.057.087.277.627.718.068.659.469.8118.83MIN: 6.38 / MAX: 9.36MIN: 6.97 / MAX: 7.95MIN: 7 / MAX: 7.94MIN: 6.74 / MAX: 8.84MIN: 7.01 / MAX: 14.37MIN: 7.15 / MAX: 9.1MIN: 7.42 / MAX: 9.25MIN: 6.64 / MAX: 544.17MIN: 7.03 / MAX: 160.39MIN: 7.16 / MAX: 389.1MIN: 6.71 / MAX: 206.111. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny3090 rep30904080 xxx4080 rep4080RTX 3070 Ti4080 zzz4090 repnv 409040903070714212835SE +/- 0.19, N = 1512.9012.9713.5213.7313.9315.0015.2615.4015.5515.9528.73MIN: 12.77 / MAX: 13.92MIN: 12.83 / MAX: 13.8MIN: 12.72 / MAX: 21.19MIN: 12.78 / MAX: 20.99MIN: 13.08 / MAX: 15.68MIN: 12.75 / MAX: 401.37MIN: 14.19 / MAX: 17.06MIN: 13 / MAX: 245.79MIN: 12.87 / MAX: 342.3MIN: 13.38 / MAX: 245.18MIN: 12.83 / MAX: 264.491. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet50

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet503090 rep30904080 rep4080 xxx40804090 rep4090RTX 3070 Ti4080 zzznv 40903070612182430SE +/- 0.25, N = 159.9810.0711.0711.2211.4811.5112.4012.4212.5013.1323.44MIN: 9.85 / MAX: 11.35MIN: 9.95 / MAX: 10.88MIN: 10.16 / MAX: 13.16MIN: 10.33 / MAX: 12.81MIN: 10.56 / MAX: 12.93MIN: 10.56 / MAX: 13.22MIN: 11.44 / MAX: 14.43MIN: 10.23 / MAX: 444.76MIN: 11.47 / MAX: 14.56MIN: 10.18 / MAX: 247.5MIN: 10.17 / MAX: 219.361. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: alexnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: alexnet3090 rep30904080 rep4090nv 40904080 zzz4080 xxx40804090 repRTX 3070 Ti30703691215SE +/- 0.16, N = 154.304.324.644.674.694.704.714.985.255.3411.89MIN: 4.24 / MAX: 5.11MIN: 4.25 / MAX: 5.33MIN: 4.24 / MAX: 6MIN: 4.28 / MAX: 6MIN: 4.28 / MAX: 6.33MIN: 4.28 / MAX: 5.92MIN: 4.26 / MAX: 7.21MIN: 4.59 / MAX: 7.15MIN: 4.86 / MAX: 6.33MIN: 4.25 / MAX: 221.78MIN: 4.34 / MAX: 229.181. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet18

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet183090 rep30904080 xxx4080 rep4080 zzz4090 rep4080RTX 3070 Tinv 4090409030703691215SE +/- 0.20, N = 155.205.215.655.675.745.905.926.407.387.5211.30MIN: 5.1 / MAX: 6.09MIN: 5.09 / MAX: 6.13MIN: 5.18 / MAX: 6.76MIN: 5.19 / MAX: 7.38MIN: 5.18 / MAX: 8.08MIN: 5.43 / MAX: 7.49MIN: 5.37 / MAX: 8.24MIN: 5.1 / MAX: 457.07MIN: 5.15 / MAX: 138.85MIN: 5.45 / MAX: 290.49MIN: 5.3 / MAX: 181.71. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vgg16

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vgg163090 rep30904080 xxx4080 rep40804080 zzz4090nv 4090RTX 3070 Ti4090 rep30701122334455SE +/- 0.28, N = 1523.3823.5125.3325.5625.6726.0927.4428.1428.5329.1749.70MIN: 23.19 / MAX: 24.27MIN: 23.27 / MAX: 24.38MIN: 24.26 / MAX: 34.98MIN: 24.24 / MAX: 27.92MIN: 24.46 / MAX: 27.34MIN: 24.58 / MAX: 30.18MIN: 24.06 / MAX: 264.59MIN: 24.24 / MAX: 221.5MIN: 23.95 / MAX: 473.83MIN: 24.61 / MAX: 264.85MIN: 25.55 / MAX: 421.441. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: googlenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: googlenet30903090 rep4080 xxxnv 40904080 rep4080 zzz40804090RTX 3070 Ti4090 rep307048121620SE +/- 0.19, N = 157.847.868.268.358.428.558.799.059.9010.3916.97MIN: 7.74 / MAX: 8.72MIN: 7.75 / MAX: 8.71MIN: 7.62 / MAX: 10.47MIN: 7.7 / MAX: 10.46MIN: 7.77 / MAX: 10.52MIN: 7.86 / MAX: 10.08MIN: 8.08 / MAX: 10.27MIN: 8.26 / MAX: 13.34MIN: 7.76 / MAX: 396.66MIN: 7.87 / MAX: 391.66MIN: 7.44 / MAX: 229.931. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: blazeface

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: blazefacenv 40904080 xxxRTX 3070 Ti3090 rep30904080 zzz4090 rep4080 rep4090408030700.67281.34562.01842.69123.364SE +/- 0.03, N = 151.261.311.341.371.381.411.411.421.421.442.99MIN: 1.2 / MAX: 1.76MIN: 1.25 / MAX: 3.14MIN: 1.06 / MAX: 2.66MIN: 1.35 / MAX: 1.48MIN: 1.36 / MAX: 1.53MIN: 1.34 / MAX: 1.88MIN: 1.35 / MAX: 1.91MIN: 1.35 / MAX: 2.89MIN: 1.36 / MAX: 1.92MIN: 1.37 / MAX: 2.07MIN: 1.22 / MAX: 149.551. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b03090 rep30904080 xxx4080 rep40804080 zzznv 40904090RTX 3070 Ti4090 rep30703691215SE +/- 0.13, N = 153.853.883.974.014.044.054.104.154.374.419.81MIN: 3.78 / MAX: 4.83MIN: 3.83 / MAX: 4.72MIN: 3.79 / MAX: 5.93MIN: 3.81 / MAX: 6.04MIN: 3.84 / MAX: 4.83MIN: 3.83 / MAX: 5.42MIN: 3.87 / MAX: 6.14MIN: 3.93 / MAX: 5.94MIN: 3.85 / MAX: 366.28MIN: 4.21 / MAX: 5.82MIN: 3.87 / MAX: 165.381. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mnasnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mnasnet30903090 rep4080 xxx4080 repnv 40904080 zzz4080RTX 3070 Ti40904090 rep3070246810SE +/- 0.04, N = 152.942.972.983.063.073.083.103.104.934.996.06MIN: 2.9 / MAX: 3.34MIN: 2.94 / MAX: 3.45MIN: 2.86 / MAX: 4.47MIN: 2.93 / MAX: 5.02MIN: 2.93 / MAX: 4.52MIN: 2.95 / MAX: 3.88MIN: 2.95 / MAX: 4.05MIN: 2.61 / MAX: 4.75MIN: 2.97 / MAX: 124.96MIN: 3.02 / MAX: 235.56MIN: 2.96 / MAX: 42.71. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v230904080 xxx3090 rep4080 rep4080 zzz40804090 repnv 40904090RTX 3070 Ti30701.25782.51563.77345.03126.289SE +/- 0.20, N = 153.323.343.343.443.443.483.483.503.523.895.59MIN: 3.29 / MAX: 3.79MIN: 3.22 / MAX: 3.97MIN: 3.3 / MAX: 3.79MIN: 3.31 / MAX: 4.32MIN: 3.31 / MAX: 4.85MIN: 3.34 / MAX: 4.88MIN: 3.34 / MAX: 4.1MIN: 3.37 / MAX: 4.2MIN: 3.38 / MAX: 4.23MIN: 3.08 / MAX: 345.39MIN: 3.32 / MAX: 42.331. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v34080 xxx3090nv 40903090 rep40804080 zzz4080 rep4090 repRTX 3070 Ti409030701.34782.69564.04345.39126.739SE +/- 0.13, N = 133.053.133.173.183.263.263.313.343.443.625.99MIN: 2.94 / MAX: 3.56MIN: 3.09 / MAX: 3.68MIN: 3.04 / MAX: 4.3MIN: 3.13 / MAX: 3.61MIN: 3.13 / MAX: 4.7MIN: 3.12 / MAX: 4.74MIN: 3.16 / MAX: 3.93MIN: 3.19 / MAX: 3.99MIN: 2.65 / MAX: 361.91MIN: 3.47 / MAX: 4.24MIN: 3.05 / MAX: 26.811. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v230904080 xxx3090 rep4080 rep4080 zzz40804090 repnv 40904090RTX 3070 Ti30701.22852.4573.68554.9146.1425SE +/- 0.18, N = 153.123.143.173.273.283.293.363.423.483.665.46MIN: 3.07 / MAX: 3.62MIN: 3 / MAX: 3.85MIN: 3.12 / MAX: 3.89MIN: 3.08 / MAX: 4.68MIN: 3.11 / MAX: 4.26MIN: 3.12 / MAX: 3.99MIN: 3.17 / MAX: 4.8MIN: 3.15 / MAX: 25.1MIN: 3.32 / MAX: 4.99MIN: 2.73 / MAX: 398.42MIN: 3.27 / MAX: 38.651. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet30903090 rep4080 xxx4080 rep4080nv 40904090 rep40904080 zzzRTX 3070 Ti307048121620SE +/- 0.25, N = 158.038.058.318.388.438.939.029.169.199.5217.09MIN: 7.96 / MAX: 8.83MIN: 7.96 / MAX: 9.04MIN: 7.85 / MAX: 10.21MIN: 7.94 / MAX: 10.07MIN: 8.03 / MAX: 9.64MIN: 8.33 / MAX: 11.07MIN: 8.42 / MAX: 11.17MIN: 8.5 / MAX: 10.51MIN: 8.51 / MAX: 11.04MIN: 7.97 / MAX: 420.29MIN: 7.89 / MAX: 121.531. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

VkFFT

Test: FFT + iFFT C2C 1D batched in double precision

OpenBenchmarking.orgBenchmark Score, More Is BetterVkFFT 1.2.31Test: FFT + iFFT C2C 1D batched in double precision4090 rep4090nv 40904080 xxx4080 zzz4080 rep40803090 rep3090cbaiedhfg12K24K36K48K60KSE +/- 14.62, N = 3SE +/- 11.67, N = 3SE +/- 12.42, N = 3SE +/- 10.58, N = 35538355214549503507135058350383497431122309452084720822208161478012168121431057210561105481. (CXX) g++ options: -O3

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3g30904080 rep4080 zzzi4080 xxxnv 409040904090 repRTX 3070 Ti3070246810SE +/- 0.18, N = 153.163.163.243.243.263.263.263.303.333.626.56MIN: 3.12 / MAX: 3.58MIN: 3.11 / MAX: 3.77MIN: 3.11 / MAX: 4.37MIN: 3.1 / MAX: 3.88MIN: 3.11 / MAX: 4.7MIN: 3.13 / MAX: 4.08MIN: 3.13 / MAX: 3.96MIN: 3.14 / MAX: 4.82MIN: 3.19 / MAX: 4.79MIN: 3 / MAX: 469.9MIN: 3.07 / MAX: 110.871. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

VkFFT

Test: FFT + iFFT C2C Bluestein in single precision

OpenBenchmarking.orgBenchmark Score, More Is BetterVkFFT 1.2.31Test: FFT + iFFT C2C Bluestein in single precisionnv 40904090 rep40904080 xxx4080 rep4080 zzz40803090 rep3090acbdeihgf4K8K12K16K20KSE +/- 83.38, N = 3SE +/- 62.67, N = 3SE +/- 72.34, N = 3SE +/- 75.16, N = 152060120404203731734317287171851712114449144061134011311112731071910560100617622757475711. (CXX) g++ options: -O3

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny3090 rep3090g4080 rep4080 zzz4080 xxxi4080RTX 3070 Ti4090 rep4090nv 40903070714212835SE +/- 0.28, N = 1512.8613.1013.1413.5513.6113.6213.7713.8615.2115.4515.5517.3029.80MIN: 12.76 / MAX: 13.73MIN: 13.01 / MAX: 14.17MIN: 13 / MAX: 14.02MIN: 12.72 / MAX: 15.51MIN: 12.67 / MAX: 19.72MIN: 12.71 / MAX: 15.65MIN: 12.96 / MAX: 14.66MIN: 13.04 / MAX: 15.04MIN: 12.34 / MAX: 380.51MIN: 12.65 / MAX: 445.76MIN: 13.11 / MAX: 307.2MIN: 14.66 / MAX: 441.3MIN: 12.85 / MAX: 216.341. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3 - Model: mobilenet-v3

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3 - Model: mobilenet-v3gcda30903090 rep40804080 zzzi4080 rep4080 xxx4090 repnv 40904090RTX 3070 Ti3070246810SE +/- 0.00, N = 3SE +/- 0.00, N = 2SE +/- 0.21, N = 143.143.173.173.183.183.193.243.243.263.263.273.313.473.533.617.52MIN: 3.1 / MAX: 3.81MIN: 3.15 / MAX: 3.74MIN: 3.12 / MAX: 3.96MIN: 3.14 / MAX: 3.82MIN: 3.14 / MAX: 4.14MIN: 3.15 / MAX: 3.72MIN: 3.09 / MAX: 4.73MIN: 3.11 / MAX: 4.47MIN: 3.14 / MAX: 3.9MIN: 3.09 / MAX: 3.96MIN: 3.13 / MAX: 3.85MIN: 3.16 / MAX: 4.73MIN: 3.32 / MAX: 4.91MIN: 3.39 / MAX: 4.31MIN: 2.51 / MAX: 502.85MIN: 2.94 / MAX: 2151. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: FastestDet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: FastestDetnv 4090ig4090 rep309040903090 rep4080 rep4080 xxx4080 zzz4080RTX 3070 Ti3070246810SE +/- 0.20, N = 152.813.833.923.964.034.034.084.144.174.204.284.416.93MIN: 2.68 / MAX: 4.38MIN: 3.7 / MAX: 4.57MIN: 3.88 / MAX: 4.72MIN: 3.79 / MAX: 11.36MIN: 3.99 / MAX: 4.22MIN: 3.89 / MAX: 4.63MIN: 4.04 / MAX: 4.29MIN: 4 / MAX: 5.6MIN: 4.03 / MAX: 5.63MIN: 4.01 / MAX: 11.47MIN: 4.13 / MAX: 4.85MIN: 2.06 / MAX: 295.24MIN: 2.57 / MAX: 163.841. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: vision_transformer

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: vision_transformer3090 rep3090g4080 rep4080 zzz4080 xxx4080RTX 3070 Tii4090 rep4090nv 409030701632486480SE +/- 0.20, N = 1532.0933.2233.3234.1034.1034.2334.9137.8838.0138.1738.3839.1871.08MIN: 31.84 / MAX: 32.77MIN: 33.04 / MAX: 36.99MIN: 31.83 / MAX: 104.12MIN: 32.43 / MAX: 38.75MIN: 32.32 / MAX: 38.54MIN: 33.08 / MAX: 37.43MIN: 33.72 / MAX: 36.82MIN: 32.46 / MAX: 518.57MIN: 32.96 / MAX: 388.09MIN: 32.97 / MAX: 462.63MIN: 33.53 / MAX: 477.38MIN: 33.74 / MAX: 520.24MIN: 38.84 / MAX: 374.681. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: regnety_400m

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: regnety_400m309040904080 rep3090 repgi4080 zzz4080 xxx40804090 repRTX 3070 Tinv 4090307048121620SE +/- 0.21, N = 157.998.108.248.348.388.468.498.528.618.649.1910.0917.88MIN: 7.92 / MAX: 8.78MIN: 7.65 / MAX: 10.05MIN: 7.91 / MAX: 9.53MIN: 8.26 / MAX: 9.09MIN: 8.05 / MAX: 27.34MIN: 8.08 / MAX: 10.33MIN: 8.08 / MAX: 9.72MIN: 8.13 / MAX: 9.73MIN: 8.21 / MAX: 10.07MIN: 8.3 / MAX: 10.51MIN: 7.44 / MAX: 524.66MIN: 7.84 / MAX: 366.66MIN: 7.38 / MAX: 190.771. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd30903090 repgi4080 rep40904080 zzz40804080 xxxRTX 3070 Ti4090 repnv 4090307048121620SE +/- 0.25, N = 147.047.097.147.217.557.577.627.667.678.289.169.2115.40MIN: 6.96 / MAX: 7.7MIN: 7.01 / MAX: 7.97MIN: 7.03 / MAX: 7.99MIN: 6.73 / MAX: 8.82MIN: 6.99 / MAX: 9.08MIN: 7.02 / MAX: 9MIN: 7 / MAX: 9.93MIN: 7.09 / MAX: 8.97MIN: 7.04 / MAX: 9.1MIN: 6.38 / MAX: 381.81MIN: 6.73 / MAX: 423.75MIN: 6.83 / MAX: 203.62MIN: 6.64 / MAX: 132.681. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: resnet50

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: resnet503090 repg30904080 rep4080 xxx4080 zzz408040904090 repRTX 3070 Tiinv 40903070612182430SE +/- 0.23, N = 1510.0410.3310.3810.8010.9110.9111.4011.5312.1712.7313.1013.6323.59MIN: 9.94 / MAX: 10.89MIN: 10.2 / MAX: 11.18MIN: 9.88 / MAX: 18.75MIN: 9.89 / MAX: 12.54MIN: 9.91 / MAX: 13.07MIN: 9.94 / MAX: 14.83MIN: 10.5 / MAX: 13.51MIN: 10.59 / MAX: 13.73MIN: 11.25 / MAX: 13.79MIN: 9.84 / MAX: 518.97MIN: 10.59 / MAX: 267.95MIN: 10.52 / MAX: 488.94MIN: 9.96 / MAX: 177.631. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: alexnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: alexnet30903090 repg40804080 xxx4080 zzz4080 rep4090inv 40904090 repRTX 3070 Ti30703691215SE +/- 0.22, N = 154.314.314.354.614.674.684.724.725.105.145.335.5310.08MIN: 4.25 / MAX: 4.94MIN: 4.26 / MAX: 5.07MIN: 4.28 / MAX: 5.1MIN: 4.24 / MAX: 7.25MIN: 4.27 / MAX: 6.36MIN: 4.26 / MAX: 6.8MIN: 4.25 / MAX: 7.3MIN: 4.31 / MAX: 6.71MIN: 4.75 / MAX: 6.12MIN: 4.65 / MAX: 6.81MIN: 4.83 / MAX: 6.6MIN: 4.22 / MAX: 362.62MIN: 4.36 / MAX: 225.661. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: resnet18

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: resnet1830903090 repg4080 zzz40804080 rep4080 xxx4090i4090 repRTX 3070 Tinv 409030703691215SE +/- 0.23, N = 155.195.225.285.605.615.615.675.785.886.056.578.1612.14MIN: 5.09 / MAX: 6MIN: 5.13 / MAX: 6.1MIN: 5.16 / MAX: 6.09MIN: 5.09 / MAX: 7.51MIN: 5.09 / MAX: 7.91MIN: 5.07 / MAX: 7.08MIN: 5.1 / MAX: 8.06MIN: 5.26 / MAX: 7.24MIN: 5.36 / MAX: 8.2MIN: 5.53 / MAX: 7.66MIN: 4.91 / MAX: 391.33MIN: 5.39 / MAX: 397.44MIN: 5.28 / MAX: 151.531. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: vgg16

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: vgg163090 rep3090g4080 rep4080 zzz4080 xxx4080inv 4090RTX 3070 Ti40904090 rep30701224364860SE +/- 0.30, N = 1523.5223.5524.0425.0525.1625.4025.4827.4327.8928.6329.0529.1255.48MIN: 23.33 / MAX: 25.08MIN: 23.31 / MAX: 24.48MIN: 23.48 / MAX: 73.3MIN: 23.78 / MAX: 26.95MIN: 23.97 / MAX: 27.81MIN: 24.05 / MAX: 27.09MIN: 23.88 / MAX: 51.68MIN: 24.65 / MAX: 251.37MIN: 24.5 / MAX: 463.23MIN: 24.13 / MAX: 500.18MIN: 24.19 / MAX: 451.92MIN: 24.62 / MAX: 266.39MIN: 25.94 / MAX: 298.671. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: googlenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: googlenet30903090 repg40804080 zzz4080 xxx4080 repnv 4090RTX 3070 Ti4090 repi40903070510152025SE +/- 0.24, N = 157.867.897.968.408.428.438.498.619.8410.3810.4710.8718.60MIN: 7.74 / MAX: 8.62MIN: 7.79 / MAX: 8.84MIN: 7.81 / MAX: 9.05MIN: 7.71 / MAX: 10.64MIN: 7.78 / MAX: 10.7MIN: 7.77 / MAX: 10.4MIN: 7.74 / MAX: 10.76MIN: 7.95 / MAX: 10.07MIN: 7.3 / MAX: 438.04MIN: 7.96 / MAX: 255.68MIN: 8.21 / MAX: 350.07MIN: 8.37 / MAX: 194.11MIN: 8.02 / MAX: 292.161. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: blazeface

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: blazeface4090nv 40903090g3090 repi4080 rep4080 xxx4080 zzz40804090 repRTX 3070 Ti30700.39830.79661.19491.59321.9915SE +/- 0.12, N = 141.161.181.361.381.391.411.411.421.421.431.461.491.77MIN: 1.1 / MAX: 2MIN: 1.11 / MAX: 1.85MIN: 1.34 / MAX: 1.46MIN: 1.35 / MAX: 2.09MIN: 1.37 / MAX: 1.52MIN: 1.35 / MAX: 2.02MIN: 1.34 / MAX: 2.1MIN: 1.35 / MAX: 2MIN: 1.34 / MAX: 2.84MIN: 1.36 / MAX: 2.06MIN: 1.39 / MAX: 2.91MIN: 1.05 / MAX: 379.08MIN: 1.08 / MAX: 12.531. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b03090g3090 rep4080 rep4080 zzz4080 xxx4090 rep4080nv 4090i4090RTX 3070 Ti3070246810SE +/- 0.19, N = 153.833.843.873.984.014.024.044.064.124.194.244.608.41MIN: 3.78 / MAX: 4.4MIN: 3.78 / MAX: 4.57MIN: 3.81 / MAX: 4.62MIN: 3.77 / MAX: 5.44MIN: 3.79 / MAX: 5.39MIN: 3.8 / MAX: 5.14MIN: 3.85 / MAX: 4.9MIN: 3.85 / MAX: 4.97MIN: 3.86 / MAX: 5.39MIN: 4.01 / MAX: 5.09MIN: 3.96 / MAX: 5.55MIN: 3.79 / MAX: 336.2MIN: 3.76 / MAX: 67.731. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: mnasnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: mnasnet3090g3090 rep4080 rep4080 xxx4080i4080 zzz40904090 repnv 4090RTX 3070 Ti30701.03282.06563.09844.13125.164SE +/- 0.14, N = 152.962.972.993.033.053.063.073.083.103.133.163.344.59MIN: 2.93 / MAX: 3.31MIN: 2.93 / MAX: 3.88MIN: 2.96 / MAX: 3.32MIN: 2.91 / MAX: 4.45MIN: 2.91 / MAX: 3.67MIN: 2.94 / MAX: 3.67MIN: 2.93 / MAX: 3.84MIN: 2.93 / MAX: 4.42MIN: 2.97 / MAX: 3.71MIN: 3.01 / MAX: 3.62MIN: 3.02 / MAX: 4.6MIN: 2.68 / MAX: 393.6MIN: 2.88 / MAX: 20.121. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v23090g3090 rep4080 repi4080 xxx4080 zzz4080nv 4090RTX 3070 Ti40904090 rep3070246810SE +/- 0.16, N = 153.333.343.373.393.433.433.433.463.513.755.095.278.00MIN: 3.29 / MAX: 3.67MIN: 3.31 / MAX: 4.05MIN: 3.33 / MAX: 3.8MIN: 3.26 / MAX: 3.91MIN: 3.3 / MAX: 4.89MIN: 3.31 / MAX: 3.95MIN: 3.29 / MAX: 3.87MIN: 3.3 / MAX: 5.74MIN: 3.38 / MAX: 4.05MIN: 3.2 / MAX: 361.52MIN: 3.33 / MAX: 161.5MIN: 3.27 / MAX: 191.55MIN: 3.16 / MAX: 190.151. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v23090g3090 rep4080 rep4080 xxx40804080 zzzi40904090 repRTX 3070 Tinv 40903070246810SE +/- 0.15, N = 153.153.173.193.273.273.283.283.293.323.343.665.108.35MIN: 3.1 / MAX: 3.68MIN: 3.1 / MAX: 5.03MIN: 3.13 / MAX: 4MIN: 3.08 / MAX: 5.18MIN: 3.11 / MAX: 4.73MIN: 3.11 / MAX: 4.16MIN: 3.09 / MAX: 4.98MIN: 3.1 / MAX: 3.96MIN: 3.12 / MAX: 4.24MIN: 3.14 / MAX: 4.45MIN: 3.01 / MAX: 311.25MIN: 3.14 / MAX: 138.88MIN: 3.08 / MAX: 103.381. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: mobilenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: mobilenet3090 rep30904090 rep4080 rep4080 xxx4080 zzzg40804090nv 4090RTX 3070 Tii3070510152025SE +/- 0.27, N = 158.048.118.378.408.468.468.508.848.969.419.6210.0218.39MIN: 7.96 / MAX: 9.01MIN: 8.02 / MAX: 14.2MIN: 7.98 / MAX: 10.71MIN: 7.93 / MAX: 15.25MIN: 7.95 / MAX: 10.34MIN: 7.97 / MAX: 10.56MIN: 8.42 / MAX: 9.29MIN: 8.31 / MAX: 10.98MIN: 8.37 / MAX: 11.12MIN: 8.98 / MAX: 11.38MIN: 7.71 / MAX: 449.11MIN: 8.07 / MAX: 266.25MIN: 7.92 / MAX: 173.391. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU - Model: FastestDet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU - Model: FastestDetnv 4090bge3090 repcad309040804080 xxx4080 repfRTX 3070 Ti40904090 rep4080 zzzi3070246810SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.02, N = 3SE +/- 0.29, N = 153.934.074.074.084.084.094.104.114.114.204.204.214.244.264.394.594.795.148.41MIN: 3.8 / MAX: 5.4MIN: 4.04 / MAX: 4.53MIN: 4.02 / MAX: 4.82MIN: 4.03 / MAX: 5.29MIN: 4.04 / MAX: 4.35MIN: 4.05 / MAX: 5.5MIN: 4.06 / MAX: 4.81MIN: 4.01 / MAX: 9.72MIN: 4.07 / MAX: 4.29MIN: 4.02 / MAX: 4.97MIN: 4.03 / MAX: 6.49MIN: 4.04 / MAX: 4.97MIN: 3.88 / MAX: 24.21MIN: 2.5 / MAX: 396.93MIN: 4.25 / MAX: 5.86MIN: 2.62 / MAX: 232.18MIN: 4.64 / MAX: 6.21MIN: 3.7 / MAX: 81.79MIN: 2.89 / MAX: 487.781. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU - Model: vision_transformer

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU - Model: vision_transformercba3090 repe3090dgf4080 xxx4080 zzz4080 rep4080i4090 repRTX 3070 Ti4090nv 409030701632486480SE +/- 0.09, N = 3SE +/- 0.07, N = 3SE +/- 0.21, N = 3SE +/- 0.16, N = 1531.7931.8531.8831.9131.9331.9432.1232.4232.9234.1934.3235.0735.5636.4237.5938.0338.7639.0470.76MIN: 31.63 / MAX: 35.57MIN: 31.69 / MAX: 33.06MIN: 31.55 / MAX: 37.47MIN: 31.74 / MAX: 34.28MIN: 31.62 / MAX: 35.85MIN: 31.73 / MAX: 34.21MIN: 31.66 / MAX: 46.9MIN: 31.89 / MAX: 65.47MIN: 32.67 / MAX: 36.93MIN: 32.72 / MAX: 36.79MIN: 32.58 / MAX: 41.88MIN: 33.66 / MAX: 39.36MIN: 33.19 / MAX: 40.43MIN: 33.49 / MAX: 224.86MIN: 34.45 / MAX: 457.98MIN: 32.66 / MAX: 467.28MIN: 33.12 / MAX: 539.58MIN: 33.83 / MAX: 463.88MIN: 38.81 / MAX: 250.011. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU - Model: regnety_400m

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU - Model: regnety_400mc3090 repeadbfg309040804090 rep4080 rep4080 xxx4080 zzz4090RTX 3070 Tinv 4090i307048121620SE +/- 0.02, N = 3SE +/- 0.11, N = 3SE +/- 0.06, N = 3SE +/- 0.21, N = 158.008.028.108.168.178.218.348.368.388.398.488.568.568.588.649.079.819.8816.22MIN: 7.94 / MAX: 8.88MIN: 7.95 / MAX: 8.63MIN: 7.98 / MAX: 8.84MIN: 7.9 / MAX: 8.99MIN: 7.99 / MAX: 8.97MIN: 8.14 / MAX: 8.84MIN: 7.99 / MAX: 26.72MIN: 8.27 / MAX: 9.08MIN: 8.31 / MAX: 8.86MIN: 8 / MAX: 10.29MIN: 8.09 / MAX: 9.64MIN: 8.17 / MAX: 10.28MIN: 8.15 / MAX: 9.8MIN: 8.13 / MAX: 9.78MIN: 8.28 / MAX: 10.42MIN: 7.61 / MAX: 402.49MIN: 7.82 / MAX: 241.19MIN: 8.14 / MAX: 251.77MIN: 7.74 / MAX: 314.841. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU - Model: squeezenet_ssd

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU - Model: squeezenet_ssdec3090 repbdfag3090i40804080 xxx4080 zzz4080 rep40904090 repRTX 3070 Tinv 4090307048121620SE +/- 0.02, N = 3SE +/- 0.02, N = 3SE +/- 0.02, N = 3SE +/- 0.26, N = 157.057.067.067.077.087.087.097.107.167.467.587.627.637.647.838.228.479.1115.82MIN: 6.95 / MAX: 8MIN: 7 / MAX: 8.03MIN: 7 / MAX: 7.82MIN: 7 / MAX: 8.07MIN: 6.97 / MAX: 7.99MIN: 6.98 / MAX: 8.07MIN: 6.98 / MAX: 7.95MIN: 6.99 / MAX: 8.59MIN: 7.05 / MAX: 13.55MIN: 6.9 / MAX: 8.9MIN: 6.98 / MAX: 9.05MIN: 7.01 / MAX: 9.28MIN: 7 / MAX: 9.17MIN: 7.05 / MAX: 9.12MIN: 7.21 / MAX: 9.32MIN: 7.56 / MAX: 9.8MIN: 6.29 / MAX: 533.92MIN: 6.35 / MAX: 130.38MIN: 6.99 / MAX: 82.571. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU - Model: yolov4-tiny

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU - Model: yolov4-tinyc3090 repadbe3090fg4080 xxx4080 rep40804080 zzz4090inv 4090RTX 3070 Ti4090 rep3070714212835SE +/- 0.02, N = 3SE +/- 0.01, N = 3SE +/- 0.04, N = 3SE +/- 0.18, N = 1512.8112.8312.8412.8512.8712.8712.8813.1713.6413.6513.6713.7913.8013.9715.1615.2615.5415.7228.59MIN: 12.73 / MAX: 13.08MIN: 12.74 / MAX: 13.59MIN: 12.69 / MAX: 15.33MIN: 12.72 / MAX: 13.93MIN: 12.76 / MAX: 13.73MIN: 12.68 / MAX: 13.84MIN: 12.76 / MAX: 13.67MIN: 13.03 / MAX: 14.1MIN: 13.04 / MAX: 76.32MIN: 12.71 / MAX: 14.99MIN: 12.71 / MAX: 14.88MIN: 12.75 / MAX: 19.63MIN: 12.76 / MAX: 15.76MIN: 13.11 / MAX: 16.15MIN: 12.86 / MAX: 248.64MIN: 12.87 / MAX: 132.82MIN: 12.15 / MAX: 492.01MIN: 13.2 / MAX: 301.81MIN: 12.87 / MAX: 325.371. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU - Model: resnet50

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU - Model: resnet50bca3090 repde3090fg40804080 rep4080 xxx4080 zzznv 4090RTX 3070 Tii4090 rep40903070612182430SE +/- 0.02, N = 3SE +/- 0.09, N = 3SE +/- 0.12, N = 3SE +/- 0.26, N = 1510.0010.0010.0110.0610.1010.1010.1010.2610.3410.8110.8410.9411.0712.4512.6012.9613.0814.1323.48MIN: 9.92 / MAX: 12.35MIN: 9.91 / MAX: 11.15MIN: 9.88 / MAX: 11.4MIN: 9.95 / MAX: 11.04MIN: 9.86 / MAX: 11.08MIN: 9.84 / MAX: 11.72MIN: 9.97 / MAX: 11.42MIN: 10.09 / MAX: 11.22MIN: 10.14 / MAX: 11.37MIN: 9.95 / MAX: 12.78MIN: 9.93 / MAX: 12.81MIN: 9.95 / MAX: 12.7MIN: 10.1 / MAX: 13.23MIN: 11.55 / MAX: 14.48MIN: 9.82 / MAX: 418.4MIN: 10.23 / MAX: 424.46MIN: 10.11 / MAX: 444.45MIN: 10.63 / MAX: 167.28MIN: 10.06 / MAX: 112.911. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU - Model: alexnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU - Model: alexnetade3090 repbc30904080f40904080 rep4080 xxx4080 zzzgnv 4090RTX 3070 Tii4090 rep30703691215SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.00, N = 3SE +/- 0.18, N = 154.314.314.314.314.334.334.354.624.644.644.654.684.684.875.205.256.536.7910.88MIN: 4.24 / MAX: 5.2MIN: 4.25 / MAX: 5.28MIN: 4.23 / MAX: 11.03MIN: 4.26 / MAX: 5.26MIN: 4.28 / MAX: 5.16MIN: 4.26 / MAX: 10.59MIN: 4.28 / MAX: 7.49MIN: 4.26 / MAX: 6.15MIN: 4.57 / MAX: 5.49MIN: 4.26 / MAX: 5.98MIN: 4.26 / MAX: 6.53MIN: 4.26 / MAX: 6.61MIN: 4.26 / MAX: 6.23MIN: 4.8 / MAX: 5.62MIN: 4.82 / MAX: 7.07MIN: 4.23 / MAX: 375.94MIN: 4.57 / MAX: 242.16MIN: 4.23 / MAX: 262.43MIN: 4.38 / MAX: 52.991. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU - Model: resnet18

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU - Model: resnet183090 repcebd3090af4080 zzz4080 rep4080 xxx40804090i4090 repgRTX 3070 Tinv 409030703691215SE +/- 0.01, N = 3SE +/- 0.03, N = 3SE +/- 0.01, N = 3SE +/- 0.20, N = 155.205.215.225.235.235.275.285.485.595.615.625.675.695.826.016.226.287.4412.68MIN: 5.09 / MAX: 5.98MIN: 5.11 / MAX: 6.04MIN: 5.09 / MAX: 11.15MIN: 5.13 / MAX: 6.18MIN: 5.08 / MAX: 6.28MIN: 5.15 / MAX: 6.19MIN: 5.17 / MAX: 6.16MIN: 5.33 / MAX: 6.16MIN: 5.06 / MAX: 6.95MIN: 5.11 / MAX: 7.44MIN: 5.1 / MAX: 7.65MIN: 5.18 / MAX: 7.22MIN: 5.16 / MAX: 8.22MIN: 5.28 / MAX: 7.02MIN: 5.44 / MAX: 8.18MIN: 6.11 / MAX: 7MIN: 4.94 / MAX: 298.06MIN: 5.29 / MAX: 320.54MIN: 5.39 / MAX: 262.621. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU - Model: vgg16

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU - Model: vgg163090 repac3090bdegf40804080 xxx4080 rep4080 zzz4090 repi4090RTX 3070 Tinv 409030701122334455SE +/- 0.04, N = 3SE +/- 0.10, N = 3SE +/- 0.14, N = 3SE +/- 0.24, N = 1523.4823.5123.5423.5523.5623.5623.6024.2024.5525.0025.0125.0425.8227.0427.8328.8229.0629.2948.29MIN: 23.24 / MAX: 29.21MIN: 23.29 / MAX: 24.68MIN: 23.33 / MAX: 24.61MIN: 23.3 / MAX: 24.45MIN: 23.34 / MAX: 24.72MIN: 23.24 / MAX: 24.78MIN: 23.17 / MAX: 24.71MIN: 23.56 / MAX: 58.31MIN: 23.62 / MAX: 97.69MIN: 23.93 / MAX: 26.69MIN: 23.8 / MAX: 26.41MIN: 24.06 / MAX: 27.35MIN: 24.35 / MAX: 62.94MIN: 24.22 / MAX: 296.13MIN: 24.98 / MAX: 262.23MIN: 24.35 / MAX: 214.1MIN: 24.11 / MAX: 541.55MIN: 24.63 / MAX: 296.95MIN: 24.97 / MAX: 183.121. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU - Model: googlenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU - Model: googlenetc3090 repbde3090af4080 rep4080 zzz40804080 xxxnv 4090igRTX 3070 Ti40904090 rep3070510152025SE +/- 0.04, N = 3SE +/- 0.01, N = 3SE +/- 0.02, N = 3SE +/- 0.22, N = 157.807.827.857.857.857.877.908.158.408.418.428.428.708.758.969.8710.6210.6519.49MIN: 7.72 / MAX: 8.74MIN: 7.69 / MAX: 8.61MIN: 7.76 / MAX: 8.76MIN: 7.71 / MAX: 8.85MIN: 7.71 / MAX: 8.76MIN: 7.76 / MAX: 10.36MIN: 7.74 / MAX: 9.54MIN: 8.02 / MAX: 9.02MIN: 7.77 / MAX: 9.78MIN: 7.72 / MAX: 9.9MIN: 7.79 / MAX: 10.01MIN: 7.73 / MAX: 10.06MIN: 7.96 / MAX: 10.01MIN: 8.08 / MAX: 16.01MIN: 8.82 / MAX: 9.87MIN: 7.33 / MAX: 399.24MIN: 7.83 / MAX: 323.31MIN: 8.29 / MAX: 236.11MIN: 7.4 / MAX: 200.011. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU - Model: blazeface

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU - Model: blazefacebc3090 repadef30904090i4090 repnv 4090g40804080 rep4080 xxx4080 zzzRTX 3070 Ti30700.89551.7912.68653.5824.4775SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.16, N = 151.371.371.371.381.381.381.381.391.391.401.401.401.411.411.411.421.421.603.98MIN: 1.35 / MAX: 1.75MIN: 1.35 / MAX: 1.82MIN: 1.36 / MAX: 1.46MIN: 1.34 / MAX: 1.85MIN: 1.34 / MAX: 2.25MIN: 1.34 / MAX: 1.88MIN: 1.35 / MAX: 2.08MIN: 1.37 / MAX: 1.82MIN: 1.33 / MAX: 1.94MIN: 1.33 / MAX: 2MIN: 1.33 / MAX: 1.93MIN: 1.34 / MAX: 1.87MIN: 1.38 / MAX: 2.09MIN: 1.35 / MAX: 2.01MIN: 1.35 / MAX: 1.9MIN: 1.36 / MAX: 1.93MIN: 1.36 / MAX: 2.01MIN: 1.11 / MAX: 436.01MIN: 1.31 / MAX: 228.41. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU - Model: efficientnet-b0

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU - Model: efficientnet-b0bceafg3090 repd309040804080 xxx4080 zzz4080 rep4090 repnv 40904090RTX 3070 Tii30703691215SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.02, N = 3SE +/- 0.18, N = 153.823.833.843.863.863.863.863.873.884.014.044.044.054.094.104.344.535.888.99MIN: 3.78 / MAX: 4.39MIN: 3.79 / MAX: 4.61MIN: 3.79 / MAX: 4.76MIN: 3.8 / MAX: 4.6MIN: 3.78 / MAX: 10.45MIN: 3.82 / MAX: 4.22MIN: 3.82 / MAX: 4.34MIN: 3.77 / MAX: 9.91MIN: 3.84 / MAX: 4.39MIN: 3.78 / MAX: 5.34MIN: 3.82 / MAX: 5.33MIN: 3.8 / MAX: 5.31MIN: 3.83 / MAX: 6.11MIN: 3.87 / MAX: 5.46MIN: 3.86 / MAX: 5.46MIN: 4.14 / MAX: 5.84MIN: 3.75 / MAX: 396.62MIN: 4.04 / MAX: 364.21MIN: 3.71 / MAX: 129.991. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU - Model: mnasnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU - Model: mnasnetbcedf3090 repag30904080 zzz40804080 xxx4080 rep4090 rep4090iRTX 3070 Tinv 409030701.14532.29063.43594.58125.7265SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.16, N = 152.952.962.962.972.972.972.982.982.993.063.073.073.083.153.183.203.404.705.09MIN: 2.92 / MAX: 3.42MIN: 2.93 / MAX: 3.41MIN: 2.91 / MAX: 5.9MIN: 2.92 / MAX: 3.34MIN: 2.93 / MAX: 3.66MIN: 2.94 / MAX: 3.28MIN: 2.92 / MAX: 4.03MIN: 2.94 / MAX: 3.65MIN: 2.96 / MAX: 3.14MIN: 2.92 / MAX: 3.73MIN: 2.93 / MAX: 4.63MIN: 2.95 / MAX: 4.19MIN: 2.94 / MAX: 3.67MIN: 3 / MAX: 4.54MIN: 3.05 / MAX: 4.64MIN: 3.07 / MAX: 3.86MIN: 2.72 / MAX: 432.18MIN: 3 / MAX: 188.08MIN: 2.86 / MAX: 53.751. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU - Model: shufflenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU - Model: shufflenet-v2cbeadg3090 rep3090f40804080 rep40904080 zzzi4090 rep4080 xxxnv 4090RTX 3070 Ti3070246810SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.20, N = 153.323.333.333.353.353.353.363.393.403.433.443.453.463.493.493.503.513.987.07MIN: 3.29 / MAX: 4.19MIN: 3.3 / MAX: 3.59MIN: 3.28 / MAX: 4.14MIN: 3.29 / MAX: 3.85MIN: 3.3 / MAX: 3.82MIN: 3.3 / MAX: 4.02MIN: 3.32 / MAX: 4.06MIN: 3.35 / MAX: 3.69MIN: 3.35 / MAX: 5.89MIN: 3.3 / MAX: 4.22MIN: 3.3 / MAX: 5.36MIN: 3.32 / MAX: 3.99MIN: 3.32 / MAX: 5.24MIN: 3.35 / MAX: 4.24MIN: 3.36 / MAX: 4.33MIN: 3.37 / MAX: 4.85MIN: 3.37 / MAX: 4MIN: 3.14 / MAX: 529.82MIN: 3.25 / MAX: 243.321. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2cfbeda3090 repg309040804080 xxxi4080 rep4080 zzz40904090 repnv 4090RTX 3070 Ti3070246810SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.14, N = 153.133.133.143.143.163.173.173.183.183.263.263.293.293.293.303.313.433.567.81MIN: 3.08 / MAX: 3.85MIN: 3.07 / MAX: 3.82MIN: 3.1 / MAX: 3.73MIN: 3.08 / MAX: 4.06MIN: 3.09 / MAX: 3.92MIN: 3.09 / MAX: 3.78MIN: 3.12 / MAX: 3.64MIN: 3.13 / MAX: 3.9MIN: 3.14 / MAX: 3.63MIN: 3.1 / MAX: 4.12MIN: 3.1 / MAX: 3.87MIN: 3.12 / MAX: 3.93MIN: 3.12 / MAX: 4.14MIN: 3.11 / MAX: 3.98MIN: 3.12 / MAX: 4.82MIN: 3.14 / MAX: 4.92MIN: 3.25 / MAX: 4.81MIN: 3.09 / MAX: 345.01MIN: 3.07 / MAX: 154.751. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU - Model: mobilenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU - Model: mobilenetdc3090 repbea3090gf4080 xxx4080nv 40904080 zzz4080 repRTX 3070 Ti4090 repi40903070510152025SE +/- 0.00, N = 3SE +/- 0.02, N = 3SE +/- 0.02, N = 3SE +/- 0.25, N = 158.028.038.038.048.048.058.078.178.278.378.448.458.478.579.3510.2310.4010.5521.11MIN: 7.95 / MAX: 9.81MIN: 7.98 / MAX: 8.84MIN: 7.96 / MAX: 8.77MIN: 7.95 / MAX: 14.33MIN: 7.95 / MAX: 9.09MIN: 7.95 / MAX: 8.89MIN: 7.99 / MAX: 8.8MIN: 8.08 / MAX: 9.37MIN: 8.17 / MAX: 9.04MIN: 7.97 / MAX: 16.09MIN: 7.98 / MAX: 10.55MIN: 8.03 / MAX: 12.61MIN: 8.04 / MAX: 10.17MIN: 7.98 / MAX: 10MIN: 7.49 / MAX: 474.12MIN: 8.13 / MAX: 386.42MIN: 7.97 / MAX: 455.46MIN: 8.22 / MAX: 303.1MIN: 7.98 / MAX: 322.431. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU - Model: FastestDet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU - Model: FastestDetgiaRTX 3070 Tibd3090 repc4080 zzz4080 xxx3090f4080 rep4080nv 40904090 rep40903070246810SE +/- 0.45, N = 3SE +/- 0.23, N = 15SE +/- 0.01, N = 32.572.663.623.944.054.084.084.114.164.174.214.224.344.424.515.275.488.65MIN: 2.53 / MAX: 3.21MIN: 2.54 / MAX: 3.41MIN: 2.7 / MAX: 4.54MIN: 2.43 / MAX: 267.02MIN: 4.02 / MAX: 4.35MIN: 4.02 / MAX: 4.28MIN: 4.05 / MAX: 4.84MIN: 4.08 / MAX: 4.4MIN: 4 / MAX: 4.69MIN: 4.05 / MAX: 4.74MIN: 4.19 / MAX: 4.41MIN: 4.18 / MAX: 4.97MIN: 4.19 / MAX: 5.77MIN: 4.25 / MAX: 6.71MIN: 4.34 / MAX: 5.96MIN: 4.05 / MAX: 247.02MIN: 2.67 / MAX: 259.34MIN: 3.94 / MAX: 185.211. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU - Model: blazeface

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU - Model: blazeface4090nv 4090fabdg30903090 repci40804080 zzz4080 rep4080 xxx4090 repRTX 3070 Ti30700.67051.3412.01152.6823.3525SE +/- 0.00, N = 3SE +/- 0.01, N = 3SE +/- 0.14, N = 151.271.331.371.381.381.381.381.381.381.391.401.401.401.421.421.451.602.98MIN: 1.21 / MAX: 1.95MIN: 1.27 / MAX: 1.77MIN: 1.34 / MAX: 2.11MIN: 1.35 / MAX: 2.06MIN: 1.35 / MAX: 1.67MIN: 1.35 / MAX: 2.05MIN: 1.36 / MAX: 1.62MIN: 1.35 / MAX: 2.23MIN: 1.36 / MAX: 1.71MIN: 1.36 / MAX: 1.53MIN: 1.34 / MAX: 2MIN: 1.34 / MAX: 2.15MIN: 1.34 / MAX: 2.1MIN: 1.35 / MAX: 1.88MIN: 1.36 / MAX: 2.02MIN: 1.38 / MAX: 2.96MIN: 0.95 / MAX: 433.24MIN: 1.29 / MAX: 144.961. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v3-v3 - Model: mobilenet-v3

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v3-v3 - Model: mobilenet-v3nv 40904090fg3090 repbc4080 zzzi40804080 rep4080 xxxRTX 3070 Ti4090 rep3070246810SE +/- 0.18, N = 152.613.123.153.153.153.163.173.203.263.273.273.333.654.908.06MIN: 2.5 / MAX: 3.12MIN: 2.99 / MAX: 5.09MIN: 3.1 / MAX: 3.8MIN: 3.1 / MAX: 3.87MIN: 3.11 / MAX: 3.83MIN: 3.11 / MAX: 3.75MIN: 3.11 / MAX: 8.89MIN: 3.06 / MAX: 3.84MIN: 3.12 / MAX: 4.19MIN: 3.12 / MAX: 5.24MIN: 3.14 / MAX: 3.99MIN: 3.19 / MAX: 4.2MIN: 2.87 / MAX: 347.75MIN: 3.17 / MAX: 120.84MIN: 2.96 / MAX: 219.871. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3 - Model: mobilenet-v3

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3 - Model: mobilenet-v3f30903090 repbcg40904080 zzz4080i4080 xxxnv 40904090 repRTX 3070 Ti30701.21052.4213.63154.8426.0525SE +/- 0.22, N = 143.153.153.153.163.163.163.253.273.283.293.313.353.533.765.38MIN: 3.11 / MAX: 3.48MIN: 3.11 / MAX: 3.71MIN: 3.11 / MAX: 3.6MIN: 3.12 / MAX: 3.69MIN: 3.12 / MAX: 3.7MIN: 3.11 / MAX: 3.93MIN: 3.11 / MAX: 4.74MIN: 3.14 / MAX: 4.63MIN: 3.14 / MAX: 3.89MIN: 3.15 / MAX: 4.32MIN: 3.16 / MAX: 5.3MIN: 3.21 / MAX: 5.23MIN: 3.2 / MAX: 40.81MIN: 2.89 / MAX: 366.04MIN: 2.74 / MAX: 121.291. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU - Model: vision_transformer

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU - Model: vision_transformerc3090 repbdag3090f4080 zzz4080 xxx40804080 rep4090 repiRTX 3070 Ti4090nv 4090307020406080100SE +/- 0.39, N = 3SE +/- 0.29, N = 3SE +/- 0.12, N = 1531.7731.8031.9532.4332.4932.7333.0133.5634.1034.2735.0735.2837.0537.8037.9138.2538.9075.34MIN: 31.61 / MAX: 35.68MIN: 31.66 / MAX: 32.23MIN: 31.79 / MAX: 32.33MIN: 31.56 / MAX: 37.69MIN: 31.67 / MAX: 40.11MIN: 31.44 / MAX: 81.32MIN: 32.88 / MAX: 33.42MIN: 32.98 / MAX: 51.93MIN: 32.65 / MAX: 37.64MIN: 32.82 / MAX: 39.79MIN: 33.14 / MAX: 43.26MIN: 33.9 / MAX: 38.67MIN: 33.89 / MAX: 407.84MIN: 33.74 / MAX: 321.51MIN: 32.08 / MAX: 541.11MIN: 33.04 / MAX: 447.7MIN: 34.2 / MAX: 300.84MIN: 38.72 / MAX: 418.011. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU - Model: regnety_400m

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU - Model: regnety_400mf4090abd40803090 rep3090cg4080 zzz4080 xxx4080 rep4090 repRTX 3070 Tiinv 4090307048121620SE +/- 0.04, N = 3SE +/- 0.06, N = 3SE +/- 0.19, N = 158.088.138.188.188.238.248.248.258.278.308.378.458.678.788.839.9410.1718.00MIN: 7.98 / MAX: 10.87MIN: 7.78 / MAX: 9.98MIN: 8.07 / MAX: 9.68MIN: 8.12 / MAX: 8.86MIN: 8.03 / MAX: 8.9MIN: 7.89 / MAX: 9.52MIN: 8.17 / MAX: 8.84MIN: 8.17 / MAX: 8.9MIN: 8.22 / MAX: 9.18MIN: 8.22 / MAX: 9.1MIN: 8.05 / MAX: 10.19MIN: 8.12 / MAX: 9.68MIN: 8.22 / MAX: 15.29MIN: 8.45 / MAX: 10.05MIN: 7.65 / MAX: 351.08MIN: 7.43 / MAX: 166.02MIN: 8.12 / MAX: 209.53MIN: 7.91 / MAX: 176.281. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU - Model: squeezenet_ssd

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU - Model: squeezenet_ssdbadc3090 repgf4090 rep30904080 zzz4080 xxx40804080 rep4090RTX 3070 Tiinv 409030703691215SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.24, N = 157.067.077.097.107.127.137.237.317.527.557.627.737.867.868.138.969.1113.20MIN: 7.01 / MAX: 7.55MIN: 7.01 / MAX: 8.07MIN: 6.99 / MAX: 9.39MIN: 7.05 / MAX: 7.65MIN: 7.05 / MAX: 7.63MIN: 7.04 / MAX: 8.43MIN: 7.15 / MAX: 8.02MIN: 6.71 / MAX: 9.3MIN: 7.45 / MAX: 7.74MIN: 7 / MAX: 8.72MIN: 7.01 / MAX: 8.84MIN: 7.13 / MAX: 9.7MIN: 7.22 / MAX: 10.84MIN: 7.25 / MAX: 8.98MIN: 6.37 / MAX: 399.11MIN: 6.92 / MAX: 244.02MIN: 6.77 / MAX: 101.58MIN: 6.9 / MAX: 68.611. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU - Model: yolov4-tiny

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU - Model: yolov4-tinyb3090 repcadf4080 xxx4080 zzz409040804080 rep3090iRTX 3070 Ti4090 repnv 4090g3070714212835SE +/- 0.11, N = 3SE +/- 0.05, N = 3SE +/- 0.18, N = 1512.7412.7712.8112.9012.9513.3213.6013.6313.6813.8514.0314.2614.6515.2015.3815.4017.2327.66MIN: 12.66 / MAX: 13.28MIN: 12.7 / MAX: 13.02MIN: 12.74 / MAX: 13.2MIN: 12.69 / MAX: 15.88MIN: 12.75 / MAX: 18.88MIN: 12.95 / MAX: 35.49MIN: 12.8 / MAX: 16.23MIN: 12.77 / MAX: 15.36MIN: 12.83 / MAX: 14.63MIN: 12.84 / MAX: 16.75MIN: 13.15 / MAX: 15.97MIN: 14.17 / MAX: 14.53MIN: 12.44 / MAX: 202.68MIN: 12.69 / MAX: 431.37MIN: 12.32 / MAX: 188.07MIN: 12.35 / MAX: 321.43MIN: 12.99 / MAX: 196.66MIN: 12.74 / MAX: 294.91. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU - Model: resnet50

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU - Model: resnet503090 repdbca3090g4080 xxxf4080 zzz4080nv 40904080 repiRTX 3070 Ti4090 rep40903070510152025SE +/- 0.01, N = 3SE +/- 0.23, N = 3SE +/- 0.26, N = 159.9510.0010.0110.1110.2010.3010.7210.8211.0511.1011.1611.4111.7612.0912.7312.7314.1021.50MIN: 9.85 / MAX: 10.72MIN: 9.86 / MAX: 11.02MIN: 9.85 / MAX: 11.06MIN: 9.95 / MAX: 16.18MIN: 9.84 / MAX: 12.48MIN: 9.82 / MAX: 17.56MIN: 10.1 / MAX: 108.3MIN: 9.9 / MAX: 12.26MIN: 10.14 / MAX: 162.88MIN: 10.2 / MAX: 13.06MIN: 10.29 / MAX: 15.03MIN: 10.57 / MAX: 12.22MIN: 10.68 / MAX: 44.94MIN: 11.16 / MAX: 13.48MIN: 10.18 / MAX: 541.92MIN: 10.22 / MAX: 181.72MIN: 10.27 / MAX: 287MIN: 10.24 / MAX: 116.851. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU - Model: alexnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU - Model: alexnetd3090c3090 repbgfa4080 rep4080 xxx4080 zzz408040904090 repnv 4090iRTX 3070 Ti30703691215SE +/- 0.01, N = 3SE +/- 0.11, N = 3SE +/- 0.21, N = 144.304.304.314.314.324.324.364.414.674.684.694.755.145.145.185.305.499.62MIN: 4.23 / MAX: 5.32MIN: 4.25 / MAX: 4.83MIN: 4.26 / MAX: 4.98MIN: 4.26 / MAX: 5.18MIN: 4.26 / MAX: 5.15MIN: 4.25 / MAX: 5.17MIN: 4.29 / MAX: 5.7MIN: 4.24 / MAX: 5.16MIN: 4.27 / MAX: 5.88MIN: 4.28 / MAX: 6.37MIN: 4.29 / MAX: 5.78MIN: 4.31 / MAX: 13.88MIN: 4.73 / MAX: 6.32MIN: 4.76 / MAX: 6.26MIN: 4.75 / MAX: 7.12MIN: 4.92 / MAX: 7.18MIN: 4.26 / MAX: 363.39MIN: 4.31 / MAX: 147.61. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU - Model: resnet18

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU - Model: resnet18b3090dca3090 repg4080 xxxi4080 zzz4080 repf40804090RTX 3070 Ti4090 repnv 4090307048121620SE +/- 0.02, N = 3SE +/- 0.07, N = 3SE +/- 0.17, N = 155.205.215.235.245.295.295.555.565.605.635.685.695.696.006.086.777.8214.03MIN: 5.1 / MAX: 5.9MIN: 5.09 / MAX: 6.04MIN: 5.1 / MAX: 6.28MIN: 5.15 / MAX: 6.09MIN: 5.09 / MAX: 6.29MIN: 5.18 / MAX: 6.19MIN: 5.19 / MAX: 25.4MIN: 5.09 / MAX: 6.84MIN: 5.13 / MAX: 6.83MIN: 5.08 / MAX: 7.55MIN: 5.17 / MAX: 7.45MIN: 5.22 / MAX: 92.59MIN: 5.16 / MAX: 7.68MIN: 5.47 / MAX: 7.29MIN: 4.97 / MAX: 245.95MIN: 6.16 / MAX: 8.42MIN: 5.54 / MAX: 303.05MIN: 5 / MAX: 303.381. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU - Model: vgg16

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU - Model: vgg1630903090 repcbdagf4080 xxx40804080 zzz4080 rep40904090 repRTX 3070 Tinv 4090i30701326395265SE +/- 0.05, N = 3SE +/- 0.30, N = 3SE +/- 0.23, N = 1523.4323.4323.4523.4923.5123.7523.7824.1925.0325.3725.4026.1127.7528.1928.3629.5430.9656.64MIN: 23.2 / MAX: 24.1MIN: 23.23 / MAX: 24.39MIN: 23.26 / MAX: 24.51MIN: 23.36 / MAX: 24.62MIN: 23.19 / MAX: 24.68MIN: 23.31 / MAX: 25.12MIN: 23.52 / MAX: 24.89MIN: 23.99 / MAX: 30.98MIN: 23.85 / MAX: 28.9MIN: 24.26 / MAX: 36.52MIN: 24.09 / MAX: 32.86MIN: 24.54 / MAX: 30.29MIN: 24.58 / MAX: 282.59MIN: 24.69 / MAX: 205.72MIN: 24.13 / MAX: 449.57MIN: 24.77 / MAX: 364.86MIN: 25.92 / MAX: 328.63MIN: 25.75 / MAX: 367.741. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU - Model: googlenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU - Model: googlenetbd30903090 repfcag4080 xxx4080 zzz40804080 repnv 40904090 repRTX 3070 Ti4090i307048121620SE +/- 0.01, N = 3SE +/- 0.11, N = 3SE +/- 0.22, N = 157.827.857.867.907.927.937.947.988.388.408.428.528.939.539.6910.2710.3018.25MIN: 7.73 / MAX: 8.65MIN: 7.71 / MAX: 8.83MIN: 7.76 / MAX: 8.74MIN: 7.79 / MAX: 8.74MIN: 7.8 / MAX: 8.96MIN: 7.82 / MAX: 8.91MIN: 7.71 / MAX: 8.73MIN: 7.86 / MAX: 8.78MIN: 7.72 / MAX: 10.05MIN: 7.72 / MAX: 10.5MIN: 7.75 / MAX: 9.96MIN: 7.84 / MAX: 10.21MIN: 8.27 / MAX: 10.68MIN: 8.86 / MAX: 11.44MIN: 7.29 / MAX: 407.61MIN: 7.95 / MAX: 115.68MIN: 8.19 / MAX: 349.57MIN: 7.5 / MAX: 267.891. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU - Model: efficientnet-b0

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU - Model: efficientnet-b0bd3090 repf3090cag40804080 zzz4090 rep4080 xxxi4080 rep4090nv 4090RTX 3070 Ti30703691215SE +/- 0.00, N = 3SE +/- 0.05, N = 3SE +/- 0.19, N = 153.853.853.863.873.873.883.903.913.993.994.034.044.054.094.234.374.729.23MIN: 3.81 / MAX: 4.42MIN: 3.81 / MAX: 4.46MIN: 3.81 / MAX: 4.75MIN: 3.81 / MAX: 4.97MIN: 3.83 / MAX: 4.69MIN: 3.84 / MAX: 4.41MIN: 3.82 / MAX: 4.51MIN: 3.85 / MAX: 4.64MIN: 3.79 / MAX: 5.83MIN: 3.8 / MAX: 5.69MIN: 3.86 / MAX: 4.82MIN: 3.83 / MAX: 5.71MIN: 3.78 / MAX: 5.45MIN: 3.86 / MAX: 5.59MIN: 3.98 / MAX: 12.23MIN: 4.15 / MAX: 5.96MIN: 3.37 / MAX: 486.93MIN: 3.43 / MAX: 156.191. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU - Model: mnasnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU - Model: mnasnetiabf3090 repdc3090g4080 zzz40804080 rep4080 xxx40904090 repRTX 3070 Tinv 40903070246810SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.14, N = 152.742.972.972.972.972.982.992.993.003.043.053.063.063.123.183.264.776.88MIN: 2.62 / MAX: 4.22MIN: 2.92 / MAX: 3.48MIN: 2.93 / MAX: 3.45MIN: 2.93 / MAX: 3.95MIN: 2.93 / MAX: 3.28MIN: 2.94 / MAX: 3.83MIN: 2.96 / MAX: 3.44MIN: 2.95 / MAX: 3.88MIN: 2.96 / MAX: 3.68MIN: 2.91 / MAX: 4.47MIN: 2.92 / MAX: 3.82MIN: 2.94 / MAX: 4.51MIN: 2.94 / MAX: 4.45MIN: 2.98 / MAX: 3.79MIN: 3.05 / MAX: 3.8MIN: 2.46 / MAX: 277.54MIN: 3.07 / MAX: 97.57MIN: 3.05 / MAX: 110.251. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU - Model: shufflenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU - Model: shufflenet-v2ab3090cd3090 rep40804080 zzz4080 rep4080 xxxnv 40904090 repf4090gRTX 3070 Tii3070246810SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.19, N = 153.343.343.343.353.353.353.413.423.433.453.453.473.553.553.593.775.036.82MIN: 3.3 / MAX: 3.85MIN: 3.31 / MAX: 3.77MIN: 3.3 / MAX: 4.19MIN: 3.31 / MAX: 3.8MIN: 3.3 / MAX: 3.82MIN: 3.31 / MAX: 3.68MIN: 3.28 / MAX: 4.87MIN: 3.28 / MAX: 4.19MIN: 3.3 / MAX: 4.15MIN: 3.32 / MAX: 3.85MIN: 3.32 / MAX: 4.91MIN: 3.33 / MAX: 4.93MIN: 3.27 / MAX: 22.86MIN: 3.39 / MAX: 5.48MIN: 3.3 / MAX: 25.28MIN: 3.02 / MAX: 511.95MIN: 3.07 / MAX: 228.55MIN: 3.16 / MAX: 64.721. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v2-v2 - Model: mobilenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v2-v2 - Model: mobilenet-v2fabgd30903090 repc4080 zzz40804080 rep4080 xxx40904090 repinv 4090RTX 3070 Ti30703691215SE +/- 0.00, N = 3SE +/- 0.02, N = 3SE +/- 0.17, N = 153.153.163.163.163.173.173.173.183.253.283.283.283.303.303.523.603.769.67MIN: 3.1 / MAX: 3.65MIN: 3.1 / MAX: 3.8MIN: 3.11 / MAX: 3.61MIN: 3.11 / MAX: 3.83MIN: 3.1 / MAX: 8.86MIN: 3.12 / MAX: 4.05MIN: 3.11 / MAX: 4.94MIN: 3.13 / MAX: 3.84MIN: 3.09 / MAX: 4.51MIN: 3.11 / MAX: 3.88MIN: 3.11 / MAX: 4MIN: 3.1 / MAX: 4.05MIN: 3.11 / MAX: 4.81MIN: 3.13 / MAX: 3.97MIN: 3.29 / MAX: 19.18MIN: 3.43 / MAX: 4.62MIN: 2.6 / MAX: 364.73MIN: 3.19 / MAX: 225.841. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU - Model: mobilenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU - Model: mobilenetb3090 repcadnv 4090i4080 xxx4080 zzz4080 rep4090 repf30904080RTX 3070 Ti40903070g510152025SE +/- 0.03, N = 3SE +/- 0.06, N = 3SE +/- 0.22, N = 157.978.018.028.058.108.158.378.378.388.418.438.458.608.739.4310.0817.8122.74MIN: 7.94 / MAX: 8.26MIN: 7.96 / MAX: 9.85MIN: 7.98 / MAX: 8.33MIN: 7.97 / MAX: 9.07MIN: 7.94 / MAX: 14.4MIN: 7.73 / MAX: 9.34MIN: 8.15 / MAX: 9.75MIN: 7.96 / MAX: 9.72MIN: 7.94 / MAX: 10.16MIN: 8.14 / MAX: 11.03MIN: 8.04 / MAX: 18.04MIN: 8.37 / MAX: 9.44MIN: 8.5 / MAX: 13.72MIN: 8.15 / MAX: 10.96MIN: 7.95 / MAX: 398.1MIN: 8.1 / MAX: 118.32MIN: 8.05 / MAX: 159.41MIN: 8.24 / MAX: 1264.671. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3 - Model: FastestDet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3 - Model: FastestDet4090fg4080 zzz3090nv 4090bc3090 rep4090 rep4080 rep4080 xxx4080RTX 3070 Tii30703691215SE +/- 0.27, N = 142.933.853.974.044.044.064.074.084.114.164.184.194.204.335.699.18MIN: 2.84 / MAX: 3.38MIN: 3.8 / MAX: 4.65MIN: 3.92 / MAX: 4.75MIN: 3.89 / MAX: 5.01MIN: 4.01 / MAX: 4.15MIN: 3.91 / MAX: 5.78MIN: 4.03 / MAX: 5.83MIN: 4.05 / MAX: 4.36MIN: 4.07 / MAX: 4.21MIN: 4 / MAX: 5.58MIN: 4.03 / MAX: 5.07MIN: 4.04 / MAX: 5.47MIN: 4.06 / MAX: 4.86MIN: 2.59 / MAX: 433.58MIN: 3.69 / MAX: 261.71MIN: 3.64 / MAX: 122.651. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3 - Model: vision_transformer

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3 - Model: vision_transformerbc30903090 repgf40804080 rep4080 xxx4080 zzziRTX 3070 Ti4090nv 40904090 rep307020406080100SE +/- 0.18, N = 1531.6531.7831.8931.9333.3933.4734.2034.2734.3734.4736.5537.8638.7938.9939.1281.77MIN: 31.53 / MAX: 32.23MIN: 31.64 / MAX: 34.51MIN: 31.66 / MAX: 39.97MIN: 31.76 / MAX: 33.09MIN: 32.73 / MAX: 88.83MIN: 32.89 / MAX: 74.09MIN: 32.92 / MAX: 36.19MIN: 33.07 / MAX: 37.01MIN: 33.01 / MAX: 38.7MIN: 33.32 / MAX: 37.42MIN: 33 / MAX: 209.38MIN: 32.9 / MAX: 463.9MIN: 33.95 / MAX: 457.41MIN: 34.17 / MAX: 473.06MIN: 33.92 / MAX: 465.83MIN: 44.4 / MAX: 460.281. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3 - Model: regnety_400m

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3 - Model: regnety_400m3090bg3090 rep4090ci4080f4080 zzz4080 rep4080 xxxRTX 3070 Tinv 40904090 rep3070510152025SE +/- 0.21, N = 157.958.058.078.098.138.148.218.338.348.478.578.759.029.5517.1519.66MIN: 7.88 / MAX: 8.67MIN: 8 / MAX: 8.58MIN: 7.97 / MAX: 8.81MIN: 7.99 / MAX: 14.25MIN: 7.75 / MAX: 10.05MIN: 8.08 / MAX: 8.69MIN: 7.9 / MAX: 9.99MIN: 8.02 / MAX: 9.64MIN: 8.26 / MAX: 9.3MIN: 8.13 / MAX: 10.27MIN: 8.21 / MAX: 10.39MIN: 8.35 / MAX: 10.08MIN: 7.69 / MAX: 501.76MIN: 7.5 / MAX: 193.79MIN: 8.02 / MAX: 773.45MIN: 7.5 / MAX: 235.361. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3 - Model: squeezenet_ssd

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3 - Model: squeezenet_ssdfbc30903090 repg4080 zzz4080 rep40804080 xxxiRTX 3070 Ti4090 rep4090nv 4090307048121620SE +/- 0.24, N = 156.977.037.047.047.097.267.357.597.647.648.168.399.309.329.3717.75MIN: 6.83 / MAX: 13.87MIN: 6.97 / MAX: 7.88MIN: 6.96 / MAX: 7.83MIN: 6.96 / MAX: 7.74MIN: 7.02 / MAX: 7.99MIN: 7.14 / MAX: 8.59MIN: 6.79 / MAX: 9.82MIN: 7.02 / MAX: 8.87MIN: 7.05 / MAX: 9.9MIN: 7.03 / MAX: 9.19MIN: 7.51 / MAX: 9.94MIN: 6.53 / MAX: 436.05MIN: 6.92 / MAX: 310.91MIN: 7.1 / MAX: 172.56MIN: 7.07 / MAX: 281.92MIN: 6.47 / MAX: 272.111. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3 - Model: yolov4-tiny

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3 - Model: yolov4-tiny3090 rep3090cbfg4080 rep4080 xxx40804080 zzzi40904090 repRTX 3070 Tinv 40903070714212835SE +/- 0.14, N = 1512.8212.8712.8912.9813.0713.0813.5513.6913.8113.8315.1115.3015.3415.4215.6229.34MIN: 12.72 / MAX: 13.48MIN: 12.75 / MAX: 13.58MIN: 12.84 / MAX: 13.19MIN: 12.73 / MAX: 35.55MIN: 12.95 / MAX: 14.55MIN: 12.96 / MAX: 13.83MIN: 12.75 / MAX: 14.74MIN: 12.73 / MAX: 15.68MIN: 12.84 / MAX: 15.1MIN: 12.89 / MAX: 15.4MIN: 12.93 / MAX: 151.45MIN: 12.87 / MAX: 144.73MIN: 12.94 / MAX: 157.95MIN: 12.21 / MAX: 414.81MIN: 12.99 / MAX: 184MIN: 12.17 / MAX: 245.341. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3 - Model: resnet50

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3 - Model: resnet50b30903090 repc4080 rep4080 xxxf40804080 zzzg4090RTX 3070 Tinv 40904090 repi3070612182430SE +/- 0.22, N = 1510.0110.0510.0710.3310.7910.9111.0511.1111.2111.2511.3912.1113.6813.8214.0524.07MIN: 9.89 / MAX: 10.86MIN: 9.85 / MAX: 12.64MIN: 9.94 / MAX: 11.06MIN: 10.16 / MAX: 13.97MIN: 9.91 / MAX: 12.75MIN: 9.91 / MAX: 13.1MIN: 10.46 / MAX: 112.6MIN: 10.19 / MAX: 13.03MIN: 10.3 / MAX: 13.25MIN: 10.55 / MAX: 118.12MIN: 10.48 / MAX: 13.29MIN: 10.16 / MAX: 382.56MIN: 10.25 / MAX: 566.67MIN: 10.34 / MAX: 245.6MIN: 11.69 / MAX: 252.21MIN: 10.02 / MAX: 218.351. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3 - Model: alexnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3 - Model: alexnetcb3090 rep30904080 rep4080 xxx40804080 zzznv 4090gf4090i4090 repRTX 3070 Ti30703691215SE +/- 0.23, N = 154.284.294.304.314.654.654.664.674.674.714.834.945.015.275.5511.00MIN: 4.24 / MAX: 5.12MIN: 4.24 / MAX: 5.64MIN: 4.24 / MAX: 4.99MIN: 4.25 / MAX: 5.13MIN: 4.26 / MAX: 6.13MIN: 4.28 / MAX: 6.42MIN: 4.29 / MAX: 6.1MIN: 4.28 / MAX: 6.29MIN: 4.28 / MAX: 5.7MIN: 4.65 / MAX: 5.57MIN: 4.76 / MAX: 5.74MIN: 4.51 / MAX: 6.64MIN: 4.6 / MAX: 6.68MIN: 4.78 / MAX: 7.7MIN: 4.2 / MAX: 281.58MIN: 4.33 / MAX: 199.921. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3 - Model: resnet18

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3 - Model: resnet1830903090 repbcg4080 rep4080 xxx40804080 zzz4090ifRTX 3070 Tinv 40904090 rep30703691215SE +/- 0.19, N = 155.195.205.215.265.485.635.665.705.715.815.866.136.187.617.7511.14MIN: 5.09 / MAX: 6.13MIN: 5.1 / MAX: 5.97MIN: 5.12 / MAX: 6.22MIN: 5.18 / MAX: 6.27MIN: 5.37 / MAX: 6.51MIN: 5.09 / MAX: 7.75MIN: 5.14 / MAX: 7.49MIN: 5.15 / MAX: 7.9MIN: 5.12 / MAX: 8.19MIN: 5.27 / MAX: 7.16MIN: 5.35 / MAX: 7.79MIN: 5.41 / MAX: 151.51MIN: 5.17 / MAX: 262.79MIN: 5.23 / MAX: 90.18MIN: 5.57 / MAX: 125.43MIN: 4.79 / MAX: 65.121. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3 - Model: vgg16

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3 - Model: vgg16b30903090 repcf4080 repg4080 xxx40804080 zzznv 40904090 repRTX 3070 Ti4090i30701122334455SE +/- 0.24, N = 1523.5023.5023.5423.9924.4524.9124.9225.0025.1025.4527.0427.2528.4028.5529.1249.75MIN: 23.3 / MAX: 24.41MIN: 23.17 / MAX: 24.44MIN: 23.33 / MAX: 24.41MIN: 23.72 / MAX: 24.98MIN: 24.26 / MAX: 25.26MIN: 23.8 / MAX: 26.87MIN: 24.58 / MAX: 31.89MIN: 23.91 / MAX: 27.99MIN: 24.12 / MAX: 27.57MIN: 24.22 / MAX: 27.73MIN: 24.33 / MAX: 215.56MIN: 24.14 / MAX: 379.93MIN: 24.12 / MAX: 509.06MIN: 24.05 / MAX: 201.8MIN: 26.33 / MAX: 310.23MIN: 25.45 / MAX: 273.861. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3 - Model: googlenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3 - Model: googlenet3090b3090 repcf40804080 xxx4080 rep4080 zzznv 4090g4090 repRTX 3070 Ti4090i307048121620SE +/- 0.24, N = 157.837.847.857.888.078.458.508.528.559.029.159.299.659.9710.1717.00MIN: 7.71 / MAX: 8.8MIN: 7.74 / MAX: 8.7MIN: 7.75 / MAX: 8.69MIN: 7.79 / MAX: 8.78MIN: 7.92 / MAX: 8.86MIN: 7.79 / MAX: 10.32MIN: 7.79 / MAX: 9.94MIN: 7.81 / MAX: 10.78MIN: 7.85 / MAX: 10.35MIN: 8.41 / MAX: 11.08MIN: 7.84 / MAX: 198.46MIN: 7.98 / MAX: 83.03MIN: 7.59 / MAX: 472.81MIN: 7.67 / MAX: 258.52MIN: 7.94 / MAX: 150.01MIN: 7.35 / MAX: 277.791. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3 - Model: blazeface

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3 - Model: blazefacenv 40904090i4090 rep3090bg3090 repc4080 zzz4080 rep4080 xxxf4080RTX 3070 Ti30700.68181.36362.04542.72723.409SE +/- 0.19, N = 151.161.171.281.341.361.371.371.371.381.411.421.421.431.441.793.03MIN: 1.11 / MAX: 1.67MIN: 1.11 / MAX: 1.9MIN: 1.23 / MAX: 1.73MIN: 1.27 / MAX: 1.95MIN: 1.34 / MAX: 1.46MIN: 1.35 / MAX: 1.52MIN: 1.34 / MAX: 2.07MIN: 1.35 / MAX: 1.46MIN: 1.36 / MAX: 1.58MIN: 1.34 / MAX: 1.91MIN: 1.36 / MAX: 2.2MIN: 1.36 / MAX: 1.92MIN: 1.4 / MAX: 1.77MIN: 1.37 / MAX: 3.45MIN: 1.13 / MAX: 312.12MIN: 1.28 / MAX: 96.941. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3 - Model: efficientnet-b0

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3 - Model: efficientnet-b0b30903090 repc40804080 rep4080 zzzfnv 40904080 xxx4090g4090 repiRTX 3070 Ti30703691215SE +/- 0.22, N = 153.823.833.853.894.024.024.034.044.044.064.094.144.344.684.789.01MIN: 3.79 / MAX: 4.34MIN: 3.78 / MAX: 4.41MIN: 3.81 / MAX: 4.53MIN: 3.83 / MAX: 9.72MIN: 3.82 / MAX: 5.66MIN: 3.82 / MAX: 5.39MIN: 3.82 / MAX: 5.43MIN: 3.99 / MAX: 4.82MIN: 3.78 / MAX: 4.9MIN: 3.83 / MAX: 5.55MIN: 3.86 / MAX: 4.83MIN: 4.09 / MAX: 5.13MIN: 4.16 / MAX: 5.28MIN: 4.48 / MAX: 6.02MIN: 3.82 / MAX: 411.19MIN: 3.98 / MAX: 188.571. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3 - Model: mnasnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3 - Model: mnasnet30903090 repbcg4080 zzz4080 xxx40804080 repRTX 3070 Tif40904090 repinv 40903070246810SE +/- 0.04, N = 152.952.962.972.973.053.063.073.083.093.113.123.193.233.394.616.87MIN: 2.92 / MAX: 3.29MIN: 2.94 / MAX: 3.38MIN: 2.94 / MAX: 3.43MIN: 2.94 / MAX: 3.43MIN: 3.01 / MAX: 3.88MIN: 2.93 / MAX: 3.64MIN: 2.94 / MAX: 3.6MIN: 2.94 / MAX: 4.52MIN: 2.95 / MAX: 4.52MIN: 2.8 / MAX: 4.98MIN: 3.08 / MAX: 3.86MIN: 3.06 / MAX: 3.75MIN: 3.1 / MAX: 3.75MIN: 3.26 / MAX: 4.86MIN: 2.78 / MAX: 222.99MIN: 2.93 / MAX: 216.411. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3 - Model: shufflenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3 - Model: shufflenet-v23090b3090 repcgf4080 rep4080 zzz4080nv 40904080 xxxi4090 repRTX 3070 Ti40903070246810SE +/- 0.21, N = 153.323.333.333.343.383.403.433.433.463.463.473.523.594.095.188.13MIN: 3.28 / MAX: 3.66MIN: 3.3 / MAX: 3.79MIN: 3.3 / MAX: 3.67MIN: 3.32 / MAX: 3.79MIN: 3.34 / MAX: 4.15MIN: 3.35 / MAX: 4.17MIN: 3.3 / MAX: 4.03MIN: 3.31 / MAX: 3.94MIN: 3.34 / MAX: 3.93MIN: 3.32 / MAX: 5.2MIN: 3.33 / MAX: 5.01MIN: 3.39 / MAX: 4.05MIN: 3.46 / MAX: 4.09MIN: 3.12 / MAX: 435.28MIN: 3.34 / MAX: 283.54MIN: 3.09 / MAX: 147.211. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v2-v2 - Model: mobilenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v2-v2 - Model: mobilenet-v2c3090bgf3090 rep4080 rep4080 zzzi4080 xxx4080nv 4090RTX 3070 Ti4090 rep409030703691215SE +/- 0.10, N = 153.143.143.153.153.163.173.273.283.303.303.313.393.413.453.469.19MIN: 3.1 / MAX: 3.67MIN: 3.08 / MAX: 3.7MIN: 3.1 / MAX: 3.68MIN: 3.1 / MAX: 3.63MIN: 3.09 / MAX: 3.89MIN: 3.11 / MAX: 4.5MIN: 3.1 / MAX: 4.34MIN: 3.1 / MAX: 4MIN: 3.14 / MAX: 4.82MIN: 3.12 / MAX: 4.03MIN: 3.12 / MAX: 4.76MIN: 3.21 / MAX: 4.24MIN: 2.99 / MAX: 184.91MIN: 3.23 / MAX: 4.55MIN: 3.29 / MAX: 4.38MIN: 3.04 / MAX: 232.121. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3 - Model: mobilenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3 - Model: mobilenetcb3090 rep30904080 zzz40804080 xxx4080 repf4090 repnv 40904090gRTX 3070 Tii307048121620SE +/- 0.23, N = 158.008.018.038.068.408.438.448.488.568.748.918.968.989.6210.0817.82MIN: 7.96 / MAX: 8.63MIN: 7.95 / MAX: 8.95MIN: 7.98 / MAX: 8.77MIN: 7.94 / MAX: 13.92MIN: 8.12 / MAX: 10.11MIN: 7.99 / MAX: 10.44MIN: 7.97 / MAX: 10.71MIN: 7.96 / MAX: 10.32MIN: 8.04 / MAX: 75.44MIN: 8.25 / MAX: 10.5MIN: 8.33 / MAX: 10.07MIN: 8.39 / MAX: 10.77MIN: 8.1 / MAX: 124.43MIN: 7.76 / MAX: 454.91MIN: 8.08 / MAX: 286.28MIN: 7.57 / MAX: 211.621. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3 - Model: regnety_400m

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3 - Model: regnety_400mnv 4090cgi30904080 zzz3090 repb4080 rep4080f4080 xxxRTX 3070 Ti40904090 rep307048121620SE +/- 0.25, N = 147.737.987.997.998.018.108.258.278.448.458.508.588.899.6010.3417.23MIN: 7.43 / MAX: 9.41MIN: 7.93 / MAX: 8.65MIN: 7.91 / MAX: 8.8MIN: 7.62 / MAX: 9.27MIN: 7.93 / MAX: 8.35MIN: 7.77 / MAX: 15.42MIN: 8.12 / MAX: 14MIN: 8.22 / MAX: 9.01MIN: 8.04 / MAX: 10.17MIN: 8.05 / MAX: 10.3MIN: 8.04 / MAX: 30.12MIN: 8.23 / MAX: 10.39MIN: 7.74 / MAX: 476.28MIN: 7.66 / MAX: 210.23MIN: 8.21 / MAX: 214.16MIN: 7.8 / MAX: 193.141. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3 - Model: FastestDet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3 - Model: FastestDetc4090g3090b4080 rep3090 rep4080 zzz4090 repf4080RTX 3070 Ti4080 xxxinv 40903070246810SE +/- 0.15, N = 153.693.943.974.044.064.094.104.124.164.204.204.264.314.435.928.63MIN: 3.66 / MAX: 3.92MIN: 3.8 / MAX: 5.41MIN: 3.93 / MAX: 4.73MIN: 4 / MAX: 4.15MIN: 4.03 / MAX: 4.3MIN: 3.92 / MAX: 5.5MIN: 4.06 / MAX: 4.21MIN: 3.97 / MAX: 6.99MIN: 4.03 / MAX: 4.73MIN: 4.15 / MAX: 4.92MIN: 4.04 / MAX: 5.82MIN: 2.71 / MAX: 347.03MIN: 4.14 / MAX: 6.11MIN: 4.28 / MAX: 5.01MIN: 4.25 / MAX: 103.26MIN: 4.27 / MAX: 144.31. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3 - Model: vision_transformer

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3 - Model: vision_transformercb30903090 repgf4080 zzz40804080 rep4080 xxxRTX 3070 Tiinv 40904090 rep409030701632486480SE +/- 0.13, N = 1531.6631.7131.8632.1132.3833.3634.0534.1334.2935.4038.2938.3338.5838.7339.0173.51MIN: 31.52 / MAX: 32.14MIN: 31.56 / MAX: 33.03MIN: 31.58 / MAX: 35.84MIN: 31.94 / MAX: 33.01MIN: 32.04 / MAX: 51.55MIN: 32.83 / MAX: 76.21MIN: 32.83 / MAX: 38.57MIN: 32.98 / MAX: 36.11MIN: 33.11 / MAX: 40.12MIN: 33.93 / MAX: 39.3MIN: 32.31 / MAX: 557.38MIN: 34.14 / MAX: 246.43MIN: 33.77 / MAX: 476.18MIN: 33.81 / MAX: 362.17MIN: 33.91 / MAX: 411.66MIN: 39.27 / MAX: 288.21. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3 - Model: squeezenet_ssd

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3 - Model: squeezenet_ssd3090c3090 repfbg4080 zzz4080 rep40804080 xxxnv 40904090 rep4090RTX 3070 Tii307048121620SE +/- 0.19, N = 157.047.077.087.097.147.197.517.637.667.707.727.817.938.298.3316.15MIN: 6.97 / MAX: 7.76MIN: 7.01 / MAX: 7.75MIN: 7.01 / MAX: 7.93MIN: 6.98 / MAX: 8.01MIN: 7.06 / MAX: 7.95MIN: 6.99 / MAX: 23.11MIN: 6.94 / MAX: 9.51MIN: 7.02 / MAX: 9.71MIN: 7.02 / MAX: 9.08MIN: 7.11 / MAX: 9.19MIN: 7.12 / MAX: 23.25MIN: 7.24 / MAX: 9.04MIN: 7.31 / MAX: 9.45MIN: 6.37 / MAX: 448.22MIN: 6.32 / MAX: 222.03MIN: 7.25 / MAX: 210.691. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3 - Model: yolov4-tiny

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3 - Model: yolov4-tinyb3090 repc3090g4080 zzz4080 rep40804080 xxxfiRTX 3070 Ti40904090 repnv 40903070714212835SE +/- 0.23, N = 1512.7712.8412.8612.8812.8913.6213.6813.7913.9514.3415.4315.4415.4416.3916.6129.49MIN: 12.69 / MAX: 13.71MIN: 12.76 / MAX: 13.7MIN: 12.76 / MAX: 13.98MIN: 12.75 / MAX: 13.79MIN: 12.65 / MAX: 27.99MIN: 12.75 / MAX: 15.79MIN: 12.77 / MAX: 15.57MIN: 12.79 / MAX: 15.92MIN: 13.03 / MAX: 15.9MIN: 14.23 / MAX: 15.12MIN: 13.1 / MAX: 210.2MIN: 12.61 / MAX: 387.62MIN: 12.92 / MAX: 211.43MIN: 12.97 / MAX: 369.64MIN: 12.32 / MAX: 375.99MIN: 13.03 / MAX: 182.991. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3 - Model: resnet50

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3 - Model: resnet50b30903090 repcgf4080 rep40804080 zzzi4080 xxxRTX 3070 Tinv 40904090 rep40903070612182430SE +/- 0.27, N = 159.879.9710.0110.0310.1810.2510.8410.9511.0911.1511.5012.3513.1313.5714.5823.11MIN: 9.79 / MAX: 10.73MIN: 9.86 / MAX: 10.84MIN: 9.91 / MAX: 10.74MIN: 9.93 / MAX: 10.96MIN: 10.01 / MAX: 11.25MIN: 10.05 / MAX: 11.08MIN: 9.93 / MAX: 12.83MIN: 9.91 / MAX: 17.11MIN: 10.18 / MAX: 13.12MIN: 10.31 / MAX: 12.97MIN: 10.5 / MAX: 13.47MIN: 9.83 / MAX: 424.28MIN: 10.56 / MAX: 323.44MIN: 10.45 / MAX: 199.55MIN: 10.67 / MAX: 324.82MIN: 10.22 / MAX: 140.411. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3 - Model: alexnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3 - Model: alexnetc30903090 repfgb4080 zzz40804080 repi4080 xxxRTX 3070 Ti4090nv 40904090 rep30703691215SE +/- 0.23, N = 154.304.304.304.354.354.424.654.694.694.995.215.676.116.116.589.86MIN: 4.26 / MAX: 5.16MIN: 4.25 / MAX: 5.08MIN: 4.25 / MAX: 4.7MIN: 4.27 / MAX: 5.16MIN: 4.26 / MAX: 5.85MIN: 4.32 / MAX: 5.1MIN: 4.26 / MAX: 5.97MIN: 4.26 / MAX: 6.15MIN: 4.26 / MAX: 7.17MIN: 4.59 / MAX: 6.56MIN: 4.79 / MAX: 6.66MIN: 4.21 / MAX: 365.75MIN: 4.73 / MAX: 81.72MIN: 4.83 / MAX: 124.76MIN: 4.61 / MAX: 91.07MIN: 4.25 / MAX: 157.021. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3 - Model: resnet18

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3 - Model: resnet18c30903090 repfgb4080 zzz40804080 rep4090 repnv 4090i4080 xxx4090RTX 3070 Ti30703691215SE +/- 0.16, N = 155.235.235.245.305.305.425.595.655.695.815.845.855.895.976.2313.38MIN: 5.11 / MAX: 6.03MIN: 5.1 / MAX: 6.07MIN: 5.14 / MAX: 5.99MIN: 5.17 / MAX: 5.93MIN: 5.19 / MAX: 6.24MIN: 5.36 / MAX: 6.27MIN: 5.09 / MAX: 7.7MIN: 5.14 / MAX: 6.93MIN: 5.11 / MAX: 6.94MIN: 5.3 / MAX: 6.82MIN: 5.35 / MAX: 7.72MIN: 5.3 / MAX: 8.27MIN: 5.36 / MAX: 7.53MIN: 5.46 / MAX: 7.02MIN: 4.99 / MAX: 309.18MIN: 5.43 / MAX: 208.421. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3 - Model: vgg16

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3 - Model: vgg16b3090 rep3090cgf40804080 rep4080 zzz4080 xxx4090 repnv 40904090RTX 3070 Tii30701224364860SE +/- 0.27, N = 1523.4223.4323.5023.5423.8224.1225.0425.0425.2626.0827.5927.7728.2128.4029.0755.42MIN: 23.27 / MAX: 24.32MIN: 23.26 / MAX: 24.3MIN: 23.23 / MAX: 24.26MIN: 23.32 / MAX: 24.54MIN: 23.62 / MAX: 24.63MIN: 23.57 / MAX: 46.44MIN: 23.87 / MAX: 28.04MIN: 23.81 / MAX: 27.15MIN: 24.14 / MAX: 27.73MIN: 24.52 / MAX: 27.73MIN: 24.34 / MAX: 396.09MIN: 24.82 / MAX: 264.66MIN: 24.57 / MAX: 270.76MIN: 23.98 / MAX: 456MIN: 24.45 / MAX: 263.33MIN: 25.32 / MAX: 281.461. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3 - Model: googlenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3 - Model: googlenet3090c3090 repfgb4080 zzz40804080 rep40904090 rep4080 xxxRTX 3070 Tinv 4090i3070510152025SE +/- 0.21, N = 157.827.837.857.947.947.978.378.498.588.878.908.999.8610.0110.1920.72MIN: 7.69 / MAX: 8.6MIN: 7.74 / MAX: 8.61MIN: 7.75 / MAX: 8.64MIN: 7.8 / MAX: 8.78MIN: 7.79 / MAX: 9.59MIN: 7.89 / MAX: 8.7MIN: 7.76 / MAX: 10.31MIN: 7.82 / MAX: 11.98MIN: 7.79 / MAX: 10.48MIN: 8.18 / MAX: 11.09MIN: 8.22 / MAX: 11.07MIN: 8.25 / MAX: 10.27MIN: 7.54 / MAX: 396.21MIN: 7.29 / MAX: 259.11MIN: 7.73 / MAX: 212.36MIN: 7.49 / MAX: 355.331. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3 - Model: blazeface

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3 - Model: blazefacenv 4090i4090c3090bfg3090 rep4080 zzz4090 rep40804080 xxx4080 repRTX 3070 Ti30700.71551.4312.14652.8623.5775SE +/- 0.18, N = 151.071.251.331.361.361.371.371.371.381.391.411.421.421.451.713.18MIN: 1.02 / MAX: 1.52MIN: 1.19 / MAX: 2.61MIN: 1.27 / MAX: 1.98MIN: 1.34 / MAX: 1.44MIN: 1.34 / MAX: 1.61MIN: 1.35 / MAX: 1.39MIN: 1.35 / MAX: 1.62MIN: 1.34 / MAX: 1.7MIN: 1.36 / MAX: 1.9MIN: 1.34 / MAX: 1.89MIN: 1.35 / MAX: 1.89MIN: 1.35 / MAX: 2.15MIN: 1.36 / MAX: 1.92MIN: 1.36 / MAX: 8.73MIN: 1.09 / MAX: 448.17MIN: 1.31 / MAX: 185.031. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3 - Model: efficientnet-b0

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3 - Model: efficientnet-b0cbfg30903090 rep4080 zzz4080 rep40804090i4080 xxxRTX 3070 Tinv 40904090 rep3070246810SE +/- 0.18, N = 153.823.853.853.853.853.853.954.044.054.184.214.224.735.266.287.81MIN: 3.78 / MAX: 4.53MIN: 3.82 / MAX: 4.48MIN: 3.8 / MAX: 4.6MIN: 3.8 / MAX: 4.62MIN: 3.8 / MAX: 4.43MIN: 3.81 / MAX: 4.62MIN: 3.76 / MAX: 4.84MIN: 3.81 / MAX: 5.08MIN: 3.83 / MAX: 5MIN: 4 / MAX: 5.25MIN: 3.96 / MAX: 4.94MIN: 4 / MAX: 5.58MIN: 3.79 / MAX: 418.72MIN: 3.48 / MAX: 250.88MIN: 3.91 / MAX: 337.73MIN: 3.73 / MAX: 159.471. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3 - Model: mnasnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3 - Model: mnasnetnv 4090gbcf30903090 repi40904080 zzz4080 rep40804090 rep4080 xxxRTX 3070 Ti3070246810SE +/- 0.16, N = 142.542.952.962.962.962.972.972.993.003.013.073.093.103.133.376.02MIN: 2.44 / MAX: 3.58MIN: 2.91 / MAX: 3.64MIN: 2.93 / MAX: 3.4MIN: 2.93 / MAX: 3.41MIN: 2.92 / MAX: 3.81MIN: 2.92 / MAX: 3.28MIN: 2.94 / MAX: 3.39MIN: 2.86 / MAX: 4.38MIN: 2.89 / MAX: 3.46MIN: 2.91 / MAX: 3.6MIN: 2.94 / MAX: 3.72MIN: 2.94 / MAX: 3.79MIN: 2.97 / MAX: 3.72MIN: 3 / MAX: 5.1MIN: 2.86 / MAX: 278.87MIN: 2.79 / MAX: 50.491. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3 - Model: shufflenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3 - Model: shufflenet-v2nv 4090bcfg30904090i3090 rep4080 zzz4090 rep40804080 rep4080 xxxRTX 3070 Ti30701.10032.20063.30094.40125.5015SE +/- 0.22, N = 153.173.333.333.333.333.343.343.363.363.373.403.443.443.513.954.89MIN: 3.04 / MAX: 3.78MIN: 3.3 / MAX: 3.77MIN: 3.31 / MAX: 3.81MIN: 3.29 / MAX: 3.99MIN: 3.29 / MAX: 4.1MIN: 3.31 / MAX: 3.6MIN: 3.23 / MAX: 4.78MIN: 3.25 / MAX: 4.02MIN: 3.32 / MAX: 3.7MIN: 3.25 / MAX: 3.95MIN: 3.26 / MAX: 4.84MIN: 3.31 / MAX: 4.88MIN: 3.32 / MAX: 4.16MIN: 3.37 / MAX: 4.26MIN: 3.19 / MAX: 410.41MIN: 3.04 / MAX: 18.321. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3-v2-v2 - Model: mobilenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3-v2-v2 - Model: mobilenet-v2gbcf30903090 rep4080 zzzi40804080 rep4090 rep4080 xxxRTX 3070 Tinv 409040903070246810SE +/- 0.18, N = 153.143.153.153.163.163.173.233.283.293.303.313.403.664.454.997.24MIN: 3.09 / MAX: 3.61MIN: 3.11 / MAX: 3.88MIN: 3.11 / MAX: 3.85MIN: 3.1 / MAX: 3.71MIN: 3.11 / MAX: 3.51MIN: 3.12 / MAX: 4.03MIN: 3.06 / MAX: 4.66MIN: 3.09 / MAX: 5.28MIN: 3.12 / MAX: 4.64MIN: 3.12 / MAX: 4.7MIN: 3.12 / MAX: 4.6MIN: 3.23 / MAX: 4.8MIN: 3.01 / MAX: 437.59MIN: 2.65 / MAX: 216.76MIN: 3.1 / MAX: 201.8MIN: 3.04 / MAX: 261.681. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: Vulkan GPU-v3-v3-v3 - Model: mobilenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: Vulkan GPU-v3-v3-v3 - Model: mobilenetcb30903090 repg4080 zzz40804080 repf40904080 xxxi4090 repRTX 3070 Tinv 4090307048121620SE +/- 0.26, N = 157.958.008.008.018.048.388.438.468.658.818.889.059.549.6210.5416.34MIN: 7.89 / MAX: 8.79MIN: 7.95 / MAX: 8.99MIN: 7.94 / MAX: 8.78MIN: 7.95 / MAX: 8.35MIN: 7.93 / MAX: 8.86MIN: 7.95 / MAX: 10.41MIN: 7.99 / MAX: 10.66MIN: 7.99 / MAX: 10.62MIN: 8.55 / MAX: 9.53MIN: 8.32 / MAX: 10.7MIN: 8.31 / MAX: 10.01MIN: 8.48 / MAX: 11.28MIN: 8.94 / MAX: 10.54MIN: 7.76 / MAX: 502.83MIN: 8.41 / MAX: 134.08MIN: 8.13 / MAX: 80.691. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

VkFFT

Test: FFT + iFFT C2C 1D batched in single precision

OpenBenchmarking.orgBenchmark Score, More Is BetterVkFFT 1.2.31Test: FFT + iFFT C2C 1D batched in single precision4090 rep4090nv 40903090 rep309040804080 zzz4080 xxx4080 repifghcbaed30K60K90K120K150KSE +/- 25.50, N = 3SE +/- 9.54, N = 3SE +/- 1.67, N = 3SE +/- 2.73, N = 31539391538961521701414371413571045561045431045281044916973856476564555643147971479484788742651426451. (CXX) g++ options: -O3

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: FastestDet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: FastestDet3090 repRTX 3070 Ti4090 rep4090nv 40903070246810SE +/- 0.15, N = 34.074.144.594.625.867.23MIN: 4.04 / MAX: 4.25MIN: 3.73 / MAX: 5.07MIN: 4.44 / MAX: 5.2MIN: 4.48 / MAX: 5.16MIN: 3.9 / MAX: 190.17MIN: 3.75 / MAX: 121.711. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vision_transformer

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vision_transformer3090 repnv 4090RTX 3070 Ti4090 rep409030701632486480SE +/- 0.10, N = 331.9437.1338.5038.6539.3570.53MIN: 31.73 / MAX: 32.75MIN: 33.97 / MAX: 443.1MIN: 33.7 / MAX: 418.06MIN: 33.07 / MAX: 476.08MIN: 34.22 / MAX: 466.65MIN: 39.2 / MAX: 276.331. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: regnety_400m

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: regnety_400m3090 repnv 40904090 repRTX 3070 Ti4090307048121620SE +/- 0.54, N = 38.068.348.709.1410.1117.02MIN: 7.98 / MAX: 8.6MIN: 8.01 / MAX: 12.36MIN: 8.29 / MAX: 12.6MIN: 8.14 / MAX: 400.02MIN: 8.03 / MAX: 259.38MIN: 7.65 / MAX: 216.631. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd3090 rep4090RTX 3070 Tinv 40904090 rep307048121620SE +/- 0.14, N = 37.077.437.458.269.4415.32MIN: 6.98 / MAX: 9.71MIN: 6.84 / MAX: 8.82MIN: 6.59 / MAX: 9.11MIN: 7.64 / MAX: 11.08MIN: 7.17 / MAX: 94.63MIN: 6.66 / MAX: 139.171. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny3090 repRTX 3070 Ti4090 rep4090nv 40903070714212835SE +/- 0.94, N = 312.9214.6415.4116.0516.3029.38MIN: 12.79 / MAX: 18.5MIN: 12.77 / MAX: 383.28MIN: 12.75 / MAX: 226.87MIN: 12.93 / MAX: 474.03MIN: 14.11 / MAX: 184.46MIN: 12.95 / MAX: 201.311. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet50

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet503090 rep4090 rep4090RTX 3070 Tinv 40903070510152025SE +/- 0.30, N = 310.2710.9613.0013.1513.2522.15MIN: 10.12 / MAX: 11.19MIN: 10.09 / MAX: 12.99MIN: 10.34 / MAX: 397.57MIN: 10.26 / MAX: 349.93MIN: 10.61 / MAX: 154.12MIN: 10.11 / MAX: 123.041. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: alexnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: alexnet3090 rep40904090 repRTX 3070 Tinv 409030703691215SE +/- 0.57, N = 34.315.145.346.256.3211.43MIN: 4.26 / MAX: 4.83MIN: 4.75 / MAX: 7.34MIN: 4.87 / MAX: 6.57MIN: 4.27 / MAX: 334.55MIN: 4.26 / MAX: 195.95MIN: 4.24 / MAX: 178.831. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet18

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet183090 repnv 40904090 repRTX 3070 Ti409030703691215SE +/- 0.05, N = 35.275.585.875.946.9612.13MIN: 5.15 / MAX: 6.11MIN: 5.09 / MAX: 6.98MIN: 5.41 / MAX: 7.58MIN: 5.32 / MAX: 8.32MIN: 5.3 / MAX: 242.18MIN: 5.32 / MAX: 123.41. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vgg16

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vgg163090 repnv 40904090RTX 3070 Ti4090 rep30701224364860SE +/- 0.28, N = 323.7227.2527.3227.8629.8553.48MIN: 23.56 / MAX: 24.59MIN: 24.12 / MAX: 252.53MIN: 24.36 / MAX: 262.38MIN: 24.17 / MAX: 416.36MIN: 24.25 / MAX: 400.86MIN: 25.52 / MAX: 296.521. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: googlenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: googlenet3090 rep4090RTX 3070 Tinv 40904090 rep3070510152025SE +/- 0.55, N = 37.868.559.9710.1410.4718.80MIN: 7.75 / MAX: 8.57MIN: 7.85 / MAX: 11.39MIN: 8.16 / MAX: 381.49MIN: 7.85 / MAX: 257.61MIN: 7.86 / MAX: 191.94MIN: 7.78 / MAX: 141.461. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: blazeface

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: blazeface40903090 repRTX 3070 Tinv 40904090 rep30700.60531.21061.81592.42123.0265SE +/- 0.04, N = 31.351.381.401.401.422.69MIN: 1.28 / MAX: 1.84MIN: 1.36 / MAX: 1.73MIN: 1.28 / MAX: 1.91MIN: 1.34 / MAX: 1.86MIN: 1.36 / MAX: 2.03MIN: 1.35 / MAX: 48.811. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b03090 rep4090 repRTX 3070 Ti4090nv 40903070246810SE +/- 0.08, N = 33.844.104.174.635.826.63MIN: 3.8 / MAX: 4.67MIN: 3.88 / MAX: 5.04MIN: 3.86 / MAX: 5.52MIN: 4.38 / MAX: 6.01MIN: 3.98 / MAX: 197.79MIN: 3.75 / MAX: 22.341. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mnasnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mnasnet3090 repnv 4090RTX 3070 Ti4090 rep40903070246810SE +/- 0.02, N = 32.963.103.123.123.238.55MIN: 2.92 / MAX: 3.27MIN: 2.97 / MAX: 3.73MIN: 2.97 / MAX: 4.65MIN: 3 / MAX: 4.1MIN: 3.08 / MAX: 4.73MIN: 2.99 / MAX: 185.51. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v23090 repnv 4090RTX 3070 Ti40904090 rep30701.32532.65063.97595.30126.6265SE +/- 0.02, N = 33.323.433.483.565.235.89MIN: 3.29 / MAX: 3.62MIN: 3.29 / MAX: 5.31MIN: 3.33 / MAX: 5.22MIN: 3.43 / MAX: 4.24MIN: 3.34 / MAX: 185.57MIN: 3.19 / MAX: 97.881. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v33090 repRTX 3070 Ti4090 rep4090nv 40903070246810SE +/- 0.04, N = 33.193.243.353.364.967.34MIN: 3.13 / MAX: 3.61MIN: 3.05 / MAX: 5.14MIN: 3.22 / MAX: 3.99MIN: 3.22 / MAX: 4.62MIN: 3.14 / MAX: 189.43MIN: 3.09 / MAX: 155.331. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v23090 repnv 40904090 repRTX 3070 Ti409030701.3322.6643.9965.3286.66SE +/- 0.53, N = 33.153.293.383.834.755.92MIN: 3.1 / MAX: 3.75MIN: 3.12 / MAX: 4.27MIN: 3.2 / MAX: 4MIN: 3.11 / MAX: 343.21MIN: 2.93 / MAX: 147.66MIN: 3.16 / MAX: 103.241. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet3090 rep4090 repRTX 3070 Ti4090nv 4090307048121620SE +/- 0.13, N = 38.038.2210.0210.5610.6417.06MIN: 7.97 / MAX: 8.91MIN: 7.75 / MAX: 9.41MIN: 7.8 / MAX: 372.36MIN: 8.32 / MAX: 239.95MIN: 8.4 / MAX: 127.99MIN: 8 / MAX: 101.451. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v34080 zzz4080 xxx3090 rep4080 rep4090 rep4090RTX 3070 Tinv 40903070246810SE +/- 0.53, N = 33.063.083.173.283.303.333.704.976.43MIN: 2.94 / MAX: 3.94MIN: 2.97 / MAX: 3.67MIN: 3.12 / MAX: 3.75MIN: 3.13 / MAX: 4.78MIN: 3.15 / MAX: 3.91MIN: 3.2 / MAX: 4.4MIN: 2.98 / MAX: 261.6MIN: 3.15 / MAX: 291.01MIN: 2.85 / MAX: 164.911. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

VkFFT

Test: FFT + iFFT C2C 1D batched in single precision, no reshuffling

OpenBenchmarking.orgBenchmark Score, More Is BetterVkFFT 1.2.31Test: FFT + iFFT C2C 1D batched in single precision, no reshuffling4090 repnv 4090409030903090 rep40804080 rep4080 xxx4080 zzzifgbcaed30K60K90K120K150KSE +/- 8.89, N = 3SE +/- 2.08, N = 3SE +/- 2.33, N = 315593615514815265614396914395610621010620510609910592671163571105709450643505965050443365433651. (CXX) g++ options: -O3

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3 - Model: FastestDet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3 - Model: FastestDet40904090 rep4080 xxx4080 zzznv 40903090 rep3090RTX 3070 Ti4080 rep3070246810SE +/- 0.87, N = 32.853.123.803.823.934.074.104.184.207.12MIN: 2.74 / MAX: 4.36MIN: 2.97 / MAX: 4.42MIN: 3.65 / MAX: 6.08MIN: 3.65 / MAX: 9.77MIN: 3.76 / MAX: 11.77MIN: 4.03 / MAX: 4.2MIN: 4.07 / MAX: 4.34MIN: 2.53 / MAX: 295.11MIN: 4.04 / MAX: 5.63MIN: 3.72 / MAX: 188.71. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3 - Model: vision_transformer

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3 - Model: vision_transformer3090 rep30904080 xxx4080 rep4080 zzzRTX 3070 Ti40904090 repnv 409030701530456075SE +/- 0.11, N = 331.8532.1034.1434.2234.4738.0438.7638.7938.9565.41MIN: 31.67 / MAX: 35.74MIN: 31.9 / MAX: 33.03MIN: 32.5 / MAX: 37.13MIN: 33.01 / MAX: 37.09MIN: 33.05 / MAX: 39.69MIN: 33.11 / MAX: 346.94MIN: 33.38 / MAX: 423.24MIN: 34.02 / MAX: 460.15MIN: 34.04 / MAX: 486.96MIN: 39.08 / MAX: 230.591. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3 - Model: regnety_400m

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3 - Model: regnety_400m3090 rep3090nv 40904080 zzz4080 xxxRTX 3070 Ti4080 rep40904090 rep307048121620SE +/- 0.29, N = 38.038.228.258.348.388.428.7210.0910.2318.25MIN: 7.97 / MAX: 8.65MIN: 8.14 / MAX: 8.67MIN: 7.87 / MAX: 10.07MIN: 8.03 / MAX: 10.23MIN: 8.04 / MAX: 9.63MIN: 7.66 / MAX: 10.74MIN: 8.32 / MAX: 10.48MIN: 8.01 / MAX: 418.58MIN: 8.22 / MAX: 197.1MIN: 7.8 / MAX: 238.291. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd3090 rep30904080 zzz4080 xxxnv 4090RTX 3070 Ti4080 rep4090 rep4090307048121620SE +/- 0.23, N = 37.097.127.257.277.487.577.679.389.5114.27MIN: 7.02 / MAX: 7.86MIN: 7.04 / MAX: 7.97MIN: 6.72 / MAX: 8.05MIN: 6.73 / MAX: 8.77MIN: 6.85 / MAX: 9.67MIN: 6.69 / MAX: 10MIN: 7.06 / MAX: 9.96MIN: 6.77 / MAX: 224.11MIN: 7.11 / MAX: 307.17MIN: 7.01 / MAX: 51.131. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny3090 rep30904080 zzz4080 xxx4080 rep4090 repRTX 3070 Ti4090nv 40903070714212835SE +/- 0.81, N = 312.8112.8213.4213.6313.7113.8814.5715.8517.6728.41MIN: 12.7 / MAX: 13.69MIN: 12.72 / MAX: 13.66MIN: 12.65 / MAX: 16.19MIN: 12.77 / MAX: 16.93MIN: 12.78 / MAX: 15.62MIN: 13.09 / MAX: 14.77MIN: 12.33 / MAX: 312.42MIN: 13.26 / MAX: 253.23MIN: 14.92 / MAX: 343.93MIN: 12.49 / MAX: 151.041. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3 - Model: resnet50

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3 - Model: resnet5030903090 rep4080 rep4080 zzz4080 xxx40904090 repRTX 3070 Tinv 40903070510152025SE +/- 0.04, N = 310.0310.0610.8611.1011.2611.7212.4712.8113.2922.19MIN: 9.93 / MAX: 10.87MIN: 9.86 / MAX: 11.9MIN: 9.98 / MAX: 12.46MIN: 10.19 / MAX: 18.3MIN: 10.32 / MAX: 13.29MIN: 10.8 / MAX: 12.8MIN: 11.5 / MAX: 14.68MIN: 10.06 / MAX: 349.03MIN: 10.54 / MAX: 456.82MIN: 10.16 / MAX: 181.741. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3 - Model: alexnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3 - Model: alexnet3090 rep30904080 zzz4080 rep4080 xxx40904090 repRTX 3070 Tinv 409030703691215SE +/- 0.43, N = 34.314.334.664.684.694.995.456.176.6210.59MIN: 4.26 / MAX: 5.07MIN: 4.26 / MAX: 5.19MIN: 4.24 / MAX: 5.97MIN: 4.27 / MAX: 6.08MIN: 4.26 / MAX: 6.07MIN: 4.56 / MAX: 6.91MIN: 4.93 / MAX: 7.98MIN: 4.5 / MAX: 261.75MIN: 4.28 / MAX: 339.62MIN: 4.3 / MAX: 177.681. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3 - Model: resnet18

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3 - Model: resnet1830903090 rep4080 rep4080 zzz4080 xxxnv 4090RTX 3070 Ti40904090 rep30703691215SE +/- 0.30, N = 35.205.305.645.775.786.076.227.748.1412.64MIN: 5.1 / MAX: 6.16MIN: 5.21 / MAX: 6.24MIN: 5.11 / MAX: 7.51MIN: 5.22 / MAX: 7.06MIN: 5.21 / MAX: 6.97MIN: 5.49 / MAX: 15.12MIN: 5.3 / MAX: 8.22MIN: 5.25 / MAX: 312.09MIN: 5.39 / MAX: 122.47MIN: 5.3 / MAX: 53.811. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3 - Model: vgg16

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3 - Model: vgg163090 rep30904080 rep4080 zzz4080 xxxnv 4090RTX 3070 Ti40904090 rep30701122334455SE +/- 0.54, N = 323.4023.5825.0125.2625.4427.6127.9830.1630.7450.32MIN: 23.2 / MAX: 24.07MIN: 23.35 / MAX: 24.43MIN: 23.88 / MAX: 26.66MIN: 24.29 / MAX: 27.75MIN: 24.27 / MAX: 27.68MIN: 24.67 / MAX: 401.29MIN: 24.35 / MAX: 423.63MIN: 24.66 / MAX: 332.49MIN: 25.36 / MAX: 428.68MIN: 25.92 / MAX: 281.061. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3 - Model: googlenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3 - Model: googlenet30903090 rep4080 zzz4080 xxx40904080 rep4090 repRTX 3070 Tinv 40903070510152025SE +/- 0.80, N = 37.907.918.298.328.388.528.979.6810.7519.20MIN: 7.8 / MAX: 8.73MIN: 7.81 / MAX: 8.62MIN: 7.63 / MAX: 9.87MIN: 7.71 / MAX: 10.39MIN: 7.78 / MAX: 10.43MIN: 7.85 / MAX: 10.56MIN: 8.22 / MAX: 10.51MIN: 8.16 / MAX: 382.41MIN: 7.92 / MAX: 447.83MIN: 7.84 / MAX: 193.361. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3 - Model: blazeface

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3 - Model: blazeface40904080 zzz4080 xxx3090 rep30904090 repnv 40904080 repRTX 3070 Ti30700.56931.13861.70792.27722.8465SE +/- 0.48, N = 31.301.311.321.381.391.421.421.432.482.53MIN: 1.24 / MAX: 1.92MIN: 1.25 / MAX: 1.76MIN: 1.26 / MAX: 2.03MIN: 1.35 / MAX: 1.64MIN: 1.37 / MAX: 1.48MIN: 1.34 / MAX: 2.37MIN: 1.34 / MAX: 1.99MIN: 1.36 / MAX: 2.02MIN: 1.17 / MAX: 344.52MIN: 1.08 / MAX: 118.731. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b03090 rep30904080 zzz4080 xxx4080 rep4090 rep4090RTX 3070 Tinv 409030703691215SE +/- 0.46, N = 33.853.883.954.014.074.354.474.745.949.19MIN: 3.81 / MAX: 4.75MIN: 3.83 / MAX: 4.61MIN: 3.79 / MAX: 4.59MIN: 3.83 / MAX: 5.28MIN: 3.85 / MAX: 4.79MIN: 4.08 / MAX: 5.62MIN: 4.23 / MAX: 5.82MIN: 3.68 / MAX: 295.7MIN: 3.97 / MAX: 208.59MIN: 3.85 / MAX: 131.421. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3 - Model: mnasnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3 - Model: mnasnet4080 zzz3090 rep30904080 xxx4080 repnv 4090RTX 3070 Ti4090 rep40903070246810SE +/- 0.13, N = 32.962.972.983.003.093.123.245.115.196.07MIN: 2.85 / MAX: 3.82MIN: 2.93 / MAX: 3.3MIN: 2.95 / MAX: 3.9MIN: 2.88 / MAX: 4.37MIN: 2.96 / MAX: 4.98MIN: 2.98 / MAX: 3.71MIN: 2.9 / MAX: 5.34MIN: 2.96 / MAX: 247.47MIN: 3.04 / MAX: 436.91MIN: 2.94 / MAX: 129.11. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2nv 40903090 rep4080 zzz30904080 xxx4090 rep4080 rep4090RTX 3070 Ti3070246810SE +/- 0.60, N = 33.323.333.363.363.403.423.473.484.027.81MIN: 3.19 / MAX: 4.76MIN: 3.3 / MAX: 3.78MIN: 3.23 / MAX: 3.99MIN: 3.32 / MAX: 3.66MIN: 3.28 / MAX: 3.87MIN: 3.29 / MAX: 3.94MIN: 3.33 / MAX: 5.39MIN: 3.35 / MAX: 4.05MIN: 3.27 / MAX: 328.59MIN: 3.3 / MAX: 131.261. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v24080 zzz30903090 rep4080 xxxnv 40904080 rep40904090 repRTX 3070 Ti3070246810SE +/- 0.53, N = 33.163.163.163.203.293.303.363.443.917.22MIN: 3.01 / MAX: 5.17MIN: 3.11 / MAX: 3.95MIN: 3.09 / MAX: 4.06MIN: 3.05 / MAX: 4.67MIN: 3.13 / MAX: 4.29MIN: 3.11 / MAX: 4.01MIN: 3.21 / MAX: 4.78MIN: 3.27 / MAX: 4.93MIN: 3.04 / MAX: 394.66MIN: 3.17 / MAX: 69.661. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

NCNN

Target: CPU-v3-v3-v3-v3-v3-v3 - Model: mobilenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20230517Target: CPU-v3-v3-v3-v3-v3-v3 - Model: mobilenet3090 rep30904080 zzz4080 xxx4080 rep4090RTX 3070 Ti4090 repnv 4090307048121620SE +/- 0.14, N = 38.068.078.258.348.459.0410.0310.6112.1216.52MIN: 8 / MAX: 8.96MIN: 8.01 / MAX: 8.62MIN: 7.78 / MAX: 9.61MIN: 7.89 / MAX: 9.42MIN: 8.01 / MAX: 10.86MIN: 8.49 / MAX: 10.96MIN: 7.86 / MAX: 346.64MIN: 8.34 / MAX: 225.97MIN: 9.16 / MAX: 505.01MIN: 7.9 / MAX: 82.531. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread

VkFFT

Test: FFT + iFFT C2C 1D batched in half precision

OpenBenchmarking.orgBenchmark Score, More Is BetterVkFFT 1.2.31Test: FFT + iFFT C2C 1D batched in half precisionnv 409040904090 rep3090 rep309040804080 rep4080 zzz4080 xxxihgfbcaed60K120K180K240K300KSE +/- 133.47, N = 3SE +/- 83.55, N = 3SE +/- 26.03, N = 3SE +/- 18.50, N = 329276829034228765126517125520721107621105821099121071313227010429810417110414691812917449159785191851811. (CXX) g++ options: -O3

VkFFT

Test: FFT + iFFT C2C multidimensional in single precision

OpenBenchmarking.orgBenchmark Score, More Is BetterVkFFT 1.2.31Test: FFT + iFFT C2C multidimensional in single precisionnv 409040904090 rep4080 rep4080 zzz4080 xxx40803090 rep3090ediacbgf20K40K60K80K100KSE +/- 555.86, N = 3SE +/- 437.33, N = 3SE +/- 116.12, N = 3SE +/- 57.83, N = 382875814068099970068700406788765869548145100537090363283468633001328123275126541262381. (CXX) g++ options: -O3

VkFFT

Test: FFT + iFFT R2C / C2R

OpenBenchmarking.orgBenchmark Score, More Is BetterVkFFT 1.2.31Test: FFT + iFFT R2C / C2Rnv 409040904090 rep4080 xxx4080 rep4080 zzz408030903090 repcbadeigfh20K40K60K80K100KSE +/- 796.66, N = 3SE +/- 200.55, N = 3SE +/- 118.74, N = 3SE +/- 3.71, N = 38488784351813296906868279676896647355347544324302142163421053539935304337272663826593265241. (CXX) g++ options: -O3

VkResample

Upscale: 2x - Precision: Single

OpenBenchmarking.orgms, Fewer Is BetterVkResample 1.0Upscale: 2x - Precision: Single4090 repnv 4090409030903090 repacb4080 zzz40804080 rep4080 xxxi3070fgRTX 3070 Tied816243240SE +/- 0.001, N = 3SE +/- 0.029, N = 3SE +/- 0.000, N = 3SE +/- 0.004, N = 38.9628.9679.28410.39910.42811.68611.68811.69013.12613.13613.13613.13720.93022.06426.73826.76927.18332.85032.8551. (CXX) g++ options: -O3


Phoronix Test Suite v10.8.5