vulkan-benchmarks AMD Ryzen 9 7950X 16-Core testing with a ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS) and NVIDIA GeForce RTX 4090 24GB on Ubuntu 23.04 via the Phoronix Test Suite. a: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: AMD Radeon RX 6700 XT (2855/1000MHz), Audio: AMD Navi 21/23, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7 + Wayland, OpenGL: 4.6 Mesa 23.3~git2307260600.87109c~oibaf~l (git-87109c3 2023-07-26 lunar-oibaf-ppa) (LLVM 15.0.7 DRM 3.52), Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 b: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: AMD Radeon RX 6700 XT (2855/1000MHz), Audio: AMD Navi 21/23, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7 + Wayland, OpenGL: 4.6 Mesa 23.3~git2307260600.87109c~oibaf~l (git-87109c3 2023-07-26 lunar-oibaf-ppa) (LLVM 15.0.7 DRM 3.52), Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 c: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: AMD Radeon RX 6700 XT (2855/1000MHz), Audio: AMD Navi 21/23, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7 + Wayland, OpenGL: 4.6 Mesa 23.3~git2307260600.87109c~oibaf~l (git-87109c3 2023-07-26 lunar-oibaf-ppa) (LLVM 15.0.7 DRM 3.52), Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 d: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: MSI NVIDIA GeForce RTX 4060 8GB, Audio: NVIDIA Device 22be, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 e: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: MSI NVIDIA GeForce RTX 4060 8GB, Audio: NVIDIA Device 22be, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 f: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: eVGA NVIDIA GeForce RTX 3060 12GB, Audio: NVIDIA GA106 HD Audio, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 g: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: eVGA NVIDIA GeForce RTX 3060 12GB, Audio: NVIDIA GA106 HD Audio, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 h: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: eVGA NVIDIA GeForce RTX 3060 12GB, Audio: NVIDIA GA106 HD Audio, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 i: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: NVIDIA GeForce RTX 3060 Ti 8GB, Audio: NVIDIA GA104 HD Audio, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 2560x1440 4080: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: NVIDIA GeForce RTX 4080 16GB, Audio: NVIDIA Device 22bb, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 4080 rep: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: NVIDIA GeForce RTX 4080 16GB, Audio: NVIDIA Device 22bb, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 4080 xxx: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: NVIDIA GeForce RTX 4080 16GB, Audio: NVIDIA Device 22bb, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 4080 zzz: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: NVIDIA GeForce RTX 4080 16GB, Audio: NVIDIA Device 22bb, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 3090: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: NVIDIA GeForce RTX 3090 24GB, Audio: NVIDIA GA102 HD Audio, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 3090 rep: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: NVIDIA GeForce RTX 3090 24GB, Audio: NVIDIA GA102 HD Audio, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 3070: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: NVIDIA GeForce RTX 3070 8GB, Audio: NVIDIA GA104 HD Audio, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 2560x1440 RTX 3070 Ti: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: NVIDIA GeForce RTX 3070 Ti 8GB, Audio: NVIDIA GA104 HD Audio, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 2560x1440 4090: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: NVIDIA GeForce RTX 4090 24GB, Audio: NVIDIA AD102 HD Audio, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 4090 rep: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: NVIDIA GeForce RTX 4090 24GB, Audio: NVIDIA AD102 HD Audio, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 nv 4090: Processor: AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: Western Digital WD_BLACK SN850X 1000GB + 4001GB, Graphics: NVIDIA GeForce RTX 4090 24GB, Audio: NVIDIA AD102 HD Audio, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 23.04, Kernel: 6.4.6-060406-generic (x86_64), Desktop: GNOME Shell 44.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 535.86.05, OpenGL: 4.6.0, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 vkpeak 20230730 int32-scalar GIOPS > Higher Is Better g ........... 6824.21 |================== 3090 rep .... 20613.41 |====================================================== a ........... 2272.62 |====== b ........... 2269.25 |====== c ........... 2269.06 |====== d ........... 8520.02 |====================== e ........... 8505.20 |====================== f ........... 6827.92 |================== g ........... 6824.29 |================== h ........... 6800.60 |================== 3090 ........ 20909.02 |======================================================= 3090 rep .... 20767.64 |======================================================= vkpeak 20230730 int32-vec4 GIOPS > Higher Is Better g ........... 6795.39 |================== 3090 rep .... 20517.45 |====================================================== a ........... 2658.73 |======= b ........... 2640.08 |======= c ........... 2638.69 |======= d ........... 8465.82 |====================== e ........... 8465.71 |====================== f ........... 6800.17 |================== g ........... 6794.92 |================== h ........... 6772.98 |================== 3090 ........ 20820.09 |======================================================= 3090 rep .... 20517.68 |====================================================== VkFFT 1.2.31 Test: FFT + iFFT C2C 1D batched in double precision Benchmark Score > Higher Is Better a ........... 20816 |====================== b ........... 20822 |====================== c ........... 20847 |====================== d ........... 12143 |============= e ........... 12168 |============= f ........... 10561 |=========== g ........... 10548 |=========== h ........... 10572 |=========== i ........... 14780 |=============== 4080 ........ 34974 |===================================== 4080 rep .... 35038 |===================================== 4080 xxx .... 35071 |===================================== 4080 zzz .... 35058 |===================================== 3090 ........ 30945 |================================ 3090 rep .... 31122 |================================= 4090 ........ 55214 |========================================================== 4090 rep .... 55383 |========================================================== nv 4090 ..... 54950 |========================================================== VkFFT 1.2.31 Test: FFT + iFFT C2C Bluestein benchmark in double precision Benchmark Score > Higher Is Better a ........... 4717 |================================== b ........... 4695 |================================== c ........... 4670 |================================== d ........... 2346 |================= e ........... 2343 |================= f ........... 1814 |============= g ........... 1818 |============= i ........... 2417 |================== 4080 ........ 5579 |======================================== 4080 rep .... 5583 |========================================= 4080 xxx .... 5587 |========================================= 4080 zzz .... 5584 |========================================= 3090 ........ 4282 |=============================== 3090 rep .... 4289 |=============================== 4090 ........ 8039 |========================================================== 4090 rep .... 8119 |=========================================================== nv 4090 ..... 8132 |=========================================================== vkpeak 20230730 fp64-vec4 GFLOPS > Higher Is Better a ........... 841.80 |========================================================= b ........... 836.55 |========================================================= c ........... 836.16 |========================================================= d ........... 267.74 |================== e ........... 267.25 |================== f ........... 214.23 |=============== g ........... 213.95 |============== h ........... 210.96 |============== 3090 ........ 653.15 |============================================ vkpeak 20230730 fp64-scalar GFLOPS > Higher Is Better 3090 rep .... 648.71 |============================================ a ........... 841.40 |========================================================= b ........... 839.20 |========================================================= c ........... 839.01 |========================================================= d ........... 267.43 |================== e ........... 267.41 |================== f ........... 214.17 |=============== g ........... 213.96 |============== h ........... 213.37 |============== 3090 ........ 653.13 |============================================ 3090 rep .... 653.63 |============================================ vkpeak 20230730 int16-vec4 GIOPS > Higher Is Better g ........... 5956.24 |============== 3090 rep .... 16878.20 |======================================== a ........... 23123.77 |====================================================== b ........... 23396.59 |======================================================= c ........... 23385.44 |======================================================= d ........... 7352.85 |================= e ........... 7336.25 |================= f ........... 5959.75 |============== g ........... 5956.38 |============== h ........... 5978.38 |============== 3090 ........ 16886.66 |======================================== 3090 rep .... 16881.47 |======================================== VkResample 1.0 Upscale: 2x - Precision: Single ms < Lower Is Better a ........... 11.686 |==================== b ........... 11.690 |==================== c ........... 11.688 |==================== d ........... 32.855 |========================================================= e ........... 32.850 |========================================================= f ........... 26.738 |============================================== g ........... 26.769 |============================================== i ........... 20.930 |==================================== 4080 ........ 13.136 |======================= 4080 rep .... 13.136 |======================= 4080 xxx .... 13.137 |======================= 4080 zzz .... 13.126 |======================= 3090 ........ 10.399 |================== 3090 rep .... 10.428 |================== 3070 ........ 22.064 |====================================== RTX 3070 Ti . 27.183 |=============================================== 4090 ........ 9.284 |================ 4090 rep .... 8.962 |================ nv 4090 ..... 8.967 |================ VkFFT 1.2.31 Test: FFT + iFFT C2C 1D batched in single precision Benchmark Score > Higher Is Better a ........... 47887 |================== b ........... 47948 |================== c ........... 47971 |================== d ........... 42645 |================ e ........... 42651 |================ f ........... 56476 |===================== g ........... 56455 |===================== h ........... 56431 |===================== i ........... 69738 |========================== 4080 ........ 104556 |======================================= 4080 rep .... 104491 |======================================= 4080 xxx .... 104528 |======================================= 4080 zzz .... 104543 |======================================= 3090 ........ 141357 |==================================================== 3090 rep .... 141437 |==================================================== 4090 ........ 153896 |========================================================= 4090 rep .... 153939 |========================================================= nv 4090 ..... 152170 |======================================================== VkFFT 1.2.31 Test: FFT + iFFT C2C 1D batched in single precision, no reshuffling Benchmark Score > Higher Is Better a ........... 50504 |================== b ........... 50643 |=================== c ........... 50596 |================== d ........... 43365 |================ e ........... 43365 |================ f ........... 57110 |===================== g ........... 57094 |===================== i ........... 71163 |========================== 4080 ........ 106210 |======================================= 4080 rep .... 106205 |======================================= 4080 xxx .... 106099 |======================================= 4080 zzz .... 105926 |======================================= 3090 ........ 143969 |===================================================== 3090 rep .... 143956 |===================================================== 4090 ........ 152656 |======================================================== 4090 rep .... 155936 |========================================================= nv 4090 ..... 155148 |========================================================= VkFFT 1.2.31 Test: FFT + iFFT C2C 1D batched in half precision Benchmark Score > Higher Is Better a ........... 91597 |================== b ........... 91812 |================== c ........... 91744 |================== d ........... 85181 |================= e ........... 85191 |================= f ........... 104146 |==================== g ........... 104171 |==================== h ........... 104298 |==================== i ........... 132270 |========================== 4080 ........ 211076 |========================================= 4080 rep .... 211058 |========================================= 4080 xxx .... 210713 |========================================= 4080 zzz .... 210991 |========================================= 3090 ........ 255207 |================================================== 3090 rep .... 265171 |==================================================== 4090 ........ 290342 |========================================================= 4090 rep .... 287651 |======================================================== nv 4090 ..... 292768 |========================================================= NCNN 20230517 Target: CPU - Model: FastestDet ms < Lower Is Better a ........... 3.62 |========================= b ........... 4.05 |============================ c ........... 4.11 |============================ d ........... 4.08 |============================ f ........... 4.22 |============================= g ........... 2.57 |================== i ........... 2.66 |================== 4080 ........ 4.42 |============================== 4080 rep .... 4.34 |============================== 4080 xxx .... 4.17 |============================ 4080 zzz .... 4.16 |============================ 3090 ........ 4.21 |============================= 3090 rep .... 4.08 |============================ 3070 ........ 8.65 |=========================================================== RTX 3070 Ti . 3.94 |=========================== 4090 ........ 5.48 |===================================== 4090 rep .... 5.27 |==================================== nv 4090 ..... 4.51 |=============================== VkFFT 1.2.31 Test: FFT + iFFT R2C / C2R Benchmark Score > Higher Is Better a ........... 42105 |============================= b ........... 42163 |============================= c ........... 43021 |============================= d ........... 35399 |======================== e ........... 35304 |======================== f ........... 26593 |================== g ........... 26638 |================== h ........... 26524 |================== i ........... 33727 |======================= 4080 ........ 66473 |============================================= 4080 rep .... 68279 |=============================================== 4080 xxx .... 69068 |=============================================== 4080 zzz .... 67689 |============================================== 3090 ........ 55347 |====================================== 3090 rep .... 54432 |===================================== 4090 ........ 84351 |========================================================== 4090 rep .... 81329 |======================================================== nv 4090 ..... 84887 |========================================================== VkFFT 1.2.31 Test: FFT + iFFT C2C multidimensional in single precision Benchmark Score > Higher Is Better a ........... 33001 |======================= b ........... 32751 |======================= c ........... 32812 |======================= d ........... 36328 |========================= e ........... 37090 |========================== f ........... 26238 |================== g ........... 26541 |=================== i ........... 34686 |======================== 4080 ........ 65869 |============================================== 4080 rep .... 70068 |================================================= 4080 xxx .... 67887 |================================================ 4080 zzz .... 70040 |================================================= 3090 ........ 51005 |==================================== 3090 rep .... 54814 |====================================== 4090 ........ 81406 |========================================================= 4090 rep .... 80999 |========================================================= nv 4090 ..... 82875 |========================================================== NCNN 20230517 Target: CPU-v3-v3-v3 - Model: FastestDet ms < Lower Is Better b ........... 4.07 |========================== c ........... 4.08 |========================== f ........... 3.85 |========================= g ........... 3.97 |========================== i ........... 5.69 |===================================== 4080 ........ 4.20 |=========================== 4080 rep .... 4.18 |=========================== 4080 xxx .... 4.19 |=========================== 4080 zzz .... 4.04 |========================== 3090 ........ 4.04 |========================== 3090 rep .... 4.11 |========================== 3070 ........ 9.18 |=========================================================== RTX 3070 Ti . 4.33 |============================ 4090 ........ 2.93 |=================== 4090 rep .... 4.16 |=========================== nv 4090 ..... 4.06 |========================== vkpeak 20230730 fp32-scalar GFLOPS > Higher Is Better g ........... 6832.74 |================== 3090 rep .... 20708.84 |====================================================== a ........... 13190.09 |================================== b ........... 12807.06 |================================= c ........... 12860.56 |================================= d ........... 8531.96 |====================== e ........... 8515.58 |====================== f ........... 6837.94 |================== g ........... 6812.99 |================== h ........... 6810.73 |================== 3090 ........ 21269.72 |======================================================= 3090 rep .... 20925.30 |====================================================== vkpeak 20230730 fp32-vec4 GFLOPS > Higher Is Better g ........... 9003.12 |================== 3090 rep .... 27393.20 |====================================================== a ........... 12730.08 |========================= b ........... 12808.59 |========================= c ........... 12822.01 |========================= d ........... 11251.17 |====================== e ........... 11231.72 |====================== f ........... 9006.57 |================== g ........... 9002.59 |================== h ........... 9036.17 |================== 3090 ........ 27797.80 |======================================================= 3090 rep .... 27807.58 |======================================================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better b ........... 3.16 |======================= c ........... 3.17 |======================= f ........... 3.15 |======================= g ........... 3.15 |======================= i ........... 3.26 |======================== 4080 ........ 3.27 |======================== 4080 rep .... 3.27 |======================== 4080 xxx .... 3.33 |======================== 4080 zzz .... 3.20 |======================= 3090 rep .... 3.15 |======================= 3070 ........ 8.06 |=========================================================== RTX 3070 Ti . 3.65 |=========================== 4090 ........ 3.12 |======================= 4090 rep .... 4.90 |==================================== nv 4090 ..... 2.61 |=================== vkpeak 20230730 fp16-scalar GFLOPS > Higher Is Better g ........... 6810.55 |================== 3090 rep .... 20640.67 |====================================================== a ........... 13154.15 |=================================== b ........... 13145.19 |=================================== c ........... 13136.79 |================================== d ........... 8412.33 |====================== e ........... 8397.80 |====================== f ........... 6812.52 |================== g ........... 6811.35 |================== h ........... 6838.32 |================== 3090 ........ 20845.09 |======================================================= 3090 rep .... 20953.30 |======================================================= NCNN 20230517 Target: CPU-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better 4090 rep .... 3.30 |==================== a ........... 3.16 |=================== b ........... 3.16 |=================== c ........... 3.18 |=================== d ........... 3.17 |=================== f ........... 3.15 |=================== g ........... 3.16 |=================== i ........... 3.52 |===================== 4080 ........ 3.28 |==================== 4080 rep .... 3.28 |==================== 4080 xxx .... 3.28 |==================== 4080 zzz .... 3.25 |==================== 3090 ........ 3.17 |=================== 3090 rep .... 3.17 |=================== 3070 ........ 9.67 |=========================================================== RTX 3070 Ti . 3.76 |======================= 4090 ........ 3.30 |==================== 4090 rep .... 4.74 |============================= nv 4090 ..... 3.60 |====================== vkpeak 20230730 fp16-vec4 GFLOPS > Higher Is Better g ........... 13438.40 |================== 3090 rep .... 40876.12 |======================================================= a ........... 23232.42 |=============================== b ........... 23390.44 |=============================== c ........... 23387.26 |=============================== d ........... 16864.47 |======================= e ........... 16865.29 |======================= f ........... 13440.97 |================== g ........... 13438.47 |================== h ........... 13490.24 |================== 3090 ........ 41149.10 |======================================================= 3090 rep .... 41188.02 |======================================================= vkpeak 20230730 int16-scalar GIOPS > Higher Is Better g ........... 4478.41 |================== 3090 rep .... 13606.79 |======================================================= a ........... 13102.75 |===================================================== b ........... 13070.81 |==================================================== c ........... 13063.86 |==================================================== d ........... 5676.02 |======================= e ........... 5675.99 |======================= f ........... 4480.59 |================== g ........... 4479.22 |================== h ........... 4495.98 |================== 3090 ........ 13710.88 |======================================================= 3090 rep .... 13608.57 |======================================================= VkResample 1.0 Upscale: 2x - Precision: Double ms < Lower Is Better d ........... 500.01 |========================================================= e ........... 500.02 |========================================================= f ........... 500.01 |========================================================= g ........... 500.01 |========================================================= i ........... 500.01 |========================================================= 4080 ........ 288.20 |================================= 4080 rep .... 288.17 |================================= 4080 xxx .... 288.04 |================================= 4080 zzz .... 288.03 |================================= 3090 ........ 371.70 |========================================== 3090 rep .... 371.42 |========================================== 3070 ........ 24.75 |=== RTX 3070 Ti . 24.81 |=== 4090 ........ 172.88 |==================== 4090 rep .... 173.04 |==================== nv 4090 ..... 172.89 |==================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: blazeface ms < Lower Is Better g ........... 1.37 |========================= b ........... 1.37 |========================= c ........... 1.36 |========================= f ........... 1.37 |========================= g ........... 1.38 |========================== i ........... 1.25 |======================= 4080 ........ 1.42 |========================== 4080 rep .... 1.45 |=========================== 4080 xxx .... 1.42 |========================== 4080 zzz .... 1.39 |========================== 3090 ........ 1.36 |========================= 3090 rep .... 1.38 |========================== 3070 ........ 3.18 |=========================================================== RTX 3070 Ti . 1.71 |================================ 4090 ........ 1.33 |========================= 4090 rep .... 1.41 |========================== nv 4090 ..... 1.07 |==================== NCNN 20230517 Target: CPU-v3-v3-v3-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better b ........... 3.15 |==================== c ........... 3.14 |==================== f ........... 3.16 |==================== g ........... 3.15 |==================== i ........... 3.30 |===================== 4080 ........ 3.31 |===================== 4080 rep .... 3.27 |===================== 4080 xxx .... 3.30 |===================== 4080 zzz .... 3.28 |===================== 3090 ........ 3.14 |==================== 3090 rep .... 3.17 |==================== 3070 ........ 9.19 |=========================================================== RTX 3070 Ti . 3.41 |====================== 4090 ........ 3.46 |====================== 4090 rep .... 3.45 |====================== nv 4090 ..... 3.39 |====================== NCNN 20230517 Target: Vulkan GPU - Model: blazeface ms < Lower Is Better a ........... 1.38 |==================== b ........... 1.37 |==================== c ........... 1.37 |==================== d ........... 1.38 |==================== e ........... 1.38 |==================== f ........... 1.38 |==================== g ........... 1.41 |===================== i ........... 1.40 |===================== 4080 ........ 1.41 |===================== 4080 rep .... 1.41 |===================== 4080 xxx .... 1.42 |===================== 4080 zzz .... 1.42 |===================== 3090 ........ 1.39 |===================== 3090 rep .... 1.37 |==================== 3070 ........ 3.98 |=========================================================== RTX 3070 Ti . 1.60 |======================== 4090 ........ 1.39 |===================== 4090 rep .... 1.40 |===================== nv 4090 ..... 1.40 |===================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mnasnet ms < Lower Is Better 3090 rep .... 2.96 |==================== 3070 ........ 8.55 |=========================================================== RTX 3070 Ti . 3.12 |====================== 4090 ........ 3.23 |====================== 4090 rep .... 3.12 |====================== nv 4090 ..... 3.10 |===================== NCNN 20230517 Target: CPU - Model: mobilenet ms < Lower Is Better 4090 rep .... 10.75 |=========================== a ........... 8.05 |===================== b ........... 7.97 |==================== c ........... 8.02 |==================== d ........... 8.10 |===================== f ........... 8.45 |====================== g ........... 22.74 |========================================================== i ........... 8.37 |===================== 4080 ........ 8.73 |====================== 4080 rep .... 8.41 |===================== 4080 xxx .... 8.37 |===================== 4080 zzz .... 8.38 |===================== 3090 ........ 8.60 |====================== 3090 rep .... 8.01 |==================== 3070 ........ 17.81 |============================================= RTX 3070 Ti . 9.43 |======================== 4090 ........ 10.08 |========================== 4090 rep .... 8.43 |====================== nv 4090 ..... 8.15 |===================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: alexnet ms < Lower Is Better 4080 ........ 4.98 |======================== 4080 rep .... 4.64 |======================= 4080 xxx .... 4.71 |======================= 4080 zzz .... 4.70 |======================= 3090 ........ 4.32 |===================== 3090 rep .... 4.30 |===================== 3070 ........ 11.89 |========================================================== RTX 3070 Ti . 5.34 |========================== 4090 ........ 4.67 |======================= 4090 rep .... 5.25 |========================== nv 4090 ..... 4.69 |======================= NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mnasnet ms < Lower Is Better 3090 ........ 2.97 |====================== 3090 rep .... 2.98 |====================== 3070 ........ 8.15 |=========================================================== RTX 3070 Ti . 3.25 |======================== 4090 ........ 3.19 |======================= 4090 rep .... 3.28 |======================== nv 4090 ..... 3.10 |====================== VkFFT 1.2.31 Test: FFT + iFFT C2C Bluestein in single precision Benchmark Score > Higher Is Better a ........... 11340 |================================ b ........... 11273 |================================ c ........... 11311 |================================ d ........... 10719 |============================== e ........... 10560 |============================== f ........... 7571 |===================== g ........... 7574 |===================== h ........... 7622 |===================== i ........... 10061 |============================ 4080 ........ 17121 |================================================ 4080 rep .... 17287 |================================================= 4080 xxx .... 17343 |================================================= 4080 zzz .... 17185 |================================================ 3090 ........ 14406 |========================================= 3090 rep .... 14449 |========================================= 4090 ........ 20373 |========================================================= 4090 rep .... 20404 |========================================================= nv 4090 ..... 20601 |========================================================== NCNN 20230517 Target: CPU - Model: resnet18 ms < Lower Is Better 4090 rep .... 6.77 |============================ a ........... 5.29 |====================== b ........... 5.20 |===================== c ........... 5.24 |====================== d ........... 5.23 |====================== f ........... 5.69 |======================== g ........... 5.55 |======================= i ........... 5.60 |======================= 4080 ........ 5.69 |======================== 4080 rep .... 5.68 |======================= 4080 xxx .... 5.56 |======================= 4080 zzz .... 5.63 |======================= 3090 ........ 5.21 |====================== 3090 rep .... 5.29 |====================== 3070 ........ 14.03 |========================================================== RTX 3070 Ti . 6.08 |========================= 4090 ........ 6.00 |========================= 4090 rep .... 8.07 |================================= nv 4090 ..... 7.82 |================================ NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd ms < Lower Is Better 4080 ........ 7.71 |======================== 4080 rep .... 7.62 |======================= 4080 xxx .... 7.27 |====================== 4080 zzz .... 8.06 |========================= 3090 ........ 7.05 |====================== 3090 rep .... 7.08 |====================== 3070 ........ 18.83 |========================================================== RTX 3070 Ti . 8.65 |=========================== 4090 ........ 9.81 |============================== 4090 rep .... 9.46 |============================= nv 4090 ..... 7.02 |====================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: alexnet ms < Lower Is Better 3090 rep .... 4.31 |====================== 3070 ........ 11.43 |========================================================== RTX 3070 Ti . 6.25 |================================ 4090 ........ 5.14 |========================== 4090 rep .... 5.34 |=========================== nv 4090 ..... 6.32 |================================ NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better 4090 ........ 3.41 |======================== g ........... 3.17 |====================== i ........... 3.29 |======================= 4080 ........ 3.28 |======================= 4080 rep .... 3.27 |======================= 4080 xxx .... 3.27 |======================= 4080 zzz .... 3.28 |======================= 3090 ........ 3.15 |====================== 3090 rep .... 3.19 |======================= 3070 ........ 8.35 |=========================================================== RTX 3070 Ti . 3.66 |========================== 4090 ........ 3.32 |======================= 4090 rep .... 3.34 |======================== nv 4090 ..... 5.10 |==================================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: googlenet ms < Lower Is Better g ........... 7.94 |====================== b ........... 7.97 |====================== c ........... 7.83 |====================== f ........... 7.94 |====================== g ........... 8.35 |======================= i ........... 10.19 |============================= 4080 ........ 8.49 |======================== 4080 rep .... 8.58 |======================== 4080 xxx .... 8.99 |========================= 4080 zzz .... 8.37 |======================= 3090 ........ 7.82 |====================== 3090 rep .... 7.85 |====================== 3070 ........ 20.72 |========================================================== RTX 3070 Ti . 9.86 |============================ 4090 ........ 8.87 |========================= 4090 rep .... 8.90 |========================= nv 4090 ..... 10.01 |============================ NCNN 20230517 Target: Vulkan GPU - Model: mobilenet ms < Lower Is Better a ........... 8.05 |====================== b ........... 8.04 |====================== c ........... 8.03 |====================== d ........... 8.02 |====================== e ........... 8.04 |====================== f ........... 8.27 |======================= g ........... 8.17 |====================== i ........... 10.40 |============================= 4080 ........ 8.44 |======================= 4080 rep .... 8.57 |======================== 4080 xxx .... 8.37 |======================= 4080 zzz .... 8.47 |======================= 3090 ........ 8.07 |====================== 3090 rep .... 8.03 |====================== 3070 ........ 21.11 |========================================================== RTX 3070 Ti . 9.35 |========================== 4090 ........ 10.55 |============================= 4090 rep .... 10.23 |============================ nv 4090 ..... 8.45 |======================= NCNN 20230517 Target: CPU-v3-v3-v3 - Model: blazeface ms < Lower Is Better b ........... 1.37 |=========================== c ........... 1.38 |=========================== f ........... 1.43 |============================ g ........... 1.37 |=========================== i ........... 1.28 |========================= 4080 ........ 1.44 |============================ 4080 rep .... 1.42 |============================ 4080 xxx .... 1.42 |============================ 4080 zzz .... 1.41 |=========================== 3090 ........ 1.36 |========================== 3090 rep .... 1.37 |=========================== 3070 ........ 3.03 |=========================================================== RTX 3070 Ti . 1.79 |=================================== 4090 ........ 1.17 |======================= 4090 rep .... 1.34 |========================== nv 4090 ..... 1.16 |======================= NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: blazeface ms < Lower Is Better 3090 ........ 1.39 |======================= 3090 rep .... 1.38 |======================= 3070 ........ 3.57 |=========================================================== RTX 3070 Ti . 1.51 |========================= 4090 ........ 1.45 |======================== 4090 rep .... 1.38 |======================= nv 4090 ..... 2.91 |================================================ NCNN 20230517 Target: CPU-v3-v3-v3 - Model: vision_transformer ms < Lower Is Better b ........... 31.65 |====================== c ........... 31.78 |======================= f ........... 33.47 |======================== g ........... 33.39 |======================== i ........... 36.55 |========================== 4080 ........ 34.20 |======================== 4080 rep .... 34.27 |======================== 4080 xxx .... 34.37 |======================== 4080 zzz .... 34.47 |======================== 3090 ........ 31.89 |======================= 3090 rep .... 31.93 |======================= 3070 ........ 81.77 |========================================================== RTX 3070 Ti . 37.86 |=========================== 4090 ........ 38.79 |============================ 4090 rep .... 39.12 |============================ nv 4090 ..... 38.99 |============================ NCNN 20230517 Target: CPU-v3-v3-v3 - Model: alexnet ms < Lower Is Better b ........... 4.29 |======================= c ........... 4.28 |======================= f ........... 4.83 |========================= g ........... 4.71 |========================= i ........... 5.01 |========================== 4080 ........ 4.66 |========================= 4080 rep .... 4.65 |========================= 4080 xxx .... 4.65 |========================= 4080 zzz .... 4.67 |========================= 3090 ........ 4.31 |======================= 3090 rep .... 4.30 |======================= 3070 ........ 11.00 |========================================================== RTX 3070 Ti . 5.55 |============================= 4090 ........ 4.94 |========================== 4090 rep .... 5.27 |============================ nv 4090 ..... 4.67 |========================= NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet18 ms < Lower Is Better 3090 ........ 5.20 |======================= 3090 rep .... 5.20 |======================= 3070 ........ 13.34 |========================================================== RTX 3070 Ti . 6.69 |============================= 4090 ........ 7.78 |================================== 4090 rep .... 5.84 |========================= nv 4090 ..... 5.97 |========================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: resnet18 ms < Lower Is Better g ........... 5.30 |======================= b ........... 5.42 |======================= c ........... 5.23 |======================= f ........... 5.30 |======================= g ........... 5.50 |======================== i ........... 5.85 |========================= 4080 ........ 5.65 |======================== 4080 rep .... 5.69 |========================= 4080 xxx .... 5.89 |========================== 4080 zzz .... 5.59 |======================== 3090 ........ 5.23 |======================= 3090 rep .... 5.24 |======================= 3070 ........ 13.38 |========================================================== RTX 3070 Ti . 6.23 |=========================== 4090 ........ 5.97 |========================== 4090 rep .... 5.81 |========================= nv 4090 ..... 5.84 |========================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0 ms < Lower Is Better 4080 ........ 4.04 |======================== 4080 rep .... 4.01 |======================== 4080 xxx .... 3.97 |======================== 4080 zzz .... 4.05 |======================== 3090 ........ 3.88 |======================= 3090 rep .... 3.85 |======================= 3070 ........ 9.81 |=========================================================== RTX 3070 Ti . 4.37 |========================== 4090 ........ 4.15 |========================= 4090 rep .... 4.41 |=========================== nv 4090 ..... 4.10 |========================= NCNN 20230517 Target: CPU-v3-v3-v3 - Model: squeezenet_ssd ms < Lower Is Better b ........... 7.03 |======================= c ........... 7.04 |======================= f ........... 6.97 |======================= g ........... 7.26 |======================== i ........... 8.16 |=========================== 4080 ........ 7.64 |========================= 4080 rep .... 7.59 |========================= 4080 xxx .... 7.64 |========================= 4080 zzz .... 7.35 |======================== 3090 ........ 7.04 |======================= 3090 rep .... 7.09 |======================= 3070 ........ 17.75 |========================================================== RTX 3070 Ti . 8.39 |=========================== 4090 ........ 9.32 |============================== 4090 rep .... 9.30 |============================== nv 4090 ..... 9.37 |=============================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: FastestDet ms < Lower Is Better 4080 ........ 4.19 |===================================== 4080 rep .... 4.17 |===================================== 4080 xxx .... 3.75 |================================= 4080 zzz .... 4.61 |========================================= 3090 ........ 3.83 |================================== 3090 rep .... 4.07 |==================================== 3070 ........ 6.71 |=========================================================== RTX 3070 Ti . 4.32 |====================================== 4090 ........ 4.45 |======================================= 4090 rep .... 4.11 |==================================== nv 4090 ..... 2.64 |======================= NCNN 20230517 Target: Vulkan GPU - Model: alexnet ms < Lower Is Better a ........... 4.31 |======================= b ........... 4.33 |======================= c ........... 4.33 |======================= d ........... 4.31 |======================= e ........... 4.31 |======================= f ........... 4.64 |========================= g ........... 4.87 |========================== i ........... 6.53 |=================================== 4080 ........ 4.62 |========================= 4080 rep .... 4.65 |========================= 4080 xxx .... 4.68 |========================= 4080 zzz .... 4.68 |========================= 3090 ........ 4.35 |======================= 3090 rep .... 4.31 |======================= 3070 ........ 10.88 |========================================================== RTX 3070 Ti . 5.25 |============================ 4090 ........ 4.64 |========================= 4090 rep .... 6.79 |==================================== nv 4090 ..... 5.20 |============================ NCNN 20230517 Target: CPU - Model: mnasnet ms < Lower Is Better 4090 rep .... 3.18 |=========================== a ........... 2.97 |========================= b ........... 2.97 |========================= c ........... 2.99 |========================== d ........... 2.98 |========================== f ........... 2.97 |========================= g ........... 3.00 |========================== i ........... 2.74 |======================= 4080 ........ 3.05 |========================== 4080 rep .... 3.06 |========================== 4080 xxx .... 3.06 |========================== 4080 zzz .... 3.04 |========================== 3090 ........ 2.99 |========================== 3090 rep .... 2.97 |========================= 3070 ........ 6.88 |=========================================================== RTX 3070 Ti . 3.26 |============================ 4090 ........ 3.12 |=========================== 4090 rep .... 3.22 |============================ nv 4090 ..... 4.77 |========================================= NCNN 20230517 Target: Vulkan GPU - Model: googlenet ms < Lower Is Better a ........... 7.90 |======================== b ........... 7.85 |======================= c ........... 7.80 |======================= d ........... 7.85 |======================= e ........... 7.85 |======================= f ........... 8.15 |======================== g ........... 8.96 |=========================== i ........... 8.75 |========================== 4080 ........ 8.42 |========================= 4080 rep .... 8.40 |========================= 4080 xxx .... 8.42 |========================= 4080 zzz .... 8.41 |========================= 3090 ........ 7.87 |======================= 3090 rep .... 7.82 |======================= 3070 ........ 19.49 |========================================================== RTX 3070 Ti . 9.87 |============================= 4090 ........ 10.62 |================================ 4090 rep .... 10.65 |================================ nv 4090 ..... 8.70 |========================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: FastestDet ms < Lower Is Better 4080 rep .... 4.20 |=================================== 4080 xxx .... 3.80 |=============================== 4080 zzz .... 3.82 |================================ 3090 ........ 4.10 |================================== 3090 rep .... 4.07 |================================== 3070 ........ 7.12 |=========================================================== RTX 3070 Ti . 4.18 |=================================== 4090 ........ 2.85 |======================== 4090 rep .... 3.12 |========================== nv 4090 ..... 3.93 |================================= NCNN 20230517 Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better a ........... 3.17 |======================== b ........... 3.14 |======================== c ........... 3.13 |======================== d ........... 3.16 |======================== e ........... 3.14 |======================== f ........... 3.13 |======================== g ........... 3.18 |======================== i ........... 3.29 |========================= 4080 ........ 3.26 |========================= 4080 rep .... 3.29 |========================= 4080 xxx .... 3.26 |========================= 4080 zzz .... 3.29 |========================= 3090 ........ 3.18 |======================== 3090 rep .... 3.17 |======================== 3070 ........ 7.81 |=========================================================== RTX 3070 Ti . 3.56 |=========================== 4090 ........ 3.30 |========================= 4090 rep .... 3.31 |========================= nv 4090 ..... 3.43 |========================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: alexnet ms < Lower Is Better 3090 ........ 4.30 |======================= 3090 rep .... 4.30 |======================= 3070 ........ 10.69 |========================================================== RTX 3070 Ti . 5.41 |============================= 4090 ........ 4.94 |=========================== 4090 rep .... 5.16 |============================ nv 4090 ..... 6.54 |=================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0 ms < Lower Is Better 3090 ........ 3.86 |======================== 3090 rep .... 3.85 |======================== 3070 ........ 9.53 |=========================================================== RTX 3070 Ti . 4.55 |============================ 4090 ........ 4.14 |========================== 4090 rep .... 4.44 |=========================== nv 4090 ..... 5.88 |==================================== NCNN 20230517 Target: CPU-v3-v3-v3 - Model: regnety_400m ms < Lower Is Better b ........... 8.05 |======================== c ........... 8.14 |======================== f ........... 8.34 |========================= g ........... 8.07 |======================== i ........... 8.21 |======================== 4080 ........ 8.33 |========================= 4080 rep .... 8.57 |========================= 4080 xxx .... 8.75 |========================== 4080 zzz .... 8.47 |========================= 3090 ........ 7.95 |======================= 3090 rep .... 8.09 |======================== 3070 ........ 19.66 |========================================================== RTX 3070 Ti . 9.02 |=========================== 4090 ........ 8.13 |======================== 4090 rep .... 17.15 |=================================================== nv 4090 ..... 9.55 |============================ NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: FastestDet ms < Lower Is Better 4090 ........ 4.13 |=================================== g ........... 3.92 |================================= i ........... 3.83 |================================= 4080 ........ 4.28 |==================================== 4080 rep .... 4.14 |=================================== 4080 xxx .... 4.17 |==================================== 4080 zzz .... 4.20 |==================================== 3090 ........ 4.03 |================================== 3090 rep .... 4.08 |=================================== 3070 ........ 6.93 |=========================================================== RTX 3070 Ti . 4.41 |====================================== 4090 ........ 4.03 |================================== 4090 rep .... 3.96 |================================== nv 4090 ..... 2.81 |======================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: alexnet ms < Lower Is Better 4080 rep .... 4.68 |========================== 4080 xxx .... 4.69 |========================== 4080 zzz .... 4.66 |========================== 3090 ........ 4.33 |======================== 3090 rep .... 4.31 |======================== 3070 ........ 10.59 |========================================================== RTX 3070 Ti . 6.17 |================================== 4090 ........ 4.99 |=========================== 4090 rep .... 5.45 |============================== nv 4090 ..... 6.62 |==================================== NCNN 20230517 Target: CPU-v3-v3-v3 - Model: shufflenet-v2 ms < Lower Is Better b ........... 3.33 |======================== c ........... 3.34 |======================== f ........... 3.40 |========================= g ........... 3.38 |========================= i ........... 3.52 |========================== 4080 ........ 3.46 |========================= 4080 rep .... 3.43 |========================= 4080 xxx .... 3.47 |========================= 4080 zzz .... 3.43 |========================= 3090 ........ 3.32 |======================== 3090 rep .... 3.33 |======================== 3070 ........ 8.13 |=========================================================== RTX 3070 Ti . 4.09 |============================== 4090 ........ 5.18 |====================================== 4090 rep .... 3.59 |========================== nv 4090 ..... 3.46 |========================= NCNN 20230517 Target: Vulkan GPU - Model: resnet18 ms < Lower Is Better a ........... 5.28 |======================== b ........... 5.23 |======================== c ........... 5.21 |======================== d ........... 5.23 |======================== e ........... 5.22 |======================== f ........... 5.48 |========================= g ........... 6.22 |============================ i ........... 5.82 |=========================== 4080 ........ 5.67 |========================== 4080 rep .... 5.61 |========================== 4080 xxx .... 5.62 |========================== 4080 zzz .... 5.59 |========================== 3090 ........ 5.27 |======================== 3090 rep .... 5.20 |======================== 3070 ........ 12.68 |========================================================== RTX 3070 Ti . 6.28 |============================= 4090 ........ 5.69 |========================== 4090 rep .... 6.01 |=========================== nv 4090 ..... 7.44 |================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: resnet18 ms < Lower Is Better 4080 rep .... 5.64 |========================== 4080 xxx .... 5.78 |=========================== 4080 zzz .... 5.77 |========================== 3090 ........ 5.20 |======================== 3090 rep .... 5.30 |======================== 3070 ........ 12.64 |========================================================== RTX 3070 Ti . 6.22 |============================= 4090 ........ 7.74 |==================================== 4090 rep .... 8.14 |===================================== nv 4090 ..... 6.07 |============================ NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: googlenet ms < Lower Is Better 4080 rep .... 8.52 |========================== 4080 xxx .... 8.32 |========================= 4080 zzz .... 8.29 |========================= 3090 ........ 7.90 |======================== 3090 rep .... 7.91 |======================== 3070 ........ 19.20 |========================================================== RTX 3070 Ti . 9.68 |============================= 4090 ........ 8.38 |========================= 4090 rep .... 8.97 |=========================== nv 4090 ..... 10.75 |================================ NCNN 20230517 Target: CPU - Model: vgg16 ms < Lower Is Better 4090 rep .... 29.24 |============================== a ........... 23.75 |======================== b ........... 23.49 |======================== c ........... 23.45 |======================== d ........... 23.51 |======================== f ........... 24.19 |========================= g ........... 23.78 |======================== i ........... 30.96 |================================ 4080 ........ 25.37 |========================== 4080 rep .... 26.11 |=========================== 4080 xxx .... 25.03 |========================== 4080 zzz .... 25.40 |========================== 3090 ........ 23.43 |======================== 3090 rep .... 23.43 |======================== 3070 ........ 56.64 |========================================================== RTX 3070 Ti . 28.36 |============================= 4090 ........ 27.75 |============================ 4090 rep .... 28.19 |============================= nv 4090 ..... 29.54 |============================== NCNN 20230517 Target: CPU-v3-v3-v3 - Model: resnet50 ms < Lower Is Better b ........... 10.01 |======================== c ........... 10.33 |========================= f ........... 11.05 |=========================== g ........... 11.25 |=========================== i ........... 14.05 |================================== 4080 ........ 11.11 |=========================== 4080 rep .... 10.79 |========================== 4080 xxx .... 10.91 |========================== 4080 zzz .... 11.21 |=========================== 3090 ........ 10.05 |======================== 3090 rep .... 10.07 |======================== 3070 ........ 24.07 |========================================================== RTX 3070 Ti . 12.11 |============================= 4090 ........ 11.39 |=========================== 4090 rep .... 13.82 |================================= nv 4090 ..... 13.68 |================================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2 ms < Lower Is Better 4090 ........ 5.17 |====================================== g ........... 3.34 |========================= i ........... 3.43 |========================= 4080 ........ 3.46 |========================== 4080 rep .... 3.39 |========================= 4080 xxx .... 3.43 |========================= 4080 zzz .... 3.43 |========================= 3090 ........ 3.33 |========================= 3090 rep .... 3.37 |========================= 3070 ........ 8.00 |=========================================================== RTX 3070 Ti . 3.75 |============================ 4090 ........ 5.09 |====================================== 4090 rep .... 5.27 |======================================= nv 4090 ..... 3.51 |========================== NCNN 20230517 Target: CPU - Model: efficientnet-b0 ms < Lower Is Better 4090 rep .... 4.30 |=========================== a ........... 3.90 |========================= b ........... 3.85 |========================= c ........... 3.88 |========================= d ........... 3.85 |========================= f ........... 3.87 |========================= g ........... 3.91 |========================= i ........... 4.05 |========================== 4080 ........ 3.99 |========================== 4080 rep .... 4.09 |========================== 4080 xxx .... 4.04 |========================== 4080 zzz .... 3.99 |========================== 3090 ........ 3.87 |========================= 3090 rep .... 3.86 |========================= 3070 ........ 9.23 |=========================================================== RTX 3070 Ti . 4.72 |============================== 4090 ........ 4.23 |=========================== 4090 rep .... 4.03 |========================== nv 4090 ..... 4.37 |============================ NCNN 20230517 Target: CPU-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better 4090 rep .... 3.31 |========================== a ........... 3.18 |========================= c ........... 3.17 |========================= d ........... 3.17 |========================= g ........... 3.14 |========================= i ........... 3.26 |========================== 4080 ........ 3.24 |========================= 4080 rep .... 3.26 |========================== 4080 xxx .... 3.27 |========================== 4080 zzz .... 3.24 |========================= 3090 ........ 3.18 |========================= 3090 rep .... 3.19 |========================= 3070 ........ 7.52 |=========================================================== RTX 3070 Ti . 3.61 |============================ 4090 ........ 3.53 |============================ 4090 rep .... 3.41 |=========================== nv 4090 ..... 3.47 |=========================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: googlenet ms < Lower Is Better 3090 rep .... 7.86 |======================== 3070 ........ 18.80 |========================================================== RTX 3070 Ti . 9.97 |=============================== 4090 ........ 8.55 |========================== 4090 rep .... 10.47 |================================ nv 4090 ..... 10.14 |=============================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0 ms < Lower Is Better 4080 rep .... 4.07 |========================== 4080 xxx .... 4.01 |========================== 4080 zzz .... 3.95 |========================= 3090 ........ 3.88 |========================= 3090 rep .... 3.85 |========================= 3070 ........ 9.19 |=========================================================== RTX 3070 Ti . 4.74 |============================== 4090 ........ 4.47 |============================= 4090 rep .... 4.35 |============================ nv 4090 ..... 5.94 |====================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: googlenet ms < Lower Is Better 3090 ........ 7.83 |======================== 3090 rep .... 7.82 |======================== 3070 ........ 18.66 |========================================================== RTX 3070 Ti . 9.58 |============================== 4090 ........ 8.91 |============================ 4090 rep .... 10.18 |================================ nv 4090 ..... 8.85 |============================ NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: blazeface ms < Lower Is Better 4080 ........ 1.44 |============================ 4080 rep .... 1.42 |============================ 4080 xxx .... 1.31 |========================== 4080 zzz .... 1.41 |============================ 3090 ........ 1.38 |=========================== 3090 rep .... 1.37 |=========================== 3070 ........ 2.99 |=========================================================== RTX 3070 Ti . 1.34 |========================== 4090 ........ 1.42 |============================ 4090 rep .... 1.41 |============================ nv 4090 ..... 1.26 |========================= NCNN 20230517 Target: CPU - Model: vision_transformer ms < Lower Is Better 4090 rep .... 37.05 |============================= a ........... 32.49 |========================= b ........... 31.95 |========================= c ........... 31.77 |======================== d ........... 32.43 |========================= f ........... 33.56 |========================== g ........... 32.73 |========================= i ........... 37.80 |============================= 4080 ........ 35.07 |=========================== 4080 rep .... 35.28 |=========================== 4080 xxx .... 34.27 |========================== 4080 zzz .... 34.10 |========================== 3090 ........ 33.01 |========================= 3090 rep .... 31.80 |======================== 3070 ........ 75.34 |========================================================== RTX 3070 Ti . 37.91 |============================= 4090 ........ 38.25 |============================= 4090 rep .... 37.81 |============================= nv 4090 ..... 38.90 |============================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: mnasnet ms < Lower Is Better g ........... 2.95 |============================= b ........... 2.96 |============================= c ........... 2.96 |============================= f ........... 2.96 |============================= g ........... 2.98 |============================= i ........... 2.99 |============================= 4080 ........ 3.09 |============================== 4080 rep .... 3.07 |============================== 4080 xxx .... 3.13 |=============================== 4080 zzz .... 3.01 |============================== 3090 ........ 2.97 |============================= 3090 rep .... 2.97 |============================= 3070 ........ 6.02 |=========================================================== RTX 3070 Ti . 3.37 |================================= 4090 ........ 3.00 |============================= 4090 rep .... 3.10 |============================== nv 4090 ..... 2.54 |========================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: googlenet ms < Lower Is Better 4090 ........ 11.30 |=================================== g ........... 7.96 |========================= i ........... 10.47 |================================= 4080 ........ 8.40 |========================== 4080 rep .... 8.49 |========================== 4080 xxx .... 8.43 |========================== 4080 zzz .... 8.42 |========================== 3090 ........ 7.86 |========================= 3090 rep .... 7.89 |========================= 3070 ........ 18.60 |========================================================== RTX 3070 Ti . 9.84 |=============================== 4090 ........ 10.87 |================================== 4090 rep .... 10.38 |================================ nv 4090 ..... 8.61 |=========================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: vgg16 ms < Lower Is Better g ........... 23.82 |========================= b ........... 23.42 |========================= c ........... 23.54 |========================= f ........... 24.12 |========================= g ........... 24.71 |========================== i ........... 29.07 |============================== 4080 ........ 25.04 |========================== 4080 rep .... 25.04 |========================== 4080 xxx .... 26.08 |=========================== 4080 zzz .... 25.26 |========================== 3090 ........ 23.50 |========================= 3090 rep .... 23.43 |========================= 3070 ........ 55.42 |========================================================== RTX 3070 Ti . 28.40 |============================== 4090 ........ 28.21 |============================== 4090 rep .... 27.59 |============================= nv 4090 ..... 27.77 |============================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: vgg16 ms < Lower Is Better 4090 ........ 29.05 |============================== g ........... 24.04 |========================= i ........... 27.43 |============================= 4080 ........ 25.48 |=========================== 4080 rep .... 25.05 |========================== 4080 xxx .... 25.40 |=========================== 4080 zzz .... 25.16 |========================== 3090 ........ 23.55 |========================= 3090 rep .... 23.52 |========================= 3070 ........ 55.48 |========================================================== RTX 3070 Ti . 28.63 |============================== 4090 ........ 31.57 |================================= 4090 rep .... 29.12 |============================== nv 4090 ..... 27.89 |============================= NCNN 20230517 Target: CPU-v3-v3-v3 - Model: efficientnet-b0 ms < Lower Is Better b ........... 3.82 |========================= c ........... 3.89 |========================= f ........... 4.04 |========================== g ........... 4.14 |=========================== i ........... 4.68 |=============================== 4080 ........ 4.02 |========================== 4080 rep .... 4.02 |========================== 4080 xxx .... 4.06 |=========================== 4080 zzz .... 4.03 |========================== 3090 ........ 3.83 |========================= 3090 rep .... 3.85 |========================= 3070 ........ 9.01 |=========================================================== RTX 3070 Ti . 4.78 |=============================== 4090 ........ 4.09 |=========================== 4090 rep .... 4.34 |============================ nv 4090 ..... 4.04 |========================== NCNN 20230517 Target: Vulkan GPU - Model: efficientnet-b0 ms < Lower Is Better a ........... 3.86 |========================= b ........... 3.82 |========================= c ........... 3.83 |========================= d ........... 3.87 |========================= e ........... 3.84 |========================= f ........... 3.86 |========================= g ........... 3.86 |========================= i ........... 5.88 |======================================= 4080 ........ 4.01 |========================== 4080 rep .... 4.05 |=========================== 4080 xxx .... 4.04 |=========================== 4080 zzz .... 4.04 |=========================== 3090 ........ 3.88 |========================= 3090 rep .... 3.86 |========================= 3070 ........ 8.99 |=========================================================== RTX 3070 Ti . 4.53 |============================== 4090 ........ 4.34 |============================ 4090 rep .... 4.09 |=========================== nv 4090 ..... 4.10 |=========================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2 ms < Lower Is Better 4080 rep .... 3.47 |========================== 4080 xxx .... 3.40 |========================== 4080 zzz .... 3.36 |========================= 3090 ........ 3.36 |========================= 3090 rep .... 3.33 |========================= 3070 ........ 7.81 |=========================================================== RTX 3070 Ti . 4.02 |============================== 4090 ........ 3.48 |========================== 4090 rep .... 3.42 |========================== nv 4090 ..... 3.32 |========================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: resnet50 ms < Lower Is Better 4090 ........ 11.53 |============================ g ........... 10.33 |========================= i ........... 13.10 |================================ 4080 ........ 11.40 |============================ 4080 rep .... 10.80 |=========================== 4080 xxx .... 10.91 |=========================== 4080 zzz .... 10.91 |=========================== 3090 ........ 10.38 |========================== 3090 rep .... 10.04 |========================= 3070 ........ 23.59 |========================================================== RTX 3070 Ti . 12.73 |=============================== 4090 ........ 14.08 |=================================== 4090 rep .... 12.17 |============================== nv 4090 ..... 13.63 |================================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet50 ms < Lower Is Better 4080 ........ 11.48 |============================ 4080 rep .... 11.07 |=========================== 4080 xxx .... 11.22 |============================ 4080 zzz .... 12.50 |=============================== 3090 ........ 10.07 |========================= 3090 rep .... 9.98 |========================= 3070 ........ 23.44 |========================================================== RTX 3070 Ti . 12.42 |=============================== 4090 ........ 12.40 |=============================== 4090 rep .... 11.51 |============================ nv 4090 ..... 13.13 |================================ NCNN 20230517 Target: Vulkan GPU - Model: resnet50 ms < Lower Is Better a ........... 10.01 |========================= b ........... 10.00 |========================= c ........... 10.00 |========================= d ........... 10.10 |========================= e ........... 10.10 |========================= f ........... 10.26 |========================= g ........... 10.34 |========================== i ........... 12.96 |================================ 4080 ........ 10.81 |=========================== 4080 rep .... 10.84 |=========================== 4080 xxx .... 10.94 |=========================== 4080 zzz .... 11.07 |=========================== 3090 ........ 10.10 |========================= 3090 rep .... 10.06 |========================= 3070 ........ 23.48 |========================================================== RTX 3070 Ti . 12.60 |=============================== 4090 ........ 14.13 |=================================== 4090 rep .... 13.08 |================================ nv 4090 ..... 12.45 |=============================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet50 ms < Lower Is Better 3090 ........ 10.03 |========================= 3090 rep .... 10.04 |========================= 3070 ........ 23.54 |========================================================== RTX 3070 Ti . 12.52 |=============================== 4090 ........ 12.98 |================================ 4090 rep .... 11.24 |============================ nv 4090 ..... 13.46 |================================= NCNN 20230517 Target: CPU - Model: blazeface ms < Lower Is Better a ........... 1.38 |=========================== b ........... 1.38 |=========================== c ........... 1.39 |============================ d ........... 1.38 |=========================== f ........... 1.37 |=========================== g ........... 1.38 |=========================== i ........... 1.40 |============================ 4080 ........ 1.40 |============================ 4080 rep .... 1.42 |============================ 4080 xxx .... 1.42 |============================ 4080 zzz .... 1.40 |============================ 3090 ........ 1.38 |=========================== 3090 rep .... 1.38 |=========================== 3070 ........ 2.98 |=========================================================== RTX 3070 Ti . 1.60 |================================ 4090 ........ 1.27 |========================= 4090 rep .... 1.45 |============================= nv 4090 ..... 1.33 |========================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: resnet50 ms < Lower Is Better g ........... 10.18 |========================== b ........... 9.87 |========================= c ........... 10.03 |========================= f ........... 10.25 |========================== g ........... 10.43 |========================== i ........... 11.15 |============================ 4080 ........ 10.95 |=========================== 4080 rep .... 10.84 |=========================== 4080 xxx .... 11.50 |============================= 4080 zzz .... 11.09 |============================ 3090 ........ 9.97 |========================= 3090 rep .... 10.01 |========================= 3070 ........ 23.11 |========================================================== RTX 3070 Ti . 12.35 |=============================== 4090 ........ 14.58 |===================================== 4090 rep .... 13.57 |================================== nv 4090 ..... 13.13 |================================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: resnet18 ms < Lower Is Better 4090 ........ 5.78 |============================ g ........... 5.28 |========================= i ........... 5.88 |============================ 4080 ........ 5.61 |=========================== 4080 rep .... 5.61 |=========================== 4080 xxx .... 5.67 |=========================== 4080 zzz .... 5.60 |=========================== 3090 ........ 5.19 |========================= 3090 rep .... 5.22 |========================= 3070 ........ 12.14 |========================================================== RTX 3070 Ti . 6.57 |=============================== 4090 ........ 6.58 |=============================== 4090 rep .... 6.05 |============================= nv 4090 ..... 8.16 |======================================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: FastestDet ms < Lower Is Better g ........... 4.06 |============================ b ........... 4.06 |============================ c ........... 3.69 |========================= f ........... 4.20 |============================= g ........... 3.97 |=========================== i ........... 4.43 |============================== 4080 ........ 4.20 |============================= 4080 rep .... 4.09 |============================ 4080 xxx .... 4.31 |============================= 4080 zzz .... 4.12 |============================ 3090 ........ 4.04 |============================ 3090 rep .... 4.10 |============================ 3070 ........ 8.63 |=========================================================== RTX 3070 Ti . 4.26 |============================= 4090 ........ 3.94 |=========================== 4090 rep .... 4.16 |============================ nv 4090 ..... 5.92 |======================================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: alexnet ms < Lower Is Better 4090 ........ 4.72 |=========================== g ........... 4.35 |========================= i ........... 5.10 |============================= 4080 ........ 4.61 |=========================== 4080 rep .... 4.72 |=========================== 4080 xxx .... 4.67 |=========================== 4080 zzz .... 4.68 |=========================== 3090 ........ 4.31 |========================= 3090 rep .... 4.31 |========================= 3070 ........ 10.08 |========================================================== RTX 3070 Ti . 5.53 |================================ 4090 ........ 5.14 |============================== 4090 rep .... 5.33 |=============================== nv 4090 ..... 5.14 |============================== NCNN 20230517 Target: CPU - Model: googlenet ms < Lower Is Better 4090 rep .... 10.86 |=================================== a ........... 7.94 |========================= b ........... 7.82 |========================= c ........... 7.93 |========================= d ........... 7.85 |========================= f ........... 7.92 |========================= g ........... 7.98 |========================= i ........... 10.30 |================================= 4080 ........ 8.42 |=========================== 4080 rep .... 8.52 |=========================== 4080 xxx .... 8.38 |=========================== 4080 zzz .... 8.40 |=========================== 3090 ........ 7.86 |========================= 3090 rep .... 7.90 |========================= 3070 ........ 18.25 |========================================================== RTX 3070 Ti . 9.69 |=============================== 4090 ........ 10.27 |================================= 4090 rep .... 9.53 |============================== nv 4090 ..... 8.93 |============================ NCNN 20230517 Target: CPU-v3-v3-v3 - Model: mnasnet ms < Lower Is Better b ........... 2.97 |========================== c ........... 2.97 |========================== f ........... 3.12 |=========================== g ........... 3.05 |========================== i ........... 3.39 |============================= 4080 ........ 3.08 |========================== 4080 rep .... 3.09 |=========================== 4080 xxx .... 3.07 |========================== 4080 zzz .... 3.06 |========================== 3090 ........ 2.95 |========================= 3090 rep .... 2.96 |========================= 3070 ........ 6.87 |=========================================================== RTX 3070 Ti . 3.11 |=========================== 4090 ........ 3.19 |=========================== 4090 rep .... 3.23 |============================ nv 4090 ..... 4.61 |======================================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: vision_transformer ms < Lower Is Better g ........... 32.38 |========================== b ........... 31.71 |========================= c ........... 31.66 |========================= f ........... 33.36 |========================== g ........... 32.68 |========================== i ........... 38.33 |============================== 4080 ........ 34.13 |=========================== 4080 rep .... 34.29 |=========================== 4080 xxx .... 35.40 |============================ 4080 zzz .... 34.05 |=========================== 3090 ........ 31.86 |========================= 3090 rep .... 32.11 |========================= 3070 ........ 73.51 |========================================================== RTX 3070 Ti . 38.29 |============================== 4090 ........ 39.01 |=============================== 4090 rep .... 38.73 |=============================== nv 4090 ..... 38.58 |============================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny ms < Lower Is Better g ........... 13.14 |========================== i ........... 13.77 |=========================== 4080 ........ 13.86 |=========================== 4080 rep .... 13.55 |========================== 4080 xxx .... 13.62 |=========================== 4080 zzz .... 13.61 |========================== 3090 ........ 13.10 |========================= 3090 rep .... 12.86 |========================= 3070 ........ 29.80 |========================================================== RTX 3070 Ti . 15.21 |============================== 4090 ........ 15.55 |============================== 4090 rep .... 15.45 |============================== nv 4090 ..... 17.30 |================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet ms < Lower Is Better 3090 ........ 8.01 |========================= 3090 rep .... 8.05 |========================= 3070 ........ 18.54 |========================================================== RTX 3070 Ti . 9.98 |=============================== 4090 ........ 8.46 |========================== 4090 rep .... 8.83 |============================ nv 4090 ..... 10.15 |================================ NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: yolov4-tiny ms < Lower Is Better g ........... 12.89 |========================= b ........... 12.77 |========================= c ........... 12.86 |========================= f ........... 14.34 |============================ g ........... 13.35 |========================== i ........... 15.43 |============================== 4080 ........ 13.79 |=========================== 4080 rep .... 13.68 |=========================== 4080 xxx .... 13.95 |=========================== 4080 zzz .... 13.62 |=========================== 3090 ........ 12.88 |========================= 3090 rep .... 12.84 |========================= 3070 ........ 29.49 |========================================================== RTX 3070 Ti . 15.44 |============================== 4090 ........ 15.44 |============================== 4090 rep .... 16.39 |================================ nv 4090 ..... 16.61 |================================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better g ........... 3.14 |========================== b ........... 3.15 |========================== c ........... 3.15 |========================== f ........... 3.16 |========================== g ........... 3.17 |========================== i ........... 3.28 |=========================== 4080 ........ 3.29 |=========================== 4080 rep .... 3.30 |=========================== 4080 xxx .... 3.40 |============================ 4080 zzz .... 3.23 |========================== 3090 ........ 3.16 |========================== 3090 rep .... 3.17 |========================== 3070 ........ 7.24 |=========================================================== RTX 3070 Ti . 3.66 |============================== 4090 ........ 4.99 |========================================= 4090 rep .... 3.31 |=========================== nv 4090 ..... 4.45 |==================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet18 ms < Lower Is Better 3090 rep .... 5.27 |========================= 3070 ........ 12.13 |========================================================== RTX 3070 Ti . 5.94 |============================ 4090 ........ 6.96 |================================= 4090 rep .... 5.87 |============================ nv 4090 ..... 5.58 |=========================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better 3090 rep .... 3.19 |========================== 3070 ........ 7.34 |=========================================================== RTX 3070 Ti . 3.24 |========================== 4090 ........ 3.36 |=========================== 4090 rep .... 3.35 |=========================== nv 4090 ..... 4.96 |======================================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: squeezenet_ssd ms < Lower Is Better g ........... 7.19 |========================== b ........... 7.14 |========================== c ........... 7.07 |========================= f ........... 7.09 |========================= g ........... 7.31 |========================== i ........... 8.33 |============================== 4080 ........ 7.66 |============================ 4080 rep .... 7.63 |=========================== 4080 xxx .... 7.70 |============================ 4080 zzz .... 7.51 |=========================== 3090 ........ 7.04 |========================= 3090 rep .... 7.08 |========================= 3070 ........ 16.15 |========================================================== RTX 3070 Ti . 8.29 |============================== 4090 ........ 7.93 |============================ 4090 rep .... 7.81 |============================ nv 4090 ..... 7.72 |============================ NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: alexnet ms < Lower Is Better g ........... 4.35 |========================== b ........... 4.42 |========================== c ........... 4.30 |========================== f ........... 4.35 |========================== g ........... 4.86 |============================= i ........... 4.99 |============================== 4080 ........ 4.69 |============================ 4080 rep .... 4.69 |============================ 4080 xxx .... 5.21 |=============================== 4080 zzz .... 4.65 |============================ 3090 ........ 4.30 |========================== 3090 rep .... 4.30 |========================== 3070 ........ 9.86 |=========================================================== RTX 3070 Ti . 5.67 |================================== 4090 ........ 6.11 |===================================== 4090 rep .... 6.58 |======================================= nv 4090 ..... 6.11 |===================================== NCNN 20230517 Target: CPU-v3-v3-v3 - Model: yolov4-tiny ms < Lower Is Better b ........... 12.98 |========================== c ........... 12.89 |========================= f ........... 13.07 |========================== g ........... 13.08 |========================== i ........... 15.11 |============================== 4080 ........ 13.81 |=========================== 4080 rep .... 13.55 |=========================== 4080 xxx .... 13.69 |=========================== 4080 zzz .... 13.83 |=========================== 3090 ........ 12.87 |========================= 3090 rep .... 12.82 |========================= 3070 ........ 29.34 |========================================================== RTX 3070 Ti . 15.42 |============================== 4090 ........ 15.30 |============================== 4090 rep .... 15.34 |============================== nv 4090 ..... 15.62 |=============================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: mobilenet ms < Lower Is Better 4090 ........ 8.96 |============================ g ........... 8.50 |=========================== i ........... 10.02 |================================ 4080 ........ 8.84 |============================ 4080 rep .... 8.40 |========================== 4080 xxx .... 8.46 |=========================== 4080 zzz .... 8.46 |=========================== 3090 ........ 8.11 |========================== 3090 rep .... 8.04 |========================= 3070 ........ 18.39 |========================================================== RTX 3070 Ti . 9.62 |============================== 4090 ........ 10.18 |================================ 4090 rep .... 8.37 |========================== nv 4090 ..... 9.41 |============================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better 4080 rep .... 3.30 |=========================== 4080 xxx .... 3.20 |========================== 4080 zzz .... 3.16 |========================== 3090 ........ 3.16 |========================== 3090 rep .... 3.16 |========================== 3070 ........ 7.22 |=========================================================== RTX 3070 Ti . 3.91 |================================ 4090 ........ 3.36 |=========================== 4090 rep .... 3.44 |============================ nv 4090 ..... 3.29 |=========================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny ms < Lower Is Better 3090 rep .... 12.92 |========================== 3070 ........ 29.38 |========================================================== RTX 3070 Ti . 14.64 |============================= 4090 ........ 16.05 |================================ 4090 rep .... 15.41 |============================== nv 4090 ..... 16.30 |================================ NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: regnety_400m ms < Lower Is Better 4080 rep .... 8.72 |============================ 4080 xxx .... 8.38 |=========================== 4080 zzz .... 8.34 |=========================== 3090 ........ 8.22 |========================== 3090 rep .... 8.03 |========================== 3070 ........ 18.25 |========================================================== RTX 3070 Ti . 8.42 |=========================== 4090 ........ 10.09 |================================ 4090 rep .... 10.23 |================================= nv 4090 ..... 8.25 |========================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vgg16 ms < Lower Is Better 3090 rep .... 23.72 |========================== 3070 ........ 53.48 |========================================================== RTX 3070 Ti . 27.86 |============================== 4090 ........ 27.32 |============================== 4090 rep .... 29.85 |================================ nv 4090 ..... 27.25 |============================== NCNN 20230517 Target: Vulkan GPU - Model: squeezenet_ssd ms < Lower Is Better a ........... 7.09 |========================== b ........... 7.07 |========================== c ........... 7.06 |========================== d ........... 7.08 |========================== e ........... 7.05 |========================== f ........... 7.08 |========================== g ........... 7.10 |========================== i ........... 7.46 |=========================== 4080 ........ 7.58 |============================ 4080 rep .... 7.64 |============================ 4080 xxx .... 7.62 |============================ 4080 zzz .... 7.63 |============================ 3090 ........ 7.16 |========================== 3090 rep .... 7.06 |========================== 3070 ........ 15.82 |========================================================== RTX 3070 Ti . 8.47 |=============================== 4090 ........ 7.83 |============================= 4090 rep .... 8.22 |============================== nv 4090 ..... 9.11 |================================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: regnety_400m ms < Lower Is Better 4090 ........ 10.10 |================================= g ........... 8.38 |=========================== i ........... 8.46 |=========================== 4080 ........ 8.61 |============================ 4080 rep .... 8.24 |=========================== 4080 xxx .... 8.52 |============================ 4080 zzz .... 8.49 |============================ 3090 ........ 7.99 |========================== 3090 rep .... 8.34 |=========================== 3070 ........ 17.88 |========================================================== RTX 3070 Ti . 9.19 |============================== 4090 ........ 8.10 |========================== 4090 rep .... 8.64 |============================ nv 4090 ..... 10.09 |================================= NCNN 20230517 Target: CPU - Model: alexnet ms < Lower Is Better 4090 rep .... 5.23 |================================ a ........... 4.41 |=========================== b ........... 4.32 |========================== c ........... 4.31 |========================== d ........... 4.30 |========================== f ........... 4.36 |=========================== g ........... 4.32 |========================== i ........... 5.30 |================================= 4080 ........ 4.75 |============================= 4080 rep .... 4.67 |============================= 4080 xxx .... 4.68 |============================= 4080 zzz .... 4.69 |============================= 3090 ........ 4.30 |========================== 3090 rep .... 4.31 |========================== 3070 ........ 9.62 |=========================================================== RTX 3070 Ti . 5.49 |================================== 4090 ........ 5.14 |================================ 4090 rep .... 5.14 |================================ nv 4090 ..... 5.18 |================================ NCNN 20230517 Target: Vulkan GPU - Model: yolov4-tiny ms < Lower Is Better a ........... 12.84 |========================== b ........... 12.87 |========================== c ........... 12.81 |========================== d ........... 12.85 |========================== e ........... 12.87 |========================== f ........... 13.17 |=========================== g ........... 13.64 |============================ i ........... 15.16 |=============================== 4080 ........ 13.79 |============================ 4080 rep .... 13.67 |============================ 4080 xxx .... 13.65 |============================ 4080 zzz .... 13.80 |============================ 3090 ........ 12.88 |========================== 3090 rep .... 12.83 |========================== 3070 ........ 28.59 |========================================================== RTX 3070 Ti . 15.54 |================================ 4090 ........ 13.97 |============================ 4090 rep .... 15.72 |================================ nv 4090 ..... 15.26 |=============================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: regnety_400m ms < Lower Is Better b ........... 8.27 |============================ c ........... 7.98 |=========================== f ........... 8.50 |============================= g ........... 7.99 |=========================== i ........... 7.99 |=========================== 4080 ........ 8.45 |============================ 4080 rep .... 8.44 |============================ 4080 xxx .... 8.58 |============================= 4080 zzz .... 8.10 |=========================== 3090 ........ 8.01 |=========================== 3090 rep .... 8.25 |============================ 3070 ........ 17.23 |========================================================== RTX 3070 Ti . 8.89 |============================== 4090 ........ 9.60 |================================ 4090 rep .... 10.34 |=================================== nv 4090 ..... 7.73 |========================== NCNN 20230517 Target: CPU - Model: regnety_400m ms < Lower Is Better 4090 rep .... 8.78 |============================ a ........... 8.18 |========================== b ........... 8.18 |========================== c ........... 8.27 |=========================== d ........... 8.23 |=========================== f ........... 8.08 |========================== g ........... 8.30 |=========================== i ........... 9.94 |================================ 4080 ........ 8.24 |=========================== 4080 rep .... 8.67 |============================ 4080 xxx .... 8.45 |=========================== 4080 zzz .... 8.37 |=========================== 3090 ........ 8.25 |=========================== 3090 rep .... 8.24 |=========================== 3070 ........ 18.00 |========================================================== RTX 3070 Ti . 8.83 |============================ 4090 ........ 8.13 |========================== 4090 rep .... 9.66 |=============================== nv 4090 ..... 10.17 |================================= NCNN 20230517 Target: CPU-v3-v3-v3 - Model: mobilenet ms < Lower Is Better b ........... 8.01 |========================== c ........... 8.00 |========================== f ........... 8.56 |============================ g ........... 8.98 |============================= i ........... 10.08 |================================= 4080 ........ 8.43 |=========================== 4080 rep .... 8.48 |============================ 4080 xxx .... 8.44 |=========================== 4080 zzz .... 8.40 |=========================== 3090 ........ 8.06 |========================== 3090 rep .... 8.03 |========================== 3070 ........ 17.82 |========================================================== RTX 3070 Ti . 9.62 |=============================== 4090 ........ 8.96 |============================= 4090 rep .... 8.74 |============================ nv 4090 ..... 8.91 |============================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny ms < Lower Is Better 4080 ........ 13.93 |============================ 4080 rep .... 13.73 |============================ 4080 xxx .... 13.52 |=========================== 4080 zzz .... 15.26 |=============================== 3090 ........ 12.97 |========================== 3090 rep .... 12.90 |========================== 3070 ........ 28.73 |========================================================== RTX 3070 Ti . 15.00 |============================== 4090 ........ 15.95 |================================ 4090 rep .... 15.40 |=============================== nv 4090 ..... 15.55 |=============================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: regnety_400m ms < Lower Is Better 3090 ........ 8.20 |========================== 3090 rep .... 8.19 |========================== 3070 ........ 18.24 |========================================================== RTX 3070 Ti . 9.10 |============================= 4090 ........ 10.05 |================================ 4090 rep .... 8.45 |=========================== nv 4090 ..... 8.37 |=========================== NCNN 20230517 Target: Vulkan GPU - Model: vision_transformer ms < Lower Is Better a ........... 31.88 |========================== b ........... 31.85 |========================== c ........... 31.79 |========================== d ........... 32.12 |========================== e ........... 31.93 |========================== f ........... 32.92 |=========================== g ........... 32.42 |=========================== i ........... 36.42 |============================== 4080 ........ 35.56 |============================= 4080 rep .... 35.07 |============================= 4080 xxx .... 34.19 |============================ 4080 zzz .... 34.32 |============================ 3090 ........ 31.94 |========================== 3090 rep .... 31.91 |========================== 3070 ........ 70.76 |========================================================== RTX 3070 Ti . 38.03 |=============================== 4090 ........ 38.76 |================================ 4090 rep .... 37.59 |=============================== nv 4090 ..... 39.04 |================================ NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny ms < Lower Is Better 4080 rep .... 13.71 |============================ 4080 xxx .... 13.63 |============================ 4080 zzz .... 13.42 |=========================== 3090 ........ 12.82 |========================== 3090 rep .... 12.81 |========================== 3070 ........ 28.41 |========================================================== RTX 3070 Ti . 14.57 |============================== 4090 ........ 15.85 |================================ 4090 rep .... 13.88 |============================ nv 4090 ..... 17.67 |==================================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: vision_transformer ms < Lower Is Better 4090 ........ 39.06 |================================ g ........... 33.32 |=========================== i ........... 38.01 |=============================== 4080 ........ 34.91 |============================ 4080 rep .... 34.10 |============================ 4080 xxx .... 34.23 |============================ 4080 zzz .... 34.10 |============================ 3090 ........ 33.22 |=========================== 3090 rep .... 32.09 |========================== 3070 ........ 71.08 |========================================================== RTX 3070 Ti . 37.88 |=============================== 4090 ........ 38.38 |=============================== 4090 rep .... 38.17 |=============================== nv 4090 ..... 39.18 |================================ NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: resnet50 ms < Lower Is Better 4080 rep .... 10.86 |============================ 4080 xxx .... 11.26 |============================= 4080 zzz .... 11.10 |============================= 3090 ........ 10.03 |========================== 3090 rep .... 10.06 |========================== 3070 ........ 22.19 |========================================================== RTX 3070 Ti . 12.81 |================================= 4090 ........ 11.72 |=============================== 4090 rep .... 12.47 |================================= nv 4090 ..... 13.29 |=================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vision_transformer ms < Lower Is Better 3090 rep .... 31.94 |========================== 3070 ........ 70.53 |========================================================== RTX 3070 Ti . 38.50 |================================ 4090 ........ 39.35 |================================ 4090 rep .... 38.65 |================================ nv 4090 ..... 37.13 |=============================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vision_transformer ms < Lower Is Better 4080 ........ 35.60 |============================= 4080 rep .... 33.93 |============================ 4080 xxx .... 33.90 |============================ 4080 zzz .... 35.36 |============================= 3090 ........ 31.94 |========================== 3090 rep .... 31.97 |========================== 3070 ........ 70.29 |========================================================== RTX 3070 Ti . 38.27 |================================ 4090 ........ 38.62 |================================ 4090 rep .... 39.03 |================================ nv 4090 ..... 38.46 |================================ NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0 ms < Lower Is Better 4090 ........ 4.24 |============================== g ........... 3.84 |=========================== i ........... 4.19 |============================= 4080 ........ 4.06 |============================ 4080 rep .... 3.98 |============================ 4080 xxx .... 4.02 |============================ 4080 zzz .... 4.01 |============================ 3090 ........ 3.83 |=========================== 3090 rep .... 3.87 |=========================== 3070 ........ 8.41 |=========================================================== RTX 3070 Ti . 4.60 |================================ 4090 ........ 4.36 |=============================== 4090 rep .... 4.04 |============================ nv 4090 ..... 4.12 |============================= NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd ms < Lower Is Better 3090 ........ 7.05 |========================== 3090 rep .... 7.07 |=========================== 3070 ........ 15.46 |========================================================== RTX 3070 Ti . 8.31 |=============================== 4090 ........ 7.40 |============================ 4090 rep .... 9.34 |=================================== nv 4090 ..... 7.72 |============================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd ms < Lower Is Better 4090 ........ 9.28 |=================================== g ........... 7.14 |=========================== i ........... 7.21 |=========================== 4080 ........ 7.66 |============================= 4080 rep .... 7.55 |============================ 4080 xxx .... 7.67 |============================= 4080 zzz .... 7.62 |============================= 3090 ........ 7.04 |=========================== 3090 rep .... 7.09 |=========================== 3070 ........ 15.40 |========================================================== RTX 3070 Ti . 8.28 |=============================== 4090 ........ 7.57 |============================= 4090 rep .... 9.16 |================================== nv 4090 ..... 9.21 |=================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vgg16 ms < Lower Is Better 3090 ........ 23.50 |=========================== 3090 rep .... 23.47 |=========================== 3070 ........ 51.28 |========================================================== RTX 3070 Ti . 28.53 |================================ 4090 ........ 27.31 |=============================== 4090 rep .... 29.35 |================================= nv 4090 ..... 29.40 |================================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: regnety_400m ms < Lower Is Better 4080 ........ 8.67 |============================= 4080 rep .... 8.35 |============================ 4080 xxx .... 8.25 |=========================== 4080 zzz .... 8.37 |============================ 3090 ........ 8.33 |=========================== 3090 rep .... 8.07 |=========================== 3070 ........ 17.61 |========================================================== RTX 3070 Ti . 9.05 |============================== 4090 ........ 9.87 |================================= 4090 rep .... 10.69 |=================================== nv 4090 ..... 10.03 |================================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet18 ms < Lower Is Better 4080 ........ 5.92 |============================== 4080 rep .... 5.67 |============================= 4080 xxx .... 5.65 |============================= 4080 zzz .... 5.74 |============================= 3090 ........ 5.21 |=========================== 3090 rep .... 5.20 |=========================== 3070 ........ 11.30 |========================================================== RTX 3070 Ti . 6.40 |================================= 4090 ........ 7.52 |======================================= 4090 rep .... 5.90 |============================== nv 4090 ..... 7.38 |====================================== NCNN 20230517 Target: CPU-v3-v3-v3 - Model: googlenet ms < Lower Is Better b ........... 7.84 |=========================== c ........... 7.88 |=========================== f ........... 8.07 |============================ g ........... 9.15 |=============================== i ........... 10.17 |=================================== 4080 ........ 8.45 |============================= 4080 rep .... 8.52 |============================= 4080 xxx .... 8.50 |============================= 4080 zzz .... 8.55 |============================= 3090 ........ 7.83 |=========================== 3090 rep .... 7.85 |=========================== 3070 ........ 17.00 |========================================================== RTX 3070 Ti . 9.65 |================================= 4090 ........ 9.97 |================================== 4090 rep .... 9.29 |================================ nv 4090 ..... 9.02 |=============================== NCNN 20230517 Target: CPU - Model: yolov4-tiny ms < Lower Is Better 4090 rep .... 16.79 |=================================== a ........... 12.90 |=========================== b ........... 12.74 |=========================== c ........... 12.81 |=========================== d ........... 12.95 |=========================== f ........... 13.32 |============================ g ........... 17.23 |==================================== i ........... 14.65 |=============================== 4080 ........ 13.85 |============================= 4080 rep .... 14.03 |============================= 4080 xxx .... 13.60 |============================= 4080 zzz .... 13.63 |============================= 3090 ........ 14.26 |============================== 3090 rep .... 12.77 |=========================== 3070 ........ 27.66 |========================================================== RTX 3070 Ti . 15.20 |================================ 4090 ........ 13.68 |============================= 4090 rep .... 15.38 |================================ nv 4090 ..... 15.40 |================================ NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd ms < Lower Is Better 3090 rep .... 7.07 |=========================== 3070 ........ 15.32 |========================================================== RTX 3070 Ti . 7.45 |============================ 4090 ........ 7.43 |============================ 4090 rep .... 9.44 |==================================== nv 4090 ..... 8.26 |=============================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: googlenet ms < Lower Is Better 4080 ........ 8.79 |============================== 4080 rep .... 8.42 |============================= 4080 xxx .... 8.26 |============================ 4080 zzz .... 8.55 |============================= 3090 ........ 7.84 |=========================== 3090 rep .... 7.86 |=========================== 3070 ........ 16.97 |========================================================== RTX 3070 Ti . 9.90 |================================== 4090 ........ 9.05 |=============================== 4090 rep .... 10.39 |==================================== nv 4090 ..... 8.35 |============================= NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vision_transformer ms < Lower Is Better 3090 ........ 32.16 |=========================== 3090 rep .... 32.13 |=========================== 3070 ........ 69.48 |========================================================== RTX 3070 Ti . 38.32 |================================ 4090 ........ 38.82 |================================ 4090 rep .... 38.69 |================================ nv 4090 ..... 38.58 |================================ NCNN 20230517 Target: CPU - Model: resnet50 ms < Lower Is Better 4090 rep .... 13.73 |===================================== a ........... 10.20 |============================ b ........... 10.01 |=========================== c ........... 10.11 |=========================== d ........... 10.00 |=========================== f ........... 11.05 |============================== g ........... 10.72 |============================= i ........... 12.09 |================================= 4080 ........ 11.16 |============================== 4080 rep .... 11.76 |================================ 4080 xxx .... 10.82 |============================= 4080 zzz .... 11.10 |============================== 3090 ........ 10.30 |============================ 3090 rep .... 9.95 |=========================== 3070 ........ 21.50 |========================================================== RTX 3070 Ti . 12.73 |================================== 4090 ........ 14.10 |====================================== 4090 rep .... 12.73 |================================== nv 4090 ..... 11.41 |=============================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet50 ms < Lower Is Better 3090 rep .... 10.27 |=========================== 3070 ........ 22.15 |========================================================== RTX 3070 Ti . 13.15 |================================== 4090 ........ 13.00 |================================== 4090 rep .... 10.96 |============================= nv 4090 ..... 13.25 |=================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: vgg16 ms < Lower Is Better 4080 rep .... 25.01 |============================= 4080 xxx .... 25.44 |============================= 4080 zzz .... 25.26 |============================= 3090 ........ 23.58 |=========================== 3090 rep .... 23.40 |=========================== 3070 ........ 50.32 |========================================================== RTX 3070 Ti . 27.98 |================================ 4090 ........ 30.16 |=================================== 4090 rep .... 30.74 |=================================== nv 4090 ..... 27.61 |================================ NCNN 20230517 Target: CPU-v3-v3-v3 - Model: resnet18 ms < Lower Is Better b ........... 5.21 |=========================== c ........... 5.26 |=========================== f ........... 6.13 |================================ g ........... 5.48 |============================= i ........... 5.86 |=============================== 4080 ........ 5.70 |============================== 4080 rep .... 5.63 |============================= 4080 xxx .... 5.66 |============================= 4080 zzz .... 5.71 |============================== 3090 ........ 5.19 |=========================== 3090 rep .... 5.20 |=========================== 3070 ........ 11.14 |========================================================== RTX 3070 Ti . 6.18 |================================ 4090 ........ 5.81 |============================== 4090 rep .... 7.75 |======================================== nv 4090 ..... 7.61 |======================================== NCNN 20230517 Target: Vulkan GPU - Model: FastestDet ms < Lower Is Better a ........... 4.10 |============================= b ........... 4.07 |============================= c ........... 4.09 |============================= d ........... 4.11 |============================= e ........... 4.08 |============================= f ........... 4.24 |============================== g ........... 4.07 |============================= i ........... 5.14 |==================================== 4080 ........ 4.20 |============================= 4080 rep .... 4.21 |============================== 4080 xxx .... 4.20 |============================= 4080 zzz .... 4.79 |================================== 3090 ........ 4.11 |============================= 3090 rep .... 4.08 |============================= 3070 ........ 8.41 |=========================================================== RTX 3070 Ti . 4.26 |============================== 4090 ........ 4.39 |=============================== 4090 rep .... 4.59 |================================ nv 4090 ..... 3.93 |============================ NCNN 20230517 Target: Vulkan GPU - Model: shufflenet-v2 ms < Lower Is Better a ........... 3.35 |============================ b ........... 3.33 |============================ c ........... 3.32 |============================ d ........... 3.35 |============================ e ........... 3.33 |============================ f ........... 3.40 |============================ g ........... 3.35 |============================ i ........... 3.49 |============================= 4080 ........ 3.43 |============================= 4080 rep .... 3.44 |============================= 4080 xxx .... 3.50 |============================= 4080 zzz .... 3.46 |============================= 3090 ........ 3.39 |============================ 3090 rep .... 3.36 |============================ 3070 ........ 7.07 |=========================================================== RTX 3070 Ti . 3.98 |================================= 4090 ........ 3.45 |============================= 4090 rep .... 3.49 |============================= nv 4090 ..... 3.51 |============================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet ms < Lower Is Better 4080 ........ 8.43 |============================= 4080 rep .... 8.38 |============================ 4080 xxx .... 8.31 |============================ 4080 zzz .... 9.19 |=============================== 3090 ........ 8.03 |=========================== 3090 rep .... 8.05 |=========================== 3070 ........ 17.09 |========================================================== RTX 3070 Ti . 9.52 |================================ 4090 ........ 9.16 |=============================== 4090 rep .... 9.02 |=============================== nv 4090 ..... 8.93 |============================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vgg16 ms < Lower Is Better 4080 ........ 25.67 |============================== 4080 rep .... 25.56 |============================== 4080 xxx .... 25.33 |============================== 4080 zzz .... 26.09 |============================== 3090 ........ 23.51 |=========================== 3090 rep .... 23.38 |=========================== 3070 ........ 49.70 |========================================================== RTX 3070 Ti . 28.53 |================================= 4090 ........ 27.44 |================================ 4090 rep .... 29.17 |================================== nv 4090 ..... 28.14 |================================= NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet ms < Lower Is Better 3090 rep .... 8.03 |=========================== 3070 ........ 17.06 |========================================================== RTX 3070 Ti . 10.02 |================================== 4090 ........ 10.56 |==================================== 4090 rep .... 8.22 |============================ nv 4090 ..... 10.64 |==================================== NCNN 20230517 Target: CPU-v3-v3-v3 - Model: vgg16 ms < Lower Is Better b ........... 23.50 |=========================== c ........... 23.99 |============================ f ........... 24.45 |============================= g ........... 24.92 |============================= i ........... 29.12 |================================== 4080 ........ 25.10 |============================= 4080 rep .... 24.91 |============================= 4080 xxx .... 25.00 |============================= 4080 zzz .... 25.45 |============================== 3090 ........ 23.50 |=========================== 3090 rep .... 23.54 |=========================== 3070 ........ 49.75 |========================================================== RTX 3070 Ti . 28.40 |================================= 4090 ........ 28.55 |================================= 4090 rep .... 27.25 |================================ nv 4090 ..... 27.04 |================================ NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: regnety_400m ms < Lower Is Better 3090 rep .... 8.06 |=========================== 3070 ........ 17.02 |========================================================== RTX 3070 Ti . 9.14 |=============================== 4090 ........ 10.11 |================================== 4090 rep .... 8.70 |============================== nv 4090 ..... 8.34 |============================ NCNN 20230517 Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better a ........... 3.17 |============================ c ........... 3.20 |============================= d ........... 3.17 |============================ e ........... 3.18 |============================ f ........... 3.14 |============================ i ........... 4.87 |============================================ 4080 zzz .... 3.28 |============================= 3090 ........ 3.19 |============================= 3090 rep .... 3.16 |============================ 3070 ........ 6.60 |=========================================================== RTX 3070 Ti . 3.64 |================================= 4090 ........ 3.28 |============================= 4090 rep .... 3.30 |============================== nv 4090 ..... 3.36 |============================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better 4080 rep .... 3.28 |============================== 4080 xxx .... 3.08 |============================ 4080 zzz .... 3.06 |============================ 3090 rep .... 3.17 |============================= 3070 ........ 6.43 |=========================================================== RTX 3070 Ti . 3.70 |================================== 4090 ........ 3.33 |=============================== 4090 rep .... 3.30 |============================== nv 4090 ..... 4.97 |============================================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better 4090 ........ 3.34 |============================== g ........... 3.16 |============================ i ........... 3.26 |============================= 4080 rep .... 3.24 |============================= 4080 xxx .... 3.26 |============================= 4080 zzz .... 3.24 |============================= 3090 ........ 3.16 |============================ 3070 ........ 6.56 |=========================================================== RTX 3070 Ti . 3.62 |================================= 4090 ........ 3.30 |============================== 4090 rep .... 3.33 |============================== nv 4090 ..... 3.26 |============================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mnasnet ms < Lower Is Better 4080 ........ 3.10 |============================== 4080 rep .... 3.06 |============================== 4080 xxx .... 2.98 |============================= 4080 zzz .... 3.08 |============================== 3090 ........ 2.94 |============================= 3090 rep .... 2.97 |============================= 3070 ........ 6.06 |=========================================================== RTX 3070 Ti . 3.10 |============================== 4090 ........ 4.93 |================================================ 4090 rep .... 4.99 |================================================= nv 4090 ..... 3.07 |============================== NCNN 20230517 Target: Vulkan GPU - Model: vgg16 ms < Lower Is Better a ........... 23.51 |============================ b ........... 23.56 |============================ c ........... 23.54 |============================ d ........... 23.56 |============================ e ........... 23.60 |============================ f ........... 24.55 |============================= g ........... 24.20 |============================= i ........... 27.83 |================================= 4080 ........ 25.00 |============================== 4080 rep .... 25.04 |============================== 4080 xxx .... 25.01 |============================== 4080 zzz .... 25.82 |=============================== 3090 ........ 23.55 |============================ 3090 rep .... 23.48 |============================ 3070 ........ 48.29 |========================================================== RTX 3070 Ti . 29.06 |=================================== 4090 ........ 28.82 |=================================== 4090 rep .... 27.04 |================================ nv 4090 ..... 29.29 |=================================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: mobilenet ms < Lower Is Better g ........... 8.04 |============================= b ........... 8.00 |============================ c ........... 7.95 |============================ f ........... 8.65 |=============================== g ........... 8.20 |============================= i ........... 9.05 |================================ 4080 ........ 8.43 |============================== 4080 rep .... 8.46 |============================== 4080 xxx .... 8.88 |================================ 4080 zzz .... 8.38 |============================== 3090 ........ 8.00 |============================ 3090 rep .... 8.01 |============================ 3070 ........ 16.34 |========================================================== RTX 3070 Ti . 9.62 |================================== 4090 ........ 8.81 |=============================== 4090 rep .... 9.54 |================================== nv 4090 ..... 10.54 |===================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: vision_transformer ms < Lower Is Better 4080 rep .... 34.22 |============================== 4080 xxx .... 34.14 |============================== 4080 zzz .... 34.47 |=============================== 3090 ........ 32.10 |============================ 3090 rep .... 31.85 |============================ 3070 ........ 65.41 |========================================================== RTX 3070 Ti . 38.04 |================================== 4090 ........ 38.76 |================================== 4090 rep .... 38.79 |================================== nv 4090 ..... 38.95 |=================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: mnasnet ms < Lower Is Better 4080 rep .... 3.09 |============================== 4080 xxx .... 3.00 |============================= 4080 zzz .... 2.96 |============================= 3090 ........ 2.98 |============================= 3090 rep .... 2.97 |============================= 3070 ........ 6.07 |=========================================================== RTX 3070 Ti . 3.24 |=============================== 4090 ........ 5.19 |================================================== 4090 rep .... 5.11 |================================================== nv 4090 ..... 3.12 |============================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: mobilenet ms < Lower Is Better 4080 rep .... 8.45 |============================== 4080 xxx .... 8.34 |============================= 4080 zzz .... 8.25 |============================= 3090 ........ 8.07 |============================ 3090 rep .... 8.06 |============================ 3070 ........ 16.52 |========================================================== RTX 3070 Ti . 10.03 |=================================== 4090 ........ 9.04 |================================ 4090 rep .... 10.61 |===================================== nv 4090 ..... 12.12 |=========================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny ms < Lower Is Better 3090 ........ 12.86 |============================ 3090 rep .... 12.89 |============================ 3070 ........ 26.33 |========================================================== RTX 3070 Ti . 15.56 |================================== 4090 ........ 15.69 |=================================== 4090 rep .... 16.60 |===================================== nv 4090 ..... 15.67 |=================================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: efficientnet-b0 ms < Lower Is Better g ........... 3.85 |============================= b ........... 3.85 |============================= c ........... 3.82 |============================= f ........... 3.85 |============================= g ........... 4.63 |=================================== i ........... 4.21 |================================ 4080 ........ 4.05 |=============================== 4080 rep .... 4.04 |=============================== 4080 xxx .... 4.22 |================================ 4080 zzz .... 3.95 |============================== 3090 ........ 3.85 |============================= 3090 rep .... 3.85 |============================= 3070 ........ 7.81 |=========================================================== RTX 3070 Ti . 4.73 |==================================== 4090 ........ 4.18 |================================ 4090 rep .... 6.28 |=============================================== nv 4090 ..... 5.26 |======================================== NCNN 20230517 Target: CPU - Model: shufflenet-v2 ms < Lower Is Better 4090 rep .... 3.47 |============================== a ........... 3.34 |============================= b ........... 3.34 |============================= c ........... 3.35 |============================= d ........... 3.35 |============================= f ........... 3.55 |=============================== g ........... 3.59 |=============================== i ........... 5.03 |============================================ 4080 ........ 3.41 |============================== 4080 rep .... 3.43 |============================== 4080 xxx .... 3.45 |============================== 4080 zzz .... 3.42 |============================== 3090 ........ 3.34 |============================= 3090 rep .... 3.35 |============================= 3070 ........ 6.82 |=========================================================== RTX 3070 Ti . 3.77 |================================= 4090 ........ 3.55 |=============================== 4090 rep .... 3.51 |============================== nv 4090 ..... 3.45 |============================== NCNN 20230517 Target: Vulkan GPU - Model: regnety_400m ms < Lower Is Better a ........... 8.16 |============================= b ........... 8.21 |============================= c ........... 8.00 |============================= d ........... 8.17 |============================= e ........... 8.10 |============================= f ........... 8.34 |============================== g ........... 8.36 |============================== i ........... 9.88 |=================================== 4080 ........ 8.39 |============================== 4080 rep .... 8.56 |=============================== 4080 xxx .... 8.56 |=============================== 4080 zzz .... 8.58 |=============================== 3090 ........ 8.38 |============================== 3090 rep .... 8.02 |============================= 3070 ........ 16.22 |========================================================== RTX 3070 Ti . 9.07 |================================ 4090 ........ 8.64 |=============================== 4090 rep .... 8.48 |============================== nv 4090 ..... 9.81 |=================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd ms < Lower Is Better 4080 rep .... 7.67 |=============================== 4080 xxx .... 7.27 |============================== 4080 zzz .... 7.25 |============================= 3090 ........ 7.12 |============================= 3090 rep .... 7.09 |============================= 3070 ........ 14.27 |========================================================== RTX 3070 Ti . 7.57 |=============================== 4090 ........ 9.51 |======================================= 4090 rep .... 9.38 |====================================== nv 4090 ..... 7.48 |============================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: blazeface ms < Lower Is Better 3090 rep .... 1.38 |============================== 3070 ........ 2.69 |=========================================================== RTX 3070 Ti . 1.40 |=============================== 4090 ........ 1.35 |============================== 4090 rep .... 1.42 |=============================== nv 4090 ..... 1.40 |=============================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better 4080 ........ 3.26 |================================ 4080 rep .... 3.31 |================================= 4080 xxx .... 3.05 |============================== 4080 zzz .... 3.26 |================================ 3090 ........ 3.13 |=============================== 3090 rep .... 3.18 |=============================== 3070 ........ 5.99 |=========================================================== RTX 3070 Ti . 3.44 |================================== 4090 ........ 3.62 |==================================== 4090 rep .... 3.34 |================================= nv 4090 ..... 3.17 |=============================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: blazeface ms < Lower Is Better 4080 rep .... 1.43 |================================= 4080 xxx .... 1.32 |=============================== 4080 zzz .... 1.31 |=============================== 3090 ........ 1.39 |================================ 3090 rep .... 1.38 |================================ 3070 ........ 2.53 |=========================================================== RTX 3070 Ti . 2.48 |========================================================== 4090 ........ 1.30 |============================== 4090 rep .... 1.42 |================================= nv 4090 ..... 1.42 |================================= NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better 3090 ........ 3.16 |=============================== 3070 ........ 5.97 |=========================================================== RTX 3070 Ti . 3.52 |=================================== 4090 ........ 3.36 |================================= 4090 rep .... 3.44 |================================== nv 4090 ..... 4.81 |================================================ NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better 3090 rep .... 3.15 |=============================== 3070 ........ 5.92 |=========================================================== RTX 3070 Ti . 3.83 |====================================== 4090 ........ 4.75 |=============================================== 4090 rep .... 3.38 |================================== nv 4090 ..... 3.29 |================================= NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2 ms < Lower Is Better 3090 ........ 3.36 |=============================== 3090 rep .... 3.36 |=============================== 3070 ........ 6.30 |=========================================================== RTX 3070 Ti . 3.92 |===================================== 4090 ........ 3.47 |================================ 4090 rep .... 5.18 |================================================= nv 4090 ..... 3.37 |================================ NCNN 20230517 Target: CPU - Model: squeezenet_ssd ms < Lower Is Better 4090 rep .... 7.98 |=================================== a ........... 7.07 |=============================== b ........... 7.06 |=============================== c ........... 7.10 |=============================== d ........... 7.09 |=============================== f ........... 7.23 |================================ g ........... 7.13 |=============================== i ........... 8.96 |======================================= 4080 ........ 7.73 |================================== 4080 rep .... 7.86 |=================================== 4080 xxx .... 7.62 |================================= 4080 zzz .... 7.55 |================================= 3090 ........ 7.52 |================================= 3090 rep .... 7.12 |=============================== 3070 ........ 13.20 |========================================================== RTX 3070 Ti . 8.13 |==================================== 4090 ........ 7.86 |=================================== 4090 rep .... 7.31 |================================ nv 4090 ..... 9.11 |======================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: FastestDet ms < Lower Is Better 3090 rep .... 4.07 |================================= 3070 ........ 7.23 |=========================================================== RTX 3070 Ti . 4.14 |================================== 4090 ........ 4.62 |====================================== 4090 rep .... 4.59 |===================================== nv 4090 ..... 5.86 |================================================ NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2 ms < Lower Is Better 3090 rep .... 3.32 |================================= 3070 ........ 5.89 |=========================================================== RTX 3070 Ti . 3.48 |=================================== 4090 ........ 3.56 |==================================== 4090 rep .... 5.23 |==================================================== nv 4090 ..... 3.43 |================================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better 4080 ........ 3.29 |==================================== 4080 rep .... 3.27 |=================================== 4080 xxx .... 3.14 |================================== 4080 zzz .... 3.28 |=================================== 3090 ........ 3.12 |================================== 3090 rep .... 3.17 |================================== 3070 ........ 5.46 |=========================================================== RTX 3070 Ti . 3.66 |======================================== 4090 ........ 3.48 |====================================== 4090 rep .... 3.36 |==================================== nv 4090 ..... 3.42 |===================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better 3090 ........ 3.15 |================================== 3090 rep .... 3.17 |================================== 3070 ........ 5.49 |=========================================================== RTX 3070 Ti . 3.69 |======================================== 4090 ........ 5.25 |======================================================== 4090 rep .... 3.60 |======================================= nv 4090 ..... 3.27 |=================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0 ms < Lower Is Better 3090 rep .... 3.84 |================================== 3070 ........ 6.63 |=========================================================== RTX 3070 Ti . 4.17 |===================================== 4090 ........ 4.63 |========================================= 4090 rep .... 4.10 |==================================== nv 4090 ..... 5.82 |==================================================== NCNN 20230517 Target: Vulkan GPU - Model: mnasnet ms < Lower Is Better a ........... 2.98 |=================================== b ........... 2.95 |================================== c ........... 2.96 |================================== d ........... 2.97 |================================== e ........... 2.96 |================================== f ........... 2.97 |================================== g ........... 2.98 |=================================== i ........... 3.20 |===================================== 4080 ........ 3.07 |==================================== 4080 rep .... 3.08 |==================================== 4080 xxx .... 3.07 |==================================== 4080 zzz .... 3.06 |=================================== 3090 ........ 2.99 |=================================== 3090 rep .... 2.97 |================================== 3070 ........ 5.09 |=========================================================== RTX 3070 Ti . 3.40 |======================================= 4090 ........ 3.18 |===================================== 4090 rep .... 3.15 |===================================== nv 4090 ..... 4.70 |====================================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better b ........... 3.16 |=================================== c ........... 3.16 |=================================== f ........... 3.15 |=================================== g ........... 3.16 |=================================== i ........... 3.29 |==================================== 4080 ........ 3.28 |==================================== 4080 xxx .... 3.31 |==================================== 4080 zzz .... 3.27 |==================================== 3090 ........ 3.15 |=================================== 3090 rep .... 3.15 |=================================== 3070 ........ 5.38 |=========================================================== RTX 3070 Ti . 3.76 |========================================= 4090 ........ 3.25 |==================================== 4090 rep .... 3.53 |======================================= nv 4090 ..... 3.35 |===================================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2 ms < Lower Is Better 4080 ........ 3.48 |===================================== 4080 rep .... 3.44 |==================================== 4080 xxx .... 3.34 |=================================== 4080 zzz .... 3.44 |==================================== 3090 ........ 3.32 |=================================== 3090 rep .... 3.34 |=================================== 3070 ........ 5.59 |=========================================================== RTX 3070 Ti . 3.89 |========================================= 4090 ........ 3.52 |===================================== 4090 rep .... 3.48 |===================================== nv 4090 ..... 3.50 |===================================== NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: FastestDet ms < Lower Is Better 3090 ........ 4.08 |====================================================== 3090 rep .... 4.10 |====================================================== 3070 ........ 4.48 |=========================================================== RTX 3070 Ti . 4.25 |======================================================== 4090 ........ 2.82 |===================================== 4090 rep .... 3.91 |=================================================== nv 4090 ..... 4.01 |===================================================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: mnasnet ms < Lower Is Better 4090 ........ 3.10 |======================================== g ........... 2.97 |====================================== i ........... 3.07 |======================================= 4080 ........ 3.06 |======================================= 4080 rep .... 3.03 |======================================= 4080 xxx .... 3.05 |======================================= 4080 zzz .... 3.08 |======================================== 3090 ........ 2.96 |====================================== 3090 rep .... 2.99 |====================================== 3070 ........ 4.59 |=========================================================== RTX 3070 Ti . 3.34 |=========================================== 4090 ........ 3.17 |========================================= 4090 rep .... 3.13 |======================================== nv 4090 ..... 3.16 |========================================= NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: shufflenet-v2 ms < Lower Is Better g ........... 3.33 |======================================== b ........... 3.33 |======================================== c ........... 3.33 |======================================== f ........... 3.33 |======================================== g ........... 3.35 |======================================== i ........... 3.36 |========================================= 4080 ........ 3.44 |========================================== 4080 rep .... 3.44 |========================================== 4080 xxx .... 3.51 |========================================== 4080 zzz .... 3.37 |========================================= 3090 ........ 3.34 |======================================== 3090 rep .... 3.36 |========================================= 3070 ........ 4.89 |=========================================================== RTX 3070 Ti . 3.95 |================================================ 4090 ........ 3.34 |======================================== 4090 rep .... 3.40 |========================================= nv 4090 ..... 3.17 |====================================== NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: blazeface ms < Lower Is Better 4090 ........ 1.43 |================================================ g ........... 1.38 |============================================== i ........... 1.41 |=============================================== 4080 ........ 1.43 |================================================ 4080 rep .... 1.41 |=============================================== 4080 xxx .... 1.42 |=============================================== 4080 zzz .... 1.42 |=============================================== 3090 ........ 1.36 |============================================= 3090 rep .... 1.39 |============================================== 3070 ........ 1.77 |=========================================================== RTX 3070 Ti . 1.49 |================================================== 4090 ........ 1.16 |======================================= 4090 rep .... 1.46 |================================================= nv 4090 ..... 1.18 |=======================================