tflite ARMv8 Cortex-A78E testing with a NVIDIA Jetson AGX Orin Developer Kit (36.3.0-gcid-36191598 BIOS) and Orin on Ubuntu 22.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2408262-NE-TFLITE58672 .
tflite Processor Motherboard Memory Disk Graphics Network OS Kernel Desktop Display Server Display Driver Vulkan Compiler File-System Screen Resolution all of them ARMv8 Cortex-A78E @ 2.20GHz (12 Cores) NVIDIA Jetson AGX Orin Developer Kit (36.3.0-gcid-36191598 BIOS) 30GB 1000GB Samsung SSD 960 EVO 1TB + 64GB G1M15M Orin Realtek RTL8822CE 802.11ac PCIe Ubuntu 22.04 5.15.136-tegra (aarch64) GNOME Shell 42.9 X Server 1.21.1.4 NVIDIA 1.3.251 GCC 11.4.0 + CUDA 12.2 ext4 6582x1234 OpenBenchmarking.org - Transparent Huge Pages: always - Scaling Governor: tegra194 performance - gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of __user pointer sanitization + spectre_v2: Mitigation of CSV2 but not BHB + srbds: Not affected + tsx_async_abort: Not affected
tflite tensorflow-lite: SqueezeNet tensorflow-lite: Inception V4 tensorflow-lite: NASNet Mobile tensorflow-lite: Mobilenet Float tensorflow-lite: Mobilenet Quant tensorflow-lite: Inception ResNet V2 all of them 6212.21 84257.5 21324.4 4698.18 2522.19 78285.9 OpenBenchmarking.org
TensorFlow Lite Model: SqueezeNet OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: SqueezeNet all of them 1300 2600 3900 5200 6500 SE +/- 6.38, N = 3 6212.21
TensorFlow Lite Model: Inception V4 OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: Inception V4 all of them 20K 40K 60K 80K 100K SE +/- 131.77, N = 3 84257.5
TensorFlow Lite Model: NASNet Mobile OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: NASNet Mobile all of them 5K 10K 15K 20K 25K SE +/- 292.35, N = 3 21324.4
TensorFlow Lite Model: Mobilenet Float OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: Mobilenet Float all of them 1000 2000 3000 4000 5000 SE +/- 16.55, N = 3 4698.18
TensorFlow Lite Model: Mobilenet Quant OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: Mobilenet Quant all of them 500 1000 1500 2000 2500 SE +/- 4.68, N = 3 2522.19
TensorFlow Lite Model: Inception ResNet V2 OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: Inception ResNet V2 all of them 20K 40K 60K 80K 100K SE +/- 274.21, N = 3 78285.9
Phoronix Test Suite v10.8.5