xeon emr 2 x INTEL XEON PLATINUM 8592+ testing with a Quanta Cloud QuantaGrid D54Q-2U S6Q-MB-MPS (3B05.TEL4P1 BIOS) and ASPEED on Ubuntu 23.10 via the Phoronix Test Suite. a: Processor: 2 x INTEL XEON PLATINUM 8592+ @ 3.90GHz (128 Cores / 256 Threads), Motherboard: Quanta Cloud QuantaGrid D54Q-2U S6Q-MB-MPS (3B05.TEL4P1 BIOS), Chipset: Intel Device 1bce, Memory: 1008GB, Disk: 3201GB Micron_7450_MTFDKCC3T2TFS, Graphics: ASPEED, Network: 2 x Intel X710 for 10GBASE-T OS: Ubuntu 23.10, Kernel: 6.6.0-rc5-phx-patched (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server 1.21.1.7, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200 b: Processor: 2 x INTEL XEON PLATINUM 8592+ @ 3.90GHz (128 Cores / 256 Threads), Motherboard: Quanta Cloud QuantaGrid D54Q-2U S6Q-MB-MPS (3B05.TEL4P1 BIOS), Chipset: Intel Device 1bce, Memory: 1008GB, Disk: 3201GB Micron_7450_MTFDKCC3T2TFS, Graphics: ASPEED, Network: 2 x Intel X710 for 10GBASE-T OS: Ubuntu 23.10, Kernel: 6.6.0-rc5-phx-patched (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server 1.21.1.7, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200 c: Processor: 2 x INTEL XEON PLATINUM 8592+ @ 3.90GHz (128 Cores / 256 Threads), Motherboard: Quanta Cloud QuantaGrid D54Q-2U S6Q-MB-MPS (3B05.TEL4P1 BIOS), Chipset: Intel Device 1bce, Memory: 1008GB, Disk: 3201GB Micron_7450_MTFDKCC3T2TFS, Graphics: ASPEED, Network: 2 x Intel X710 for 10GBASE-T OS: Ubuntu 23.10, Kernel: 6.6.0-rc5-phx-patched (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server 1.21.1.7, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200 d: Processor: 2 x INTEL XEON PLATINUM 8592+ @ 3.90GHz (128 Cores / 256 Threads), Motherboard: Quanta Cloud QuantaGrid D54Q-2U S6Q-MB-MPS (3B05.TEL4P1 BIOS), Chipset: Intel Device 1bce, Memory: 1008GB, Disk: 3201GB Micron_7450_MTFDKCC3T2TFS, Graphics: ASPEED, Network: 2 x Intel X710 for 10GBASE-T OS: Ubuntu 23.10, Kernel: 6.6.0-rc5-phx-patched (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server 1.21.1.7, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200 SVT-AV1 2.0 Encoder Mode: Preset 4 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 7.750 |==================================================================== b . 7.531 |================================================================== c . 7.541 |================================================================== d . 7.586 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 8 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 72.86 |=================================================================== b . 73.71 |==================================================================== c . 72.83 |=================================================================== d . 71.51 |================================================================== SVT-AV1 2.0 Encoder Mode: Preset 12 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 172.72 |=================================================================== b . 168.30 |================================================================= c . 169.91 |================================================================== d . 169.39 |================================================================== SVT-AV1 2.0 Encoder Mode: Preset 13 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 168.80 |=================================================================== b . 167.05 |================================================================== c . 168.20 |=================================================================== d . 168.29 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 4 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 22.03 |=================================================================== b . 22.16 |=================================================================== c . 22.39 |==================================================================== d . 22.24 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 8 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 152.48 |================================================================== b . 152.79 |================================================================== c . 153.10 |================================================================== d . 154.50 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 12 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 457.74 |=================================================================== b . 439.12 |================================================================ c . 441.42 |================================================================= d . 442.34 |================================================================= SVT-AV1 2.0 Encoder Mode: Preset 13 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 643.53 |=================================================================== b . 640.89 |=================================================================== c . 620.87 |================================================================= d . 627.59 |================================================================= Stockfish 16.1 Chess Benchmark Nodes Per Second > Higher Is Better a . 212133268 |============================================================= b . 207635904 |============================================================ c . 221316167 |================================================================ d . 214224198 |============================================================== Timed Linux Kernel Compilation 6.8 Build: defconfig Seconds < Lower Is Better a . 26.07 |=================================================================== b . 26.36 |==================================================================== c . 26.04 |=================================================================== d . 26.13 |=================================================================== Timed Linux Kernel Compilation 6.8 Build: allmodconfig Seconds < Lower Is Better a . 181.56 |================================================================== b . 183.15 |=================================================================== c . 182.43 |=================================================================== d . 181.28 |================================================================== Parallel BZIP2 Compression 1.1.13 FreeBSD-13.0-RELEASE-amd64-memstick.img Compression Seconds < Lower Is Better a . 1.249184 |================================================================= b . 1.235156 |================================================================ c . 1.235287 |================================================================ d . 1.252094 |================================================================= Primesieve 12.1 Length: 1e12 Seconds < Lower Is Better a . 2.213 |==================================================================== b . 2.163 |================================================================== c . 2.159 |================================================================== d . 2.163 |================================================================== Primesieve 12.1 Length: 1e13 Seconds < Lower Is Better a . 25.03 |==================================================================== b . 24.98 |==================================================================== c . 25.08 |==================================================================== d . 24.93 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 137.95 |=================================================================== b . 137.28 |=================================================================== c . 136.56 |================================================================== d . 137.82 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 461.38 |================================================================== b . 462.52 |=================================================================== c . 465.46 |=================================================================== d . 460.91 |================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 34.06 |==================================================================== b . 34.12 |==================================================================== c . 33.99 |==================================================================== d . 33.89 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 29.35 |==================================================================== b . 29.29 |==================================================================== c . 29.41 |==================================================================== d . 29.50 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 4590.23 |================================================================== b . 4598.88 |================================================================== c . 4605.99 |================================================================== d . 4589.15 |================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 13.92 |==================================================================== b . 13.89 |==================================================================== c . 13.87 |==================================================================== d . 13.92 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 304.76 |=================================================================== b . 304.80 |=================================================================== c . 304.90 |=================================================================== d . 303.84 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 3.2778 |=================================================================== b . 3.2774 |=================================================================== c . 3.2761 |=================================================================== d . 3.2877 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 1789.70 |================================================================== b . 1792.69 |================================================================== c . 1781.89 |================================================================== d . 1789.81 |================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 35.72 |==================================================================== b . 35.67 |==================================================================== c . 35.87 |==================================================================== d . 35.71 |==================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 342.28 |=================================================================== b . 342.90 |=================================================================== c . 341.44 |=================================================================== d . 343.53 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 2.9188 |=================================================================== b . 2.9135 |=================================================================== c . 2.9257 |=================================================================== d . 2.9080 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 11719.01 |================================================================= b . 11722.59 |================================================================= c . 11702.18 |================================================================= d . 11674.25 |================================================================= Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 5.4463 |=================================================================== b . 5.4445 |=================================================================== c . 5.4544 |=================================================================== d . 5.4675 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 1060.95 |================================================================= b . 1079.57 |================================================================== c . 1065.97 |================================================================= d . 1055.18 |================================================================= Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 0.9394 |=================================================================== b . 0.9232 |================================================================= c . 0.9356 |================================================================== d . 0.9449 |=================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 3.1028 |================================================================= b . 2.9676 |============================================================== c . 2.9506 |============================================================== d . 3.2137 |=================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 16509.14 |============================================================== b . 17153.23 |================================================================= c . 17224.36 |================================================================= d . 16021.35 |============================================================ Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 14.50 |==================================================================== b . 14.29 |=================================================================== c . 14.34 |=================================================================== d . 14.28 |=================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 68.89 |=================================================================== b . 69.88 |==================================================================== c . 69.64 |==================================================================== d . 69.95 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 1800.71 |================================================================== b . 1788.80 |================================================================= c . 1803.23 |================================================================== d . 1794.85 |================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 35.50 |==================================================================== b . 35.73 |==================================================================== c . 35.45 |=================================================================== d . 35.61 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 343.46 |=================================================================== b . 342.21 |=================================================================== c . 342.44 |=================================================================== d . 344.05 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 2.9085 |=================================================================== b . 2.9192 |=================================================================== c . 2.9173 |=================================================================== d . 2.9035 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 864.35 |=================================================================== b . 863.67 |=================================================================== c . 863.85 |=================================================================== d . 863.93 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 73.97 |==================================================================== b . 74.00 |==================================================================== c . 74.04 |==================================================================== d . 74.03 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 197.61 |=================================================================== b . 195.87 |================================================================== c . 195.83 |================================================================== d . 197.01 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 5.0580 |================================================================== b . 5.1030 |=================================================================== c . 5.1037 |=================================================================== d . 5.0734 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 1238.06 |================================================================== b . 1241.55 |================================================================== c . 1233.71 |================================================================== d . 1236.95 |================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 51.66 |==================================================================== b . 51.51 |==================================================================== c . 51.84 |==================================================================== d . 51.71 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 199.07 |=================================================================== b . 198.34 |================================================================== c . 200.28 |=================================================================== d . 198.38 |================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 5.0199 |=================================================================== b . 5.0378 |=================================================================== c . 4.9896 |================================================================== d . 5.0371 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 181.79 |=================================================================== b . 182.11 |=================================================================== c . 182.81 |=================================================================== d . 182.55 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 351.87 |=================================================================== b . 350.38 |=================================================================== c . 349.80 |=================================================================== d . 350.44 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 35.83 |==================================================================== b . 35.85 |==================================================================== c . 35.19 |=================================================================== d . 35.77 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 27.88 |=================================================================== b . 27.87 |=================================================================== c . 28.39 |==================================================================== d . 27.93 |=================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 1904.08 |================================================================== b . 1902.91 |================================================================== c . 1902.18 |================================================================== d . 1902.93 |================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 33.57 |==================================================================== b . 33.59 |==================================================================== c . 33.59 |==================================================================== d . 33.60 |==================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 100.21 |=================================================================== b . 99.71 |================================================================== c . 100.95 |=================================================================== d . 98.75 |================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 9.9739 |================================================================= b . 10.0233 |================================================================= c . 9.8997 |================================================================= d . 10.1216 |================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 138.31 |=================================================================== b . 137.79 |=================================================================== c . 137.97 |=================================================================== d . 137.97 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 460.73 |=================================================================== b . 461.22 |=================================================================== c . 461.99 |=================================================================== d . 462.03 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 34.18 |==================================================================== b . 34.23 |==================================================================== c . 34.15 |==================================================================== d . 34.20 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 29.25 |==================================================================== b . 29.21 |==================================================================== c . 29.28 |==================================================================== d . 29.24 |==================================================================== Google Draco 1.5.6 Model: Lion ms < Lower Is Better a . 4060 |==================================================================== b . 4041 |==================================================================== c . 4092 |===================================================================== d . 4059 |==================================================================== Google Draco 1.5.6 Model: Church Facade ms < Lower Is Better a . 4837 |===================================================================== b . 4793 |==================================================================== c . 4765 |==================================================================== d . 4820 |===================================================================== srsRAN Project 23.10.1-20240219 Test: PDSCH Processor Benchmark, Throughput Total Mbps > Higher Is Better a . 43297.4 |======================================================= a . 46252.5 |========================================================== b . 50572.1 |================================================================ c . 52220.4 |================================================================== d . 49150.3 |============================================================== srsRAN Project 23.10.1-20240219 Test: PDSCH Processor Benchmark, Throughput Thread Mbps > Higher Is Better a . 755.6 |==================================================================== b . 728.7 |================================================================== c . 755.3 |==================================================================== d . 756.0 |====================================================================