OpenVINO GenAI
This is a benchmark of OpenVINO GenAI that makes use of Intel OpenVINO, a toolkit around neural networks / AI. Complementing the pts/openvino test profile, pts/openvino-genai is focused on Generative AI (GenAI) LLM performance.
OpenVINO GenAI 2024.5
Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU
OpenBenchmarking.org metrics for this test profile configuration based on 135 public results since 23 November 2024 with the latest data as of 13 January 2025.
Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.
Component
Details
Percentile Rank
# Compatible Public Results
tokens/s (Average)
Zen 5 [64 Cores / 128 Threads]
100th
7
66.0
Zen 4 [192 Cores / 384 Threads]
81st
6
56.6 +/- 0.6
Zen 5 [96 Cores / 192 Threads]
78th
15
56.0 +/- 0.5
Zen 4 [192 Cores / 384 Threads]
64th
4
46.8 +/- 0.5
Zen 4 [32 Cores / 64 Threads]
58th
4
36.5 +/- 0.1
Zen 4 [64 Cores / 128 Threads]
55th
3
34.8
Arrow Lake [24 Cores / 24 Threads]
53rd
3
27.5 +/- 0.1
Zen 5 [16 Cores / 32 Threads]
50th
4
25.2
Zen 5 [8 Cores / 16 Threads]
47th
4
21.6
Zen 4 [12 Cores / 24 Threads]
39th
7
20.2 +/- 0.2
Zen 4 [12 Cores / 24 Threads]
38th
5
19.8
Zen 4 [16 Cores / 32 Threads]
30th
7
19.7
Zen 4 [16 Cores / 32 Threads]
26th
7
19.3 +/- 0.2
Zen 4 [8 Cores / 16 Threads]
21st
3
19.1 +/- 0.2
Zen 4 [8 Cores / 16 Threads]
19th
4
19.1 +/- 0.1
Zen 5 [10 Cores / 20 Threads]
16th
3
18.9 +/- 0.9
Zen 5 [12 Cores / 24 Threads]
13th
5
17.0 +/- 0.1
Zen 4 [4 Cores / 8 Threads]
10th
3
16.7
Alder Lake [14 Cores / 20 Threads]
2nd
3
9.4 +/- 1.1
