This is a benchmark of Apache Spark with its PySpark interface. Apache Spark is an open-source unified analytics engine for large-scale data processing and dealing with big data. This test profile benchmars the Apache Spark in a single-system configuration using spark-submit. The test makes use of DIYBigData's pyspark-benchmark (https://github.com/DIYBigData/pyspark-benchmark/) for generating of test data and various Apache Spark operations.
Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.
Component
Details
Percentile Rank
# Compatible Public Results
Seconds (Average)
Zen 3 [128 Cores / 256 Threads]
99th
10
16
Zen 3 [128 Cores / 256 Threads]
97th
4
18
Zen 2 [128 Cores / 256 Threads]
95th
4
18
Zen 3 [64 Cores / 128 Threads]
94th
4
24
Zen 3 [64 Cores / 128 Threads]
92nd
4
32
Ice Lake [80 Cores / 160 Threads]
92nd
3
32
Zen 2 [64 Cores / 128 Threads]
90th
5
38 +/- 1
Zen 3 [24 Cores / 48 Threads]
86th
5
50 +/- 1
Raptor Lake [24 Cores / 32 Threads]
84th
5
52 +/- 1
Zen 2 [32 Cores / 64 Threads]
81st
3
56
Zen 4 [16 Cores / 32 Threads]
81st
10
56 +/- 1
Zen 4 [16 Cores / 32 Threads]
78th
8
59 +/- 1
Zen+ [32 Cores / 64 Threads]
77th
4
61
Cascade Lake [48 Cores / 96 Threads]
75th
6
66
Zen 2 [24 Cores / 48 Threads]
72nd
8
70 +/- 1
Broadwell [44 Cores / 88 Threads]
71st
4
71 +/- 1
Zen 4 [12 Cores / 24 Threads]
68th
11
73 +/- 2
Zen 4 [12 Cores / 24 Threads]
65th
8
78 +/- 1
Zen [32 Cores / 64 Threads]
64th
4
79 +/- 1
Zen+ [24 Cores / 48 Threads]
64th
3
79 +/- 1
Zen 3 [16 Cores / 32 Threads]
62nd
23
81 +/- 6
Alder Lake [16 Cores / 24 Threads]
62nd
4
82 +/- 1
Zen 4 [12 Cores / 24 Threads]
62nd
4
82 +/- 1
Zen 2 [16 Cores / 32 Threads]
56th
3
88
Cascade Lake [18 Cores / 36 Threads]
53rd
4
103
Zen 4 [8 Cores / 16 Threads]
52nd
8
107 +/- 1
Zen+ [16 Cores / 32 Threads]
51st
3
109 +/- 1
Zen 4 [8 Cores / 16 Threads]
50th
5
112
Zen 4 [8 Cores / 16 Threads]
48th
6
117 +/- 2
Zen 2 [12 Cores / 24 Threads]
47th
3
122 +/- 1
Alder Lake [10 Cores / 16 Threads]
46th
3
124
Zen 3 [8 Cores / 16 Threads]
46th
3
136 +/- 1
Zen 2 [12 Cores / 24 Threads]
43rd
4
139
Zen 3 [8 Cores / 16 Threads]
39th
6
147 +/- 2
Zen 4 [6 Cores / 12 Threads]
39th
12
148 +/- 10
Zen 3 [8 Cores / 16 Threads]
38th
7
157 +/- 5
Cascade Lake [16 Cores / 32 Threads]
35th
4
165
Zen 2 [8 Cores / 16 Threads]
31st
8
187 +/- 1
Rocket Lake [8 Cores / 16 Threads]
26th
7
229
Alder Lake [14 Cores / 20 Threads]
25th
9
230 +/- 23
Zen 3 [8 Cores / 16 Threads]
24th
7
231 +/- 3
Zen 3 [8 Cores / 16 Threads]
22nd
8
242 +/- 6
[8 Cores / 8 Threads]
20th
5
251 +/- 1
Kaby/Coffee/Whiskey Lake [6 Cores / 12 Threads]
19th
3
265 +/- 1
Zen 2 [8 Cores / 16 Threads]
16th
3
305 +/- 3
Zen 2 [8 Cores / 8 Threads]
15th
7
317 +/- 15
Rocket Lake [6 Cores / 12 Threads]
13th
5
334 +/- 1
Zen 2 [6 Cores / 6 Threads]
12th
6
359 +/- 22
Comet Lake [8 Cores / 16 Threads]
12th
3
374 +/- 1
[4 Cores / 8 Threads]
8th
10
405 +/- 29
Kaby/Coffee/Whiskey Lake [4 Cores / 8 Threads]
8th
3
405 +/- 1
Skylake [8 Cores / 8 Threads]
7th
4
441 +/- 1
Alder Lake [4 Cores / 8 Threads]
4th
4
584 +/- 6
Ice Lake [4 Cores / 8 Threads]
2nd
3
709 +/- 50
