Intel® Select Solutions for BigDL on Apache Spark*

Accelerate and simplify deep learning development and deployment on an optimized, verified infrastructure.
View the solution brief ›

Base Configuration for Intel® Select Solutions for BigDL on Apache Spark*

Ingredient

Intel® Select Solutions for BigDL on Apache Spark* 
Base Configuration

Minimum of One Master Node

Processor

Intel® Xeon® Gold 6148 processor (2.40 GHz, 20 cores, 40 threads), Intel® Xeon® Gold 6248 processor (2.5 GHz, 20 cores), or a higher number Intel® Xeon® Scalable processor

Memory

192 GB or higher (12 x 16 GB DDR4-2666)

Boot Drive**

1 x 240 GB or larger Intel® SSD DC S4500/S4510 (M.2 or 2.5-inch) or higher

Storage HBA Controller Per Node**

Not applicable (N/A)

Data Tier

4 x 960 GB or larger Intel® SSD DC S4500/S4510 or higher

Data Network

10 Gb Intel® Ethernet Converged Network Adapter X722

Management Network Per Node

Integrated 1 GbE port 0/RMM port

Minimum of Four Worker Nodes

Processor

Intel® Xeon® Gold 6148 processor (2.40 GHz, 20 cores, 40 threads), Intel® Xeon® Gold 6248 processor (2.5 GHz, 20 cores), or a higher number Intel® Xeon® Scalable processor

Memory

384 GB or higher (12 x 32 GB DDR4-2666)

Boot Drive**

1 x 240 GB or larger Intel® SSD DC S4500/S4510 (M.2 or 2.5-inch) or higher

Storage HBA Controller Per Node**

N/A

Data Tier

4 x 3.8 TB or larger Intel® SSD DC S4500/S4510 or higher

Data Network

10 Gb Intel® Ethernet Converged Network Adapter X722 (Intel® Ethernet CNA X722)

Management Network Per Node

Integrated 1 GbE port 0/RMM port

Network Switches

Top of the Rack (ToR) Switch

10 Gbps 48x port switch

Management Switch

1 Gbps 48x port switch

Software

Linux* OS

CentOS Linux* release 7.5.1804 or Red Hat Enterprise Linux* (RHEL*) 7

Apache Spark*

2.2.0

Apache Hadoop*

2.7.3

Java Development Kit* (JDK*)

Oracle* JDK 1.8.0 update 181 or later

BigDL

0.6

Analytics Zoo

0.2

Intel® Distribution for Python*

2.7

Intel® Math Kernel Library (Intel® MKL)

Intel® MKL version 2018 Update 3

Applies to All Nodes

Trusted Platform Module (TPM)

TPM 1.2 discrete or firmware TPM (Intel® Platform Trust Technology [Intel® PTT])

Firmware and Software Optimizations

Intel® Volume Management Device (Intel® VMD) enabled**

Intel® Boot Guard enabled**

Intel® Hyper-Threading Technology (Intel® HT Technology) disabled

Intel® Turbo Boost Technology enabled

P-states enabled**

C-states enabled**

Power-management settings set to performance**

Workload configuration set to balanced**

Memory Latency Checker (MLC) streamer enabled**

MLC spatial prefetch enabled**

Data Cache Unit (DCU) data prefetch enabled**

DCU instruction prefetch enabled**

Last-level cache (LLC) prefetch disabled**

Uncore frequency scaling enabled**

Minimum Performance Standards

Verified to meet or exceed the following minimum performance capabilities:

ImageNet* Training Throughput

375 images per second with Top-5 Accuracy of 85% or higher1
**Recommended, not required

Product and Performance Information

1

Performance results are based on testing as of October 10, 2018, and may not reflect all publicly available security updates. See configuration disclosure for details. No product can be absolutely secure. Configuration: One master node: 2 x Intel® Xeon® Gold 6148 processor, Intel® Server Board S2600WFT, total memory: 192 GB, 12 slots/16 GB/2,666 megatransfers per second (MT/s) DDR4 RDIMM, Intel® Hyper-Threading Technology (Intel® HT Technology) disabled, Intel® Turbo Boost Technology enabled; storage (boot): 1 x 240 GB Intel® SSD DC S4500; storage: 4 x 960 GB Intel® SSD DC S4500; network devices: 1 x 10 Gb Intel® Ethernet Network Adapter X722, network speed: 10 GbE, ucode: 0x043, OS/software: CentOS* Linux release 7.5.1804; four worker nodes: 2 x Intel® Xeon® Gold 6148 processor, Intel® Server Board S2600WFT, total memory: 384 GB, 12 slots/32 GB/2,666 megatransfers per second (MT/s) DDR4 RDIMM, Intel® HT Technology disabled, Intel® Turbo Boost Technology enabled; storage (boot): 1 x 240 GB Intel® SSD DC S4500; storage: 4 x 3.8 TB Intel® SSD DC S4500; network devices: 1 x 10 Gb Intel® Ethernet Network Adapter X722, network speed: 10 GbE, ucode: 0x043, OS/software: CentOS* Linux release 7.5.1804.