SPEC® MPIM2007 Result

AMD, QLogic Corporation, Rackable Systems, IWILL

AMD Emerald Cluster: AMD Opteron CPUs,
QLogic InfiniPath/SilverStorm Interconnect

SPECmpiM_peak2007 = Not Run

SPECmpiM_base2007 = 17.3

MPI2007 license:	0018	Test date:	May-2007
Test sponsor:	QLogic Corporation	Hardware Availability:	Nov-2006
Tested by:	QLogic Performance Engineering	Software Availability:	Jul-2007

Results Table

Benchmark	Base							Peak
Benchmark	Ranks	Seconds	Ratio	Seconds	Ratio	Seconds	Ratio	Ranks	Seconds	Ratio	Seconds	Ratio	Seconds	Ratio
Results appear in the order in which they were run. Bold underlined text indicates a median measurement.
104.milc	256	83.0	18.8	85.4	18.3	82.3	19.0
107.leslie3d	256	316	16.5	351	14.9	361	14.4
113.GemsFDTD	256	867	7.28	889	7.10	886	7.12
115.fds4	256	84.9	23.0	87.0	22.4	86.3	22.6
121.pop2	256	339	12.2	373	11.1	375	11.0
122.tachyon	256	207	13.5	195	14.3	203	13.8
126.lammps	256	229	12.8	230	12.7	229	12.7
127.wrf2	256	292	26.7	294	26.5	296	26.3
128.GAPgeofem	256	93.6	22.1	94.9	21.7	95.2	21.7
129.tera_tf	256	164	16.9	164	16.9	164	16.9
130.socorro	256	197	19.4	196	19.5	193	19.8
132.zeusmp2	256	142	21.9	143	21.7	142	21.8
137.lu	256	102	36.1	109	33.8	111	33.1

Hardware Summary
Type of System:	Homogenous
Compute Node:	Rackable, IWILL, AMD
Interconnects:	QLogic InfiniBand HCAs and switches Broadcom NICs, Force10 switches
File Server Node:	Headnode NFS filesystem
Head Node:	Rackable, IWILL, AMD
Other Node:	Headnode NFS filesystem
Total Compute Nodes:	64
Total Chips:	128
Total Cores:	256
Total Threads:	256
Total Memory:	512 GB
Base Ranks Run:	256
Minimum Peak Ranks:	--
Maximum Peak Ranks:	--

Software Summary
C Compiler:	QLogic PathScale C Compiler 3.0
C++ Compiler:	QLogic PathScale C++ Compiler 3.0
Fortran Compiler:	QLogic PathScale Fortran Compiler 3.0
Base Pointers:	64-bit
Peak Pointers:	64-bit
MPI Library:	QLogic InfiniPath MPI 2.1
Other MPI Info:	None
Pre-processors:	No
Other Software:	None

Node Description: Rackable, IWILL, AMD

Hardware
Number of nodes:	64
Uses of the node:	compute, head
Vendor:	Rackable Systems, IWILL, AMD
Model:	Rackable Systems C1000 chassis, IWILL DK8-HTX motherboard
CPU Name:	AMD Opteron 290
CPU(s) orderable:	1-2 chips
Chips enabled:	2
Cores enabled:	4
Cores per chip:	2
Threads per core:	1
CPU Characteristics:	--
CPU MHz:	2800
Primary Cache:	64 KB I + 64 KB D on chip per core
Secondary Cache:	1 MB I+D on chip per core
L3 Cache:	None
Other Cache:	None
Memory:	8 GB (8 x 1 GB DDR400)
Disk Subsystem:	250 GB, SATA
Other Hardware:	Nodes custom-built by Rackable Systems. The Rackable C1000 chassis is half-depth with 450W, 48 VDC Power Supply. Integrated Gigabit Ethernet for admin/filesystem.
Adapter:	Intel 82541PI Gigabit Ethernet controller
Number of Adapters:	1
Slot Type:	integrated on motherboard
Data Rate:	1 Gbps Ethernet
Ports Used:	1
Interconnect Type:	Ethernet
Adapter:	QLogic InfiniPath QHT7140
Number of Adapters:	1
Slot Type:	HTX
Data Rate:	InfiniBand 4x SDR
Ports Used:	1
Interconnect Type:	InfiniBand

Software
Adapter:	Intel 82541PI Gigabit Ethernet controller
Adapter Driver:	Part of Linux kernel modules
Adapter Firmware:	None
Adapter:	QLogic InfiniPath QHT7140
Adapter Driver:	InfiniPath 2.1
Adapter Firmware:	None
Operating System:	ClusterCorp Rocks 4.2.1 (Based on RedHat Enterprise Linux 4.0 Update 4)
Local File System:	Linux ext3
Shared File System:	NFS
System State:	Multi-User
Other Software:	Sun Grid Engine 6.0

Node Description: Headnode NFS filesystem

Hardware
Number of nodes:	1
Uses of the node:	file server, other
Vendor:	Tyan
Model:	Thunder K8QSD Pro (S4882) motherboard
CPU Name:	AMD Opteron 885
CPU(s) orderable:	1-4 chips
Chips enabled:	4
Cores enabled:	8
Cores per chip:	2
Threads per core:	1
CPU Characteristics:	--
CPU MHz:	2600
Primary Cache:	64 KB I + 64 KB D on chip per core
Secondary Cache:	1 MB I+D on chip per core
L3 Cache:	None
Other Cache:	None
Memory:	16 GB (16 x 1 GB DDR400 dimms)
Disk Subsystem:	250 GB, SATA, 7200 RPM
Other Hardware:	None
Adapter:	Broadcom BCM5704C
Number of Adapters:	2
Slot Type:	integrated on motherboard
Data Rate:	1 Gbps Ethernet
Ports Used:	2
Interconnect Type:	Ethernet

Software
Adapter:	Broadcom BCM5704C
Adapter Driver:	Part of Linux kernel modules
Adapter Firmware:	None
Operating System:	ClusterCorp Rocks 4.2.1 (Based on RedHat Enterprise Linux 4.0 Update 4)
Local File System:	Linux ext3
Shared File System:	NFS
System State:	Multi-User
Other Software:	Sun Grid Engine 6.0

General Notes

"other" purposes of this node: login, compile, job submission
and queuing.
This node assembled with a 2U chassis and 700 watt ATX 12V Power Supply.

Interconnect Description: QLogic InfiniBand HCAs and switches

Hardware
Vendor:	QLogic
Model:	InfiniPath and Silverstorm
Switch Model:	QLogic SilverStorm 9120 Fabric Director
Number of Switches:	1
Number of Ports:	144
Data Rate:	InfiniBand 4x SDR and InfiniBand 4x DDR
Firmware:	3.4.0.5.2
Topology:	Single switch (star)
Primary Use:	MPI traffic

General Notes

The data rate between InifniPath HCAs and SilverStorm switches
is SDR. However, DDR is used for inter-switch links.

Interconnect Description: Broadcom NICs, Force10 switches

Hardware
Vendor:	Force10
Model:	E300
Switch Model:	Force10 E300 Gig-E switch
Number of Switches:	1
Number of Ports:	288
Data Rate:	1 Gbps Ethernet
Firmware:	N/A
Topology:	Single switch (star)
Primary Use:	file system traffic

Base Compiler Invocation

C benchmarks:

/usr/bin/mpicc -cc=pathcc

C++ benchmarks:

126.lammps:

/usr/bin/mpicxx -CC=pathCC

Fortran benchmarks:

107.leslie3d:	/usr/bin/mpif90 -f90=pathf90
113.GemsFDTD:	/usr/bin/mpif90 -f90=pathf90
115.fds4:	/usr/bin/mpif90 -f90=pathf90
129.tera_tf:	/usr/bin/mpif90 -f90=pathf90
132.zeusmp2:	/usr/bin/mpif90 -f90=pathf90
137.lu:	/usr/bin/mpif90 -f90=pathf90

Benchmarks using both Fortran and C (except as noted below):

/usr/bin/mpicc -cc=pathcc /usr/bin/mpif90 -f90=pathf90

Base Portability Flags

104.milc:	-DSPEC_MPI_LP64
121.pop2:	-DSPEC_MPI_DOUBLE_UNDERSCORE -DSPEC_MPI_LP64
122.tachyon:	-DSPEC_MPI_LP64
127.wrf2:	-DF2CSTYLE -DSPEC_MPI_DOUBLE_UNDERSCORE -DSPEC_MPI_LINUX -DSPEC_MPI_LP64
128.GAPgeofem:	-DSPEC_MPI_LP64
130.socorro:	-fno-second-underscore -DSPEC_MPI_LP64

Base Optimization Flags

C benchmarks:

-march=opteron -Ofast -OPT:malloc_alg=1

C++ benchmarks:

126.lammps:

-march=opteron -O3 -OPT:Ofast -CG:local_fwd_sched=on

Fortran benchmarks:

107.leslie3d:	-march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off
113.GemsFDTD:	-march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off
115.fds4:	-march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off
129.tera_tf:	-march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off
132.zeusmp2:	-march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off
137.lu:	-march=opteron -O3 -OPT:Ofast -OPT:malloc_alg=1 -LANG:copyinout=off

Benchmarks using both Fortran and C:

121.pop2:	-march=opteron -Ofast -OPT:malloc_alg=1 -O3 -OPT:Ofast -LANG:copyinout=off
127.wrf2:	Same as 121.pop2
128.GAPgeofem:	Same as 121.pop2
130.socorro:	Same as 121.pop2

Base Other Flags

C benchmarks:

-IPA:max_jobs=4

C++ benchmarks:

126.lammps:

-IPA:max_jobs=4

Fortran benchmarks:

107.leslie3d:	-IPA:max_jobs=4
113.GemsFDTD:	-IPA:max_jobs=4
115.fds4:	-IPA:max_jobs=4
129.tera_tf:	-IPA:max_jobs=4
132.zeusmp2:	-IPA:max_jobs=4
137.lu:	-IPA:max_jobs=4

Benchmarks using both Fortran and C (except as noted below):

-IPA:max_jobs=4

The flags file that was used to format this result can be browsed at
http://www.spec.org/mpi2007/flags/MPI2007_flags.20070717.01.html.

You can also download the XML flags source by saving the following link:
http://www.spec.org/mpi2007/flags/MPI2007_flags.20070717.01.xml.