1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

Preview: A Cheap 12-Core, 30-Watt Ubuntu Cluster

Ubuntu

Published on 07 June 2012 09:40 AM EDT
Written by Michael Larabel in Ubuntu
18 Comments

It's quite easy these days to build a many-core compute cluster that is low-powered, running Linux, and performant-friendly. Here's a small cluster build that's begun at Phoronix and has twelve 1.2GHz cores while the total system power consumption under load is just about 30 Watts.

More details will be shared in the next week or two, but I'm in the process of assembling a small ARM cluster running Ubuntu Linux. This cluster will be used for delivering some interesting ARM Linux benchmarks, of course! It's also being used for Phoronix Test Suite purposes for creating more MPI/cluster benchmarks for some of the commercial/enterprise clients, coming up with more ARM Linux benchmarks in general, and this hardware is also going to be part of a much larger (~96+ ARM core cluster - details to be shared at a later time) cluster.

With low-powered ARM hardware and the onslaught of affordable development boards, it's rather easy to build a modestly powerful cluster. At the heart of this current cluster build is the PandaBoard ES. I'm quite a fan of the PandaBoard ES development board and have been benchmarking it since last year. The PandaBoard ES has the OMAP4460 SoC from Texas Instruments with a dual-core Cortex-A9 1.2GHz processor and PowerVR graphics (fortunately that's irrelevant for cluster purposes). There's 1GB of RAM on the PandaBoard ES, two USB 2.0 ports, HDMI output, 10/100 Ethernet, and there's an SDHC slot for storage. More information on the PandaBoard ES, which is an upgraded version of the original PandaBoard (TI OMAP4430 vs. OMAP4460) is available at PandaBoard.org. What also makes the PandaBoards wonderful is that Ubuntu makes official releases and daily images of Ubuntu ARM for the OMAP4, which tend to "just work" on these TI ARM development boards.

Preview: A Cheap 12-Core, 30-Watt Ubuntu Cluster

Being used for this build are six PandaBoard ES units. Having twelve 1.2GHz Cortex-A9 cores should be rather interesting to benchmark and be sufficient for the aforementioned purposes, but of course I'm quite excited to see the huge Calxeda ARM servers that are forthcoming and hopefully one at least will end up at Phoronix. The much larger PandaBoard cluster to be talked about later should also be quite interesting to see and quite powerful with a low power budget.

Preview: A Cheap 12-Core, 30-Watt Ubuntu Cluster

The PandaBoard ES is currently retailing for $182 USD, so six of them will set you back around $1,000 plus the cost of SDHC cards and power supplies. Still, this is rather affordable for a 12-core ARM cluster. For this build, special thanks go out to Texas Instruments and SVTronics for providing a discount to make this Phoronix ARM cluster build slightly more affordable.

Preview: A Cheap 12-Core, 30-Watt Ubuntu Cluster

Under load, a single PandaBoard ES draws just about five Watts, or just shy of six Watts in some really demanding workloads. This cluster therefore should be running at just around ~30 Watts when being compute-happy. There are some power consumption results and other OMAP4460 PandaBoard ES benchmarks in Ubuntu 12.10 Sets To Make ARM Even Stronger, Ubuntu 12.04 LTS ARMv7 Linux Performance Gains, Ubuntu 12.04 ARM Performance Becomes Very Compelling, and Ubuntu 12.04 Is ARM-ing Up For Better Performance. This six-board cluster will be running an Ubuntu 12.10 ARM OMAP4 snapshot for the noticeable performance improvements it offers via the Linux 3.4 kernel and GCC 4.7.

Preview: A Cheap 12-Core, 30-Watt Ubuntu Cluster

One of the unusual things I'm trying for this build is to assemble it all within a wooden dish drying rack. This isn't the first time that ARM development boards have been used in a cluster, with Ubuntu/Linaro and others using PandaBoard clusters for their build farm, etc. The other approaches to efficiently managing all of the boards with minimal space has been stacking them with spacers between the PCBs, etc. Here is the Fedora ARM build farm example. The issues I see with that though is it makes the boards not swappable at all without dismantling the entire stack, time consuming to setup, and requires special parts.

Preview: A Cheap 12-Core, 30-Watt Ubuntu Cluster

A few days ago I came up with the idea of using a wooden dish drying rack. These racks can be found for a few dollars on the Internet, can be used almost "out of the box", would allow for multiple different development boards / PCB sizes / mounting hole differences, very easy to swap out boards, could be fabricated from scratch quite easily, and allow for fairly high density clusters in compact space. The shape should also allow for managing cables and placing of AC power supplies (underneath) fairly easy. The size of this dish drying rack though for a current six-board cluster is a bit large, but this concept may end up working quite well for others.

Preview: A Cheap 12-Core, 30-Watt Ubuntu Cluster

I'm still in the process of assembling this PandaBoard ES configuration -- both for the hardware and software -- but will have more updates on the progress in the next week or two. It should be quite interesting (plus the continued ARM and cluster improvements coming to the open-source Phoronix Test Suite benchmarking software for Linux), especially when looking at the performance-per-Watt against various multi-core x86 systems!

Any questions or feedback about this build is welcome in the forums via the discussion link below. Thanks again to Texas Instruments and SVTronics for their support in this project.

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux Hardware Reviews
  1. Scythe Mugen MAX
  2. Intel Core i7 5960X Haswell-E On Linux
  3. Intel 80GB 530 Series M.2 SSD On Linux
  4. With A New Motherboard, The Core i7 5960X Haswell-E Lights Up
Latest Linux Articles
  1. 7-Way Linux Desktop Gaming Comparison On Ubuntu 14.10
  2. Intel P-State vs. CPUFreq Benchmarks On The i7-5960X
  3. RadeonSI GLAMOR Benchmarks With X.Org Server 1.16
  4. RadeonSI Gallium3D vs. Catalyst At 4K UHD On Linux
Latest Linux News
  1. Mesa Gets Closer To Having OpenGL 4.0 Tessellation Support
  2. Uselessd: A Stripped Down Version Of Systemd
  3. F2FS Tools Gain FSCK Support
  4. FreeBSD 10.1 Has The New VT Driver, Hardware Improvements
  5. AntiMicro 2.6 Yields Greater Compatibility For Gamepads On Linux
  6. OpenGL 3.3 / GLSL 3.30 Lands For Intel Sandy Bridge On Mesa
  7. AMD's RadeonSI Gallium3D Driver Sees Some Improvements
  8. Mesa 10.3 Released With The Latest Open-Source GPU Driver Improvements
  9. GNOME 3.13.92 Officially Released
  10. Wine 1.7.27 Is Still Working Towards Direct2D Support
Latest Forum Discussions
  1. New stress testing utility for GPU's
  2. Stop grabbing my keyboard :(
  3. New Group Calls For Boycotting Systemd
  4. X.Org Women Outreach Program Only Turns Up Two Applicants So Far
  5. SSD seems slow
  6. R. Tyler restarts work on FreeBSD launchd port, openlaunchd
  7. Can Linux kill a motherboard?
  8. Glamor now enabled in Debian radeonsi