1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking Benchmarking Platform
Phoromatic Test Orchestration

Preview: A Cheap 12-Core, 30-Watt Ubuntu Cluster

Ubuntu

Published on 07 June 2012 09:40 AM EDT
Written by Michael Larabel in Ubuntu
18 Comments

It's quite easy these days to build a many-core compute cluster that is low-powered, running Linux, and performant-friendly. Here's a small cluster build that's begun at Phoronix and has twelve 1.2GHz cores while the total system power consumption under load is just about 30 Watts.

More details will be shared in the next week or two, but I'm in the process of assembling a small ARM cluster running Ubuntu Linux. This cluster will be used for delivering some interesting ARM Linux benchmarks, of course! It's also being used for Phoronix Test Suite purposes for creating more MPI/cluster benchmarks for some of the commercial/enterprise clients, coming up with more ARM Linux benchmarks in general, and this hardware is also going to be part of a much larger (~96+ ARM core cluster - details to be shared at a later time) cluster.

With low-powered ARM hardware and the onslaught of affordable development boards, it's rather easy to build a modestly powerful cluster. At the heart of this current cluster build is the PandaBoard ES. I'm quite a fan of the PandaBoard ES development board and have been benchmarking it since last year. The PandaBoard ES has the OMAP4460 SoC from Texas Instruments with a dual-core Cortex-A9 1.2GHz processor and PowerVR graphics (fortunately that's irrelevant for cluster purposes). There's 1GB of RAM on the PandaBoard ES, two USB 2.0 ports, HDMI output, 10/100 Ethernet, and there's an SDHC slot for storage. More information on the PandaBoard ES, which is an upgraded version of the original PandaBoard (TI OMAP4430 vs. OMAP4460) is available at PandaBoard.org. What also makes the PandaBoards wonderful is that Ubuntu makes official releases and daily images of Ubuntu ARM for the OMAP4, which tend to "just work" on these TI ARM development boards.

Preview: A Cheap 12-Core, 30-Watt Ubuntu Cluster

Being used for this build are six PandaBoard ES units. Having twelve 1.2GHz Cortex-A9 cores should be rather interesting to benchmark and be sufficient for the aforementioned purposes, but of course I'm quite excited to see the huge Calxeda ARM servers that are forthcoming and hopefully one at least will end up at Phoronix. The much larger PandaBoard cluster to be talked about later should also be quite interesting to see and quite powerful with a low power budget.

Preview: A Cheap 12-Core, 30-Watt Ubuntu Cluster

The PandaBoard ES is currently retailing for $182 USD, so six of them will set you back around $1,000 plus the cost of SDHC cards and power supplies. Still, this is rather affordable for a 12-core ARM cluster. For this build, special thanks go out to Texas Instruments and SVTronics for providing a discount to make this Phoronix ARM cluster build slightly more affordable.

Preview: A Cheap 12-Core, 30-Watt Ubuntu Cluster

Under load, a single PandaBoard ES draws just about five Watts, or just shy of six Watts in some really demanding workloads. This cluster therefore should be running at just around ~30 Watts when being compute-happy. There are some power consumption results and other OMAP4460 PandaBoard ES benchmarks in Ubuntu 12.10 Sets To Make ARM Even Stronger, Ubuntu 12.04 LTS ARMv7 Linux Performance Gains, Ubuntu 12.04 ARM Performance Becomes Very Compelling, and Ubuntu 12.04 Is ARM-ing Up For Better Performance. This six-board cluster will be running an Ubuntu 12.10 ARM OMAP4 snapshot for the noticeable performance improvements it offers via the Linux 3.4 kernel and GCC 4.7.

Preview: A Cheap 12-Core, 30-Watt Ubuntu Cluster

One of the unusual things I'm trying for this build is to assemble it all within a wooden dish drying rack. This isn't the first time that ARM development boards have been used in a cluster, with Ubuntu/Linaro and others using PandaBoard clusters for their build farm, etc. The other approaches to efficiently managing all of the boards with minimal space has been stacking them with spacers between the PCBs, etc. Here is the Fedora ARM build farm example. The issues I see with that though is it makes the boards not swappable at all without dismantling the entire stack, time consuming to setup, and requires special parts.

Preview: A Cheap 12-Core, 30-Watt Ubuntu Cluster

A few days ago I came up with the idea of using a wooden dish drying rack. These racks can be found for a few dollars on the Internet, can be used almost "out of the box", would allow for multiple different development boards / PCB sizes / mounting hole differences, very easy to swap out boards, could be fabricated from scratch quite easily, and allow for fairly high density clusters in compact space. The shape should also allow for managing cables and placing of AC power supplies (underneath) fairly easy. The size of this dish drying rack though for a current six-board cluster is a bit large, but this concept may end up working quite well for others.

Preview: A Cheap 12-Core, 30-Watt Ubuntu Cluster

I'm still in the process of assembling this PandaBoard ES configuration -- both for the hardware and software -- but will have more updates on the progress in the next week or two. It should be quite interesting (plus the continued ARM and cluster improvements coming to the open-source Phoronix Test Suite benchmarking software for Linux), especially when looking at the performance-per-Watt against various multi-core x86 systems!

Any questions or feedback about this build is welcome in the forums via the discussion link below. Thanks again to Texas Instruments and SVTronics for their support in this project.

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux News
  1. Fedora 22 Is Being Released Next Tuesday
  2. OpenWRT 15.05 Preparing Improved Security & Better Networking
  3. Using The New LLVM/Clang OpenMP Support
  4. Zapcc Claims To Be A "Much Faster C++ Compiler"
  5. Godot 1.1 Engine Release Brings New 2D Engine
  6. Intel VA-API Driver 1.6 Is Coming
  7. Canonical Is Reportedly Considering An IPO
  8. GNOME 3.18 - GTK3 Now Supports RandR 1.5
  9. Fedora 22 Risks Being Delayed Beyond Next Week
  10. Systemd 220 Has Finally Been Released
  11. LibreOffice 5.0 Beta 1 Released
  12. Allwinner Publishes New CedarX Open-Source Code
Latest Articles & Reviews
  1. Btrfs RAID 0/1 Benchmarks On The Linux 4.1 Kernel
  2. The State Of Various Firefox Features
  3. Intel Iris Graphics Performance With Mesa 10.6
  4. Fedora Workstation 22 Is Looking Great, Running Fantastic
Most Viewed News This Week
  1. The Linux 4.0 Kernel Currently Has An EXT4 Corruption Issue
  2. AMDGPU Open-Source Driver Code Continues Maturing
  3. Oculus Rift Suspends Linux Development To Focus On Windows
  4. Microsoft Open-Sources The Windows Communication Foundation
  5. LibreOffice 5.0 Open-Source Office Suite Has Been Branched
  6. Another HTTPS Vulnerability Rattles The Internet
  7. Linux 4.1-rc4 Kernel Arrives A Day Late
  8. Wayland / Weston 1.8 Release Candidate Arrives