| VCC Blogs - Entry for ehawes |
Syndicate ehawes's entries |
2009/02/09
Cluster a La Cart' anyone?
What do you do with old gear? Here is one way to have a little fun and create an interesting solution for High Performance Computing.
The CALC cluster packs the following -
-2.8 TB of on-board storage
-96GB of RAM
-GBE network with trunking capability
-48 CPUs 3.06 Intel
-30 min. of backup power
-Runs on 4 110v standard power outlets
-Requires no special cooling - Can run in office environment or anywhere. Even on a ship for on-board supercomputing needs.

The reason that vCompute built this portable cluster was to experiment with the inflexibility of current high performance clusters with respect to power, special heavy-duty cooling, etc...
After making an investment in Verari blades we realized that the technology was too bulky and too hard to cool. These blades, according to Verari management, required the Verari mega-dense cabinet which was very expensive and costly to operate. I retired the gear within a year of purchase as it was too costly to operate. The Verari design was poorly conceived and was wasteful in terms of Energy consumption and cost to operate.
A friend of mine, who had a talent of packaging things, suggested we build a cabinet ourselves and design it with portability, efficiency and self sufficiency in mind. We then took off on the road of thinking about a design. We wanted to show the industry that clusters can be cheap to operate and portable. We also wanted to recover the cost of the Verari gear. We looked into selling the gear but the blades would only bring about 150 dollars each on ebay as no one seemed interested in Verari technology.

This is the sweet part. Most of the parts used to build this unit came from Home Depot and Lowes and Fry’s, see below:
-- 5 Shelf Cart: Lowes
-- Cart Wheels: The Container store (support 300 pounds per wheel)
-- Raw Aluminum Stock: Home Depot and Lowes
-- Blade Frame/Guides: Lowes: Screen Door Frames: Used for guides for the blades
-- Bolts/Nuts/Washers: Home Depot
-- 22 - 60CFM Fans on the bottom of the blades : Electronic Parts Outlet
-- 18 - 84CFM Fans: Fry’s: on the top of the blades.
-- Plexiglass: Home Depot. Used to seal top of the blades for air flow control
-- Bridge Rectifiers: Electronic Parts Outlet: Convert 110/AC to 48/DC for lower fan assembly
-- Surge Protectors: Electronic Parts Outlet
-- UPS: Fry’s
-- 1U Blade Server: which serves as a keyboard holder: Old Stock
-- 24 Port Gigabit Switch: Dell
-- Cat-6 Cables: Directron
-- 48 CPU coolers: Dynatron
-- Tie Wraps (lots of these): Walmart and the "99 cent only store"

We decided to name the beast CALC-1 which is an acronym for Cluster Al La Cart. What is really interesting,look for the pun, is the fan assemblies. A row of fans on the top of the blades and the other on the bottom: This is called a Fanwich. Get it.... Cluster a la Cart with a fanwich on the cart???? Oh well. The fanwich moves about 3000 CFM (cubic feet per minute) of air through the blades. The Dynatron blowers keep the CPU’s nice and cool and force more air across the blade. The heat dissipation increased 70% above the Verari Mega-DenseRack and with this design we decreased our cost power and cooling by 88%.


The whole CALC-1 consumes about 26 Amperes of power. The actual computer hardware is described as follows:
Each blade is configured as:
-- Dual Processor Motherboard
-- 2 x Xeon 3.06 GHz Processors
-- 4GB ECC registered 3200 memory
-- 120GB 7200 RPM EIDE Drives
-- 400W Private Power Supply
-- Dual Ethernet Gigabit Interface
-- USB 2.0 Compliant
-- IPMI Daughter Card
-- PXE Bootable
The head node is configured as:
-- Dual Processor IU server
-- 2 x Pentium IV 3.06 Processors
-- 1GB ECC registered Memory
-- 120GB SATA 7200 RPM SATA
Drives
-- 400W Private Power Supply
-- Dual Ethernet Gigabit Interface
-- USB 2.0 Compliant
Oh... I need to mention that the whole cluster operates on standard 110 volt power... about 4 curcuits without any special cooling needs.
We currently have Rock Linux running on all of the nodes.
The nodes are loaded via PXE...in other words, all of the client nodes are booted with some DHCP magic from the head node who listens for bootp requests. Since the head node is instructed to boot from any MAC address, it does so. Viola’ you don’t need any floppies or cdrom units on each client blade (called the hive)


This unit sold immediatley. It's current tasks is to run seismic jobs. It is running Landmark Graphics seismic applications and it is very fast and reliable. The latest result that we received is that it performs faster than a 64 node cluster that has been used in the past. In other words.... It rocks!!!! So now our clients want more built and who knows what this one will look like? Anyone have any old Verari blades hanging around?

The CALC cluster packs the following -
-2.8 TB of on-board storage
-96GB of RAM
-GBE network with trunking capability
-48 CPUs 3.06 Intel
-30 min. of backup power
-Runs on 4 110v standard power outlets
-Requires no special cooling - Can run in office environment or anywhere. Even on a ship for on-board supercomputing needs.

The reason that vCompute built this portable cluster was to experiment with the inflexibility of current high performance clusters with respect to power, special heavy-duty cooling, etc...
After making an investment in Verari blades we realized that the technology was too bulky and too hard to cool. These blades, according to Verari management, required the Verari mega-dense cabinet which was very expensive and costly to operate. I retired the gear within a year of purchase as it was too costly to operate. The Verari design was poorly conceived and was wasteful in terms of Energy consumption and cost to operate.
A friend of mine, who had a talent of packaging things, suggested we build a cabinet ourselves and design it with portability, efficiency and self sufficiency in mind. We then took off on the road of thinking about a design. We wanted to show the industry that clusters can be cheap to operate and portable. We also wanted to recover the cost of the Verari gear. We looked into selling the gear but the blades would only bring about 150 dollars each on ebay as no one seemed interested in Verari technology.

This is the sweet part. Most of the parts used to build this unit came from Home Depot and Lowes and Fry’s, see below:
-- 5 Shelf Cart: Lowes
-- Cart Wheels: The Container store (support 300 pounds per wheel)
-- Raw Aluminum Stock: Home Depot and Lowes
-- Blade Frame/Guides: Lowes: Screen Door Frames: Used for guides for the blades
-- Bolts/Nuts/Washers: Home Depot
-- 22 - 60CFM Fans on the bottom of the blades : Electronic Parts Outlet
-- 18 - 84CFM Fans: Fry’s: on the top of the blades.
-- Plexiglass: Home Depot. Used to seal top of the blades for air flow control
-- Bridge Rectifiers: Electronic Parts Outlet: Convert 110/AC to 48/DC for lower fan assembly
-- Surge Protectors: Electronic Parts Outlet
-- UPS: Fry’s
-- 1U Blade Server: which serves as a keyboard holder: Old Stock
-- 24 Port Gigabit Switch: Dell
-- Cat-6 Cables: Directron
-- 48 CPU coolers: Dynatron
-- Tie Wraps (lots of these): Walmart and the "99 cent only store"

We decided to name the beast CALC-1 which is an acronym for Cluster Al La Cart. What is really interesting,look for the pun, is the fan assemblies. A row of fans on the top of the blades and the other on the bottom: This is called a Fanwich. Get it.... Cluster a la Cart with a fanwich on the cart???? Oh well. The fanwich moves about 3000 CFM (cubic feet per minute) of air through the blades. The Dynatron blowers keep the CPU’s nice and cool and force more air across the blade. The heat dissipation increased 70% above the Verari Mega-DenseRack and with this design we decreased our cost power and cooling by 88%.


The whole CALC-1 consumes about 26 Amperes of power. The actual computer hardware is described as follows:
Each blade is configured as:
-- Dual Processor Motherboard
-- 2 x Xeon 3.06 GHz Processors
-- 4GB ECC registered 3200 memory
-- 120GB 7200 RPM EIDE Drives
-- 400W Private Power Supply
-- Dual Ethernet Gigabit Interface
-- USB 2.0 Compliant
-- IPMI Daughter Card
-- PXE Bootable
The head node is configured as:
-- Dual Processor IU server
-- 2 x Pentium IV 3.06 Processors
-- 1GB ECC registered Memory
-- 120GB SATA 7200 RPM SATA
Drives
-- 400W Private Power Supply
-- Dual Ethernet Gigabit Interface
-- USB 2.0 Compliant
Oh... I need to mention that the whole cluster operates on standard 110 volt power... about 4 curcuits without any special cooling needs.
We currently have Rock Linux running on all of the nodes.
The nodes are loaded via PXE...in other words, all of the client nodes are booted with some DHCP magic from the head node who listens for bootp requests. Since the head node is instructed to boot from any MAC address, it does so. Viola’ you don’t need any floppies or cdrom units on each client blade (called the hive)


This unit sold immediatley. It's current tasks is to run seismic jobs. It is running Landmark Graphics seismic applications and it is very fast and reliable. The latest result that we received is that it performs faster than a 64 node cluster that has been used in the past. In other words.... It rocks!!!! So now our clients want more built and who knows what this one will look like? Anyone have any old Verari blades hanging around?

The comments are owned by the poster. We aren't responsible for their content.

Syndicate ehawes's entries

