This guide* describes how to build a small Windows HPC cluster that you can use to run your parallel software applications, cluster-enabled Microsoft Excel workbooks, or service-oriented architecture (SOA) based applications. This guide is intended for those of you out there who want to set up a small development or proof-of-concept cluster, or for those of you who do not have an IT department to set up a cluster for you, and so you need to do it yourself.
After you complete the steps in this guide, you will have a cluster that is ready for you to install applications and start running jobs. Many parallel computing software applications provide documentation about how to install their software on a Windows HPC cluster.
If you have all the required parts and permissions, and you already have the Windows Server 2008 R2 operating system installed on your computers, the steps in this guide should take about one to two hours to complete.
In this guide:
*This guide is the companion to the video DIY Supercomputing: How to build a small Windows HPC cluster.
In this guide, we will walk through the steps to build an HPC cluster that consists of at least three nodes (computers with the server operating system installed). One node acts as a head node (to manage the cluster) and as a compute node (to run jobs). Additional nodes act only as compute nodes.
Note: The head node also acts as a WCF broker node to manage SOA service requests and responses. The WCF broker functionality is necessary if you are using the HPC Services for Excel to offload your UDF calculations or Microsoft Excel spreadsheets, or if your application uses the SOA programming model.
The cluster nodes are connected to each other on an isolated, private network. This network is basically used to coordinate the workload on your cluster. The head node has an additional network connection to the enterprise network (a network with an Active Directory domain controller that manages logging in, security, and authentication in your work environment). After your cluster is set up, you can submit and monitor jobs by logging in to the head node directly, or by connecting to the cluster from another computer on the enterprise network (if you install the HPC Pack client utilities on that computer).
Because we are building a small cluster, and to keep things simple, we will build a cluster with preconfigured compute nodes. That means that the computers that we add to the cluster already have the operating system installed, and we will manually install the HPC software on each node.
Note: If you are building a larger cluster, then read about how to deploy nodes from bare metal. In a bare metal deployment, you take some extra steps to set up automated installation of the operating system and HPC software to all your compute nodes.
The following diagram illustrates the cluster plan:
*You can see the minimum system requirements for Windows HPC here (http://go.microsoft.com/fwlink/?LinkId=194785). However, if you want to run specific software applications on your cluster, consult the software documentation for any additional system requirements.
For information about how to download the software, see Download Windows HPC Server 2008 R2 software.
Network and permissions
Note: You can reopen the Initial configuration task console by opening a command prompt window and then typing “oobe”.
Note: Both editions work for creating a head node. Select the Enterprise edition if you want to use or explore the features for Excel offloading and for adding workstation computers to your cluster (this lets you use workstation computers that are on the enterprise network to run HPC jobs during non-working hours).
Note: This step must be performed, but these credentials are only used for bare metal deployments. If you are only adding preconfigured nodes, you can use your domain credentials, even if they don’t have permissions to add nodes to the Active Directory.
Note: The node naming series does not apply to preconfigured compute nodes, it is only used for bare metal deployments.
Note: In this guide, the nodes that we are adding already have the Windows Server 2008 R2 operating system installed, so we do not need the node template to install an operating system for us. Since we are adding preconfigured nodes, we only really use the template to add nodes to the cluster. For bare metal deployment, you can use the node template to install an operating system on the compute nodes.
Note: If you include this step, your head node must be able to access the internet or a Windows Update Server on your enterprise network.
For each compute node, perform the following steps.
Note: This procedure assumes that the nodes already have the Windows Server 2008 R2 operating system installed.
Note: Both editions work for creating a compute node. Select the Enterprise edition if you want the compute nodes to be able to run Excel offloading jobs.
Note: Bringing a node to the Online state tells the HPC Job Scheduler Service that you want this node to run jobs. Keep in mind that Online only reflects the intended usage of the node, and that the node must also be healthy and reachable. If you want to stop using a node for jobs, then bring the node to the Offline state.