Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

Hyper-Cluster by chenleihor


By John D. Lambert, Microsoft SQL Server PFE

    •   Introduction
    •   Install and Configure Hyper-V Host
    •   Install WSS08 as Hyper-V Guest
    •   Install Node1
    •   Install Node2
    •   Configure iSCSI Targets
    •   Configure Node1
    •   Configure Node2
    •   Install and Configure Cluster
    •   Appendix A: Sample Network Chart

By "Hyper-Cluster" I mean a complete cluster, including shared disk space, all virtualized on a
single physical computer. A hyper-cluster is not a recommended configuration for a production
system for many reasons, but it's great for practicing with cluster-enabled services. This
document describes how I created a Windows Server 2008 R2-based Hyper-Cluster, using
Windows Storage Server 2008's iSCSI for shared disks. This minimum configuration uses a host
and 3 virtual servers:
    • Host (for Hyper-V, Active Directory/Domain Controller, DNS)
    • Storage Server (for shared disk space and iSNS)
    • Cluster Node 1
    • Cluster Node 2

Once you've got the Hyper-V host running, turn on the Remote Desktop feature, and you can log
into it from somewhere else on the planet to create or play with your hyper-cluster.

Please bear in mind that I'm not a Hyper-V expert, and this isn't an official white paper on the
best way to do everything, it's just a basic record of what I did to get my Hyper-Cluster working.
I had a bunch of folks ask me how I did it under Server 2008 (not R2) and I didn't have any notes
from that effort, so this time I figured I'd jot a few things down in case anyone's interested. I’ll
also point out that this is not a comprehensive step-by-step guide, as I skip lots of common
details, like clicking past splash screens and common Hyper-V management tasks.

Target vs. Initiator:
    An iSCSI Target is a shared hard drive or group of drives, and they are shared by an iSCSI
    Target service. iSCSI Targets are used by iSCSI Initiators, and the initiators are also called
    iSCSI clients or iSCSI target clients. It's that last one that causes a lot of confusion.

                                             Page 1 of 10
An iSCSI Qualified Name (IQN) is a globally unique identifier for both targets and initiators.

In my Hyper-Cluster, Windows Storage Server 2008 (WSS or WSS08) provides virtual disk space
as iSCSI targets which are used by the iSCSI initiators in my cluster nodes. The cluster nodes see
them as if they were local physical disks.

WSS installs with a default admin password but doesn't tell you. Once it boots up for the first
time, you have to log in as the administrator, but it never told you the default password, which
is buried in the release-notes of the Tools CD on MSDN. I list it below, but just in case you're
reading this document only because you're in your own desperate hunt for it, it's wSS2008!.

I also stumbled on the WSS firewall because I expected that enabling the iSCSI Service rules
(inbound and outbound) would cover all relevant traffic, especially since they seem to be very
loose rules. It didn't, however, and it wasted a lot of my time before troubleshooting pinned
down my communication problem to the WSS firewall, and I had to add an inbound rule for the
iSCSI TCP ports. Also, even though you define a virtual network as private in Hyper-V's Virtual
Network Manager, that doesn't mean your guest machines will see it as private. To the virtual
guests, that's a real network, and the Windows firewall will consider it a "Public" network as
opposed to "Private" or "Domain."

One area I suspect might trip up some folks is an unavailable network in WSS. This could happen
for 2 reasons: you haven't assigned a network to the virtual machine; or you haven't installed
the Hyper-V integration services. With Server 2008 R2 as your node OS, the integration services
are automatic, but you still have to make a network available in the virtual machine settings.

Similarly, it's really easy to get fouled up on which Local Area Connection is which and assign the
wrong subnets, so you really need to match the MAC addresses as I point out later on.

Install and Configure Hyper-V Host
This isn't intended as a basic Hyper-V document, but I'm listing the O/S, roles, hardware, and
virtual networks I used. If you're new to Hyper-V you should find basic instructions elsewhere,
but the general idea is to install Windows Server 2008 R2, then install the roles, and then define
the virtual networks in the Hyper-V Manager. And in case you missed it elsewhere, your
microprocessor has to support virtualization for Hyper-V to work.

The hardware listed is what I'm running this on, but you can run a Hyper-Cluster on a single hard
drive, and I think you could do it with as little as 2GB RAM although performance will suffer. I
did this once before using Server 2008 (not R2) as the host and cluster nodes using only 4GB of
RAM and just the 140GB drive, but of course performance is better with more memory and if
the load is split between 2 physical drives. With 2 drives, I put the host Hyper-V on the C: drive
and everything else on the other drive.

O/S: Windows Server 2008 R2 Enterprise, 64-bit

                                            Page 2 of 10
Roles installed:
    • Active Directory Certificate Services
    • Active Directory Domain Services
    • DNS Server
    • Hyper-V

   • 8 GB RAM
   • 140GB SATA drive
   • 500GB SATA drive

Hyper-V Networks:
   • Public Network (connects to external network card, allows management OS to share the
   • iSCSI Virtual Network (private)
   • Heartbeat Virtual Network (private)

Install WSS08 as Hyper-V Guest
1. From host Hyper-V Manager, create a new virtual machine.
                 Name: WSS08
                 Location: E:\ProgramData\Microsoft\Windows\Hyper-V\
                 Memory: 512MB
                 Connection: Public Network
                 Virtual Hard Disk: E:\ProgramData\Microsoft\Windows\Hyper-V\WSS08.vhd,
                 Install OS from ISO:
2. From host Hyper-V Manager, right-click on WSS08 (under the list of Virtual Machines) and
    choose Settings to begin adding shared disks.
        a. Click on the SCSI Controller, then click the Add button, then the New button. I
            created 3 virtual drives for WSS to share, using Fixed Size, and the default location:
            E:\Users\Public\Documents\Hyper-V\Virtual Hard Disks\, but you can put them in
            any drive and folder that the Hyper-V host can see. After adding one virtual drive,
            you have to click on SCSI Controller again before adding another one.
                 • Quorum.vhd, 1GB
                 • SqlData.vhd, 20GB
                 • SqlLog.vhd, 2GB
Remember that because this instance of WSS08 is a virtual guest, it can only see the drives
provided by Hyper-V. If you want more than a C: drive, you'll need to add or more SCSI drives via
Hyper-V (up to 64 virtual drives). You only need one virtual disk for WSS08, because you also
create virtual disks in the iSCSI Target service, but I use multiple virtual disks at this point
because I'm trying to emulate how I would be configuring things if I had a big honkin' SAN.
3. From host Hyper-V Manager, open the Settings on WSS08 and add the iSCSI Network.
        a. Click on Add Hardware, Network Adapter, then the Add button.
                                           Page 3 of 10
          b. Choose the iSCSI Virtual Network and click OK.
4.   From host Hyper-V Manager, turn on WSS08 and connect.
5.   Install Workgroup or higher edition by using a key for the edition you want to run. The Basic
     edition of WSS08 does NOT support iSCSI targeting. And if you want to play around with
     clustering the iSCSI target service, then you have to use the Enterprise edition of WSS.
6.   Log in as admin with the default admin password, which is wSS2008!. If you're new to
     Hyper-V, click the little button on the far left side of the button bar to send a Ctrl-Alt-Delete
     to the guest operating system.)
7.   On the Hyper-V container, click the Action drop-down menu and click Insert Integration
     Services Setup Disk (on type Ctrl-I) and install the Hyper-V integration services. I had to do
     this even though the service is supposed to be included already in WSS 2008, because it
     wasn't running and I couldn't find it after a little poking around.
8.   Change the IP address, mask, and gateway for the public network on WSS08, and the IP
     address and mask for the iSCSI network, according to the chart in Appendix A or your own
     requirements. When you have more than one virtual adapter to configure, you have to
     figure out which adapter goes with which virtual network. Here's how I did it:
          a. Go into the Hyper-V Manager and open the Properties for the WSS08 virtual server,
              click on the network adapter that says "Public Network" below it, and look at the
              MAC address (even though it's grayed-out).

         b. Back in WSS08, open the Network Connections and double-click on either of either
            of the Local Area Connections, then click the Details... button and see if the MAC
            address from the previous step matches the Physical Address. If it matches, this is
            the Local Area Connection for the Public Network, and it if doesn't match it's the
            LAC for the iSCSI network.

         c. Close the Details window and then the Status window, then right-click the Local
                                              Page 4 of 10
           Area Connection and rename it to Public Network or iSCSI Network, then rename
           the other one. It's essential to get these straight; otherwise you'll put the wrong IP
           addresses in them.
        d. For both the Public and iSCSI networks, right-click and choose Properties, then
           double-click on "Internet Protocol Version 4 (TCP/IPv4)", click on "Use the following
           IP address:" and enter the values from the table in Appendix A or whatever network
           values you need.

9. Change the computer name, add the server to the domain running in your Hyper-V host,
    and set the time zone.
10. From the host's Hyper-V Manager, change the Hyper-V settings for WSS08 to assign the DVD
    drive to the iSCSI CD (en_windows_storage_server_2008_iscsi_cd_x64_x86_x15-49563).
    You can do this whether WSS08 is on or off.
11. Back inside WSS, open the DVD/CD to the x64 folder and launch iscsitarget.msi. Remember,
    the '.msi' is hidden unless you've change the default folder views. This adds "Microsoft iSCSI
    Software Target" to the Administrative Tools folder and a service with automatic startup
    that's also listed as "Microsoft iSCSI Software Target". The iscsitarget is the version for
    hosting an iSCSI session on WSS; the iscsitargetclient is for the cluster system to use the
    iscsitargetclient, but you won't need the client software since it's build into Windows Server
    2008 (and Windows 7).
12. From the Server Manager (or from the Initial Configuration Tools window if you have that
    open), click on Features, then Add Features, and add "Internet Storage Name Server". This
    is also referred to as iSNS, and it adds "iSNS Server" to Administrative Tools. Though I got a
    previous hyper-cluster to communicate fine without it (no surprise since everything is on the
    same LAN segment), it's good practice to include it since it will probably be used in most
    production environments.
13. Format the virtual shared disks so they can be seen by the iSCSI targeting service:
         a. Open Administrative Tools --> Computer Management, and click on Disk
             Management (or whatever your favorite way to get there is).
         b. Wait for the Initialize Disk window to open and click Okay to initialize the virtual
             SCSI disks you assigned to WSS08 from the Hyper-V Manager.
         c. For each virtual SCSI disk:
                    i. Right-click on the graphical representation of each of these unformatted
                       drives and chose New Simple Drive...
                   ii. Accept the maximum size and click Next.
                                           Page 5 of 10
                  iii. Assign a drive letter and click Next. My only suggestion is to use Q for the
                       quorum disk.
                  iv. Choose NTFS, the default allocation unit size (or align them if you know your
                       physical drive's allocation unit size), assign a label to match the name you
                       gave the underlying .vhd in the Hyper-V Manager (e.g. Quorum, SqlData,
                       SqlLog), choose the Perform A Quck Format option, and click Next.
14. Configure the firewall.
          a. Launch Server Manager, expand the Configuration node, then the Windows Firewall
              node, or go to Administrative Tools --> Windows Firewall.
          b. Go to Inbound Rules, scroll down to "iSCSI Service (TCP-In)", right-click it and choose
              "Enable Rule."
          c. Go to Outbound Rules and enable "iSCSI Service (TCP-Out)."
          d. Now right-click Inbound Rules and choose New Rule..., Port, then Next.
          e. Leave TCP selected and type in "135, 3260" for local ports.
          f. Leave Allow The Connection selected and click Next.
          g. Leave all profiles checked. (WSS sees the iSCSI virtual network as a Public network.)
          h. Give it a descriptive name like "iSCSI TCP Ports" and click Finish.
It seems to me like it should be enough to just enable the built-in iSCSI service rules (both
inbound and outbound), but that didn't work for me even though it's a pretty loose rule. UDP
port 138 must also be open to your iSCSI clients, but that was open in a default rule on my
installation. Of course, you could just turn off the firewall, but it's good training to learn how to
get it right.

Now install Windows on the cluster nodes. That way, when we come back to WSS and configure
the iSCSI targets, it'll be able to see the clients so we can allow our cluster node clients and only
our cluster node clients to connect to our iSCSI targets. Limiting access to iSCSI targets is
obviously necessary to keep the data secure.

Install Node1
15. From the host Hyper-V Manager, create a new virtual machine.
                Name: Node1
                Location: E:\ProgramData\Microsoft\Windows\Hyper-V\
                Memory: 2048MB
                Connection: Public Network
                Virtual Hard Disk: E:\ProgramData\Microsoft\Windows\Hyper-V\Node1.vhd,
                Install OS from ISO:
16. From the host Hyper-V Manager, open the Settings on Node1 and add the iSCSI and
    heartbeat networks. Though only one network is required for a cluster, this represents a
    real-world scenario where one private network is used exclusively for iSCSI traffic, and
    another is used exclusively for a heartbeat.
        a. Click on Add Hardware, Network Adapter, then the Add button.
        b. Choose the iSCSI Virtual Network and click OK.
        c. Repeat to add the Heartbeat Virtual Network.
                                             Page 6 of 10
17. From the host Hyper-V Manager, turn on Node1 and connect.
18. Choose Windows Server 2008 R2 Enterprise (Full Installation) x64 (Only Enterprise and
    DataCenter editions support clustering. Hyper-V integration services are included in the OS
    and don't need to be installed from the Hyper-V manager.)
19. Change the Public Network IP address, gateway, and DNS server in the network settings.
20. Add an IP Address and mask for the iSCSI and Heartbeat networks. These networks don't
    need a gateway or DNS server.
21. Change the server name and join the node to the domain.
22. Optional: Activate Windows and install all updates.
23. Run Administrative Tools --> iSCSI Initiator.
       a. Click yes to start the service and set it to autostart. This also registers Node1 with
       b. Click the Discovery tab, then the Add Server... button.
       c. Type in WSS08 as the iSNS server name and click Okay.
       d. Click yes to unblock the iSCSI service in the firewall.
       e. Click the cancel button to close the iSCSI Initiator Properties window or just leave it
            open for later.

Now we want to get Node 2 to the same state.

Install Node2
        Installation is identical to Node1 except for the virtual machine name and server name.

Configure iSCSI Targets
24. Back on WSS08, launch Administrative Tools --> Microsoft iSCSI Software Target to open the
    iSCSITarget window.
25. In the iSCSITarget window, right-click on Microsoft iSCSI Software Target and choose
26. On the Network tab, uncheck everything except the IP address for the iSCSI Network
    ( if you're following Appendix A).
27. Click on the iSNS tab, click the Add button, and enter "WSS08" as the DNS name of the iSNS
    server and hit Enter, then OK to close the properties window.
28. Right-click iSCSI Targets and choose Create iSCSI Target.
29. On the iSCSI Target Identification screen, type in a target name, which will be used by the
    iSCSI clients. I used "HyperClusterSharedDisks."
30. On the iSCSI Initiators Identifiers screen, click the Advanced button, and for each node
    you're going to have, click the Add button, then the Browse button, and choose one of the
    iSCSI initiators listed. This tells the iSCSI target service which computers are allowed to
    connect to this target. You can also edit these values later. Once you've finished this step,
    you only have an empty target. Now you have to add virtual hard drives to the target.
31. For each virtual shared disk you want to create via iSCSI:
         a. Right-click on the target you just created and choose "Create Virtual Disk for iSCSI
         b. On the File screen, enter a path and file name for a new .vhd or browse to an
                                            Page 7 of 10
           existing one. If you choose an existing one, it will warn you that its data will be
           overwritten. And yes, this will create a virtual disk inside a virtual disk file. To help
           me keep track of inner vs. outer virtual disks, I name these inner disks with a
           "Level2" suffix. E.g., For my quorum disk, I enter "Q:\QuorumDiskLevel2.vhd". The Q
           drive itself is a .vhd at the Hyper-V level, and this new QuorumDiskLevel2.vhd file is
           on the Q drive, so it's inside the quorum.vhd file. Cool, huh? This is a necessary
           characteristic of a Hyper-Cluster, but there's no need to virtualize any additional
        c. On the Size screen, note that you have to enter values in Megabytes, even though
           it's telling you the space available in Gigabytes. Annoying, isn't it?

You should now be done with WSS, and its iSCSI targets should be available to your cluster
nodes' iSCSI initiators.

Configure Node1
32. If you didn't leave the iSCSI Initiator open, launch it from the Administrative Tools folder or
     use Start -- Run... iscsicpl.
33. If everything is configured correctly, you should have 2 entries under Discovered targets on
     the Targets tab. The iSCSI Qualified Name (IQN) of hyperclustershareddisks should be
     highlighted but have an inactive status. Click the Connect button to connect it.
34. Leave the checkmark to add this target to your Favorite Targets, which will enable the iSCSI
     initiator on this node to automatically reconnect to this target after reboots.
35. Click the Advanced button.
          a. Switch the Local Adapter to "Microsoft iSCSI Initiator."
          b. Set the Initiator IP to " if you're using Appendix A.
          c. Set the Target Portal IP to " / 3260" if you using Appendix A.
          d. Click the OK button.
If you get a connection failure, you'll need to double check your network configurations and
firewall rules. You can temporarily disable your firewalls just to determine if they’re the problem.
36. Click on the Volumes and Devices tab, click the Auto Configure button, and click OK.
37. Open Disk Manager on this node and for each new disk:
          a. Find the gray box on the left side of the graphical representation of the disk where it
              says "Offline" or "Unknown."
          b. If it says "Offline," then right-click in and click "Online." It should now say
          c. When it says "Unknown," right-click the gray box and choose "Initialize Disk."

        d. Now right-click the white rectangle that represents the disk and choose "New
           Simple Volume..." Use the maximum size and assign the same drive letter and label
                                            Page 8 of 10
             you used in WSS08 for that disk and use the quick format option.
38. Open Server Manager, click on the Features node, and click on Add Features.
39. Click Failover Clustering, Next, and Install.

Configure Node2
Configure Node2 identically to Node1 with these exceptions:
   • The Initiator IP is and the Target Portal IP is
   • You can skip the Disk Manager for Node2 since these are shared disks. However, if you
       want to use them on Node2 before you create a cluster, you'll need to open the disk
       manager, right-click, and click to bring the disks online. If you do bring them online on
       Node2, you won't have to initialize them, format them, or label them. Also, if you bring
       them online this way, the drive letters probably won't match what you set them to in

Install and Configure Cluster
After installing Failover Clustering on both nodes:
40. In Server Manager on either one of the nodes, under Features, click on Failover Cluster
    Manager (or Administrative Tools --> Failover Cluster Manager).
41. Under Management, click on "Validate a Configuration."
42. Enter "node1" and click Add, then enter "node2" and click Add, then click Next.
43. Choose Run All Tests, click Next, and Next one more time to start the validation process.
    Watch the status to see all the cool tests the validator does that you don't have to!
44. If the validation report shows any problems, fix them before going on.
45. When the validation report is clear, I recommend that you make Hyper-V snapshops of your
    virtual machines (WSS08 and your cluster nodes) so you can roll back to this clean pre-
    cluster configuration. This is especially useful if you want to practice creating a cluster
    several times without having to recreate your Hyper-Cluster environment.

At this point, you're through configuring your Hyper-Cluster, and setting up a cluster and
clustered services should be exactly the same as if it were on physical systems.

If you're really anxious to create your cluster and don't have that documentation handy, just
find the "Create a Cluster..." link inside the Failover Cluster Manager on any of the cluster

One final tip: Once you're got your cluster created, consider making another set of Hyper-V
snapshots of WSS08 and each cluster node. Then if you make a change that breaks something,
you can roll it back.

                                           Page 9 of 10
Appendix A: Sample Network Chart
 Network        Setting           Host            WSS08           Node1            Node2
  Public      IP Address
  Public     Subnet Mask
  Public       Gateway
  Public      DNS Server
   iSCSI      IP Address          N/A
   iSCSI     Subnet Mask          N/A
   iSCSI       Gateway            N/A              N/A             N/A              N/A
   iSCSI      DNS Server          N/A              N/A             N/A              N/A
 Heartbeat    IP Address          N/A              N/A
 Heartbeat   Subnet Mask          N/A              N/A
 Heartbeat     Gateway            N/A              N/A             N/A              N/A
 Heartbeat    DNS Server          N/A              N/A             N/A              N/A

The default gateway and DNS server are needed on the Public network for Internet access for
Windows activation and updates; the Hyper-V host doesn't need the iSCSI or Heartbeat
networks; and only the cluster nodes need the Heartbeat network.

                                          Page 10 of 10

To top