embedded 1-9-12-ISE-Perfmon v2.1

Document Sample
embedded 1-9-12-ISE-Perfmon v2.1 Powered By Docstoc
					ISE-Perfmon v2.1
Windows Perfmon Plugin for ISE Systems
Users Guide v1.0




                                         Written by:   Hollis Beall
                                                       Director, Technical Marketing
Table of Contents
Introduction .................................................................................................................................................. 3
Theory of Operation...................................................................................................................................... 4
Installation and Configuration ...................................................................................................................... 8
   Pre-Requisites ........................................................................................................................................... 8
   Installation Steps....................................................................................................................................... 8
Example Configuration Implementation..................................................................................................... 11
Appendix A: Resources For More Information .......................................................................................... 19
Introduction

          The ISE-Perfmon application is a light-weight application that utilizes the Web Services interface
of the ISE systems to record detailed performance information from ISE systems. This application
utilizes the Microsoft Windows “Perfmon” application as the data collection and capture mechanism for
performance data from the XIO ISE systems. It is intended to be a method for detailed capture of
performance information for trouble shooting and “baseline’ing” purposes. It can be used for long term
monitoring of ISE systems, but there is not the robust reconnection logic that is included with
applications such as ISE-Analyzer or Microsoft System Center Operations Manager. Connection
problems between the ISE and the Windows host will cause the application to fail, and require it to be
restarted. If long term trending is required, the user is strongly encouraged to evaluate ISE Analyzer,
Microsoft System Center Operations Manager, and/or SNMP monitoring methods that can
accommodate unreliable TCP/IP connections.

         Performance information is generated in the ISE system constantly, and is exposed through the
Web Services interface on the ISE. This content is generated constantly by the ISE, and obtaining it
through the Web Services interface imposes an insignificant impact on the operation of the ISE (so low it
was not possible to measure in the lab). Users are encouraged to explore the Web Services interface on
their ISE systems, and the API documentation can be downloaded from:

http://cortexdeveloper.com
http://cortexdeveloper.com/cortexdeveloper/m/apispecification/37/download.aspx

Users can start examining the ISE Web Services API by opening any web browser and going to the
following web address of their systems:

http://<ISE-IP-Address>/query

This will return top level XML data from the ISE, and will be briefly explored later in the guide.

This guide is not intended to be a tutorial on using Windows Perfmon, setting up Perfmon recordings, or
storage performance troubleshooting. While these topics are discussed in this document, the user
should have a good “working knowledge” of this tool. Links for information about general operation of
Perfmon is included in Appendix A.
Theory of Operation

The design goals of ISE-Perfmon were to be:
    1. Lightweight – Small Size to allow for ease of delivery. The zip file containing the program
       embedded right here.




    2. Simple to Install
       The only configuration required is a single IP address from each ISE to be monitored. The
       Perfmon application is included with every version of Microsoft Windows Operating system, and
       provides a platform for performance monitoring that has been in the field for more than 15
       years. There is no configuration of the counters inside of Perfmon for the user to do. The
       configuration and “objects”’ in each ISE are dynamically discovered and updated in Perfmon
       without any user intervention.

        For Example: A Perfmon recording can be created to record the performance data for all of the
        Virtual Disks in the ISE:
                                   (\\<local Machine Name>\ISE Virtual Disks\*)
        The above line tells the Perfmon capture to record all data inside of the “ISE Virtual Disk” Object.
        If the user creates new Virtual Disks, these are automatically included in all future recordings
        (once the Perfmon log is stopped and restarted). Entries for the new virtual disk are
        automatically added to Perfmon when they are included in the http pollig process, (other than
        stopping and restarting the counter log) no intervention is required by the user to include the
        new Virtual Disks in the performance captures.

        When the ISE-Perfmon application is started, it checks to ensure that the Perfmon “Objects” for
        the ISE systems exist on the local machine. If they do not, the program creates them and
        continues.

    3. Easy to Operate – Once the Configuration file is set, simply start the ISE-Perfmon application.
       When monitoring is no longer needed, simply close the application window. All
       communications occur over TCP/IP, and the Windows OS where the application is running down
       not have to be connected to the Fibre Channel fabric. The names of the ISEs are discovered as
       part of the process that ISE-Perfmon goes through when it starts up. The name of the ISEs are
       used to make analysis and correlation of the systems easier. If any two ISEs have the same
       name, then the ISE serial number is used. Perfmon cannot have 2 devices with the same name,
       or they will overwrite each other. If serial numbers are used in your recordings, ensure that the
       ISE names have all be set to unique values.
    4. Capable of collecting all of the performance information exposed by the ISE Web Services
       interface
       The ISE systems generate an ENORMOUS amount of data. There are 8x Perfmon “Objects” that
       the ISE data is organized into.

        1.   ISE Physical Disks
        2.   ISE Virtual Disks
        3.   ISE Controllers
        4.   ISE Replication
        5.   ISE System
        6.   ISE Backend
        7.   ISE FCPorts

As mentioned above, instances (ex: Virtual Disks, Host Connections, etc) are dynamically created when
they are discovered in the XML data that is returned from the ISE.

Each Perfmon “Instance” has between 14-16 “counters”. These are:
       1. totaliops
       2. readiops
       3. writeiops
       4. totalkbps
       5. readkbps
       6. writekbps
       7. readlatency
       8. readlatencymax
       9. queuedepth
       10. queuedepthmax
       11. avgxfrsize
       12. readpercent
       13. writelatency
       14. writelatencymax
       15. totaliosincelastboot
       16. totalkbsincelastboot

Note: Throughput values above are actually in Bytes, not Bits. This is a known error in the labeling
(using small “b” and not big “B”), and will be addressed in future versions of ISE-Perfmon.

Recently, a capture was done of three ISE systems in a benchmark test at 5 second intervals for 5 days.
This generated a Perfmon *.blg file that was 1.5 GB is size. Heavy activity will cause the file to grow
bigger than periods of low activity. (Storing a “0” IOPS takes a lot less space than “30000” IOPS). Each
ISE in this test contained eight Virtual Disks, and all 16 counters were required to be examined. This
resulted in 384 individual graphs that were generated to examine the performance of only the Virtual
Disks in the test. This is not meant to show that they ISE is overtly complicated. Rather, the ISE will
allow for extremely detailed performance examination of all system components to enable performance
monitoring/troubleshooting at the most granular level possible. (Note: XIO would be happy to examine
these performance files for our customers at no cost. Please contact your XIO representative for this
complementary service. We are happy to help.).

By utilizing the Windows Perfmon application as the data collection and visualization “engine”, data can
be viewed in real time from the ISE systems or recorded to any Perfmon format for analysis at a later
time. Users can use Perfmon to monitor ISE systems in the same fashion as they would for any other
Perfmon “counter”.

This application collected data from the ISE systems by utilizing the Microsoft MSXML libraries and the
.NET4 framework. When the ISE-Perfmon application is started, 8 new Perfmon “Objects” are created
on the local computer where the application is running. While the ISEs are “remote” systems, the
counters are created under the “local” machine.

ISE-Perfmon will collect data as fast as it can from the monitored ISE systems, and “polls” the ISEs in a
serial manner. Each Monitored ISE system adds approximately 1.25 sec to the collection period, so
monitoring 3x ISE systems will yield a “resolution” of no less than ≈3.75 seconds. This “polling” of the
ISEs occurs no matter what the sampling interval of Perfmon is set for. This is not a large concern with
small numbers of ISE (<5), but can be a consideration for large numbers of ISEs (>10). Perfmon sampling
intervals can be set for 1 second, but ISE Performance values will only change as quickly as the ISE-
Perfmon application poll all of the ISEs.
Below is a description of the communications that occur between the Windows Server and the ISE
system.

                                                                 1. ISE-Perfmon checks to ensure that “perfmon”
                                                                 objects exist/created on the local Windows OS




                                                                 2. ISE-Perfmon reads in ISE IP connection
                                                                 information from the config file
                                                     ISE



                                                                 3. ISE-Perfmon http connects to each ISE and
                              5
                                                                 obtains the “Name” of each ISE. If any 2 ISEs have
                                                                 the same name, the Serial number of all ISEs are
                                                                 used as the ISE “names” in the “perfmon” objects.

                 3                              6


                                                                 4. Dos window will output the list
                                                                 of ISEs that are being monitored




                                                4                5. Http request to
              1, 2                                               performance page of the ISE

                                                7

                                                                 6. ISE Returns XML page with
             Windows Server (2003, 2008), Windows 7
                                                                 ISE performance information
                            32-bit or 64-bit
                       Can be a Virtual Machine
        .Net 4 Installed (see Users Guide for correct version)

                                                                 7. ISE-Perfmon parses returned XML
                                                                 data, and populates local “perfmon”
                                                                 counters on the Windows computer
Installation and Configuration
The following are the requirements and procedures for installing and configuring ISE-Perfmon. A
“Sample Environment” is configured in the next section to show exactly what to do.


Pre-Requisites
       1. Microsoft Windows Operating System
              a. 32 or 64 bit
              b. Server 2003 or 2008, or Windows 7
              c. This can be on a Virtual Machine
       2. Microsoft .Net4 installed from the below link:
       3. Ethernet Connectivity to the ISE system
       4. Each ISE User Name and Password
              a. Default: administrator/administrator
       5. A Single IP address from each ISE to be monitored
              a. Each ISE has 2, and only 1 is used
       6. Working knowledge of Windows Perfmon


Installation Steps
   1. Ensure that all Perfmon windows are closed. Problems have been see where the ISE “instances”
      to not appear if ISE-Perfmon_v2.1 is started while and Perfmon Windows are open.
   2. Install the .NET4 update from the link below. Other “lighter” versions of .NET4 have been found
      not to work, so below version should be installed (at a minimum).
                           http://msdn.microsoft.com/en-us/library/5a4x27ek.aspx

   3. Log into the Web Interface of each ISE that is being monitored. Use the IP address that you are
      going to enter into the config file for each ISE. The address is:
                                             http://<ISE-IP-Address>

      Log into the ISE with the Username and Password that you are going to enter into the config file.
      This is different than the “ISE Manager” user name and password. By manually logging in to
      the web interface of each ISE that is being monitored, the user can ensure that the user name
      and password they are going to be using are correct to get into the ISE. This is a critical step, as
      having the wrong user name and password has been the source of several misconfigurations.
   4. Save the ISE-Perfmon v2.1.zip file to a location on the computer that will be doing the
      monitoring. Unzip the ISE-Perfmon v2.1.zip file. This does not have to be in it’s own directory,
      but it makes finding it easier.
   5. Open the ISE-Perfmon.cfg file to enter the IP Address, Username, and password for each ISE to
      be monitored. Below is a sample from the ISE-Perfmon.cfg file:
           #Example of entry for the config file:                            This is meant as an example and
           #<IP Address of 1 MRC>,<ISE User Name>,<ISE-Password>             should be removed/replaced with
           #10.10.10.3,administrator,administrator                           your entries for ISEs. Do not leave
           #                                                                 this entry in the file.
           #Enter the ISE connection information between the "$" symbols
           #$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
           10.0.2.224,administrator,administrator
           #$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
           #
           #end of file, this line and the next have to exist, do not modify this line or the one after
           this.
    Above shows a “sample” entry of what the ISE entries should look like. Notice a few things:
           1. There is only 1 IP address of the ISE to be monitored. This is critically important.
                Only use one (1) IP address of the ISE.
           2. There are “,” between the IP address, username, and password
                   a. There are no spaces
           3. Only enter values between the “$” characters. The end of the file must remain un-
                modified due to how the end of the file is determined



6. Ensure that Perfmon is not running.
       a. If Perfmon is running, then the new ISE Objects will not appear in the list of objects.
            Having Perfmon running will not cause the ISE Object creation to fail, but Perfmon must
            be restarted to view the new ISE Objects.
7. Run the ISE-Perfmon_v2.1.exe application.
8. ISE Perfmon Objects are checked/created
9. A “Dos” window will appear, and text will show that each us being “monitored”
       a. After the Dos windows shows they ISEs are being monitored, Perfmon may be started.
10. Check to ensure that monitoring is occurring
       a. This step is critical. Nothing upsets anyone more than getting a week’s worth of data to
            find out that nothing was recorded. (This is an easy thing to verify, and will save a lot of
            heartburn).
            1. Start Perfmon after the ISE-Perfmon application indicated that it is recording data
                (step 8 above).
            2. Select the “+” sign to add a counter to the “Live” view
                                                                    Select this to add a counter to the live view.




                     3. Scroll up to select the “ISE Controllers” Object, and click on the arrow to expand the
                        counters under this object.
                     4. Select one of the “counters” under this object. You should see the “Instances”
                        populate with the controllers of the ISEs that you are monitoring.
Perfmon “Object”
“ISE Controllers”
                     \\SVR-1




                                                                   Select the Arrow to display the
                                                                   “Counters” under the “Object
Counter to select
under the “Object”
                     Test-ISE1
                     Test-ISE1
                     Test-ISE1
                                                   Entries for the ISEs you are
                                                   monitoring should appear
                                                   here
Example Configuration Implementation

To illustrate how to configure the performance monitoring, we will examine a sample environment
where we have three ISE systems. These systems have been “named” in the ISE:
     1. Hyper-ISE1
     2. ISE28.8-VDI
     3. SQLDB-ISE

Each of these ISEs has two IP addresses, and login information as detailed in the below configuration
diagram.

       Name: “Hyper-ISE1”                          Name: “ISE28.8-VDI”                       Name: “SQLDB-ISE”
       Serial#: ABCDEFGH                            Serial#: 12345678                         Serial#: 88888888
       MRC0: 192.168.1.1                           MRC0: 192.168.1.11                        MRC0: 192.168.1.21
       MRC1: 192.168.1.2                           MRC1: 192.168.1.12                        MRC1: 192.168.1.22




                            ISE                                                  ISE                              ISE




                                            Ethernet Network




                                         Windows Server (2003, 2008), Windows 7
                                                        32-bit or 64-bit
                                                   Can be a Virtual Machine
                                    .Net 4 Installed (see Users Guide for correct version)

We have a Windows host on the Ethernet network that has access to all three of the ISEs. We have
downloaded the required .NET4 version and the ISE-Perfmon_v2.1 files have been unzipped into a
directory on the Windows machine. The first thing we need to do is to confirm connectivity between
the server and each ISE system.
Open a Web page to the first ISE (192.168.1.1) in the configuration and log into the ISE main page. This
verifies that:
    1. The IP address that we are going to use is the right one for the ISE, and the server can connect
         to it.
    2. The username and password are correct for logging into the ISE


         Http://192.168.1.1




                                               We are using IP address
                                               192.168.1.1 for this ISE




                                 Click on “Login” to log into
                                 the ISE


The login prompt will appear. Enter the Username and Password to log into the ISE.
In this example, we have not changed the defaults so we are going to use:
Username:        administrator
Password:        administrator


                                 192.168.1.1




If the login is successful, the following screen will be displayed. This shows that the username and
password we entered is correct.

             192.168.1.1




                              192.168.1.1
                              192.168.1.2
                               ABCDEFGH




Do this same process for the other ISE IP addresses that are being used (192.168.1.11 and 192.168.1.21).
Now that we have the IP Address, Username, Passwords, and connectivity verified, the config file can be
edited to tell ISE-Perfmon_v2.1 what to monitor. Open the ISE-Perfmon.cfg file that was included as
part of the ISE-Perfmon_v2.1.zip file. In this example, we are going to add the IP addresses that we
tested in the above steps. A sample of the config file is below, with the part where we are entering the
ISE connection info called out:



                #Example of entry for the config file:                            Only the entries for the ISEs that are
                #<IP Address of 1 MRC>,<ISE User Name>,<ISE-Password>             being monitored should be in the file.
                #10.10.10.3,administrator,administrator
                #
                #Enter the ISE connection information between the "$" symbols
                #$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
                192.168.1.1,administrator,administrator
                192.168.1.11,administrator,administrator
                192.168.1.21,administrator,administrator
                #$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
                #
                #end of file, this line and the next have to exist, do not modify this line or the one after
                this.

Enter the ISE configuration information between the “$” lines.
Now that we have:
   1. Installed the correct version of .NET4
   2. Verified connectivity
   3. Modified the Config File
   4. Ensured that Perfmon is not running

We are not ready to start the ISE-Perfmon monitoring application. Remember, this application MUST
stay running to record data from the ISEs. This is the “engine” that polls the ISEs and populates the
Perfmon counters on the local machine.

Double Click on the “ISE-Perfmon_v2.1.exe” application. We will see a “Dos” window open with the
following text:




                                                                     ISE Names shown here indicate
                                                                     that we are collecting data from
                                                                     the three ISE systems.




(Note: Is the ISE names were not changed from the defaults, ISE-Perfmon will default back to using the
serial numbers of the ISE. Show below is what would be seen if any two of our ISEs are found to have
the same name)




                                                              ISE Serial Numbers are used if any
                                                              of the ISEs are found to have the
                                                              same name.




Now it’s time to look at some ISE Performance Data.
Open the Windows Perfmon application (Start>run>Perfmon>enter)




                                                           Select this to add a counter to
                                                           the live view.




Click on the “+” sign to add a counter to the live view.
     \\SVR-1

                                                                       Hyper-ISE1
                                                                       ISE28.8-VDI
                                                                       SQLDB-ISE




                                                           Selecting the Total Read IOPS for the
                                                           three ISEs (Controllers Total)
     Hyper-ISE1
      Hyper-ISE1
      Hyper-ISE1
     ISE28.8-VDI
    ISE28.8-VDI
    ISE28.8-VDI
     SQLDB-ISE




In this case, we are going to look at the Total Read IOPS (Transactions per Second) from each ISE
(Controller Total). We could have selected to see the Read IOPS (or any other counter) from each of the
controllers in each ISE (the “Controller 1” and “Controller 2” entries above). Select “ok” when you have
all the counters you want to see loaded.
                             Hyper-ISE1                                       \\SVR-1
                             ISE28.8-VDI                                      \\SVR-1
                             SQLDB-ISE                                        \\SVR-1



Please see the appendix for a video that shows how to set up Perfmon recordings to save data with
Perfmon.
Appendix A: Resources For More Information

Cortex Developer.com
http://www.cortexdeveloper.com



XIO ISE-Analyzer
http://xiostorage.com/products/ise-software/#ise_analyzer_tab



ISE SCOM Free Plug in, ISEv1
http://support.xiostorage.com
Navigate to the ISEv1 Support Matrix. The SCOM files are linked in the section at the top of the page
titled “Emprise 5000 Downloads”



Setting Up Microsoft Perfmon Performance Counter Recordings
http://www.youtube.com/watch?v=3yEEzqDE5qI



Microsoft System Center Operations Manager
http://technet.microsoft.com/en-us/systemcenter/om/bb497976

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:22
posted:2/26/2012
language:English
pages:19