Configuring Microsoft Windows IP Security to Operate with HP-UX IPSec

Document Sample
Configuring Microsoft Windows IP Security to Operate with HP-UX IPSec Powered By Docstoc
					HP-UX IPSec
Configuring Microsoft Windows IP Security
to Operate with HP-UX IPSec




 HP Part Number: J4256-90025
 Published: June 2007
 Edition: 1.0
2
Table of Contents
About This Document.........................................................................................................9
   Typographic Conventions......................................................................................................................9
   Introduction..........................................................................................................................................11
      Testing Environment.......................................................................................................................11
          Known Problem with Windows 2000 SP1 and SP2...................................................................11
      Protocol Implementation Differences..............................................................................................12
   Windows IP Security Configuration Overview....................................................................................13
   Configuring a Windows Host-to-Host Policy.......................................................................................14
      Step 1: Starting the IP Security Policies Snap-in Configuration Utility...........................................15
      Step 2: Creating a Policy..................................................................................................................15
      Step 3: Adding a Rule......................................................................................................................16
      Step 4: Creating the IP Filter List and Filters for the Rule...............................................................18
      Step 5: Configuring Filter Actions for the Rule...............................................................................21
      Step 6: Configuring the IKE Authentication Method and Preshared Key for the Rule..................25
      Step 7: Configuring the Connection Type for the Rule...................................................................26
      Step 8: Modifying IKE Parameters for the Policy............................................................................26
      Step 9: Starting the IP Security Service............................................................................................29
      Step 10: Assigning the IP Security Policy........................................................................................30
      Step 11: Verifying the Configuration...............................................................................................31
      Example...........................................................................................................................................31
          Windows Configuration.............................................................................................................31
          HP-UX Configuration................................................................................................................32
             Additional Options...............................................................................................................32
   Configuring a Windows End-to-End Tunnel Policy.............................................................................33
      Outbound Tunnel Rule Requirements............................................................................................33
      Inbound Tunnel Rule Requirements...............................................................................................33
      Configuring a Tunnel Rule..............................................................................................................33
      Example...........................................................................................................................................34
          Windows Configuration.............................................................................................................34
             Outbound Rule.....................................................................................................................34
             Inbound Rule........................................................................................................................35
             Additional Parameters..........................................................................................................36
          HP-UX Configuration................................................................................................................37
   Troubleshooting Tips............................................................................................................................38
      Using IKE Logging on HP-UX Systems..........................................................................................38
      Using IKE Logging on Windows Systems.......................................................................................38
      Additional Windows Troubleshooting Tools..................................................................................39
   Comparing HP-UX and Windows IPsec Configuration Parameters....................................................40
      Mirrored Filters...............................................................................................................................41
      Filter Selection.................................................................................................................................42
      IKE Parameter Selection..................................................................................................................42
      IKE SA Key (Master Key) Lifetime Values......................................................................................42
          HP-UX IKE SA Lifetime Values.................................................................................................42
          Windows IKE SA Lifetime Values..............................................................................................43
      Maximum Quick Modes..................................................................................................................43
      Perfect Forward Secrecy (PFS).........................................................................................................43
      IPsec SA Key (Session Key) Lifetime Values...................................................................................43
          HP-UX IPsec SA Lifetime Values...............................................................................................43
          Windows IPsec SA Lifetime Values...........................................................................................44
   Related Publications..............................................................................................................................45


                                                                                                                             Table of Contents           3
glossary.............................................................................................................................47




4     Table of Contents
List of Figures
1    IP Security Policy Wizard..............................................................................................................16
2    Rules Tab.......................................................................................................................................17
3    Rule Properties Dialog Box...........................................................................................................17
4    Creating an IP Filter List...............................................................................................................18
5    Address Tab for Filter Properties..................................................................................................19
6    Protocol Tab for Filter Properties..................................................................................................20
7    Selecting the Filter List for a Rule.................................................................................................21
8    Security Methods for Filter Action................................................................................................22
9    Security Method Dialog Box.........................................................................................................23
10   Custom Security Methods Settings Dialog Box............................................................................24
11   Selecting the Filter Action.............................................................................................................25
12   Configuring A Preshared Key.......................................................................................................26
13   General Policy Properties Dialog Box ..........................................................................................27
14   Key Exchange Settings Dialog Box ...............................................................................................28
15   IKE Security Algorithms Dialog Box ............................................................................................29
16   IPSEC Services Properties Dialog Box...........................................................................................30
17   Assigning the IP Security Policy...................................................................................................31
18   Outbound Rule Filter....................................................................................................................35
19   Outbound Rule Tunnel Settings....................................................................................................35
20   Inbound Rule Filter.......................................................................................................................36
21   Inbound Rule Tunnel Settings.......................................................................................................36




                                                                                                                                                     5
6
List of Tables
1    IPsec Parameters on Windows and HP-UX .................................................................................40




                                                                                                                            7
8
About This Document
     This document describes how to configure Microsoft Windows IP Security to operate with the
     HP-UX IPSec product.

Typographic Conventions
     This document uses the following typographical conventions:
     %, $, or #                    A percent sign represents the C shell system prompt. A dollar
                                   sign represents the system prompt for the Bourne, Korn, and
                                   POSIX shells. A number sign represents the superuser prompt.
     audit(5)                      A manpage. The manpage name is audit, and it is located in
                                   Section 5.
     Command                       A command name or qualified command phrase.
     Computer output               Text displayed by the computer.
     Ctrl+x                        A key sequence. A sequence such as Ctrl+x indicates that you
                                   must hold down the key labeled Ctrl while you press another
                                   key or mouse button.
     ENVIRONMENT VARIABLE          The name of an environment variable, for example, PATH.
     [ERROR NAME]                  The name of an error, usually returned in the errno variable.
     Key                           The name of a keyboard key. Return and Enter both refer to the
                                   same key.
     Term                          The defined use of an important word or phrase.
     User input                    Commands and other text that you type.
     Variable                      The name of a placeholder in a command, function, or other
                                   syntax display that you replace with an actual value.
     []                            The contents are optional in syntax. If the contents are a list
                                   separated by |, you must choose one of the items.
     {}                            The contents are required in syntax. If the contents are a list
                                   separated by |, you must choose one of the items.
     ...                           The preceding element can be repeated an arbitrary number of
                                   times.
                                   Indicates the continuation of a code example.
     |                             Separates items in a list of choices.
     WARNING                       A warning calls attention to important information that if not
                                   understood or followed will result in personal injury or
                                   nonrecoverable system problems.
     CAUTION                       A caution calls attention to important information that if not
                                   understood or followed will result in data loss, data corruption,
                                   or damage to hardware or software.
     IMPORTANT                     This alert provides essential information to explain a concept or
                                   to complete a task
     NOTE                          A note contains additional information to emphasize or
                                   supplement important points of the main text.




                                                                         Typographic Conventions   9
10
Introduction
       This document contains the following sections:
       • “Windows IP Security Configuration Overview” (page 13)
           This section contains a brief overview of the Windows IPsec configuration parameters and
           the terminology used in the Windows IPsec configuration utilities.
       •   “Configuring a Windows Host-to-Host Policy” (page 14)
           This section describes how to configure IP Security (IPsec) on a Windows client to secure
           IP packets sent to and received from an HP-UX system in a host-to-host topology.
       •   “Configuring a Windows End-to-End Tunnel Policy” (page 33)
           This section describes how to configure IPsec on a Windows client to secure IP packets sent
           to and received from an HP-UX system in an end-to-end tunnel topology.
       •   “Troubleshooting Tips” (page 38)
           This section contains troubleshooting tips.
       •   “Comparing HP-UX and Windows IPsec Configuration Parameters” (page 40)
           This section compares how HP-UX and Windows systems configure and use IPsec parameters.
       •   “Related Publications” (page 45)
           This section contains a list of related HP-UX and Microsoft publications.
       The procedures and examples in this document use preshared keys for IKE authentication. For
       information about using certificates for IKE authentication with Microsoft Windows, see Using
       Microsoft Windows Certificates with HP-UX IPSec, available at http://docs.hp.com.
       The intended audience for this document is an HP-UX IPSec administrator who is familiar with
       the HP-UX IPSec product and with the IP Security protocol suite. If you are not familiar with
       the HP-UX IPSec product, see the appropriate version of the HP-UX IPSec Administrator's Guide,
       available at http://docs.hp.com.

       NOTE: The IP Security protocol suite is often referred to as IPsec. The HP-UX product that
       implements the IP Security protocol suite is HP-UX IPSec.

Testing Environment
       The procedures in this white paper were tested using the following environment:

       Component                                         Description

       HP-UX IPSec                                       Versions A.02.01 and A.02.01.01

       Microsoft Windows Client                          Windows XP with Service Pack 2 (SP2)


Known Problem with Windows 2000 SP1 and SP2
       For this white paper, HP did not test with Windows 2000 systems. However, there is a known
       problem with Windows 2000 base systems and Windows 2000 systems with Service Pack 1 (SP1)
       or Service Pack 2 (SP2). The IP Security module on these systems does not properly process IPSec
       ESP packets that are fragmented across IP packets and drops these packets. The symptoms vary
       according to how the applications handle the dropped packets.
       This problem is caused by a defect in the Windows 2000 SP1/ SP2 software and is fixed in Windows
       2000 Service Pack 3 (SP3).



                                                                                            Introduction   11
       The above problem typically occurs with ESP-encrypted UDP or ICMP packets that are fragmented
       by IP. HP-UX 11i systems minimize IP fragmentation of ESP-encrypted TCP packets. You may
       still experience problems with ESP-encrypted TCP packets sent from an HP-UX system to a
       Windows 2000 system if an intermediate IP gateway fragments the ESP packet.

Protocol Implementation Differences
       HP-UX and Microsoft Windows both implement the IP Security protocol suite. However, there
       are features in the protocol suite that HP-UX implemented which Microsoft did not implement,
       and vice-versa.
       The following features are implemented by HP-UX IPSec version A.02.01 but not by Microsoft
       Windows XP:
       •   Advanced Encryption Standard (AES): HP-UX IPSec supports ESP encryption using the
           following protocols: AES, Triple Data Encryption Standard (3DES), and Data Encryption
           Standard (DES). Windows XP and Windows 2000 support 3DES and DES, but do not support
           AES.
       •   Aggressive Mode (AM): HP-UX supports AM exchanges to establish IKE Security
           Associations (SAs). AM is an optional feature and is not supported on Windows.
       The following features are implemented by Microsoft Windows XP, but not by HP-UX IPSec
       version A.02.01:
       •   Kerberos: Windows supports Internet Key Exchange (IKE) authentication using Kerberos.
           RFC 2408 defines an optional Kerberos Token payload, but does not describe how to
           implement it. This feature is not supported on HP-UX.
       •   Perfect Forward Secrecy (PFS) for keys only: HP-UX IPSec supports PFS for keys in
           conjunction with PFS for all identities, but does not support PFS for keys only. Windows
           supports PFS for keys only (“session key PFS”) and PFS for keys in conjuctions with PFS for
           all identities (“master key PFS”). See “Perfect Forward Secrecy (PFS)” (page 43) for more
           information.




12
Windows IP Security Configuration Overview
     On Microsoft Windows systems, all IP Security (IPsec) configuration data resides in a single IP
     Security policy. You can create multiple IP Security policies, but only one local policy can be
     active on the system. If the system is a member of a Windows Active Directory domain, you can
     use an IP Security policy from a Group Policy defined for the domain.
     A Windows IP Security policy defines the parameters used to negotiate Internet Key Exchange
     Security Associations (IKE SAs) and IPsec SAs. An IKE SA is a bi-directional, secure
     communication channel that two peers establish before negotiating IPSec SAs. One of the primary
     activities during the IKE SA negotiation is the authentication of each peer's identity.
     After two peers establish an IKE SA, they can negotiate IPsec SAs. Each IPsec SA is a
     uni-directional, secure communication channel. The IPsec SA operating parameters include the
     IPsec protocol used (Encapsulating Security Payload, ESP, or Authentication Header, AH) and
     the cryptographic algorithms. IPsec SAs are negotiated in pairs (one for each direction of traffic).
     Each Windows IP Security policy contains the following components:
     •   Rules
         A policy contains one or more rules. The main purpose of a rule is to assign actions for
         address filters. Each rule contains the following components:
         — IP Filter List
              An IP Filter list contains one or more filters. Each filter contains the following
              components:
              ◦   Addressing
                   The source and destination IP addresses, network masks, and a flag that indicates
                   if the filter is mirrored (bi-directional).
              ◦    Protocol
                   The upper-layer protocol, and source and destination ports, if applicable.
              ◦    Description
                   The filter name and a description.

         —    Filter Action
              The filter action specifies the action to take for the rule, and can be one of the following
              actions:
              ◦    allow: allow the packet to pass
              ◦    block: discard the packet
              ◦    negotiate security: negotiate IPsec Authentication Header (AH) or Encapsulating
                   Security Payload (ESP) Security Associations (SAs)
         —    Authentication Methods
              The authentication methods specify the type of Internet Key Exchange (IKE)
              authentication to use (preshared key or certificates with RSA signatures). If you are
              using preshared key authentication, the authentication methods also specify the value
              of the preshared key.




                                                            Windows IP Security Configuration Overview   13
          —    Tunnel Settings
               The tunnel settings specify if the rule is a tunnel rule. If it is a tunnel rule, the settings
               also specify the tunnel destination endpoint.
          —    Connection Type
               The connection type specifies the connection (link) types for the rule, such as LAN.

      •   General
          The general parameters for a policy specify IKE SA parameters, such as the IKE encryption
          algorithm, IKE hash (integrity algorithm), Diffie-Hellman Group, and IKE SA key lifetimes.
          The parameters correspond to IKE SA proposals. You can configure multiple IKE SA
          proposals and specify the preference order. The proposals are used for all rules in the policy.
      By comparison, a minimal HP-UX IPSec configuration consists of one or more IPsec host policies,
      one or more IKE policies, and one or more authentication records. The IPsec host policies specify
      address filters, and you can configure separate IKE policies for each peer. “Comparing HP-UX
      and Windows IPsec Configuration Parameters” (page 40) lists IPsec configuration parameters
      and how they are configured in the HP-UX IPSec and the Windows IP Security configuration
      utilities.

Configuring a Windows Host-to-Host Policy
      This section describes one method for configuring host-to-host policy on a Windows XP client
      using the IP Security Policies snap-in utility. Windows also supports command-line utilities to
      configure IP Security policies: ipseccmd on Windows XP systems and netsh on Windows 2003
      systems. For more information about these utilities, see the Windows documentation set.
      To use this method, complete the following steps:
      1. Start the IP Security Policies snap-in utility. See “Step 1: Starting the IP Security Policies
          Snap-in Configuration Utility” (page 15).
      2. Create an IP Security policy. See “Step 2: Creating a Policy” (page 15).
      3. Add a rule to the policy. See “Step 3: Adding a Rule” (page 16).
      4. Create a Filter List for the rule and configure filters. See “Step 4: Creating the IP Filter List
          and Filters for the Rule” (page 18).
      5. Configure filter actions for the rule. The filter actions contain IPsec transforms or other
          actions. See “Step 5: Configuring Filter Actions for the Rule” (page 21).
      6. Configure the IKE authentication method and preshared key for the rule. See “Step 6:
          Configuring the IKE Authentication Method and Preshared Key for the Rule” (page 25).
      7. Specify the network link (connection) types for the rule. See“Step 7: Configuring the
          Connection Type for the Rule” (page 26).
      8. Modify the IKE SA parameters for the policy. By default, Windows clients will use IKE SA
          parameters that are compatible with the default HP-UX IPSec parameters. If these parameters
          are acceptable, you can skip this step. See “Step 8: Modifying IKE Parameters for the Policy”
          (page 26).
      9. Start the IP Security service. The IP Security service must be running before you can assign
          the new IP Security policy. See “Step 9: Starting the IP Security Service” (page 29).
      10. Assign (activate) the new IP Security Policy. See “Step 10: Assigning the IP Security Policy”
          (page 30).
      11. Verify the configuration. See “Step 11: Verifying the Configuration” (page 31).
      Because this is a host-to-host rule, we will use the default value for the rule tunnel setting (no
      tunnel). For information about configuring a tunnel rule and the tunnel setting, see “Configuring
      a Windows End-to-End Tunnel Policy” (page 33).



14
Step 1: Starting the IP Security Policies Snap-in Configuration Utility
       Use the following procedure to start the IP Security Policies configuration utility. This utility is
       a snap-in module for the Microsoft Management Console (MMC).
       1.   Start the Microsoft Management Console (MMC). From the Microsoft Start menu, click Run
            and type MMC. Click OK.
       2.   If the IP Security Policies snap-in configuration utility is not loaded, use the following
            procedure to add it:
            a. From the MMC window, click File→Add/Remove Snap-in.
            b. From the Add/Remove Standalone Snap-in window, click Add.
            c. From the Add Standalone Snap-in window, scroll down to IP Security Policy
                 Management and select it. Click Add.
            d. In the Select Computer or Domain window, select Local computer (in this procedure,
                 we are configuring IP Security for the local computer). Click Finish.
            e. Close the Add Standalone Snap-in window by clicking Close.
            f. Close the Add/Remove Snap-in window by clicking OK.

Step 2: Creating a Policy
       Use the following procedure to create an IP Security policy. An IP Security policy is a set of IPsec
       configuration parameters. Only one local IP Security policy can be active (assigned) on a system.
       1.   In the left navigation pane of the IP Security Policy Management snap-in, click IP Security
            Policies on Local Computer to display all IP Security Policies. Depending on your Windows
            platform, there may be IP Security Policies already configured.
       2.   Right click IP Security Policies on Local Computer and select Create IP Security Policy.
       3.   The Policy Wizard starts and displays a startup message. Click Next.
       4.   The Policy Wizard opens the IP Security Policy Name window. Enter a name in the Name
            field. This name is used only for internal identification.
            Click Next.
       5.   The Policy Wizard opens the Requests for Secure Communication window. Clear the Activate
            the default response rule check box, as shown in Figure 1. (The default response rule is a
            pre-configured rule that causes the Windows system to dynamically build a filter list based
            on the receipt of IKE requests. By default, the Windows system attempts to use IPsec only
            if it receives an IKE request from a remote system.)
            Click Next.




                                                               Configuring a Windows Host-to-Host Policy   15
           Figure 1 IP Security Policy Wizard




      6.   The Policy Wizard opens the Completing the IP Security policy wizard window. Select the
           Edit properties check box if it is not already selected.
           Click Finish.
           The IP Security configuration utility opens the Policy Properties dialog box. The title of the
           window will be name Policy, where name is the policy name.

Step 3: Adding a Rule
      The primary purpose of a rule is to assign actions to filters. A rule also specifies IKE authentication
      methods. Use the following procedure to add a rule to the IP Security policy:
      1. Select the Rules tab in the Policy Properties dialog box.
           Clear the Use Add Wizard box check box if it is selected (Figure 2).
           Click Add.




16
     Figure 2 Rules Tab




2.   The IP Security configuration utility opens the Rule Properties dialog box, which has a tab
     for each category of rule configuration data: IP Filter List, Filter Action, Authentication
     Methods, Tunnel Setting, and Connection Type ( Figure 3).

     Figure 3 Rule Properties Dialog Box




                                                      Configuring a Windows Host-to-Host Policy   17
        TIP: After you have created a rule, you can open the Rules Properties dialog box by right
        clicking the rule and selecting Properties.

Step 4: Creating the IP Filter List and Filters for the Rule
        An IP filter list can contain one or more filters. IPsec uses the filters to determine which rule to
        apply to an IP packet. The IP Security configuration utility displays the rules for a policy in
        reverse alphabetical order based on the name of the IP filter list for the rule.
        Filter Order     There is no method for specifying the search or priority order for the filters in a
        rule or for the order of rules in a policy. The Windows IP Security module automatically creates
        an internal filter list and orders the filters from most specific to least specific.
        Use the following procedure to configure the IP filter list and a filter:
        1. Create an IP filter list.
            Select the IP Filter tab from the Rule Properties dialog box.
            The IP Filter List tab shows a list of filters already defined for IP Security policies. Each rule
            can have only one filter list, but the filter list can specify multiple filters. In this example,
            we will create a new filter list that contains one filter.
            Click Add at the bottom of the dialog box.
            The IP Security configuration utility opens the IP Filter List dialog box. In the Name field,
            enter a name for the filter list. This name is used only for internal identification. Optionally,
            add a description. In Figure 4, the administrator enters the name foo.
            Clear the Use Add Wizard box check box if it is selected.
            Click Add.

            Figure 4 Creating an IP Filter List




18
     The IP Security configuration utility opens a Filter Properties dialog box.
2.   Select the Addressing tab in the Filter Properties dialog box. Use the drop-down menus to
     specify the address types for the source and destination addresses. The selections are:
     • My IP Address1
     • Any IP Address
     • A specific DNS Name
     • A specific IP Address
     • A specific IP Subnet
     Enter the source and destination IP addresses or DNS names for the filter. If you selected A
     specific IP Subnet, enter the subnet mask.

     WARNING! Be careful when configuring filters that affect packets required for basic
     network operation, such as packets exchanged with DNS servers and ICMP packets
     exchanged with routers. If you configure a policy that requires IP Security for these packets
     and the remote node does not support IP Security, your system can lose network functionality.
     Leave the Mirrored check box selected, which creates a bi-directional filter that applies to
     packets to and from the destination system. See “Mirrored Filters” (page 41) for more
     information about mirrored filters.
     In Figure 5, the administrator specifies an address filter with the Windows system address
     (10.1.1.1) as the source address and the HP-UX system address (10.2.2.2) as the destination
     address. The Mirrored check box is selected, so the address filter also matches packets from
     the HP-UX system.

     Figure 5 Address Tab for Filter Properties




3.   Select the Protocol tab in the Filter Properties dialog box. By default, the filter applies to all
     protocol types. Select the protocol type (for example, TCP) from the drop-down box. If you
     select TCP or UDP, you can also specify the From (source) port and To (destination) port.
     Click OK to return to the Filter Properties dialog box.
1. HP-UX did not test the My IP Address selection with multihomed Windows systems. However, the Windows
   documentation states that in a multi-homed system, My IP Address matches every IP address on the system.

                                                               Configuring a Windows Host-to-Host Policy      19
     In Figure 6, the administrator specifies protocol information for a Windows system that will
     be a telnet client. The protocol type is TCP, the source port is a wildcard (any port), and the
     destination port is the IANA registered TCP port number for the telnet service, 23.

     Figure 6 Protocol Tab for Filter Properties




20
       4.   From the IP Filter List dialog box, you can add another filter to the filter list by clicking the
            Add button.
            Click OK in the IP Filter List dialog box to return to the IP Filter List tab in the Rule Properties
            dialog box.
       5.   Add the filter list to the rule by selecting the option button for the filter list you just created.
            In Figure 7, the administrator added the filter list foo for the rule.

            Figure 7 Selecting the Filter List for a Rule




Step 5: Configuring Filter Actions for the Rule
       The filter action specifies the action to take for the rule, such as allow (pass), block (discard), or
       negotiate security (negotiate IPsec AH or ESP Security Associations). If you select negotiate
       security, the filter action also specifies parameters for IPsec Security Association (SA) proposals:
       ESP or AH transforms and IPSec SA key lifetimes. A rule can have only one filter action, but the
       filter action can specify multiple IPsec SA proposals. You can specify the order for the IPsec SA
       proposals.
       The filter actions you configure in the Windows IP Security rule must be compatible with the
       value or values specified for the -action argument in the HP-UX ipsec_config add host
       or add tunnel command.
       Use the following procedure to configure filter actions:
       1.   Select the Filter Action tab from the Rule Properties dialog box.
            The Filter Action tab shows a list of filter actions already defined for IP Security. In this
            procedure, we will create a new filter action.
            Clear the Use Add Wizard check box if it is selected and click Add.
       2.   The IP Security configuration utility opens the Filter Action Properties dialog box with the
            following tabs:




                                                                  Configuring a Windows Host-to-Host Policy   21
          •     Security Methods
          •     General
          Select the Security Methods tab, then select Negotiate security. Verify that the following
          check boxes are not selected:2
          • Accept unsecured communication, but always respond using IPSec.
          • Allow unsecured communication with non-IPSec-aware computer.
          In addition, verify that the Session key perfect forward secrecy (PFS) check box is not selected.
          (HP-UX does not support session key PFS, also referred to as PFS for keys only. HP-UX
          supports PFS for keys only in conjunction with PFS for identities. See “Perfect Forward
          Secrecy (PFS)” (page 43) for more information.)
          For example:

          Figure 8 Security Methods for Filter Action




          Click Add.
          The IP Security configuration utility opens the Security Method dialog box (Figure 9):




     2. HP-UX IPSec does not have options that are equivalent to these check boxes. If an HP-UX IPsec policy requires IP
        security, then HP-UX always requires IP security for packets that match the policy and drops any packets that match
        the policy but are not secured.

22
Figure 9 Security Method Dialog Box




The Encryption and Integrity and Integrity only methods each correspond to a set of
predefined parameters for an IPsec SA proposal, including an IPsec transform type (such
as ESP). The transforms and additional SA parameters defined for these methods may vary
according to the Windows release installed. On Windows XP systems with SP2, these methods
are defined as follows:
• Encryption and Integrity
    Authenticated ESP using 3DES encryption and SHA1 authentication (this is equivalent
    to the HP-UX IPSec transform ESP_3DES_HMAC_SHA1). No SA lifetimes are specified,
    and these settings are compatible with the HP-UX default SA lifetimes (see “IPsec SA
    Key (Session Key) Lifetime Values” (page 43) for more information).
•   Integrity only
    Authenticated ESP using NULL encryption and SHA1 authentication. (this is equivalent
    to the HP-UX IPSec transform ESP_NULL_HMAC_SHA1) No SA lifetimes are specified,
    and these settings are compatible with the HP-UX default SA lifetimes (see “IPsec SA
    Key (Session Key) Lifetime Values” (page 43) for more information).
To use a predefined method, select the appropriate method from the list and click OK to
return to the Security Methods tab in the Filter Actions dialog box.
To create a custom method, use the following procedure:
a. Select Custom in the Security Method dialog box, then click Settings to open the
     Custom Security Method Settings dialog box.
b. In the Custom Security Method Settings dialog box (Figure 10), select the appropriate
     transform type, algorithms, and session key lifetimes. See “IPsec SA Key (Session Key)
     Lifetime Values” (page 43) for more information about IPsec session key lifetimes.




                                                 Configuring a Windows Host-to-Host Policy   23
               Figure 10 Custom Security Methods Settings Dialog Box




          c.   Click OK to return to the Security Methods tab in the Filter Actions dialog box.
               If the parameters you configured for a custom method match a predefined method, the
               configuration utility will display an informative message and select the matching
               predefined method.

     3.   From the Security Methods tab, you can add more methods (IPsec SA proposals) by clicking
          the Add button. If you have multiple IPsec SA proposals, the configuration utility lists them
          in the preference order IKE will use when negotiating IPsec SAs. You can change the order
          by using the Move up and Move down buttons. You can also use the Delete button to delete
          IPsec SA proposals (such as proposals that use DES, which has been cracked—data encrypted
          using DES has been decrypted by unauthorized parties) .
     4.   (Optional) Configure the action name. When you create a new action (transform), the
          configuration utility assigns it the name New Filter Action. To change the name, select the
          General tab from the Action Properties dialog box. Enter a new name and click OK to return
          to the Filter Action tab in the Rule Properties dialog box.
     5.   Apply the new action to the rule. In the Filter Action tab, click on the option button for the
          filter action you just created to apply the action to the rule you are configuring. For example,
          in Figure 11, the administrator created the action my_action and selected it for the rule.
          Click Apply.




24
                   1
           Figure 1 Selecting the Filter Action




Step 6: Configuring the IKE Authentication Method and Preshared Key for the Rule
       When configuring a rule to be compatible with HP-UX IPSec, the authentication method specifies
       the IKE authentication method (preshared key or certificates) for IPsec. The authentication method
       must match the value specified for the -authentication argument in the ipsec_config
       add ike command.
       Windows also allows you to configure Kerberos (Active Directory) as an authentication method
       for IKE (this is the default), but HP-UX does not support this authentication method.
       Use the following procedure to configure the IKE authentication method:
       1. Select the Authentication Methods tab from the Rule Properties dialog box.
       2. Click Add to open the Authentication Method dialog box.
       3. To use IKE authentication with a preshared key, select Use this string. This is equivalent
           to specifying -authentication PSK in the ipsec_config add ike command.
           Enter the preshared key as ASCII text. Do not enclose the key in double quotes. The preshared
           key must match the preshared key on the HP-UX system, which is configured using the
           -preshared argument in the ipsec_config add auth command. For example:




                                                              Configuring a Windows Host-to-Host Policy   25
            Figure 12 Configuring A Preshared Key




            To use IKE authentication with certificates, select Use a certificate from this certification
            authority (CA). Click Browse. The IP Security configuration utility opens a Select Certificate
            box with a list of CA certificates stored on your system. Select the CA for the appropriate
            CA and click OK. (For additional information about configuring Microsoft Windows
            certificates, see Using Microsoft Windows Certificates with HP-UX IPSec, available at
            http://docs.hp.com.
       4.   After you have specified the IKE authentication method, click OK to return to the
            Authentication Methods tab in the Rule Properties dialog box.
       5.   In the Rule Properties dialog box, remove the Kerberos authentication method from the
            authentication methods list by highlighting it and clicking Remove.
            The configuration utility will display a confirmation message (Are you sure?). Click Yes

Step 7: Configuring the Connection Type for the Rule
       The connection type specifies the types of network connection to which the rule will apply. By
       default, the IP Security configuration utility creates rules that apply to all network connection
       types. To change the connection type, use the following procedure:
       1. Select the Connection Type tab from the Rule Properties dialog box.
       2. The IP Security configuration utility opens the Connection Type dialog box with the following
           selections:
           • All network connections: the rule applies to all network connections
           • Local area network (LAN): the rule applies only to LAN connections
           • Remote access: the rule applies only to VPN and dial-up connections
            Select the appropriate connection type and click OK. If you have configured all the required
            parameters for a rule, the IP Security configuration utility will return to the Policy Properties
            dialog box.

Step 8: Modifying IKE Parameters for the Policy
       By default, HP-UX IPSec negotiates IKE SAs using a single proposal with the following parameters:
26
•    Encryption algorithm: 3DES
•    Hash algorithm: MD5
•    Diffie-Hellman Group: 2
•    Maximum lifetime: 28,800 seconds (8 hours)
•    Maximum Quick Modes: 100
You can specify alternative values for the above parameters in the ipsec_config add ike
command.
On Windows XP systems with SP2, IP Security policies are pre-configured with four IKE SA
proposals. The second IKE proposal matches the default HP-UX IPSec IKE proposal3, and will
be used by the two systems if no changes are made to the default configuration data. If these
IKE parameters meet your security requirements, you do not need to modify the IKE parameters
and can skip to “Step 10: Assigning the IP Security Policy” (page 30).
Use the following procedure to modify the Windows IKE SA parameters:
1. From the Policy Properties dialog box, select the General tag. The IP Security configuration
    utility opens the General dialog box (Figure 13).
     Click Advanced4. (Ignore the field labeled Check for policy changes. This field is used
     only when the policy is stored in an Active Directory.)

     Figure 13 General Policy Properties Dialog Box




2.   The IP Security configuration utility opens the Key Exchange Settings dialog box (Figure 14).




3. By default, the first Windows XP proposal has the following parameters: Encryption - 3DES; Hash - SHA1;
   Diffie-Hellman Group - 2. The third and fourth Windows proposals are weaker, and use DES encryption and
   Diffie-Hellman Group 1. Refer to the Windows documentation for more information.
4. On Windows 2003 servers, this button is labeled Settings.

                                                               Configuring a Windows Host-to-Host Policy     27
     Figure 14 Key Exchange Settings Dialog Box




     Configure the fields as follows:
     • Master key perfect forward secrecy (PFS)
         Selecting this check box sets the maximum number of IPsec or Quick Mode (QM)
         negotiations that IKE can perform using an IKE SA to 1. It is equivalent to specifying
         -maxqm 1 in the ipsec_config add ike command. PFS is computationally
         expensive and HP recommends that you enable it only in hostile environments. See
         “Maximum Quick Modes” (page 43) and “Perfect Forward Secrecy (PFS)” (page 43)
         for more information.
     •   Authenticate and generate a new key after every: ____ minutes
         This field specifies the maximum lifetime for an IKE SA in units of time. It is equivalent
         to the -life argument of the ipsec_add ike command.


         TIP: Note that this value is specified in minutes on Windows systems and in seconds
         on HP-UX systems.

     •   Authenticate and generate a new key after every: ____ sessions
         This field specifies the maximum QM negotiations per IKE SA. It is equivalent to the
         -maxqm argument of the ipsec_add ike command. See “Maximum Quick Modes”
         (page 43) and “Perfect Forward Secrecy (PFS)” (page 43) for more information.
     Modify the values as appropriate. If you do not want to modify the IKE encryption, hash,
     or Diffie-Hellman Group parameters, click OK to return to the General dialog box and
     continue to the next step.
     To modify IKE encryption algorithm, hash algorithm, or Diffie-Hellman Group parameters,
     click Methods. The IP Security configuration utility opens the Key Exchange Security
     Methods dialog box, which lists IKE algorithms and Diffie-Hellman Groups that correspond
     to IKE SA proposals in order of preference. You can change the order by using the Move
     up and Move down buttons. You can also delete IKE SA proposals (such as proposals that
     use DES, which has been cracked) using the Delete button.
     To add another IKE SA proposal (method), click Add.
     The IP Security configuration utility opens the IKE Security algorithms dialog box (Figure 15).




28
            Figure 15 IKE Security Algorithms Dialog Box




            Use the drop-down menus to select the appropriate integrity algorithm, encryption algorithm,
            and Diffie-Hellman Group (these are equivalent to the -hash, -encryption, and -group
            arguments of the ipsec_config add ike command).
       3.   Click OK to return to the Key Exchange Security Methods dialog box.
            Click OK to return to the Key Exchange Settings dialog box.
            Click Close to close the Policy Properties dialog box.

Step 9: Starting the IP Security Service
       Use the following procedure to start the IP Security service. The IP Security service must be
       running before you can assign a policy.
       1.   From the Microsoft Start menu, select Control Panel→Administrative Tools→Services.
       2.   Scroll down and select IPSEC Services.
       3.   The Service manager opens the IPSEC Services Properties dialog box (Figure 16).
            In the Startup type selection menu, select Automatic.
            If the Service status is not Started, click Start.
            Click OK to close the IPSEC Services Properties dialog box.
            Close the Services dialog box.




                                                                 Configuring a Windows Host-to-Host Policy   29
           Figure 16 IPSEC Services Properties Dialog Box




       Alternatively, you can manually start the IP Security service by entering the following Windows
       command:
       net start policyagent
       You can also use the following sequence of commands to manually stop and restart the IP Security
       service. This also clears any existing IPsec SAs,:
       net stop policyagent
       net start policyagent

Step 10: Assigning the IP Security Policy
       The IP Security subsystem will not use the new policy until you assign (activate) it. Only one IP
       Security policy can be assigned or active for the system. To assign the new IP Security policy,
       return to the MMC window. Right click the policy in the MMC window and select Assign, as
       shown in Figure 17.




30
       Figure 17 Assigning the IP Security Policy




      1:
Step 1 Verifying the Configuration
       To verify your configuration, generate traffic that matches the address filter.
       On the HP-UX system, enter the following command to verify that the IKE SA and IPsec SAs
       are established:
       ipsec_report -sa

Example
       In this example, IPsec secures telnet connections from the Windows system to the HP-UX system,
       using authenticated ESP.
       The Windows system's address is 10.1.1.1
       The HP-UX system's address is 10.2.2.2.

Windows Configuration
       The Windows administrator configures and assigns an IP Security policy with the following
       parameters:
       • One rule, with the following parameters:
           — Filter List: One filter, with the following parameters:
               ◦   Addressing:
                   –   Source address: the Windows system's address.
                   –   Destination address: the HP-UX system's address.
                   –   Mirrored: yes (the Mirrored box is selected).
                   These parameters are shown in Figure 5 (page 19).
               ◦   Protocol: TCP; source port any, destination port 23 (telnet).
                   –   Protocol: TCP
                   –   From port: any
                   –   To port: 23 (telnet server)
                   These parameters are shown in Figure 6 (page 20).
           —    Filter Action: Negotiate security, using the default settings for Encryption and Integrity
                (authenticated ESP using 3DES and SHA1).
           —    Authentication Method: IKE using the preshared key my_preshared_key, as shown
                in Figure 12 (page 26).
           —    Tunnel Settings: No tunnel (this is the default).
           —    Connection Type: All network connections (this is the default).
       •   General parameters: The general parameters for the policy are set to the default values (four
           IKE SA proposals, including 3DES encryption, SHA1 integrity and Diffie-Hellman Group
           2).

                                                              Configuring a Windows Host-to-Host Policy   31
HP-UX Configuration
       On the HP-UX system, the administrator configures the following policies and records:
       ipsec_config add host telnet_from_foo1 \
       -source 10.2.2.2/32/TELNET -destination 10.1.1.1 \
       -action ESP_3DES_HMAC_SHA1

       ipsec_config add ike foo1 -remote 10.1.1.1 -auth PSK

       ipsec_config add auth foo1 -remote 10.1.1.1 \
       -psk my_preshared_key
       If the HP-UX IPSec subsystem is not already started, the administrator starts it using the
       ipsec_admin -start command.

       Additional Options
       For information on additional options and commands, see the HP-UX IPSec Administrator's Guide
       and the following manpages:
       ipsec_admin(1M)
       ipsec_config(1M)
       ipsec_policy(1M)
       ipsec_report(1M)
       .




32
Configuring a Windows End-to-End Tunnel Policy
       The only IPsec tunnel topology supported between an HP-UX system and a Windows system is
       an end-to-end tunnel.5The procedure for configuring an end-to-end tunnel policy on Windows
       system is the same as procedure for configuring a host policy, except that you must configure
       two, non-mirrored rules: one rule for outbound packets and one rule for inbound packets, as
       described in the sections that follow.

       NOTE: Do not configure any other rules in the policy with the HP-UX system address as the
       destination address. This prevents the Microsoft system from applying the tunnel transform over
       a host-to-host (transport) transform. In end-to-end tunnel topologies, HP-UX IPSec does not
       support transport transforms over a tunnel transform.

Outbound Tunnel Rule Requirements
       The outbound tunnel rule must have the following parameters:
       • Filter List: One filter, with the following parameters:
           — Address:
               ◦   Source address: the HP-UX system's address.
               ◦   Destination address: this must be a specific IP address and must be the Windows
                   system's address.
               ◦   Mirrored: no (the Mirrored box is cleared).
            —    Protocol Type: none (wildcard). The Windows documentation states that the filters in
                 tunnel rules must not specify protocols or ports to ensure that IP Security can correctly
                 process IP fragments.
       •    Tunnel Setting
            — Tunnel endpoint: the HP-UX system's address. This is the address of the tunnel endpoint
               closest to the destination. Since this is an end-to-end tunnel, it is the same as the
               destination address in the address filter.

Inbound Tunnel Rule Requirements
       The inbound tunnel rule must have the following parameters:
       • Filter List: One filter, with the following parameters:
           — Address:
               ◦   Source address: the Windows system's address.
               ◦   Destination address: this must be a specific IP address and must be the HP-UX
                   system's address.
               ◦   Mirrored: no (the Mirrored box is cleared).
            —    Protocol Type: none (wildcard).
       •    Tunnel Setting
            — Tunnel endpoint: the Windows system's address. This is the address of the tunnel
               endpoint closest to the destination. Since this is an end-to-end tunnel, it is the same as
               the destination address in the address filter

Configuring a Tunnel Rule
       Use the following procedure to configure an outbound or inbound tunnel rule.


       5. You can also configure an IPsec topology where packets exchanged between an HP-UX system and a Windows
          system are tunneled through an IPsec gateway device, but neither HP-UX nor Windows systems can be configured
          as IPsec gateways. The only topology in which an HP-UX system can act as an IPsec gateway is when the HP-UX
          system is a Home Agent for Mobile IPv6 clients. The HP-UX IPSec Administrator's Guide describes how to configure
          a host-to-gateway IPsec topology using HP-UX and a Cisco router.

                                                                 Configuring a Windows End-to-End Tunnel Policy        33
       TIP: The tunnel setting is used by all packets selected using the address filters for the rule. Do
       not include any filters for host-to-host (non-tunneled) packets in the filter list for a rule with a
       tunnel.
       1.   Start the IP Security Policies snap-in if necessary.
       2.   Create an IP Security policy or modify an existing policy. To modify an existing policy, select
            the policy in the right navigation pane and right click the policy. Select Properties.
       3.   The IP Security configuration utility opens the Policy Properties dialog box. Select the Rules
            tab. Click Add to create a new rule or select a rule you want to modify and click Edit.
       4.   Configure a new rule or modify an existing rule with the appropriate address filter for the
            outbound tunnel rule or inbound tunnel rule, as described in “Outbound Rule” (page 34)
            or “Inbound Rule” (page 35). See “Step 4: Creating the IP Filter List and Filters for the Rule”
            (page 18) if you need additional information about configuring address filters.
            Record the destination address; you will need it to configure the tunnel endpoint.
       5.   Return to the Rule Properties dialog box. Select the Tunnel Setting tab.
       6.   The IP Security configuration utility opens the Tunnel Setting dialog box.
            Select The tunnel endpoint is specified by this IP address.
            Enter the IP address of the tunnel endpoint closest to the destination. Since this is an
            end-to-end tunnel, it is the same as the destination address in the address filter.
       7.   Click Close to close the Tunnel Setting dialog box.
       8.   If this is a new rule, complete the configuration by configuring the appropriate filter action,
            authentication methods, and connection type.
            Click Close to close the Rule Properties dialog box.

Example
       In this example, IPsec secures all packets between the Windows system and the HP-UX system
       using authenticated ESP.
       The Windows system's address is 10.1.1.1
       The HP-UX system's address is 10.2.2.2.

Windows Configuration
       On the Windows system, you configure one rule for outbound packets and one for inbound
       packets.

       Outbound Rule
       The outbound rule is for packets from the Windows system (source address 10.1.1.1) to the
       HP-UX system (destination address 10.2.2.2). Figure 18 shows the address filter for this rule, and
       Figure 19 shows the corresponding tunnel settings:




34
Figure 18 Outbound Rule Filter




Figure 19 Outbound Rule Tunnel Settings




Inbound Rule
The inbound rule is for packets to the Windows system (destination address 10.1.1.1) from the
HP-UX system (source address 10.2.2.2). Figure 18 shows the address filter for this rule, and
Figure 21 shows the corresponding tunnel settings:




                                               Configuring a Windows End-to-End Tunnel Policy   35
     Figure 20 Inbound Rule Filter




     Figure 21 Inbound Rule Tunnel Settings




     Additional Parameters
     You must configure the remaining rule parameters (filter action, authentication methods, and
     connection type) to be compatible with the HP-UX configuration. In addition, the general
     parameters for the rule (the IKE SA parameters) must be compatible with the HP-UX configuration.




36
HP-UX Configuration
       On the HP-UX system, the host and tunnel policies are bi-directional (mirrored), so you configure
       only one host policy and only one tunnel policy. Since this is an end-to-end tunnel, the tunnel
       policy does not have to specify the tunnel endpoints. HP-UX IPSec will use the end source and
       end destination addresses as the tunnel addresses (the tsource and tdestination values
       default to the source and destination values).
       ipsec_config add host foo1 -source 10.2.2.2 \
       -destination 10.1.1.1 -action PASS -tunnel foo1_tunnel

       ipsec_config add tunnel foo1_tunnel -source 10.2.2.2 \
       -destination 10.1.1.1 -action ESP_3DES_HMAC_SHA1
       You must also configure an IKE policy and an authentication record to complete the configuration:
       ipsec_config add ike foo1 -remote 10.1.1.1 -auth PSK

       ipsec_config add auth foo1 -remote 10.1.1.1 \
       -psk my_preshared_key




                                                        Configuring a Windows End-to-End Tunnel Policy   37
Troubleshooting Tips
      Most interoperability problems occur during IKE negotiations, so examining IKE log events is
      useful. You can use the following procedures to enable and view IKE log events:

Using IKE Logging on HP-UX Systems
      Use the following procedure to view detailed IKE log events on HP-UX systems:
      1. Enter the following command to set the HP-UX IPSec log level to debug and increase the
          maximum log file size:
           ipsec_admin -al debug -maxsize 99999
           IPSec creates the log files in the /var/adm/ipsec directory. The log file names are
           auditdateinfo.log
      2.   Reproduce the problem.
      3.   Enter the following command to format the audit file:
           ipsec_report -audit /var/adm/ipsec/auditdateinfo.log
      4.   Use the following command to set the HP-UX IPSec log level back to warning (the default
           log level):
           ipsec_admin -al warning

Using IKE Logging on Windows Systems
      Use the following procedure to view IKE log events on Windows systems:
      1. Enable IKE logging. On Windows XP systems, use the regedit utility to enable IKE logging
          in the system registry. On Windows systems, IKE logging is configured using the Oakley6
          key.

           CAUTION: Incorrectly editing the registry may severely damage the system. Before making
           changes to the registry, HP recommends that you back up the registry and any valued data
           on the computer. Refer to the article How to back up, edit, and restore the registry in Windows
           XP and Windows Server 2003 in the Windows Knowledge Base for more information. The
           Windows Knowledge Base is available at http://support.microsoft.com
           Set the
           HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\PolicyAgent\Oakley\EnableLogging
           REG_DWORD value to 1. On some Windows versions, you may need to create the Oakley
           key.
           On Windows 2003 systems, enter the following command to enable IKE logging:
           netsh ipsec dynamic set config ikelogging 1
      2.   Stop and restart the IP Security service. You can use the following commands at the Windows
           command prompt:
           net stop policyagent
           net start policyagent
           Refer to “Step 9: Starting the IP Security Service” (page 29) for more information.
      3.   Reproduce the problem.
      4.   View the IKE log file. Windows creates the log file in the directory systemroot\Debug (by
           default, this is the WINDOWS\Debug directory). The file name is Oakley.log.



      6. The Oakley protocol is a key-agreement protocol that is incorporated in the IKE protocol.

38
      5.   Disable IKE logging. On Windows XP systems, set the
           HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\PolicyAgent\Oakley\EnableLogging
           REG_DWORD value to 0.
           On Windows 2003 systems, enter the following command:
           netsh ipsec dynamic set config ikelogging 0
      6.   Stop and restart the IP Security service.

Additional Windows Troubleshooting Tools
      Windows supports an IP Secu;rity Monitor snap-in utility for the Microsoft Management Console
      (MMC) that provides IPsec statistics and information about IKE and IPsec Security Associations.
      Refer to the Windows documentation set for more information.




                                                                              Troubleshooting Tips   39
Comparing HP-UX and Windows IPsec Configuration Parameters
     This section contains Table 1, which compares how HP-UX and Windows systems configure and
     store IPsec parameters. It also contains the following subsections, which provide additional
     comparative information:
     • “Mirrored Filters” (page 41)
     • “Filter Selection” (page 42)
     • “IKE Parameter Selection” (page 42)
     • “IKE SA Key (Master Key) Lifetime Values” (page 42)
     • “Maximum Quick Modes” (page 43)
     • “Perfect Forward Secrecy (PFS)” (page 43)
     • “IPsec SA Key (Session Key) Lifetime Values” (page 43)
     Table 1 IPsec Parameters on Windows and HP-UX
      Parameter                  Windows Configuration              HP-UX Configuration            Notes

      Address Filters            Specify them in the Filter         Specify one filter per host,   Windows and HP-UX
                                 List for a rule. The Filter List   tunnel, or gateway policy.     support subnet masks for IP
                                 can contain multiple               Use the -source and            addresses and wildcards for
                                 address filters.                   -destination arguments         IP addresses, protocols, and
                                                                    in the ipsec_config add        port numbers.
                                                                    host , tunnel, or              See “Mirrored Filters”
                                                                    gateway command.               (page 41) for additional
                                                                                                   information.

      IPsec SA Proposals         Specify them in the Filter         Specify them using the         HP-UX IPSec supports ESP
                                 Action for a rule.                 -action argument in the        encryption using the
                                                                    ipsec_config add               following protocols:
                                                                    gateway, host, or tunnel       Advanced Encryption
                                                                    command.                       Standard (AES), Triple Data
                                                                                                   Encryption Standard
                                                                                                   (3DES), and Data
                                                                                                   Encryption Standard (DES).
                                                                                                   Windows XP and Windows
                                                                                                   2000 support 3DES and
                                                                                                   DES, but do not support
                                                                                                   AES.

      Filter Priority            Not applicable.                    Specify it using the           See “Filter Selection”
                                                                    -priority argument in          (page 42) for additional
                                                                    the ipsec_config add           information.
                                                                    gateway or host
                                                                    command.

      Maximum IPsec SA           Specify it in the Custom           Specify it in the transform    See “IPsec SA Key (Session
      Lifetime, measured by time Security Methods dialog            specification for the          Key) Lifetime Values”
      or by data                 box under the Filter Action        -action argument in the        (page 43) for additional
                                 for a rule.                        ipsec_config add host          information.
                                                                    or tunnel command.

      Tunnel endpoint address    Specify the destination            Specify the endpoints using See “Mirrored Filters”
                                 tunnel endpoint (the               the -tsource and            (page 41) for additional
                                 endpoint for the                   -tdestination               information.
                                 destination) in the Tunnel         arguments of
                                 Settings for a rule. You must      theipsec_config add
                                 configure two                      tunnel command.
                                 uni-directional
                                 (non-mirrored) rules.

      IKE Authentication Method Specify it in the          Specify it using the -auth
                                Authentication Methods for argument of the
                                a rule.                    ipsec_config add ike
                                                           command.



40
       Table 1 IPsec Parameters on Windows and HP-UX (continued)
        Parameter                   Windows Configuration          HP-UX Configuration         Notes

        IKE Preshared Key           Specify it in the          Specify it using the
                                    Authentication Methods for -preshared argument of
                                    a rule.                    the ipsec_config add
                                                               auth command.

        IKE Exchange Type           Windows supports only          Specify it using the
                                    Main Mode exchanges.           -exchange argument of
                                                                   the ipsec_config add
                                                                   auth command. The
                                                                   default value is MM (Main
                                                                   Mode).

        Maximum IKE SA Lifetime, Specify it in the Key             Specify it using the-life   The Windows IP Security
        measured by time         Exchange Settings dialog          argument in the             Policy snap-in utility uses
                                 box. (To navigate to the Key      ipsec_config add ike        minutes as the time unit.
                                 Exchange Setting dialog           command.                    The HP-UX ipsec_config
                                 box, select the General tab                                   command uses seconds as
                                 in the Policy Properties                                      the time unit. See “IKE SA
                                 dialog box, then select                                       Key (Master Key) Lifetime
                                 Advanced settings.)                                           Values” (page 42) for
                                                                                               additional information.

        Maximum Quick Mode          Specify it in the Key          Specify it using the-maxqm See “Maximum Quick
        (QM) negotiations per IKE   Exchange Settings dialog       argument in the            Modes” (page 43) for
        SA                          box. (To navigate to the Key   ipsec_config add ike additional information.
                                    Exchange Setting dialog        command.
                                    box, select the General tab
                                    in the Policy Properties
                                    dialog box, then select
                                    Advanced settings.)

        Perfect Forward Secrecy     Windows supports PFS for       HP-UX does not support      See “Perfect Forward
        (PFS)                       keys only (PFS for session     PFS for session keys. HP-UX Secrecy (PFS)” (page 43) for
                                    keys) and supports PFS for     supports only PFS for       more information.
                                    keys in conjunction with       master keys.
                                    PFS for all identities (PFS    Specify PFS for master keys
                                    for master keys).              using the-maxqm 1
                                    Specify PFS for master keys    argument in the
                                    in the Key Exchange            ipsec_config add ike
                                    Settings dialog box. (To       command.
                                    navigate to the Key
                                    Exchange Setting dialog
                                    box, select the General tab
                                    in the Policy Properties
                                    dialog box, then select
                                    Advanced settings.)

        IKE SA Proposals            Specify it in the General      You can specify the        See “IKE Parameter
                                    parameters for a policy. You   parameters for one IKE SA Selection” (page 42) for
                                    can configure multiple IKE     proposal in an IKE policy, additional information.
                                    SA proposals and their         using the -encryption,
                                    preference order.              -hash, and -group
                                                                   arguments in an
                                                                   ipsec_config add ike
                                                                   command.


Mirrored Filters
       Microsoft filters can be mirrored (bi-directional) or not mirrored (uni-directional). If the filter is
       mirrored, the filter will match IP packets with the source and destination addresses and ports
       reversed. For example, a filter has the following specifications:
       Source address: 10.1.1.1
       Destination address: 10.2.2.2

                                                  Comparing HP-UX and Windows IPsec Configuration Parameters             41
        The filter matches packets with the following addresses:
        Source address: 10.1.1.1
        Destination address: 10.2.2.2
        If the filter is mirrored, it also matches packets with the following addresses:
        Source address: 10.2.2.2
        Destination address: 10.1.1.1
        The mirror setting only affects Windows IP Security behavior before IPsec SAs are established.
        If the Windows IP Security module receives a packet via an existing SA, it does not verify that
        the packet address fields match the address filter used when the SA was established.
        By comparison, HP-UX IPSec host and tunnel policies are always mirrored. (Gateway policies
        are the only HP-UX IPSec policies that are not mirrored.)

Filter Selection
        Windows does not allow you to specify the search or priority order for the filters in a rule or for
        the order of rules in a policy. The Windows IP Security module automatically creates an internal
        filter list and orders the filters from most specific to least specific.
        HP-UX IPSec allows you to specify a priority value for IPsec and IKE policies. HP-UX IPSec
        searches the policies in priority order within each type of policy. Lower priority values have
        higher priority (priority value 1 is the highest priority).
        If you do not specify a priority value when creating a policy on HP-UX, ipsec_config
        automatically assigns a priority value so that the new policy is the last policy searched before
        the default policy within its policy type. The output of the ipsec_config show command
        includes the priority values for configured policies.

IKE Parameter Selection
        On HP-UX systems, only one IKE SA proposal is used for each peer. You can configure multiple
        IKE policies, but only one IKE policy is selected per peer, and each IKE policy specifies only one
        IKE SA. During IKE negotiations, IKE searches policies in priority order and selects the first
        policy with a matching remote address. IKE then uses the IKE SA parameters to send an IKE SA
        proposal, or to evaluate the IKE SA proposal(s) it receives.
        On Windows systems, you can configure a set of multiple IKE SA proposals, but only one set
        per IP Security policy, and only one IP Security policy can be in use (assigned) on the system.

IKE SA Key (Master Key) Lifetime Values
        IKE SA key lifetimes (referred to as Master key lifetimes on Windows systems) specify the
        maximum lifetimes for IKE SA keys and are specified by units of time (seconds). In addition,
        users can specify the maximum number of IPsec SA negotiations that can be completed per IKE
        SA (“Maximum Quick Modes” (page 43)).

HP-UX IKE SA Lifetime Values
        The HP-UX IPSec default preferred lifetime value for IKE SAs is 28,800 seconds (eight hours).
        If the HP-UX system initiates IKE SA negotiations, the HP-UX IKE daemon proposes the preferred
        lifetime value to the remote system. The remote system may process this value in any manner
        according to the IPsec protocol suite.
        If the remote system initiates IKE SA negotiations and sends a proposed value that is longer than
        (less secure than) the HP-UX preferred value, HP-UX sends an IKE NOTIFY message with its
        preferred value, and this value is used for the SA.
        If the remote system initiates IKE SA negotiations and sends a proposed lifetime that is the same
        or more secure (shorter than) the HP-UX preferred value, the HP-UX IKE daemon accepts the

42
        proposed value sent by the remote system if it is within the range specified by the IPsec protocol
        suite.

Windows IKE SA Lifetime Values
        By default, Windows XP systems use the following values for preferred IKE key lifetime values:
        480 minutes (eight hours)
        0 (infinite) IPsec SA negotiations (sessions)
        In testing with HP-UX IPSec, HP configured a shorter IKE SA lifetime value on the Windows
        system. When the Windows system was the initiator, it sent the configured lifetime value to the
        remote system. When the Windows system was the responder, it accepted the value sent by the
        HP-UX system but did not send a notification message.

Maximum Quick Modes
        HP-UX and Windows enable you to specify the maximum number of IPsec or Quick Mode (QM)
        negotiations that IKE can complete per IKE SA. Each IPsec SA negotiation establishes two IPsec
        SAs (one in each direction).
        The default maximum QM values are as follows:
        HP-UX: 100
        Windows: 0 (infinite)
        If the value for maximum QM is 1, Perfect Forward Secrecy (PFS) for both keys and identities is
        implemented. See “Perfect Forward Secrecy (PFS)” (page 43) for more information.

Perfect Forward Secrecy (PFS)
        With Perfect Forward Secrecy, the exposure of one key permits access only to data protected by
        that key. RFC 2409, The Internet Key Exchange (IKE), defines two forms of PFS:
        •   PFS for both the keys and the IKE identities. PFS is provided for keys in conjuction with PFS
            for identities. IKE deletes the IKE SA after the IPsec negotiation completes. Each IKE SA is
            used for only one IPsec negotiation.
            The Windows interface refers to this type of PFS as master key PFS.
        •   PFS for IPsec keys only. The IKE peers perform a key exchange (Diffie-Hellman exchange)
            to create new keying material for each IPsec negotiation. The IKE SA is re-used until the
            IKE SA lifetime expires.
            The Windows interface refers to this type of PFS as session key PFS.
        HP-UX IPSec supports PFS for both the keys and the IKE identities but does not support PFS for
        IPsec keys only. To be compatible with HP-UX IPSec, do not configure session key PFS on
        Windows systems.
        Configuring PFS is computationally expensive. In most topologies, the strength of the
        cryptographic algorithms is sufficient protection. HP recommends that you enable PFS only in
        hostile environments.

IPsec SA Key (Session Key) Lifetime Values
        IPsec SA key lifetimes (referred to as session key lifetimes on Windows systems) specify the
        maximum lifetimes for IPsec SA keys and are specified by units of time (seconds) and by data
        units transferred (kbytes).

HP-UX IPsec SA Lifetime Values
        By default, HP-UX uses the following values for preferred lifetime values:
        28,800 seconds (eight hours)
        0 (infinite) data units

                                             Comparing HP-UX and Windows IPsec Configuration Parameters   43
        If the HP-UX system initiates IPsec SA negotiations, the HP-UX IKE daemon proposes the
        preferred lifetime values to the remote system. The remote system may process these values in
        any manner according to the IPsec protocol suite.
        If the remote system initiates IPsec SA negotiations and sends proposed lifetime value that is as
        secure or more secure than the HP-UX preferred value (it is shorter than or equal to the HP-UX
        preferred value), the HP-UX IKE daemon accepts the lifetime value proposed by the remote
        system if it is within the ranges specified by the IPsec protocol suite.
        If the remote system initiates IPsec SA negotiations and a proposed lifetime value is less secure
        (shorter than) the HP-UX preferred value, HP-UX sends an IKE NOTIFY message with its
        preferred value. If this value is acceptable to the remote system, the SA negotiation succeeds and
        the value sent in the NOTIFY message is used.

Windows IPsec SA Lifetime Values
        By default, the Windows configuration does not specify any IPsec SA lifetime values and does
        not propose any during IPsec SA negotiations. This is equivalent to proposing the lifetime values
        28,800 seconds (eight hours) and 0 (infinite) data units.
        In testing with HP-UX, HP also configured specific IPsec SA lifetime values on the Windows
        system and observed behavior equivalent to HP-UX behavior. When the Windows system
        initiated the IPsec SA negotiation, it sent the configured lifetime values in the proposal. When
        the remote system initiated the IPsec SA negotiation, the Windows system accepted the proposed
        lifetime value if it was more secure than its configured value, and sent a notification message
        when its configured lifetime value was more secure than the value proposed by the remote
        system.




44
Related Publications
      The following documents are available at http://docs.hp.com:
      •   HP-UX IPSec Administrator's Guide
      •   Using Microsoft Windows Certificates with HP-UX IPSec
      •   HP-UX IPSec manpages
      The following documents are available at http://microsoft.com:
      • Step-by-Step Guide to Internet Protocol Security (IPSec)
      • IPSec troubleshooting tools




                                                                       Related Publications   45
46
glossary
3DES              Triple Data Encryption Standard. A symmetric key block encryption algorithm that encrypts
                  data three times, using a different 56-bit key each time (168 bits are used for keys). 3DES is
                  suitable for bulk data encryption.
AES               Advanced Encryption Standard. Uses a symmetric key block encryption. HP-UX IPSec supports
                  AES with a 128-bit key. AES is suitable for encrypting large amounts of data.
AH                The AH (Authentication Header) protocol provides data integrity, system-level authentication
                  for IP packets. It can also provide anti-replay protection. The AH protocol is part of the IPsec
                  protocol suite.
authentication    The process of verifying a user's identity or integrity of data, or the identity of the party that
                  sent data.
DES               Data Encryption Standard. Uses a 56-bit key for symmetric key block encryption. It is suitable
                  for encrypting large amounts of data.
                  DES has been cracked (data encoded using DES has been decoded by a third party).
Diffie-Hellman    Method to generate a symmetric key where two parties can publicly exchange values and
                  generate the same shared key. Start with prime p and generator g, which may be publicly
                  known (typically these numbers are from a well-known Diffie-Hellman Group). Each party
                  selects a private value (a and b) and generates a public value (g**a mod p) and (g**b mod p).
                  They exchange the public values. Each party then uses its private value and the other party's
                  public value to generate the same shared key, (g**a)**b mod p and (g**b)**a mod p, which both
                  evaluate to g**(a*b) mod p for future communication.
                  The Diffie-Hellman method must be combined with authentication to prevent man-in-the-middle
                  or third party attacks (spoofing) attacks. For example, Diffie-Hellman can be used with certificate
                  or preshared key authentication.
ESP               The ESP (Encapsulating Security Payload) protocol provides confidentiality (encryption), data
                  authentication, and an anti-replay service for IP packets. When used in tunnel mode, ESP also
                  provides limited traffic flow confidentiality. The ESP protocol is part of the IPsec protocol suite.
IKE               The Internet Key Exchange (IKE) protocol is used before the ESP or AH protocol exchanges to
                  determine which encryption and/or authentication services will be used. IKE also manages the
                  distribution and update of the symmetric (shared) encryption keys used by ESP and AH.
IKE               The method used by IKE peers to authenticate each party's identity. HP-UX IPSec supports two
authentication    IKE authentication methods: preshared keys and RSA signatures using certificates.
IKE SA            IKE Security Association. An IKE SA is a bi-directional, secure communication channel that
                  IKE uses to negotiate IPsec SAs. IKE can establish IKE SAs using either Main Mode or Aggressive
                  Mode negotiations. Also referred to as IKE Phase One SA, ISAKMP SA, ISAKMP/MM SA,
                  Aggressive Mode SA, Main Mode SA.
IPsec SA          IPsec Security Association. An IPsec SA is a uni-directional, secure communication channel.
                  The IPsec SA operating parameters include the IPsec protocol used (ESP or AH), the mode
                  (transport or tunnel), the cryptographic algorithms (such as AES and SHA-1), the cryptographic
                  keys, the SA lifetime, and the endpoints (IP addresses, protocol and port numbers). IKE
                  establishes IPsec SAs using Quick Mode negotiations. Also referred to as IKE Phase Two SA,
                  IPsec SA, Quick Mode SA.
Perfect Forward   With Perfect Forward Secrecy the exposure of one key permits access only to data protected
Secrecy (PFS)     by that key. HP-UX IPSec supports PFS for keys and all identities (the IKE daemon can be
                  configured to create a new IKE SA for each IPsec negotiation). HP-UX IPSec does not support
                  PFS for keys only (the IKE SA is re-used for multiple IPsec negotiations, with a new
                  Diffie-Hellman key exchange for each IPsec negotiation).
SA                See Security Association. A secure communication channel and its parameters, such as encryption
                  and authentication method, keys and lifetime..
SHA1              (Secure Hash Algorithm-1). Authentication algorithm that generates a 160-bit message digest
                  using a 160-bit key.

                                                                                                                   47
transform       A transform defines the IPsec action(s) to be taken on the IP data, such as passing the data in
                clear text, discarding the data, authenticating and encrypting the data using ESP, or
                authenticating the data using AH.




48   glossary