IPSec Implementation
Kevin Fleming
Department of Pathology
University of Washington
Outline
What is IPSec?
Why do we want to use it?
How does IPSec work?
Case study – UW Pathology’s
implementation
What is IPSec?
IP Security is a set of protocols and
standards to support the securing of data at
the IP layer. IPSec is a framework, not an
implementation.
Supports authentication and encryption of
traffic.
– Certifies the originator of the packet.
– Protects the data from interception and
tampering while in transit.
Why do we want to use IPSec?
Secure our network
Transparent operation
– IPSec allows us to secure any IP based protocol
transparent to the application.
– Support for legacy software which is inherently
insecure (telnet,ftp,SMB).
– An alternative mechanism to implementing
application level security such as using SSL.
Widest industry support e.g. Cisco, Microsoft,
Network Associates, CheckPoint Software, Bay
Networks, etc.
IETF standard – Will be mandatory in IPv6
Why not use IPSec?
Processor overhead to encrypt & verify
each packet can be great.
Added complexity in network design.
IPSec Modes
Transport – Secures the payload part of the
IP packet, leaves the IP header unsecured.
Commonly use for securing traffic on a
LAN.
Tunnel – Secures the entire IP packet and
encapsulates it within a new IP packet.
Commonly used for creating a VPN.
IPSec protocols – AH protocol
AH - Authentication Header
– Defined in RFC 1826
– Integrity: Yes, including IP header
– Authentication: Yes
– Non-repudiation: Depends on cryptography algorithm.
– Encryption: No
– Replay Protection: Yes
Transport Packet layout
IP Header AH Header Payload (TCP, UDP, etc)
Tunnel Packet layout
IP Header AH Header IP Header Payload (TCP. UDP,etc)
IPSec protocols – ESP protocol
ESP – Encapsulating Security Payload
– Defined in RFC 1827
– Integrity: Yes
– Authentication: Depends on cryptography algorithm.
– Non-repudiation: No
– Encryption: Yes
– Replay Protection: Yes
Transport Packet layout
IP Header ESP Header Payload (TCP, UDP, etc)
Tunnel Packet layout
IP Header ESP Header IP Header Payload (TCP. UDP,etc)
Unencrypted Encrypted
What protocol to use?
Differences between AH and ESP:
– ESP provides encryption, AH does not.
– AH provides integrity of the IP header, ESP does not.
– AH can provide non-repudiation. ESP does not.
However, we don’t have to choose since both
protocols can be used in together.
Why have two protocols?
– Some countries have strict laws on encryption. If you
can’t use encryption in those countries, AH still
provides good security mechanisms. Two protocols
ensures wide acceptance of IPSec on the Internet.
Security Associations
One of the most important concepts in IPSec is called a
Security Association (SA). Defined in RFC 1825.
SAs are the combination of a given Security Parameter
Index (SPI) and Destination Address.
SAs are one way. A minimum of two SAs are required for
a single IPSec connection.
SAs contain parameters including:
– Authentication algorithm and algorithm mode
– Encryption algorithm and algorithm mode
– Key(s) used with the authentication/encryption algorithm(s)
– Lifetime of the key
– Lifetime of the SA
– Source Address(es) of the SA
– Sensitivity level (ie Secret or Unclassified)
How IPSec works: Phase 1
Internet Key Exchange (IKE) is used to setup IPSec.
IKE Phase 1:
– Establishes a secure, authenticated channel between the two computers
– Authenticates and protects the identities of the peers
– Negotiates what SA policy to use
– Performs an authenticated shared secret keys exchange
– Sets up a secure tunnel for phase 2
– Two modes: Main mode or Aggressive mode
Main Mode IKE
1. Negotiate algorithms & hashes.
2. Generate shared secret keys using a Diffie-Hillman exchange.
3. Verification of Identities.
Aggressive Mode IKE
– Squeezes all negotiation, key exchange, etc. into less packets.
– Advantage: Less network traffic & faster than main mode.
– Disadvantage: Information exchanged before a secure channel is
created. Vulnerable to sniffing.
How IPSec works: Phase 2
– An AH or ESP packet is then sent using the agreed
upon “main” SA during the IKE phase 1.
– IKE Phase 2
• Negotiates IPSec SA parameters
• Establishes IPSec security associations for specific connections
(like FTP, telnet, etc)
• Renegotiates IPSec SAs periodically
• Optionally performs an additional Diffie-Hellman exchange
How IPSec works: Communication
Once Phase 2 has established an SA for a
particular connection, all traffic on that connection
is communicated using the SA.
IKE Phase 1 exchange uses UDP Port 500.
AH uses IP protocol 51.
ESP uses IP protocol 50.
Case Study: UW Pathology
A network with 200 workstations, 100 thin clients and 30
servers used by 500+ users.
Spans Health Sciences Building, UWMC, Harborview, and
Roosevelt clinics.
Network is a Windows 2000 Active Directory domain
running on top of an IP based Ethernet infrastructure.
Network spans over 30 different IP subnets.
A variety of networking infrastructure including 10Mbps
shared, 100Mbps switched, and 1Gbps switched.
Case Study: Goals
Goals:
– Initially secure VNC, MS SQL traffic, and MS
NetBIOS (SMB) traffic.
– Provide this security transparently. Could not do
application level security (e.g. SSL) with these services.
– Provide another layer of protection for workstations.
– Integrate seamlessly into existing environment.
Technologies used:
– UW Pathology runs a Windows 2000 Active Directory
infrastructure. Therefore Microsoft Windows IPSec
implementation was chosen.
Case Study: Design Concepts
Make workstation rules as simple as possible for
ease of implementation and to ensure transparent
migration to IPSec on the network.
Make server rules control what traffic should be
secured. This allows a one server at a time
migration to IPSec.
Don’t try to protect everything. Use IPSec as a
tool to protect certain services.
Authenticate systems against the AD domain.
Centralized deployment by Group Policy.
Case Study: General Design
When using IPSec security:
– Use ESP with 3DES and SHA-1.
– Use Kerberos authentication against AD.
– Accept unsecured communication, but always
reply with IPSec.
– Generate new keys ever 100MB or 60 mins.
Permit unencrypted traffic to WINS &
Domain Controller servers.
Case Study: Workstation Policy
Enable a Default IPSec response rule that allows
negotiation of policies with the servers.
Secure VNC &
NetBIOS traffic.
Case Study: Server Policy
Enable Default response.
Define different policies based upon services that
require protection.
Case Study: IPSec Helpful hints
Do not enable IPSec for the entire system (Microsoft’s
built-in “Secure Server” policy). This is a bad idea
because the system will not be able to access any systems
not running IPSec such as DNS and Web.
Every system on the AD domain must be able to logon via
the network to other systems otherwise Kerberos
authentication fails.
Remember to always add the “Default Response” to
computers, otherwise you will have to explicitly setup
every type of IPSec policy it should respond to.
Remember to open up UDP Port 88 & 500, TCP Port 88,
and IP protocols 50 & 51 through any firewalls.
Do not use IPSec on Domain Controllers. Only setup
Domain Controllers to have the “Default Response”.
Case Study: IPSec Windows Tools
Netdiag on the W2K resource kit shows a lot of
information about IPSec policies on the system.
Run: netdiag /test:ipsec /debug
IPSecmon.exe on W2K shows active SAs.
On WinXP, use the MMC “IP Security Monitor”
Enable Debug logging:
– HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\IPSEC\DiagnosticMode=DWORD:1
– HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PolicyAgent\Oakley\EnableLogging=DW
ORD:1
Case Study: Conclusions
IPSec successfully protected Pathology systems
from RPC DCOM attacks.
No noticeable slowdowns although system
processor usage increased. NIC cards that offload
encryption tasks greatly help.
Microsoft’s IPSec implementation is buggy.
– SA synchronization issue (?). Systems do not negotiate
for appx. 30 minutes.
References:
http://www.forsitesolutions.com/Techstuff/freeswan/ipsec_overview.html
http://www.informit.com/content/index.asp?product_id=%7BC7E9CB3C%2D2995%2D
4EC6%2D9CCF%2DE0D8823516AA%7D
Microsoft Help files & White papers
RFCs 1825,1826,1827