Deploying Storage Networks

Document Sample
Deploying Storage Networks Powered By Docstoc
					                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.   1




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                                       Deploying Storage Networks
                                                                                          Session CDC-201




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                     3




                           Question?


                                         “Most all data on the Internet and in
                                          Enterprise Networks is transported
                                          over this Protocol.”

                                                  IP Storage Networking Concepts and
                                                 Technical Storage Architecture Are the
                                                   Fundamentals Needed for the Next
                                                   Converging Network Technologies
                                                                                           Are You Ready?

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                     4




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Agenda—First Half



                              • Introduction of Technology Standards
                              • Understanding the Fibre Channel
                                Storage Network
                              • Open Standards in IP Storage Transports
                              • Networked Storage Essentials



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.   5




                           Agenda—Second Half


                              • Networked Storage Designs
                              • iSCSI Concepts In-Depth
                              • Designing Storage access with iSCSI and
                                the SN5420
                              • Advances in Storage Concepts
                                  (If time allows)



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.   6




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                                                        Introduction of
                                                     Technology Standards



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                               7




                           Storage Technology—
                           SCSI-3 Architecture Roadmap
                                   Common Access Method (CAM)

                                                                                                                                               Object-
                                        Reduced                                             Medium
                        Block                     Stream                                            Multi-Media Controller Enclosure            Based
                                         Block                                              Changer
                      Commands                  Commands                                            Commands Commands Commands                 Storage
                                       Commands                                            Commands
                                                                                                                                               Device



                                                                                          Primary Commands (SPC -3)
                                                                                          Primary Commands (SPC -3)

                                   Architecture Model (SAM-2)



                                                SCSI                      Serial                 Fibre        SSA        SCSI over       SCSI Virtual
                           iSCSI
                           iSCSI               Parallel                    Bus                  Channel      SCSI -3     Scheduled        Interface
                                              Interface                  Protocol               Protocol    Protocol      Transfer        Protocol

                                                                                                             SSA- TL2

                                              Ultra 2,
                                                                        IEEE 1394                Fibre     SSA- PH1 or    Scheduled         Virtual
                                             Ultra 160,
                                                                         Firewire               Channel     SSA- PH2     Transfer (ST)   Interface (VI)
                                             Ultra 320

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                               8




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Storage Technology

                             • SCSI-3 Common Access Method—CAM-3
                                       Set of SCSI standards that define the command
                                       methods for such devices as block commands, stream
                                       commands, controllers, multimedia changers, and
                                       enclosure services
                             • SCSI-3 Primary Commands—SPC-3
                                       This standard defines the SCSI commands that are
                                       mandatory and optional for all SCSI devices
                             • SCSI-3 Architecture Model—SAM-2
                                       The set of SCSI standards that specifies the interfaces,
                                       functions, and operations necessary to ensure
                                       interoperability between conforming SCSI
                                       implementations
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                   9




                           Storage Technology

                                            Initiator—SCSI device which is capable of
                                            originating SCSI commands and task
                                            management requests
                                            Target—SCSI device which is capable of
                                            executing SCSI commands and task
                                            management requests


                              Target                                                      Initiator
                                                                           FC SAN                                  User

                                                                                                      IP Network
                                                                                          Server

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                   10




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Storage Technology
                           SCSI Domain
                                                SCSI Device                                Service Delivery                SCSI Device
                                                  Initiator                                  Subsystem                       Target


                                                                                                                                 Logical




                                                                                   Port




                                                                                                                    Port
                                           Application
                                           Application                                                                            Unit 1
                                             Client
                                             Client                                        Device Service Request
                                                                                          Device Service Response                Device
                                                                                                                                 Server
                                                                                               Task Request

                                                                                              Task Response
                                                                                                                                 Task
                                                                                                                                Manager




                             • To be functional, a SCSI Domain needs to contain a
                               SCSI device that contains a target and a SCSI device
                               that contains an Initiator
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                    11




                           Storage Technology
                           Logical Unit

                           A Basic Logical Unit Is Composed of a Logical
                           Unit Number, a Device Server, and One or More
                           Task Sets
                              • LUN—a Logical Unit Number is a field
                                containing up to 64 bits that identifies the
                                Logical Unit within a target device
                              • Device Server—object that executes SCSI
                                commands and manages the task set
                              • Task—used to manage the SCSI function;
                                i.e. abort, terminate, clear or reset
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                    12




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Storage Technology
                           LUN Math

                            • The SCSI-3 Architectural Model reserves a
                              64-bit field for LUN address
                            • SCSI-3 Architecture allows for 65,536
                              addressable units in the 16 bit field. The
                              higher addressed units are seen on Fibre
                              Channel Networks.
                            • Limitations of accessible LUN’s depends on
                              Host system and specific carrier protocol; i.e.
                              Fibre Channel, Parallel Bus
                            • Target must have a LUN 0
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                             13




                           Storage Technology
                           LUN Examples


                                LUN 0



                                   LUN 1
                                                                                                            RAID
                                   LUN 2
                                                                                                            LUN 5
                                       LUN 3

                                   LUN 4


                                                                                          Disk Array With
                                                                                             Controller

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                             14




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Storage Technology
                           Masking


                            • Much the same technique as IP subnet masking
                              except it is done to block out a LUN device
                              visibility to the initiator
                            • Masking can be done by many different network
                              devices; i.e Fibre Channel Switches can mask,
                              disk array controllers and Host Bus Adapters can
                              also mask
                            • Masking extends zoning to the LUN level
                            • Facilitates sharing between incompatible
                              operating systems

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.   15




                           Storage Technology
                           Volume

                                • Storage pooling control
                                • Creates logical volumes from the user
                                  accessible storage
                                • Either physical devices or RAID devices
                                • Volumes can span physical devices
                                • Volumes may be combined into SETS to
                                  accomplish volume mirroring

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.   16




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Storage Technology
                                              Bits and Bytes - The Storage Numbers

                                  The use of Mega-bytes (MB) is used in storage
                                  networks rather than Mega-bits (Mb) that are often
                                  used to express LAN speeds.

                                           Fibre Channel                                                GigE
                                       Transfers in sequences                                    Transfers in packets
                          Xfer rate: 1.062 Gbit/sec                                        Xfer rate: 1.25Gbit/sec
                          100 MB/sec                                                       100 MB/sec
                          Error Rate = 10exp -12                                           Error Rate = 10exp -10
                          Flow control = Credit Based                                      Flow Control = Rate Based
                          Max Payload: 2112 bytes                                          Max Payload: 1518 bytes

                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                17




                                                                  Understanding the
                                                                    Fibre Channel
                                                                   Storage Network


                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                18




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Fibre Channel Protocol Architecture


                        ULP (Upper Level Protocol)                                           SCSI - 3
                                                                                             SCSI - 3                IP
                                                                                                                     IP                 ATM
                                                                                                                                        ATM




                                                           IPI - 3 Command
                                                           IPI - 3 Command                   SCSI —
                                                                                             SCSI —                FC Link
                                                                                                                   FC Link
                        FC - 4                               Set Mapping
                                                             Set Mapping                  Command Set
                                                                                          Command Set           Encapsulation
                                                                                                                Encapsulation         FC - ATM
                                                                                                                                      FC - ATM
                                                               (IPI -3 STD)
                                                               (IPI -3 STD)                 Mapping
                                                                                            Mapping                 FC-LE
                                                                                                                    FC-LE

                        FC - 3
                                                                                                  Common Services
                                                                                                  Common Services


                                                                                                                       FC - AL
                                                                                                                       FC - AL        FC - AL 2
                                                                                                                                      FC - AL 2
                        FC - 2                  Framing Protocol                            Fibre Channel
                                                                                             Fibre Channel
                        FC - 1                  Encode/Decode                                Physical and
                                                                                             Physical and           8B/10B Encoding
                                                                                          Signaling Interface
                                                                                          Signaling Interface
                        FC - 0                  Physical Variant                                                    Copper, Optical


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                           19




                           Fibre Channel
                           Introduction




                             • Fibre Channel is an integrated set of ANSI
                               standards developed to combine the best
                               characteristics of network I/O and channel
                               I/O and to overcome their inherent
                               limitations



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                           20




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Fibre Channel
                           Characteristics


                                • A serial I/O protocol designed primarily
                                  for data delivery, secondarily for voice
                                  and video
                                • Combines network I/O and channel I/O
                                  characteristics and runs over both optical and
                                  copper media
                                • Unaware of the content of the data transferred




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.   21




                           Fibre Channel
                           Characteristics




                              • Allows connectivity among hundreds of
                                thousands of devices
                              • Transports upper layer protocols
                                including: HIPPI, SCSI, IP, IEEE 802, IPI-3,
                                SBCCS, ATM, VI
                              • FC is a flat structure (no hierarchy)


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.   22




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Fibre Channel
                           Characteristics

                              • FC is divided into Classes of Service
                                         Class 1: Dedicated connection; a virtual pipe created
                                         between two N_Ports
                                         Class 2: Multiplexed; each frame finds a route to the
                                         destination, delivery or non-delivery is acknowledged
                                         Class 3: Datagram service; same as Class 2 without
                                         acknowledgement. Note: SN5420 connects at this level
                                         Class 4: Connection Oriented, Fractional bandwidth
                                         (voice and video)
                                         Class 6: Uni-directional, reliable, one -to-many
                                         multicast service


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                     23




                           Fibre Channel
                           Network

                                       • FC Topologies fall into 3 standards

                                       Point-to-point



                                  Arbitrated Loop                                         Switched Fabric




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                     24




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Fibre Channel
                           FCP Payloads and Information Sets

                                • FCP_CMND (Command set)
                                           Communicates the command to the Target
                                                    Read and write commands
                                                    Task duties like reset and clear
                                • FCP_XFER_RDY (Data Transfer set)
                                           Specifies the length and offset of the data sequence



                                                                                 Fibre Channel
                                                                                 Fibre Channel     SCSI
                                                                                                   SCSI    CRC
                                                                                                           CRC
                                                                                    Header
                                                                                     Header      Command
                                                                                                 Command


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                          25




                           Fibre Channel
                           FCP Payloads and Information Sets


                              • FCP_Data (Transfer of data)
                                          Transfers Data between initiator and target.
                                          Three different types of data, Parameter data,
                                          Response data, and Logical data
                                          Parameter e.g. Format, Mode select
                                          Response e.g. Sense, Inquire data
                                          Logical e.g. read and write data
                                                                                 Fibre Channel
                                                                                 Fibre Channel     SCSI
                                                                                                   SCSI    CRC
                                                                                                           CRC
                                                                                    Header
                                                                                     Header      Command
                                                                                                 Command


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                          26




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Fibre Channel
                           FCP Payloads and Information Sets



                                 • FCP_RSP (Command Response)
                                             Signals the completion of SCSI Command
                                             Contains status, sense, and response code
                                             information
                                             Provide task management responses

                                                                                 Fibre Channel
                                                                                 Fibre Channel     SCSI
                                                                                                   SCSI    CRC
                                                                                                           CRC
                                                                                    Header
                                                                                     Header      Command
                                                                                                 Command


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                          27




                           Fibre Channel
                           Addressing
                        The Topology of a Fibre Channel Network Is
                        Determined at Network Connection Time
                            • Every Fibre Channel entity, port, node and fabric has a 64
                              bit ID assigned by the manufacturer (MAC) and is referred
                              to as a World Wide Name
                            • Every HBA has a 64 bit address assigned to the N_Port
                              and is called the World Wide Port Name
                            • Each N_Port is locally assigned a 24 bit ID called a
                              N_Port_ID and is used by the network to route between
                              N_Ports; used to verify that received information was
                              delivered to correct port


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                          28




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Fibre Channel
                           Addressing
                        Address identifiers are assigned during initialization
                        of the N_Port into the network topology; address
                        Identifiers can be changed if N_Port is moved to a
                        different location on the network

                              • Port_Name should be used by management to determine identity of
                                N_Port
                              • The N_Port has functional requirements on-board to determine
                                characteristics of the topology (Fabric or Loop)
                              • Each port has separate transmit and receive functions for full duplex
                                capabilities




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                   29




                           Fibre Channel
                           Addressing


                             • Fibre Channel Loop and Fabric technologies assign an abbreviated
                               address called a Arbitrated Loop Physical Address at connection to
                               network (AL_PA)
                             • In a Loop topology this 1 byte address corresponds to the 8 least
                               significant bits of the N_Port 24-bit address
                             • AL_PA can equal 0 to 126. 0 can be used to connect the loop to a
                               fabric network.
                             • In a Switch Fabric topology the uppermost 16 bits of the N_Port 24-bit
                               address identifies the loop you are on and the 8 least bits identify the
                               port on that loop




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                   30




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Fibre Channel
                           Addressing

                                                                   HBA                                  Hubs and Switch




                                                  N_Port NL_Port                                        AL_Port   FL_Port

                                                                                Locally Assigned Port ID
                                                          Byte 0                               Byte 1             Byte 2
                                            23                                    16      15             8   7              0

                                                                                               N_PORT
                                                                                                                  Port on
                                                              FL_PORT (zeros if on hub)                            Loop
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                         31




                           World Wide Name
                           Recap


                            • Node Name(WWNN): describes Fibre Channel
                              world wide node name assigned to the device by
                              the vendor
                            • Port Name (WWPN): describes Fibre Channel
                              world wide port name assigned to the Host Bus
                              Adapter (HBA) by the vendor
                            • Loop ID: describes Fibre Channel loop identifier
                              assigned to the N_Node via the network


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                         32




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Fibre Channel
                           Services

                                   • Login Server—FF FF FE
                                                Handles Characteristics and Login requests
                                                Provides N_Port Address
                                   • Fabric/Switch Controller—FF FF FD
                                                Routing Management
                                                Assists in initialization and configuration
                                   • Name Server—FF FF FC
                                                Provides registration Service for N_Port to access
                                                 database information.
                                   • Time Server—FF FF FB
                                                Time Synchronization with network and time stamping

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                  33




                           Fiber Channel
                           Services

                                       • Management Server “FF FF FA”
                                                    SNMP and access to MIB data on the Network
                                                    Can use IP or natively over FC

                                       • Alias Server “FF FF F8”
                                                    Used to establish hunt groups and Multicast groups

                                       • Quality of Service Server “FF FF F9”
                                                    Used to manage QoS on Class 4 Services

                                       • Multicast Server “FF FF F5”
                                                    Manages the Multicast groups


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                  34




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Fibre Channel
                           SCSI Command Structure
                               All SCSI commands share a common structure,
                              known as the Command Descriptor Block (CDB); this
                              is a frame of at least 6 Bytes (some commands have
                              10, 12, or 16 Bytes or more) which in its simplest form
                              comprises an Operation Code, a Block Address and
                              the Number of Blocks to process

                                 Bit                 7                     6              5        4           3   2   1   0

                               Byte

                                  0                                                           Operation Code
                                  1                                                  Command Specific Parameters

                                n-1
                                  n                                                              Control

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                        35




                           Fibre Channel
                           SCSI


                            • A SCSI command or a linked set of commands is referred to as
                              a “task”; only one command in a task can be outstanding at any
                              given time
                            • SCSI command execution results in a data phase and a
                              status phase
                            • In the data phase, data travels either from the initiator to the
                              target as in a Write command, or from the target to the initiator
                              as in a READ command
                            • In the status phase, the target returns the final status of the
                              operation; a response terminates a SCSI command



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                        36




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Fibre Channel
                           SCSI


                                 • Example command op-codes
                                                                                          READ (08), WRITE (0A)

                                                                               READ the first 10 data blocks (0 to 9)
                                                                                          CDB is 08, 0, 10

                                                                       WRITE the second 10 data blocks (10 to 19)
                                                                       CDB is 0A, 10, 10 ,10, 10, 10, 10, 10, 10, 10, 10
                                 • Command specifics can be found in the T10 organizational
                                   standards for SCSI devices




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                    37




                           Fibre Channel
                           Fibre Channel Login


                              • Determine Topology with
                                Initialization (LIP)
                              • Perform Fabric Login (FLOGI) if switch network
                              • Perform N_Port Login (PLOGI)
                              • Perform Process Login (PRLI)
                              • Perform SCSI Protocol Specific actions



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                    38




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Switch and Hub Fabric Network
                                                                                           N_Port 0
                                                                                           N_Port 0                          N_Port 1
                                                                                                                             N_Port 1

                                                                                           Transmitter
                                                                                           Transmitter                     Transmitter
                                                                                                                           Transmitter
                                                                                                         F_Port
                                                                                                         F_Port   F_Port
                                                                                                                  F_Port
                                                                                            Receiver
                                                                                            Receiver                        Receiver
                                                                                                                            Receiver
                                                                                                           W
                                                                                                           W        X
                                                                                                                    X

                                                                                            Node A                          Node B
                                                                                                             Fabric
                                       NL_Port 0
                                       NL_Port 0                                                                            N_Port 3
                                                                                                                            N_Port 3

                                       Transmitter
                                       Transmitter                                                                         Transmitter
                                                                                                                           Transmitter
                                                                                                         F_Port
                                                                                                         F_Port   F_Port
                                                                                                                  F_Port
                                         Receiver
                                         Receiver                                                          Y
                                                                                                           Y        Z
                                                                                                                    Z       Receiver
                                                                                                                            Receiver

                                       Node D                                                                               Node C

                                       NL_Port 1
                                       NL_Port 1                                             NL_Port 2
                                                                                             NL_Port 2

                                       Transmitter
                                       Transmitter                                         Transmitter
                                                                                           Transmitter

                                         Receiver
                                         Receiver                                           Receiver
                                                                                            Receiver


                     CDC-201           Node E                                                Node F
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                                 39




                           Fibre Channel




                                • Determine Topology with Initialization (LIP)
                                               Arbitrated loop, switch fabric, point-to-point




                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                                 40




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Fibre Channel
                           LIP
                                                                                                         Hub
                                                                                           NL_Port 0
                                                                                           NL_Port 0
                                                                                                                           LIP
                                                                                           Transmitter
                                                                                           Transmitter
                              • Node D plugged into Loop                                                                         FL_Port Y
                                                                                                                                 FL_Port Y
                                                                                            Receiver
                                                                                            Receiver
                                and Initialization begins
                              • All ports on the loop go to                                Node D
                                Open Init state and
                                suspended any operations
                                                                                           NL_Port 1
                                                                                           NL_Port 1            NL_Port 2
                                                                                                                NL_Port 2
                                in progress
                                                                                           Transmitter
                                                                                           Transmitter         Transmitter
                                                                                                               Transmitter
                              • A Loop Master is selected;
                                in this case it will be the                                 Receiver
                                                                                            Receiver            Receiver
                                                                                                                Receiver
                                FL port
                                                                                           Node E               Node F




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                 41




                           Fibre Channel
                           LIP
                                                                                                         Hub
                                                                                           NL_Port 0
                                                                                           NL_Port 0
                                                                                                                           LIP
                            • Loop Master Builds position                                  Transmitter
                                                                                           Transmitter
                              map and assigns addresses                                                                          FL_Port Y
                                                                                                                                 FL_Port Y
                              AL_PA                                                         Receiver
                                                                                            Receiver

                            • Loop Master then transmits                                   Node D
                              the complete map to all ports
                              on the loop and initialization
                                                                                           NL_Port 1
                                                                                           NL_Port 1            NL_Port 2
                                                                                                                NL_Port 2
                              is complete
                                                                                           Transmitter
                                                                                           Transmitter         Transmitter
                                                                                                               Transmitter

                                                                                            Receiver
                                                                                            Receiver            Receiver
                                                                                                                Receiver


                                                                                           Node E               Node F
                       When do we LIP:                                  • Power on or reset
                                                                        • Entering—needs an AL_PA to Participate
                                                                          on loop
                     CDC-201
                     2878_04_2001_c1
                                                                        •       Leaving the loop
                                       © 2001, Cisco Systems, Inc. All rights reserved.                                                 42




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Fibre Channel
                           FLOGI


                             • Determine Topology with Initialization (LIP)
                                         Arbitrated Loop, Switch Fabric, Point to Point
                             • Perform Fabric Login (FLOGI) if Switch network
                                         Fabric Login is done to determine if a switch is present.
                                         Port attempts to talk to well-known address FF FF FE , which is the
                                         F_Port we are connecting to
                                         Source and destination address are identified and managed by
                                         switch
                                         Switch returns an Accept (ACC) and service parameters are
                                         exchanged, Login complete




                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                                 43




                           Switched Fabric
                           FLOGI
                                                                                           N_Port 0
                                                                                           N_Port 0                          N_Port 1
                                                                                                                             N_Port 1

                                                                                           Transmitter
                                                                                           Transmitter                     Transmitter
                                                                                                                           Transmitter
                                                                                                         F_Port
                                                                                                         F_Port   F_Port
                                                                                                                  F_Port
                                                                                            Receiver
                                                                                            Receiver                        Receiver
                                                                                                                            Receiver
                                                                                                           W
                                                                                                           W        X
                                                                                                                    X

                                                                                            Node A                          Node B
                                                                        FLOGI
                                                                                                             Fabric
                                       NL_Port 0
                                       NL_Port 0                        Accept                                              N_Port 3
                                                                                                                            N_Port 3

                                       Transmitter
                                       Transmitter                                                                         Transmitter
                                                                                                                           Transmitter
                                                                                                         F_Port
                                                                                                         F_Port   F_Port
                                                                                                                  F_Port
                                         Receiver
                                         Receiver                                                          Y
                                                                                                           Y        Z
                                                                                                                    Z       Receiver
                                                                                                                            Receiver

                                       Node D                                                                               Node C

                                       NL_Port 1
                                       NL_Port 1                                             NL_Port 2
                                                                                             NL_Port 2

                                       Transmitter
                                       Transmitter                                         Transmitter
                                                                                           Transmitter

                                         Receiver
                                         Receiver                                           Receiver
                                                                                            Receiver


                     CDC-201           Node E                                                Node F
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                                 44




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Fibre Channel
                           PLOGI



                            • Determine Topology with Initialization (LIP)
                                        Arbitrated Loop, Switch Fabric, Point to Point

                            • Perform Fabric Login (FLOGI) if Switch network
                            • Perform N_Port Login (PLOGI)
                                        N_Port to N_Port Exchange of service parameters across the
                                        Fibre Channel Network
                                        Class of service, max frame size, buffers




                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                                 45




                           Switched Fabric
                           PLOGI
                                                                                           N_Port 0
                                                                                           N_Port 0                          N_Port 1
                                                                                                                             N_Port 1

                                                                                           Transmitter
                                                                                           Transmitter                     Transmitter
                                                                                                                           Transmitter
                                                                                                         F_Port
                                                                                                         F_Port   F_Port
                                                                                                                  F_Port
                                                                                            Receiver
                                                                                            Receiver                        Receiver
                                                                                                                            Receiver
                                                                                                           W
                                                                                                           W        X
                                                                                                                    X

                                                                                            Node A                          Node B
                                                                        PLOGI                                Fabric
                                       NL_Port 0
                                       NL_Port 0                        Accept                                              N_Port 3
                                                                                                                            N_Port 3

                                       Transmitter
                                       Transmitter                                                                         Transmitter
                                                                                                                           Transmitter
                                                                                                         F_Port
                                                                                                         F_Port   F_Port
                                                                                                                  F_Port
                                         Receiver
                                         Receiver                                                          Y
                                                                                                           Y        Z
                                                                                                                    Z       Receiver
                                                                                                                            Receiver

                                       Node D                                                                               Node C

                                       NL_Port 1
                                       NL_Port 1                                             NL_Port 2
                                                                                             NL_Port 2

                                       Transmitter
                                       Transmitter                                         Transmitter
                                                                                           Transmitter

                                         Receiver
                                         Receiver                                           Receiver
                                                                                            Receiver


                     CDC-201           Node E                                                Node F
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                                 46




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Fibre Channel
                           PRLI


                              • Determine Topology with Initialization (LIP)
                                           Arbitrated Loop, Switch Fabric, Point to Point
                              • Perform Fabric Login (FLOGI) if Switch network
                              • Perform N_Port Login (PLOGI)
                                           Determine address
                                           Scan loop or Query the Name Server
                              • Perform Process Login (PRLI)
                                           Logical Connection for Upper Level Protocols found at level FC-4
                                           (Required in SCSI-FCP)




                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                                          47




                           Switched Fabric
                           PRLI
                                                                                           N_Port 0
                                                                                           N_Port 0                          N_Port 1
                                                                                                                             N_Port 1

                                                                                           Transmitter
                                                                                           Transmitter                     Transmitter
                                                                                                                           Transmitter
                                                                                                         F_Port
                                                                                                         F_Port   F_Port
                                                                                                                  F_Port
                                                                                            Receiver
                                                                                            Receiver                        Receiver
                                                                                                                            Receiver
                                                                                                           W
                                                                                                           W        X
                                                                                                                    X

                                                                                            Node A                          Node B
                      Initiator                                          PRLI                                Fabric                      Target
                                       NL_Port 0
                                       NL_Port 0                        Accept                                              N_Port 3
                                                                                                                            N_Port 3

                                       Transmitter
                                       Transmitter                                                                         Transmitter
                                                                                                                           Transmitter
                                                                                                         F_Port
                                                                                                         F_Port   F_Port
                                                                                                                  F_Port
                                         Receiver
                                         Receiver                                                          Y
                                                                                                           Y        Z
                                                                                                                    Z       Receiver
                                                                                                                            Receiver

                                       Node D                                                                               Node C

                                       NL_Port 1
                                       NL_Port 1                                             NL_Port 2
                                                                                             NL_Port 2

                                       Transmitter
                                       Transmitter                                         Transmitter
                                                                                           Transmitter

                                         Receiver
                                         Receiver                                           Receiver
                                                                                            Receiver


                     CDC-201           Node E                                                Node F
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                                          48




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Fibre Channel


                                       • Determine Topology with Initialization (LIP)
                                                   Arbitrated Loop, Switch Fabric, Point to Point
                                       • Perform Fabric Login (FLOGI) if Switch network
                                       • Perform N_Port Login (PLOGI)
                                                   Determine address
                                                   Scan loop or Query the Name Server
                                       • Perform Process Login (PRLI)
                                                   Logical Connection for Upper Level Protocols (Required in
                                                   SCSI-FCP)
                                       • Perform SCSI Protocol Specific actions
                                                   Test Unit ready, Inquiry

                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                       49




                                                         Open Standards for
                                                        IP Storage Transports



                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                       50




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           IP Storage Transports

                             • DWDM—Dense Wave Division Multiplexing
                             • FCIP—Fiber Channel over TCP/IP;
                               draft-ietf-ips-fcovertcpip
                             • iSNS—Internet Storage Name Service, provides naming,
                               discovery, and resource management services for
                               enterprise -scale IP storage networks; draft-ietf-ips-isns
                             • iFCP—Internet Fiber Channel Protocol, provides Fiber
                               Channel Services over IP; draft-ietf-ips-ifcp
                             • mFCP—Uses UDP for transport of FCP in Metro networks
                             • iSCSI—Internet Small Computer Systems Interface;
                               draft-ietf-ips-iscsi
                             • SLP - Service Location Protocol; RFC 2608
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.     51




                           IP Storage Transports
                           DWDM

                              • Growing availability of Dark Fiber
                              • Transparent (protocol independence)
                              • Scalability
                              • Fast, dynamic broadband provisioning
                              • Simplifying high speed networking
                              • Very precise but fundamentally simplistic
                                technology

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.     52




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           IP Storage Transports
                           DWDM
                           Legacy Services

                                T3/E3                         ATM/
                                                             SONET                         OC-12c
                               OC-3/STM1                                                   OC-48c

                            Data Networking
                                                                                                                    Optical
                                              Fast and Gigabit                                                     Backbone
                                                 Ethernet

                                                                                              DWDM Creates Parallel Data
                                       Storage                                                Channels over a Common Fiber
                                                                                              by Multiplexing Disparate Data
                                                           ESCON
                                                                                              Streams onto a Defined Grid of
                                                           Fiber Ch                           Wavelengths
                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                       53




                           IP Storage Transports
                           FCIP

                            • FCIP provides a standard way of
                              encapsulating FC frames within TCP/IP,
                              allowing islands of FC SANs to be
                              interconnected over an IP-based network
                            • TCP/IP is used as the underlying transport
                              to provide congestion control and in-order
                              delivery of error-free data
                            • All classes of FC frames are treated the
                              same as datagrams

                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                       54




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           IP Storage Transports
                           FCIP

                             • End-station addressing, address resolution,
                               message routing, and other fundamental
                               elements of the network architecture remain
                               unchanged from the Fibre Channel model, with
                               IP introduced exclusively as a transport protocol
                               for an inter-network bridging function
                             • IP is unaware of the Fibre Channel Payload and
                               the Fiber channel fabric is unaware of IP
                                                                                                           //

                               Ethernet                   IP          TCP FCIP            FCP              SCSI Data …                       CRC
                                                    Checksum
                                                    Checksum
                                Header

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                                    55




                           FCIP Design
                           FCIP Approach - IP Tunneling
                                                                                                                                  FC
                                                                                                           FC Tape               Server
                                                                FC Server
                                                                                                           Library

                          FC Tape                                              FSPF Routing
                          Library                                                                                                         FSPF Routing
                                                                                 Backbone                                                   Backbone
                                                                      FC Switch                                 FC Switch
                       FC Switch                                                                                                               FC Switch
                                                                              FCIP                           FCIP
                                                Fiber                        Tunnel        IP Network       Tunnel           Fiber
                                               Channel                                    Tunnel Session                    Channel
                                                                                          Tunnel Session
                                                SAN                                                                          SAN
                                                                                                                                             FC Switch
                        FC Switch                                                                               FC Switch
                                                              FC Switch

                                                                             IP Services
                                                                       Available at Aggregated
                                    FC
                                   Server
                                                                            FC SAN Level                                               FC
                                                                                                                         FC           JBOD
                                                     FC                                                                 Server
                                                    JBOD



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                                    56




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           IP Storage Transports
                           iSNS


                             • iSNS provides the framework for the registration
                               and discovery of SCSI devices and Fibre
                               Channel-based devices using iFCP, and FCIP
                             • In IP-based storage like iSCSI end devices
                               registered with iSNS
                             • In iFCP, Fibre Channel-based storage end
                               devices register with iSNS by a iFCP gateway
                             • For FCIP, only the FCIP tunnel endpoints are
                               registered in the iSNS by the FCIP gateway


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.       57




                           IP Storage Transports
                           iSNS
                           There Are Three Main Components of the iSNS
                            • Name Service —providing Storage Resource Discovery for
                              both targets and initiators; they can register in the iSNS,
                              as well as query for information about other initiators
                              and targets
                            • Discovery Domain (DD) and Login Control Service —
                              facilitates the partitioning of iSNS client devices into more
                              manageable groupings for administrative and login
                              control purposes
                            • State Change Notification Service —allows the iSNS to
                              issue notifications about network events that affect the
                              operational state of iSNS client entities

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.       58




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           IP Storage Transports
                           iSNS

                            • iSNS leverages the services of Lightweight
                              Directory Access Protocol, RFC 1777; by
                              layering iSNS on top of LDAP, the
                              capabilities of both iSNS and LDAP can be
                              leveraged to manage and scale the
                              enterprise IP storage network
                            • iSNS is a simple and lightweight protocol
                              suitable for implementation on embedded
                              devices such as switches and targets

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.   59




                           IP Storage Transports
                           iFCP


                            • iFCP is a gateway-to-gateway protocol for
                              the implementation of a fiber channel
                              fabric over a TCP/IP transport
                            • Using iFCP all traffic between fiber
                              channel devices is routed and switched by
                              TCP/IP network components instead of
                              fiber channel components


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.   60




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           IP Storage Transports
                           iFCP


                             • Uses the 24 bit FC port numbering system
                               to direct frames to remote iFCP gateways
                             • The iFCP Gateways listen and captures
                               FC SNS queries and sends them to the
                               iSNS server
                             • It has methods to dynamically create IP
                               tunnels for FC frames


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                             61




                           IP Storage Transports
                           iFCP

                             • The iFCP layer maps Fibre Channel frames to a
                               predetermined TCP connection for transport
                             • FC messaging services and routing services are
                               terminated at the gateways so the fabrics are not
                               merged to one another
                             • It has methods to dynamically create IP tunnels for
                               FC frames.


                                                                                                //

                               Ethernet                   IP          TCP iFCP            FCP   SCSI Data …   CRC
                                                    Checksum
                                                    Checksum
                                Header

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                             62




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           iFCP Design
                           iFCP Approach—Map FCP to an IP Fabric
                                                                                                                               FC Tape
                                                                         FC Server                                              Library
                                       FC Tape
                                       Library
                                                                                                                                          FC Server
                                                                                           Device-to-Device
                                                                                               Session

                                                                         iFCP                                     iFCP
                                                                        Gateway                                  Gateway


                         iSNS
                                                                                                IP Network                                 iSNS
                        Server                                                                                                            Server

                                                                        iFCP                                      iFCP
                                                                       Gateway              Device-to-Device     Gateway
                                                                                                Session



                           FC Server                                                                                                         FC
                                                                                IP Services at Individual Device Level                      JBOD
                                                                                 IETF Standards for Routing, Naming,
                                                            FC                                                             FC Server
                                                           JBOD                 Security, QoS, CoS, Discovery (iSNS)
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                            63




                           IP Storage Transports
                           mFCP


                           • mFCP protocol is a variant of iFCP that
                             provides Fibre Channel fabric services to
                             FCP-based Fibre Channel devices using a
                             high-performance, reliable IP network
                           • mFCP uses the UDP transport protocol to
                             facilitate high performance, and assumes that
                             reliability and flow control will be handled by
                             the physical infrastructure


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                            64




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           IP Storage Transports
                           Internet SCSI

                              • iSCSI is a SCSI transport protocol for mapping of
                                block oriented storage data over TCP/IP networks.
                              • SCSI storage controllers generally identified for this
                                application include disk, tape arrays and tape
                                libraries.
                              • IP networks most applicable for this purpose are
                                Gigabit Ethernet and in the future 10 Gigabit
                                Ethernet.
                              • The iSCSI protocol enables universal access to
                                storage devices and Storage Area Networks (SANs)
                                over standard TCP/IP networks.
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                                     65




                           iSCSI and FC Frames
                                                                       IP
                                                                        IP                                                         iSCSI Control Frame
                                                           Ethernet Checksum TCP iSCSI
                                                           Ethernet                                         SCSI
                                                                                                            SCSI        CRC
                                                           Header Checksum TCP Header                                   CRC
                                                            Header                                        Command
                                                                                                          Command
                                                                                                                                      iSCSI Data Frame
                                                                       Ethernet   IP
                                                                                   IP
                                                                       Ethernet         TCP iSCSI                          SCSI Data …                CRC
                                                                        Header Checksum TCP Header                                                    CRC
                                                                        Header Checksum


                                                                                          IP iSCSI Packet
                                                                                                    Ethernet



                       Fibre Channel                                                         SCSI                   iSCSI Target              iSCSI on GigE
                                                                Fibre Channel
                                                                Fibre Channel                SCSI        CRC
                                                                                                         CRC                                        to
                                                                                           Command
                       Control Frame                               Header
                                                                    Header                 Command
                                                                                                                                                FCP on FC
                                                                                                                                           FC, SCSI
                                                                                                                                           FC, SCSI


                                       Fibre Channel
                                       Fibre Channel                                      SCSI Data …
                                                                                                                                      FC

                                                                                                               CRC
                                                                                                               CRC
                                           Header
                                           Header


                                                                                                Fibre Channel
                                                                                                Data Frame
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                                     66




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Advanced Storage Concepts
                           SLP—Service Location Protocol
                           Additions to RFC2608
                                 • Serverless discovery of targets
                                             Optional, generic Directory Agent to scale services

                                 • Zero-configuration of hosts
                                             SLP makes careful use of multicast
                                 • Access list and attribute propagation
                                 • Optional message authentication
                                 • Available open source implementations


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                           67




                           Advanced Storage Concepts
                           Service Location Protocol

                               • Service Agent (SA)                                                  Host
                                                                                                Management Code
                                                                                                Management Code
                                           Advertises services
                                           Services have attributes                             SLP    iSCSI
                                                                                                UA    Initiator
                               • User Agent (UA)
                                                                                                   TCP/IP
                                           Finds services
                                           Zero configuration                             SLP
                                                                                           DA          IP
                               • Directory Agent (DA)
                                                                                                   TCP/IP
                                           Optional
                                                                                                SLP    iSCSI
                                           Propagate service adverts                            SA     Target

                               • SLP Protocol                                                   Management Code
                                                                                                Management Code
                                           UDP or TCP                                               Device
                                           Minimize multicast
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                           68




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Advanced Storage Concepts
                           Name Service Comparisons

                                               Requirement                                    SLP          iSNS

                                       Multicast Discovery                                    Yes           No
                                       Discovery Domains                                   Voluntary      Secure
                               Access List by iSCSI Name                                      Yes           Yes

                                       TLV Message Format                                     Yes           Yes
                                             Authentication                                   Yes           Yes
                                   Registration and Query                                  Target Only   All Objects

                                 State Change Notification                                     No           Yes
                                       Lightweight Protocol                                   Yes           Yes
                                       Boot Requirements                                      Yes           Yes

                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                               69




                                  Networked Storage Essentials



                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                               70




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Network Storage Essentials
                           Storage Connectivity

                                            • Knowledge of host systems and
                                              server O/S
                                                             NT and Unix platforms
                                            • Host bus adapters for fibre channel
                                                           Fibre Channel HBA’s, compatibility issues
                                                                   Emulex, Qlogic, Adaptec, LSI Logic
                                                           Firmware and drivers
                                            • Fibre Channel
                                                           Understand architecture and connectivity

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                 71




                           Network Storage Essentials
                           Hardware Areas


                                   • High performance servers
                                   • Raid storage systems
                                   • Fiber channel switches, hubs, bridges
                                   • Backup devices
                                   • IP routers and switches
                                   Interoperability concerns with all


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                 72




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Network Storage Essentials
                           Software Areas
                           Storage Management—Two Levels
                            • Network Administration—Includes bandwidth utilization,
                              provisioning redundant links in a meshed topology to
                              guarantee alternate data paths, support for multiple
                              protocols, and error-free delivery
                            • Storage Administration—Includes organization and
                              placement of data once it arrives at its destination; LUN
                              mapping, RAID levels, file integrity, tape backup, and disk
                              utilization
                            • Examples of software are:
                            HP OpenView™, Tivoli™, VERITAS™, Legato™



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.     73




                           Network Storage Essentials
                           Server Clustering



                             • Grouping together of servers for the
                               purpose of enhancing their performance
                               and/or providing failover protection in the
                               event that a member server malfunctions
                             • Uninterrupted and seamless availability of
                               data and applications within a SAN



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.     74




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Network Storage Essentials
                           Backup

                                                   • Backup Applications
                                                                  LAN-free backup operations
                                                                  Serverless backup operations


                                                                    LAN
                                                                    LAN                   LAN
                                                                                          LAN




                                                                    SAN                   SAN



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                       75




                           Network Storage Essentials
                           Data Replication
                           Create and manage multiple copies of business-
                           critical information across a global enterprise
                             • Storage replication
                                        Bulk data transfer of the files or blocks under an application from
                                        one server to one or more other servers
                                        Independent of the applications it is replicating, meaning that
                                        multiple applications can be running on a server while they are
                                        being replicated to a secondary server
                             • Application level replication
                                        Specific to an application such as a database or web server, and is
                                        typically done at transaction level by the application itself
                                        If multiple applications are being used on the same server, an
                                        application-specific replication solution must be used for each
                                        individually

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                       76




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Network Storage Essentials
                           Replication Types
                           Remote storage replication can be implemented at
                           either the data storage array or host level

                            • Array-based (or hardware)—Data is copied from one disk
                              array to another of the same make and model; a dedicated
                              channel such as ESCON (Enterprise Systems Connection)
                              is commonly required to link the two arrays
                            • Host level—Replication is implemented in software at the
                              CPU level, and is independent of the disk array used; this
                              replication is done using standard protocols such as
                              TCP/IP across an existing network infrastructure such
                              as ATM



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.    77




                           Network Storage Essentials
                           Replication Modes

                              Synchronous Replication
                            • Data must be written to the target before the
                              write operation completes on the host system
                            • Assures the highest possible level of data
                              delivery to the Target—at any point in time, it will
                              have the exact same data as the source
                            • Replication can introduce performance delays
                              on the source system, particularly if the network
                              connection between the systems is slow

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.    78




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Network Storage Essentials
                           Replication Modes (Cont.)

                             Asynchronous Replication
                             • Source system does not wait for a confirmation
                               from the target systems before proceeding
                             • Products may queue (or cache) data and send
                               batches of changes between the systems during
                               periods of network availability

                         Performance Delays Solution:
                         Combine Synchronous and Asynchronous Operations, Switching to
                         Asynchronous Replication Dynamically When There Are Problems, Then
                         Reverting to Synchronous When the Communication Problems Are Resolved


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.      79




                           Network Storage Essentials
                           Uses of Replication



                                  • Disaster recovery/off-host processing
                                  • Off-host processing
                                  • Content distribution
                                  • Data consolidation
                                  • Follow-the-sun processing


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.      80




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Network Storage Essentials
                           Volume and File Management

                             • Volume and file system management
                               software can provide numerous
                               capabilities within a SAN environment
                                       Creating software RAID volumes on JBODs
                                       Changing RAID levels “on-the-fly”
                                       Spanning disk drives or RAID systems to form
                                       larger contiguous logical volumes
                                       File system journalizing for higher efficiency
                                       and performance, and open file management

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                81




                           Network Storage Essentials
                           Test Equipment Needs

                             • Fibre Channel Analyzers
                                        Traces not compatible between different vendors; software
                                        only base decode programs available from the vendors
                                        Need to break into FC to get tool attached, no port spanning

                             • GigE Sniffer Devices
                                        SCSI decodes not available for IP yet
                                        Not many users have GigE analyzers
                             • Fibre Channel Switches and Hubs
                                        GUI and cli on equipment is good place to get status and
                                        information about port health and address status



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                82




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                                       Networked Storage Designs



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                         83




                           Networked Storage Designs
                            Campus 1                                                      •    Network Attached Storage     Backup Site
                                                                                          •    IP Access to Storage
                                 Servers
                                                               NAS                        •    Storage over WAN
                                                                                                                                        FC
                                                                                          •    Storage over Metro Optical    FC
                                                                                                                                       SAN

                           iSCSI
                          Servers                                                                                                         NAS

                                                                                                Service Provider
                                                          IP                                  IP + Optical Network
                                                       Network
                           IP
                          SAN                                                                                                         IP
                                                                                                                                   Network
                                                    Storage
                           NAS
                                                    Router


                                                                                                                                  FC
                                                                                                                                       FC SAN
                              FC SAN          FC
                                                                                          Optical         FC

                                                                                           MAN

                                                                                                                      SSP    Campus 2
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                         84




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Networked Storage Designs
                           Storage Positions

                                                               Applications                     Directories       Object Store           Media

                          Local File System                                                                                     SAN
                          Network File Server
                          Storage Appliance                                        LAN                                          SAN
                          Network File Server
                          RAID storage
                                                                                   LAN                            SAN             SAN
                          Network File Server                                      LAN                            LAN             SAN
                          DMA bypass
                                                                                                Bulk Data Trans
                          NASD
                          Object Based Storage                                            LAN               LAN
                                                                                                  Read, Write

                                                iSCSI                                                                             LAN



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                          85




                           Data Storage Scaling

                               What Is Being Scaled                                                  What Mechanisms Are Stressed
                                                                                                     Resource Discovery
                                                                                                     Resource Discovery
                               Number of Clients and Servers
                               Number of Clients and Servers                                         Network Bandwidth and Congestion Control
                                                                                                     Network Bandwidth and Congestion Control
                                                                                                     Addressing
                                                                                                     Addressing


                                                                                                     Network Congestion and Flow Control
                                                                                                     Network Congestion and Flow Control
                                                                                                     Network Delay
                                                                                                     Network Delay
                               Distance
                               Distance                                                              Resource Discovery
                                                                                                     Resource Discovery
                                                                                                     Security
                                                                                                     Security
                                                                                                     Network Routing
                                                                                                     Network Routing


                                                                                                     Application Addressing
                                                                                                     Application Addressing
                               Number and Size of Files
                               Number and Size of Files
                                                                                                     File Metadata Management
                                                                                                     File Metadata Management


                                                                                                     File Metadata Management
                                                                                                     File Metadata Management
                               Directory Size and Tree Depth
                               Directory Size and Tree Depth
                                                                                                     Round Trip Time for Repeated Name Lookup
                                                                                                     Round Trip Time for Repeated Name Lookup

                                                                                                     Interconnect Technology
                                                                                                     Interconnect Technology
                               Aggregate and Individual Bandwidth
                               Aggregate and Individual Bandwidth
                                                                                                     Protocol Processing in Client and Server
                                                                                                     Protocol Processing in Client and Server

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                          86




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Networked Storage Designs
                           SAN-NAS Data Flows
                     Abstract File Object                                  File System Data Structures                                               Storage Volume (Blocks)
                                                                                     (Files and Directories)
                                                                                                                            myfile
                                                                                      networkers                                                           networkers
                                                                                                                                                                             myfile
                       A 25KB slide is stored in
                       myfile; it was written on May
                       1st , 2001, and is located in
                       directory  /networkers



                                                                                                                                                            Welcome to
                                                                                                                                                            Net workers
                                                                                                                                                               2001
                                                                               1/05/2001 25000                                                                                  This is
                                                                                                                                                                                session
                                                                                                                                                                               CDC-2001

                            Welcome to                                                                                                                           SAN / NAS
                            Net workers      SAN / NAS
                               2001.
                                                                                                          This is session            SAN / NAS
                                                                                                             CDC-2001
                          This is session
                             CDC-2001                                                    Welcome to
                                                                                         Net workers
                                                                                            2001.


                                                                                                                                                   SAN
                                                                            NAS
                     CDC-201
                     2878_04_2001_c1        © 2001, Cisco Systems, Inc. All rights reserved.                                                                                              87




                           Networked Storage Designs
                           NAS Data
                                                                                                       File Servers and Storage                     File Servers and Storage


                         • Scalability
                         • Manageability
                         • Serviceability                                                                 Server
                                                                                                          Server
                                                                                                          Aggregation
                                                                                                          Aggregation
                         • Troubleshooting
                                                                                                                                                                    Core
                                                                                                                                                                    Core
                                  Gigabit Ethernet or
                                  Gigabit Ethernet or
                                  Gigabit EtherChannel
                                  Gigabit EtherChannel
                                  Ethernet or Fast Ethernet Port
                                  Fibre Channel
                                  Fibre Channel                                                                                                  Distribution
                                                                                                                                                 Distribution

                                                                                                                                                  Access
                                                                                                                                                  Access
                     CDC-201
                     2878_04_2001_c1        © 2001, Cisco Systems, Inc. All rights reserved.                                                                                              88




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Networked Storage Designs
                           Cisco NAS Architecture
                                             Access/Server Aggregation Layer
                                             •      Connectivity for end user
                                             •      Connectivity for servers and network attached storage

                                             Technologies
                                             •      Scalability can be achieved with VLAN technology
                                             •      Cisco EtherChannel for incremental bandwidth growth
                                             •      VLAN spanned across Server Aggregation layer for dual homed servers




                               Access
                               Access                                                                       Access
                                                                                                            Access
                                                                                   Server
                                                                                   Server
                                                                                   Aggregation
                                                                                   Aggregation
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                   89




                           Networked Storage Designs
                           Cisco NAS Architecture
                                                    Distribution Layer
                                                    • Aggregates layer-2 traffic for multi access layers
                                                    • Controls access to resources for Security
                                                    • Controls network traffic for performance

                                                    Technologies
                                                    • Scalability can be achieved with VLAN technology
                                                    • Spanning Tree for redundant layer-2 paths




                                                                                            Distribution
                                                                                            Distribution




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                   90




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Networked Storage Designs
                           Cisco NAS Architecture
                                                      Core Layer
                                                      • High speed Layer-3 backbone
                                                      • Low latency and high packet throughput
                                                      • Internet access point
                                                      • Aggregation point for all distribution layers
                                                      Technologies
                                                      • Routing protocols OSPF, BGP, EIGRP
                                                      • HSRP


                                                                                            Core
                                                                                            Core




                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                            91




                           Networked Storage Designs
                           SAN

                           Storage-to-storage over WAN Today
                                       • ESCON or Fiber Channel over IP or ATM
                                       • 3rd party WAN extension devices e.g. CNT
                                                                                                      Application/
                                                                                                       Database              Tape
                                  SCSI, FC,
                                   ESCON
                                                                                                                          Storage
                                                                       GE
                          Storage
                                              FC
                                                                                           IP, ATM                   FC




                                                                            C6500                            FC Switch
                                            FC Switch
                                                                Extender
                                                                Extender                                Extender
                                                                                                        Extender
                                       FC/ESCON                                                                    FC/ESCON
                                                                                                     C6500

                                                                                                       Backup, DW, R&D
                                   Production                                                          Customer Service
                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                            92




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Networked Storage Designs
                           NAS and SAN over IP Metro Network
                                                               • Ideal for NAS outsourcing
                                                               • 10/100 Ethernet to the premises
                                                               • DPT (Dynamic Packet Transport)
                                                               • OC-48/OC192 on DPT ring



                                                                  GE
                           Outsourced
                              NAS                                                                                               10/100 Ethernet
                                                                                      IP/DPT
                                                                                   Metro Network


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                        93




                           Networked Storage Designs
                           NAS and SAN over Service Provider DWDM

                                               • DWDM service
                                               • Provide Wavelength per customer
                                               • Fiber Channel or GE
                                                                                                                   iSCSI
                                                                                                                                  Customer 3

                           Outsourced                                                        DWDM
                            NAS,SAN                                                       Metro Network                                 NAS
                                             GE




                                                                                                          FC SAN                  Customer 2


                                                                                                                   Customer 1

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                        94




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Networked Storage Designs
                           DWDM Applications with EMC

                           FC SAN Extension
                                               • Data Center Consolidation
                                               • Fiber Channel E-Port Extension
                                                                                                                                    NAS

                               Tape
                                                                           GE
                          Symmetrix
                             EMC                                                                            GE
                           Storage                                      FC                                                             Application/
                                                   FC
                                                                                                                    FC        FC




                                                                                                                           FC Switch    Database
                                                                                                                                         Servers
                                                                                                  DWDM
                                                                                           ONS            ONS
                                                                                          15540          15540


                                       Main Site                                                         Departmental Site
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                           95




                           Networked Storage Designs
                           DWDM Applications with EMC

                                       • Sync mirroring—disaster recovery
                                       • Async mirroring—backup, archiving, app testing
                                       • SRDF over ESCON—direct connect
                                       • SRDF over FC—via Connectrix (Brocade) switch


                          Symmetrix                                                                                                Symmetrix
                             EMC                  Connectrix                                                       Connectrix         EMC
                           Storage                FC Switch                                                        FC Switch        Storage
                                                   FC
                                                                 SRDF/FC                                         SRDF/FC      FC




                                                                                                  DWDM
                                               SRDF/ESCON                                  ONS            ONS      SRDF/ESCON
                                                                                          15540          15540


                                   Production                                                            Standby Data Center
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                           96




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Networked Storage Designs
                           Remote Mirroring—Sync/Async

                                                                                                     London
                                                                                                     London
                               • Multi-hop                                    Storage

                               • Hybrid                                                           FC FC-IP

                                 sync/async                                                  FC




                                                                                                                   IP




                                                                                                                                 As er I
                                 replication                                      Remote DR




                                                                                                                                  Ov
                                                                                                                                   yn P
                                                                                                                                     c
                                                                             New York
                                                                             New York                         New Jersey
                                                                                                              New Jersey




                                                                                                                                       IP-FC
                          Symmetrix
                             EMC                  Connectrix
                           Storage                FC Switch
                                                   FC
                                                                 SRDF/FC                                               SRDF/FC    FC




                                                                                                       DWDM
                                               SRDF/ESCON                                  ONS                  ONS        SRDF/ESCON
                                                                                          15540                15540


                                   Production                                                                 Standby Data Center
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                        97




                           Why IP-Based Storage Networking?

                               • Ubiquitous nature of IP:
                                          Access storage from LAN, MAN, WAN, Internet
                                          Economy of scale
                               • Scalability:
                                          Distance, performance (GE/10GE), expertise
                               • Single technology:
                                          Enterprise and Storage Network servers already have IP/Ethernet

                               • Manageable, secure and interoperable:
                                          (IPSEC, VLANs, ACLs, Firewalls)

                               • R&D investment on Ethernet/IP far outstretches
                                 other technologies


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                        98




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Storage over IP Challenges!
                            • Latency, Host
                              throughput
                            • Standards adoption                                                           DB,
                              (iSCSI etc… )
                                                                                            IP            Apps


                            • Conservative nature                                         Storage
                                                                                                     10GE switch
                              of storage
                            • FC installed base
                            • Security                                                        SAN/10GE
                                                                                               Production

                            • Network management
                              integration
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                            99




                                                                      iSCSI
                                                                 Concepts In-Depth



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                            100




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           iSCSI Stack

                                                                              SCSI Applications (File Systems, Databases)

                      SCSI Device-Type
                                                           SCSI Block Commands
                                                           SCSI Block Commands                   SCSI Stream Commands
                                                                                                 SCSI Stream Commands   Other SCSI Commands
                                                                                                                        Other SCSI Commands
                      Commands

                      SCSI Generic
                      Commands                                                            SCSI Commands, Data, and Status

                      SCSI Transport                                                                                           iSCSI
                                                                                                                               iSCSI
                                                             Parallel SCSI Transport
                                                             Parallel SCSI Transport               FCP SCSI Over FC
                                                                                                   FCP SCSI Over FC
                      Protocols                                                                                           SCSI Over TCP/IP
                                                                                                                          SCSI Over TCP/IP


                                                                                                                               TCP
                                                                                                                               TCP
                      Layer 3
                      Network Transport
                                                                                                                                IP
                                                                                                                                IP


                      Layer 2
                                                         Parallel SCSI Interfaces
                                                         Parallel SCSI Interfaces                    Fibre Channel
                                                                                                     Fibre Channel           Ethernet
                                                                                                                             Ethernet
                      Network
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                       101




                           iSCSI
                           Internet SCSI PDU

                            • The iSCSI layer encapsulates the SCSI
                              CDB into a iSCSI Protocol Data Unit (PDU)
                              and forwards it to the Transmission
                              Control Protocol (TCP) layer
                            • It also extracts the CDB from an iSCSI
                              PDU received from the TCP layer, and
                              forwards the CDB to the SCSI layer
                            • iSCSI mapping provides the SCSI-3
                              command layer with a reliable transport

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                       102




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           iSCSI
                           iSCSI Command Header PDU


                                 Byte
                                 Byte                             0
                                                                  0                              1
                                                                                                 1                        2
                                                                                                                          2            3
                                                                                                                                       3

                                   0
                                   0                Opcode
                                                    Opcode                                       Opcode—Specific Fields
                                                                                                 Opcode—Specific Fields            Reserved
                                                                                                                                   Reserved
                                   4
                                   4                                                       Length of Data (after 48 byte Header)
                                                                                           Length of Data (after 48 byte Header)
                                   8
                                   8                                                            Logical Unit Number (LUN)
                                                                                                Logical Unit Number (LUN)
                                  12
                                  12
                                  16
                                  16                                                                 Initiator Task Tag
                                                                                                     Initiator Task Tag

                                  20
                                  20                                                          Expected Data Transfer Length
                                                                                              Expected Data Transfer Length
                                  24
                                  24                      CmdRN
                                                          CmdRN
                                  28
                                  28                   ExpStatRN
                                                       ExpStatRN
                                  32
                                  32                                                      SCSI Command Descriptor Block (CDB)
                                                                                          SCSI Command Descriptor Block (CDB)



                                  44
                                  44
                                  48
                                  48                                                      Additional Data (Command Dependent)
                                                                                          Additional Data (Command Dependent)


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                       103




                           iSCSI
                           Initiator Op Codes - Current Cisco Release

                           0X00 NOP-Out I --> T
                           0X01 SCSI Command (This Encapsulates a SCSI CDB)
                           0X02 Login Command
                           0X03 Text Command
                           0X05 SCSI Data (Write Operation)
                           0X06 Logout Command
                           0X09 Ping
                           0X10 SACK Request (retransmission request)
                           0X0a Map
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                       104




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           iSCSI
                           iSCSI Response Header PDU
                                 Byte
                                 Byte                             0
                                                                  0                              1
                                                                                                 1                        2
                                                                                                                          2              3
                                                                                                                                         3

                                   0
                                   0                Opcode
                                                    Opcode                                       Opcode—Specific Fields
                                                                                                 Opcode—Specific Fields            Reserved (0)
                                                                                                                                   Reserved (0)
                                   4
                                   4                                                       Length of Data (after 48 byte Header)
                                                                                           Length of Data (after 48 byte Header)
                                   8
                                   8                                                                   Reserved(0)
                                                                                                       Reserved(0)
                                  12
                                  12
                                  16
                                  16                                                                 Initiator Task Tag
                                                                                                     Initiator Task Tag

                                  20
                                  20                                                                 Residual Count
                                                                                                     Residual Count
                                  24
                                  24                       StatRN
                                                           StatRN
                                  28
                                  28                  ExpCmdRN
                                                      ExpCmdRN
                                  32
                                  32                  MaxCmdRN
                                                      MaxCmdRN

                                  36
                                  36            Command Status
                                                Command Status                              ISCSI Status
                                                                                            ISCSI Status           Reserved (0)
                                                                                                                   Reserved (0)

                                  40
                                  40                      Res_len
                                                          Res_len                                                    Sense_len
                                                                                                                     Sense_len
                                  44
                                  44                  Reserved (0)
                                                      Reserved (0)

                                  48
                                  48                                                      Response and/or Sense Data (Optional)
                                                                                          Response and/or Sense Data (Optional)

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                           105




                           iSCSI
                           Target Op Codes – Current Cisco Release
                                       0X40 NOP-In (from target to Initiator)
                                       0X41 Scsi Response (Status and Sense info)
                                       0X42 SCSI Task management Response
                                       0X43 Login Response
                                       0X44 Text Response
                                       0X45 SCSI Data (Read)
                                       0X46 Ready to Transfer(Target is ready to Rx Data)
                                       0X47 Async event
                                       0X48 OP code not understood
                                       0X49 Ping Response
                                       0X4a Map Response
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                           106




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Cisco iSCSI Architecture

                                                                                                                                          Host
                                                                    SN 5420
                                                                                                                          Applications         Database

                                                 SCSI/TCP Server
                                                 SCSI/TCP Server                                                                            File System
                                                                                                                                 SCSI
                                       SCSI Driver
                                       SCSI Driver                       TCP/IP
                                                                         TCP/IP                                                  Tape        SCSI Disk
                                  Fibre Channel
                                  Fibre Channel                  Gigabit Ethernet
                                                                 Gigabit Ethernet                                                    SCSI Generic
                                                                                                        TCP/IP
                                                                                                       Network                   iSCSI                SCSI
                                                                Fibre Channel                                                    Driver              Adapter
                                                                Hub or Switch                                                                        Drivers
                                                                                                                           Host TCP/IP

                                                                                                                          Network Drivers


                                                                                                                           NIC                        HBA

                                                                                                                                   Local SCSI disk




                                 Disk Array
                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                                                       107




                           iSCSI
                           iSCSI Packet
                                                                                                                  46 – 1500 bytes
                                                             Destination Source
                                   Preamble                                      Type                        IP      TCP         Data       FCS
                                                              Address    Address
                                           8                            6                     6         2                                     4 Octet
                            Well-known Ports:
                             21 FTP
                             23 Telnet
                             25 SMTP                                                                                          iSCSI
                                                                                                                              iSCSI
                             80 http                                                                                       encapsulated
                                                                                                                           encapsulated

                            5003 iSCSI                                                                      Opcode
                                                                                                            Opcode        Opcode Specific Fields
                                                                                                                          Opcode Specific Fields
                            5003 iSCSI
                                                                                                                   Length of Data (after 40Byte header)
                                                                                                                   Length of Data (after 40Byte header)
                                       Sourced Port                                 Destination Port
                                                                                    Destination Port
                                                                                                                     LUN or Opcode-specific fields
                                                                                                                     LUN or Opcode-specific fields
                                                          Sequence Number
                                                                                                                          Initiator Task Tag
                                                                                                                          Initiator Task Tag
                                                    Acknowledgment Number

                            Offset Reserved U A P R S F                                    Window                       Opcode Specific Fields
                                                                                                                        Opcode Specific Fields
                                        Checksum                                      Urgent Pointer

                                                        Options and padding
                                                                                                                            Data Field …
                                                                                                                            Data Field …

                                                            TCP Header
                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                                                       108




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                                iSCSI Encapsulation


                                Client/User                                                                                                       Data Server
                                                                            Ethernet
                                                                            Ethernet        IP
                                                                                             IP     TCP          DATA
                                                                                                                 DATA     CRC
                                                                                                                          CRC
                                                                             Header
                                                                             Header




                                                              Ethernet IP TCP iSCSI
                                                              Ethernet IP     iSCSI                     SCSI
                                                                                                        SCSI       DATA
                                                                                                                   DATA         CRC
                                                                                                                                CRC
                                                               Header
                                                               Header



                                                                                                                                      FC   Gbe
                                                                                                                                                 SN 5420




                                                            FC
                                                                           SCSI
                                                                           SCSI                   DATA
                                                                                                  DATA             CRC
                                                           Header
                                                                                                                                  Fibre Channel




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                                         109




                           iSCSI Roles
                                                                                                                                                    End Users


                                                      Data Servers

                                                                                                                                                 External
                                                                                                                                                 Network
                                         SCSI Initiator                                                      IP
                                                                                                                           File-level Access
                                                                                                          Network
                                         iSCSI Initiator
                                                          Block-level Access

                                                             iSCSI Target
                                                                                      F   Gbe SN   5420 MGM HA
                                                                                                         T
                                                                                      C




                                                                                                           SCSI Target
                                                                                 Fibre Channel

                                 Storage Devices


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                                         110




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           iSCSI Model


                                 Data Server Host                                                 SN 5420                     FC Storage Device
                                     Initiator                                                  iSCSI Target                      FC Target


                                   Application
                                   Application                                                                                         Logical
                                                                                                                                       Logical
                                      Client
                                      Client                                                                                            Unit 1
                                                                                                                                        Unit 1
                                                                                                     Target
                                                                                                     Target




                                                                                                                           Port
                                                                         Port

                                    requests
                                                                         Port

                                    requests


                                                                                          ge2
                                                                                          ge2
                                                                                                    Mapping
                                                                                            Device Service Request
                                                                                                    Mapping




                                                                                                                     fc1
                                                                                                                     fc1
                                    data from
                                    data from                                                                                           Device
                                                                                                                                        Device
                                     LUN 1
                                      LUN 1                                               Device Service Response                       Server
                                                                                                                                        Server
                                                                                                 LUN 1 = LUN 2
                                                                                                 LUN 1 = LUN 2


                                                                                                                                       Logical
                                                                                                                                       Logical
                                                                                                                                        Unit 2
                                                                                                                                        Unit 2

                                                                                                                                        Device
                                                                                                                                        Device
                                                                                                                                        Server
                                                                                                                                        Server




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                           111




                     iSCSI Storage Device
                     Discovery Process – Current Cisco Release


                              • Host driver requests available iSCSI targets from the SCSI
                                router
                              • SCSI router sends available iSCSI target names to host (Based
                                on Access Lists)
                              • Host logs in to iSCSI targets that were received
                              • SCSI router accepts the login and sends target identifiers to
                                Host (numbers)
                              • Host queries targets for device information
                              • Targets respond with device information
                              • Host creates table of internal devices (/dev/…)


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                           112




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           iSCSI
                           Internet SCSI
                             Initiator                                                    Single TCP Session                     Target

                                            TCP                                 Establish normal TCP Session                   TCP port
                                                                                                                               5003
                                                                                          0X03 Command—Login
                                                                                              com.cisco .version: #
                                                                                          com.cisco .senTargets;please              This
                           iSCSI Driver                                                                                            Device
                                                              0X43 Login Response—Reject Login Status 1                              Has
                                                                            In Text Area, List of Accessible Target Names         Already
                                                                                       Keeps TCP Session up.                     Initialized
                                                                                                                                  Onto the
                                                                                           0X03 Command—Login                      Fibre
                                                                                             List of Target Names Sent            Channel

                                                                                           0X43 Login Response
                                                                                      Response With Target Drive Mapping

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                        113




                           TRACES
                               iSCSC Logon command                                                  iSCSI Response with reject and targets




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                        114




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           TRACES

                          Second Login With Target Names Attached




                                            Three targets can be seen target:disk1
                                                                                          target:disk2
                                                                                          target:disk3
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                       115




                           TRACES
                                                                                                   Disk1/0
                        Second and Final Response Back                                             Disk2/1
                        to Initiator With Drive Mappings
                                                                                                   Disk 3/2




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                       116




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                                 Designing Storage access with
                                    iSCSI and Cisco SN5420
                                        Storage Router


                     CDC-201
                      Presentation_ID
                     2878_04_2001_c1    © 1999, Cisco Systems, Inc. All rights reserved.
                                         © 2001, Cisco Systems, Inc.                                            117




                           Why IP-SANs?

                            • Preserve existing
                              network infrastructure
                            • Gigabit Ethernet ports
                              are 1/3 the cost of
                              Fiber Channel Ports                                            Cost-Effective
                                                                                           Ethernet Switching
                            • Decrease TCO




                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                        118




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Storage Service Provider Application
                           Current


                                                                  IP                                                      IP
                                                               Network                                                 Network


                                                                                                                FC HBA



                                                                                                                    Fibre Channel
                                               Customer A                                 Metro Area                             Customer B
                                                                                           Network
                                                                                                                                      SSP

                                 Customer A’s                                                                                     Customer
                                 Storage                                                                                         B’s Storage




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                        119




                           Storage Service Provider
                           Application with SN5420
                                                                        Ethernet                                Ethernet

                                                                  IP                                                      IP
                                                               Network                                                 Network




                                                                                          Metro Area
                                                                                           Network
                                               Customer A                                                                        Customer B
                                                                                                                                    SSP


                                                                                           FC   Gbe
                                                                                                      SN 5420



                                                                                                                       Customer A LUN
                                                                                                                       Customer B LUN

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                        120




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Enterprise Network
                           Current



                             • Disadvantages:
                                                                                                     TCP/IP
                                       Requires Fibre Channel                                       Network
                                       in servers
                                       Servers are both
                                       application and I/O
                                       processors


                                                                                          FC SAN                        FC SAN



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                          121




                           Enterprise Network with SN 5420


                         • Advantages:
                                   Fibre Channel used for
                                   storage only                                                     TCP/IP
                                   IP accessible storage from                                      Network
                                   anywhere
                                   Servers are dedicated to
                                   application processing
                                                                                                   FC   Gbe
                                                                                                              SN 5420
                                   Storage is pooled by
                                   linking SAN islands
                                                                                                   Fibre Channel



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                          122




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Backup over IP WAN Using iSCSI

                            • Backup or archiving to disk or tape
                            • FC or IP attached tape and backup storage

                                                                                                           iSCSI
                                           SN5420                                                          Driver
                                                                    GE                                     Backup
                                                                                                           Manager
                                                             Blo                                       IP
                                                                ck
                                                                   A
                          Storage                              Ov cces
                                                                                          IP WAN
                                                                                                                     FC


                                                                 er I  s                           C6500
                                                                     P
                                           FC




                                       FC Switch
                                                                                                                Backup
                                                                                                                Storage

                                   Production                                                               Backup
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                   123




                           SN5420 Configuration Techniques



                              • Simple to configure
                                        IP address for GigE interface, used as destination
                                        address by host Initiator
                                        Second IP address for management interface (Must be
                                        separate subnet address from GigE)
                                        Third IP Subnet address for HA Network (Must also be
                                        different subnet from other interfaces)
                                        SN5420 does no IP routing. Default or explicate routes
                                        may be configured in route table for off net routes


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                   124




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           SN5420 Configuration

                            •    Scsirouter object is built in SN5420 (4 total on release 1.0) along with IP address
                                 of GigEinterface and Fibre Channel interface
                            •    Target names are built using the acquired FC devices found during initialization
                                 on to the FC network
                            •    Target names are mapped to Scsirouter object and access lists are attached


                                               [SN5420]# sh scsirouter
                                               mfrase@cisco.com description “ Networkers_01"
                                               mfrase@cisco.com primary "none"
                                               mfrase@cisco.com serverif ge2 110.30.30.2/24
                                               mfrase@cisco.com deviceif fc1
                                               mfrase@cisco.com target SN5420 description “mystorage "
                                               mfrase@cisco.com target SN5420 enabled "TRUE"
                                               mfrase@cisco.com target SN5420 accesslist "all"
                                               mfrase@cisco.com target SN5420 loopid "0"
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                  125




                           Device mapping

                                           Show Device from JBOD on a FC Hub


                                 Show device
                                   wwnn                                          wwpn           loopid   lun    vendor
                                 2000002037a7a1bd                              2100002037a7a1bd 1          0   SEAGATE
                                 2000002037a7a19f                              2100002037a7a19f    2      0    SEAGATE
                                 2000002037a7a0c8                              2100002037a7a0c8    3       0   SEAGATE
                                 2000002037a7a19b                              2100002037a7a19b    4       0   SEAGATE




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                  126




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           What’s required for High Availability


                              • SN5420 HA interfaces are networked together on 10/100
                                HA network; management network also used for
                                redundancy in HA clusters
                              • The Cluster Name is configured on member SN5420’s
                              • SN5420 configurations are merged and HA management
                                protocol keeps adds and changes in sync across
                                the cluster
                              • Ready for failover…

                                             SCSI Routers Are Either Load Balanced
                                              Across the Cluster or in Hot Standby
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.   127




                           HA Redundancy

                             • HA conversation flows over HA network and over
                               Management Network
                             • HA state Machine monitors a heartbeat sent between the
                               Clusters as seen in “show ha”
                             • As changes and additions are made in configurations on
                               master the redundant unit is updated through a hash
                               mechanism and update packets are set in the heartbeat
                               packet; data is sent in XML format
                             • If state machine is notified of an interface failure or
                               application failure the slave HA unit takes over
                             • If Heartbeat does not respond slave HA takes over


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.   128




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           HA Redundancy (Cont.)


                              • Driver sends busy’s to Host application while waiting for
                                recovery
                              • The failover is almost instant; TCP session
                                does drop
                              • GigE MAC and IP address are transferred to new Master
                                SN5420; this is different from HSRP
                              • New Master SN5420 does gratuitous ARP to Switch for
                                fast cache update for port location of moved IP address
                              • Host Driver finds lost GigE peer and mini
                                login occurs

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                         129




                           Cisco SN5420 Storage Router
                           High Availability


                                                                                                                iSCSI Servers “B”

                       iSCSI Servers “A”

                                                                                                                 Mgmt

                                         SN5420 “A”
                                          Cluster A                         X                      HA
                                                                                                              SN5420 “B”
                                                                                                               Cluster A



                                                                                                        FC

                                                                       FC




                                                                                          FC SAN             FC SAN


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                         130




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Cisco SN5420 Storage Router
                           High Availability

                                         What Happens During a HA Failover?
                                Initiator                                                                                Target
                                                                                                       LAN         ARP
                                                                                                      Switch

                              Application                                                 Recover TCP Session               This
                                                                                                                           Device
                                       Busy                                                                                  Has
                                        Busy
                                         Busy                                             0X03 Command—Login              Already
                                                                                            Saved list of Target         Initialized
                              iSCSI Driver                                                     Names sent                 Onto the
                                                                                          0X43 Login Response               Fibre
                                        Login                                                                             Channel
                                       Retries                                    Response with Target Drive Mapping




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                131




                           SN 5420 and
                           iSCSI Maximum Limitations

                              • Each SN 5420 provides a 100MBps pipe
                              • One SN 5420 Storage Router may have up to
                                4 scsirouter services
                              • Each scsirouter service may accommodate up to 32
                                iSCSI hosts
                              • Each iSCSI host may access up to 8 targets
                              • Each target may include multiple LUNs
                                         NT = 8 LUNs
                                         UNIX = 32 LUNs



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                132




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Maximum Limitations Summary

                            • A UNIX iSCSI host could discover 256
                              iSCSI devices (8-targets x 32-LUNs)
                            • An NT iSCSI host could discover 64 iSCSI                                                 IP Network
                              devices (8-targets x 8-LUNs)
                            • For UNIX environments, one scsirouter                                                         SN 5420

                              service would map up to 8192 devices
                              (32-hosts x 8-targets x 32-LUNs)
                            • For NT environments, 2048 devices                                                     Fiber Channel
                              (32-hosts x 8-targets x 8-LUNs)
                            • A single SN 5420 would support 4x those
                              numbers due to multiple active scsirouters


                              Remember: The SN 5420 provides a means to share 100MB of
                              bandwidth between iSCSI hosts and fibre channel devices

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                               133




                           SN5420
                           Serviceability


                                Helpful Debugs and Shows
                                          Show Interface
                                          Show Interface                                  Show Debug Scsirouter
                                                                                          Show Debug Scsirouter
                                         Show Net
                                         Show Net                                         Show Debug Rawlundatabase
                                                                                          Show Debug Rawlundatabase
                                         Show System
                                         Show System                                      Show Debug Scsirouter Tfestatus
                                                                                          Show Debug Scsirouter Tfestatus
                                         Show Scsirouter Connection
                                         Show Scsirouter Connection                       Show Log
                                                                                          Show Log



                                   • What does TAC need from Network for troubleshooting?
                                                 SN5420 configurations
                                                 Show tech-support
                                                 Logs from host system
                                                 Network topology

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                               134




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Lessons Learned

                                 • Loss of IP path issues and concerns with NT
                                 • Portfast settings on GigE switches
                                 • HA unit must have healthy interfaces to switch
                                 • HA interface must be active to do changes in cluster
                                 • VLAN and zoning
                                 • Save drive configuration in registry on NT
                                 • Some disk subsystem array controllers are LUN 0
                                 • Flow Control on
                                 • If master and slave units can not talk over HA both become
                                   masters


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.         135




                                Advances in Storage Concepts



                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.         136




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Advances in Storage Networking

                                   • Multipath
                                   • Storage Virtualization
                                   • Network Data Management
                                     Protocol (NDMP)
                                   • InfiniBand
                                   • Direct Access File System and Virtual
                                     Interface

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                                       137




                           Advanced Storage Concepts
                           How Multipath Drivers Work
                            File Systems, Databases, and User                                                      A Multipath Driver May Be Implemented
                                  Applications Are Unaware of                                                              At One of Several Layers
                                                Multiple Paths
                                                                                                 Host                 Multipath Drivers May Be Used For:
                              Some Volume Managers Can                                     User Applications          •   Path Failover
                                  Also Do Multipath If the                                                            •   Load Balancing
                                          Drivers Do Not                                  File System/Database
                                                                                                                      Multipath Drivers May Be Provided By:
                          Multipath Drivers, If Present, Can Detect                          Volume Manager
                                                                                                                      •   HBA Vendors
                                That Both Disks Are the Same, and
                            Present Them As One Logical Disk to                                                       •   Disk Vendors
                               the Volume Manager or File System                                Drivers
                                                                                                                      •   Volume Managers Vendors
                                                                                                                      •   Host OS Vendors
                        Each NIC/HBA Presents the Disk Without
                                                                                          NIC/HBA      NIC/HBA        •   Filesystems and Databases
                                  Knowing About the Other Path
                                                                                                                 Multipath drivers must have some means
                                     The Host Has a Separate Path                                                of ascertaining that two devices are really
                                 Or Network to Each Disk Interface                                               different paths to the same device
                                                                                                                      Some Common Ways Are:
                                                                                                                      •   Configured Fiber Channel WWPN
                               Many Disk Arrays Can Present the                                                       •   SCSI LUN WWN
                                                                                                Disk
                             Same Disk on Two or More Interfaces
                                                                                                                      •   SCSI Serial Number
                                                                                                                      •   SCSI Device ID
                                                                                                                      •   Proprietary Lable on Disk
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                               •   Other Proprietary Means             138




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Advanced Storage Concepts
                           HBA Vendor Multipath Drivers

                                                                                               FC/SCSI Host
                                                                                             User Applications


                                                                                             Filesystem, VM, etc.
                           • Fibre Channel and SCSI HBA
                             vendors often provide their                                  SCSI Port/Generic Driver
                             own multipath drivers
                                                                                      .
                                                                                          HBA Multipath driver
                           • A disadvantage of this
                             approach is that all HBAs
                             must be of the same type;                                    HBA Driver HBA Driver
                             these drivers cannot be                                         HBA           HBA
                             used for iSCSI                                                                                 Example HBA Vendors:
                                                                                               Fibre   Fibre
                                                                                              Channel Channel
                                                                                                                            • Qlogic
                                                                                                                            • JNI
                                                                                                    Disk
                                                                                                                            • Emulex

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                                        139




                           Advanced Storage Concepts
                           Disk Vendor Multipath Drivers

                                                                                                   Method 1                            Method 2
                           •   Disk vendors are very creative in
                                                                                                   FC/SCSI/iSCSI Host                 FC/SCSI/iSCSI Host
                               identifying multiple paths to their devices
                           •   Some use Fibre Channel -specific                                   User Applications                  User Applications
                               information Channel -specific
                                                                                                  File System/Database               File System/Database
                               information like WWPNs; these will
                               not work with iSCSI                                                                                     Volume Manager
                                                                                                    Volume Manager
                           •   Happily, most of these drivers use things                                                               SCSI Disk Driver
                               like the device's serial number or                               Replacement Disk Driver
                               proprietary device ID; some use the                                                                    Multipath "shim"
                               standard LUN world -wide names, but not
                               many                                                             SCSI Port/Generic Driver
                                                                                                                                    SCSI Port/Generic Driver
                                                                                                   iSCSI or HBA Driver                iSCSI or HBA Driver
                               Example Vendors:
                                                                                                 NIC/HBA          NIC/HBA           NIC/HBA         NIC/HBA
                               EMC Symmetrix (PowerPath)
                               Hitachi                                                               FC or    FC or                     FC or    FC or
                               EMC Clariion (ATF)                                                   Ethernet Ethernet                  Ethernet Ethernet
                               XioTech Magnitude
                               IBM Shark
                                                                                                           Disk                              Disk
                               DotHill
                               Mylex (IBM)

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                                        140




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Advanced Storage Concepts
                           Volume Manager Multipath Drivers

                                                                                                                                  FC/SCSI/iSCSI Host
                            • A volume manager can provide multipath I/O                                                            User Applications
                              in its lower layers, below its virtualization features
                                                                                                                                   File System/Database
                            • Volume managers normally use the first one or
                                                                                                                                 Volume Mgr Virtualization
                              more sectors of the disk to write a label specifying
                              the disk's use; if the label matches on two different                                              Volume Mgr Multipath I/O
                              “disks”, the VM will assume that they are really.
                              two paths to the same device                                                                           SCSI Disk Driver
                                                                                                                                 SCSI Port/Generic Driver
                                                                                                                                    iSCSI or HBA Driver
                                Example vendors:
                                  Veritas Dynamic Multipath                                                                       NIC/HBA         NIC/HBA

                                  Legeto                                                                                             FC or    FC or
                                                                                                                                    Ethernet Ethernet



                                                                                                                                           Disk




                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                                                     141




                           Advanced Storage Concepts
                           SN5420 Approach to Multipath
                              Multipath drivers within the SCSI
                              driver stack or volume manager                                       Host
                              present the two paths as a single
                                                                                            User Applications
                              device to the upper layers

                                                                                           File System/Database

                                                                                             Volume Manager
                              The host's iSCSI driver sees
                                                                                                                      Note that multipath drivers must be
                              two disks, one from each
                                                                                                   SCSI               independent of the SCSI transport;
                              5420, and presents the both
                                                                                                   iSCSI              they must not rely on Fiber Channel
                              as SCSI devices
                                                                                                                      or parallel SCSI information to
                                                                                                   TCP/IP
                                                                                  .                                   match the paths
                                                                                             NIC              NIC
                             Each 5420 sees a disk on its
                             interface, and presents it over                               Ethernet      Ethernet
                             iSCSI ; it need not know about
                                                                                           SN5420            SN5420
                             the other
                                                                                               Fibre   Fibre
                                                                                              Channel Channel
                                       The disk is presented on both
                                       controller interfaces, as usual

                                                                                                      Disk




                     CDC-201
                     2878_04_2001_c1    © 2001, Cisco Systems, Inc. All rights reserved.                                                                     142




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Advanced Storage Concepts
                           Storage Virtualization

                             • Storage Administration Management Tool
                                       Used to Discover SAN objects
                                       Maintains virtual-to-physical relationships
                                       Allows sorting of all devices into pools
                                               Service-level
                                               Location
                                               Virtual volumes
                                               Other criteria

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.   143




                           Advanced Storage Concepts
                           Storage Virtualization

                             • Virtualization can pool any of your SCSI or
                               fiber-attached storage, from the JBOD up to the
                               most sophisticated enterprise RAID subsystems
                             • You can integrate your existing assets, mix
                               heterogeneous technologies, and evolve your
                               storage and server assets independent of each
                               other, transparent to the application
                             • Virtualization enables you to introduce new
                               technologies with minimal regard for backward
                               compatibility

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.   144




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Advanced Storage Concepts
                           Storage Virtualization


                                   Servers

                            LAN, WAN, MAN


                              Storage Pool
                              with Virtual
                              Volumes

                              Storage

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.   145




                           Advanced Storage Concepts
                           NDMP

                             • The Network Data Management Protocol (NDMP)
                               is a mechanism and protocol for controlling
                               backup, recovery, and other transfers of data
                               between primary and secondary storage
                             • The NDMP architecture separates the network
                               attached Data Management Application (DMA),
                               Data Servers and Tape Servers participating in
                               archival, recovery or data migration operations;
                               NDMP also provides low level control of tape
                               devices and SCSI media changers


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.   146




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Advanced Storage Concepts
                           NDMP (Cont.)

                                       • The TCP/IP and XDR record marking protocols are
                                         foundations for NDMP
                                       • The key goals of NDMP include interoperability,
                                         contemporary functionality, and extensibility
                                       • Client/server architecture
                                                   NDMP Clients:
                                                           Data Management Applications (DMA)
                                                   NDMP Servers:
                                                           Primary storage systems (file system access)
                                                           Secondary storage systems (tape and media
                                                           changer devices)

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                           147




                           Advanced Storage Concepts
                           NDMP Local configuration
                                           Unix/NT Host
                                           Unix/NT Host


                                             NDMP DMA
                                             NDMP DMA



                                                                                  NDMP Control Connection




                                                                                                            NDMP Server

                                                                                                                    Backup Data
                                                                                                                    Backup Data
                                                                                                   Backup Data
                                                                                                   Backup Data


                                                                                                                      Backup
                                                                                                                      Device


                                                                                                      Primary/Secondary Storage
                                                                                                      Primary/Secondary Storage

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                           148




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Advanced Storage Concepts
                           NDMP 3 Way
                                           Unix/NT Host
                                           Unix/NT Host


                                             NDMP DMA
                                             NDMP DMA



                                                                                          NDMP Control Connection




                                                                                                  Backup Data Only
                                                                                                  Backup Data Only
                                                                                 NDMP Server                              NDMP server


                                                                                                 NDMP Data Connection
                                                                                                                          Backup Data
                                                                                                                          Backup Data
                                                                                 Backup Data
                                                                                 Backup Data


                                                                                                                             Backup
                                                                                                                             Device


                                                                             Primary Storage                            Secondary Storage
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                     149




                           Advanced Storage Concepts
                           InfiniBand
                                       • Emerging standard for system area network.
                                       • Converges I/O, clustering, IP
                                         with one unified architecture
                                       • Integrates with existing technology
                                         PCI, fiber channel, Ethernet etc...
                                       • Switched architecture—scalable, reliable
                                       • Partitions for security
                                       • Channel-based
                                                     Reliable connections, bulk data

                                       • End-point architecture

                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                     150




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Advanced Storage Concepts
                           InfiniBand Architecture
                                                                              Target Channel Adapter
                                                                             Interface to I/O Controller                   e.g. Gb Enet
                                                                                                                    I/O
                                                                                                                    I/O                    Mgmt.
                                                                                                                                           Mgmt.
                                                                                 SCSI, FC-AL, etc.                 Cntlr
                                                                                                                   Cntlr                  Services
                                                                                                                                          Services
                                                                                     Link
                                                                                                                   TCA
                                                                                                                   TCA
                                                                                     High Speed Serial




                                                                                                                    Link
                                                                                                                    Link
                                                                                     X1, x4, and x12
                                              Systems Bus




                         CPU
                                                                                                                  Multi
                                                                                                                  Multi              I/O
                                                                                                                                     I/O
                                                                     Mem
                                                                                              HCA
                                                                                              HCA     Link
                                                                                                      Link       Stage
                                                                                                                 Stage     Link TCA Cntlr
                                                                                                                           Link TCA Cntlr
                                                                     Cntrl
                                                                                                                 Switch
                                                                                                                 Switch




                                                                                                                    Link
                                                                                                                    Link
                         CPU
                                                                     Sys
                                                                     Mem                  Host Channel Adapter             Switch
                                                                                                                   TCA
                                                                                                                   TCA     Simple, Low Cost,
                                                                                          Protocol Engine
                                                                                                                           Multistage
                                                                                          Moves Data Via            I/O
                                                                                                                    I/O    Network
                                                                                          Messages Queued in       Cntlr
                                                                                                                   Cntlr
                                                                                          Memory                               e.g. SCSI
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                          151




                           Advanced Storage Concepts
                           VI—Virtual Interface

                             • VI provides:
                                       Memory-to-memory RDMA capability
                                       Transport independence

                             • Vl improves CPU utilization
                             • Vl reduce latency
                             • VI is optimized for high speed, low latency
                               networks (local to the data center, not
                               the WAN)
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                          152




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           Advanced Storage Concepts
                           DAFS

                             • Protocol designed for local file sharing or
                               clustered environment
                             • Provide low latency, high throughput data
                               movement
                             • Uses memory-to-memory networking
                               technology (VI as standard transport
                               mechanism)
                             • DAFS product expected for end of 2001
                               (client and server)
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                         153




                           DAFS
                           Increasing Efficiency of File Access
                                          Local FS                                                   NFS                 DAFS
                                        Application
                                        Application                                            Application
                                                                                               Application              Application
                                                                                                                        Application
                      User
                                  Buffers                                                  Buffers                   Buffers
                                                                                                                     Buffers
                                                                                                                     Buffers
                                                                                                                                DAFS
                                                      FS Switch
                                                      FS Switch                                        FS Switch
                                                                                                       FS Switch                VIPL
                                                                                                                                VIPL

                                  Buffer                  File
                                                           File                            Buffer           NFS
                                                                                                            NFS
                                  Cache                  System
                                                         System                            Cache
                     Kernel                                                                                         SINGLE
                                                                                          Multi Copy                Copy
                                                          SCSI
                                                          SCSI                             Multi Copy
                                                          Driver
                                                          Driver
                                                                                            Multi CopyTCP/IP
                                                                                                      TCP/IP
                                                                                           Packet
                                                                                           Buffers                                     VI NIC
                                                                                                                                       VI NIC
                                                                                                            NIC                        Driver
                                                                                                                                       Driver
                                                          HBA
                                                          HBA                                               NIC
                                                          Driver
                                                          Driver                                           Driver
                                                                                                           Driver


                      HW                                                                             NIC
                                                                                                     NIC                  NIC
                                                           HBA
                                                           HBA


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                                                         154




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                                       Deploying Storage Networks
                                                                                          Session CDC-201




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                     155




                                                      Please Complete Your
                                                         Evaluation Form
                                                                                          Session CSC-201




                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                     156




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                     157




                           References

                               • Technical Reading on FC and SCSI?
                                           http://www.ietf.org/html.charters/ips-charter.html
                                           http://www.t10.org/
                                           http://www.ndmp.org/
                                           http://www1.cern.ch/HSI/fcs/
                                           http://www.t11.org/index.html
                                           http://www.iol.unh.edu/consortiums/fc/index.html
                                           http://www.dafscollaborative.org/
                                           http://www.umich.edu/~dirsvcs/ldap/doc/
                                           http://www.tivoli.com/products/solutions/san/san_redbooks.html
                                           http://www.InfiniBandta.org/home.php3
                                           http://www.fiberalliance.com/


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                     158




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
                           References

                                • Cisco SN5420 Info?
                                           http://www.cisco.com/warp/public/cc/pd/rt/5420/

                                • SAN Tutorials?
                                           http://www.sanacademy.com/
                                           http://www.storagesolutions.com/
                                           http://www.dothill.com/tutorial/tutorialmovie.htm

                                • SAN News?
                                           http://searchstorage.techtarget.com/searchStorage_News_Page
                                           http://is.pennnet.com/home.cfm
                                           http://www.snia.org/English/News/News_FS.html


                     CDC-201
                     2878_04_2001_c1   © 2001, Cisco Systems, Inc. All rights reserved.                  159




Copyright © 2001, Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr