Docstoc

ppt - University of California_ Berkeley - PowerPoint

Document Sample
ppt - University of California_ Berkeley - PowerPoint Powered By Docstoc
					          Internet in a Box
          Building Distributed System Testbed in RAMP




    Zhangxi Tan                         Xiaofan Jiang
Computer Science Department          Electrical Engineering Department
University of California, Berkeley    University of California, Berkeley
Introduction
   Testing and debugging large scale distributed
    systems is difficult
   Problem with existing approaches
                – O(100) nodes
     Scalability
     Reproducibility - PlanetLab
     Observability - Don’t know what’s going on inside
     Cost, Space and Power

   Build distributed system testbed with over 1,000
    nodes using multi modular FPGA system
     Buildingblock, HW/SW and networking infrastructure
      this semester
                         CS268 Spring 2006                 2
Methodology
                          Intermediate platform –
                           XUP
                                 Fit 6/8 RISC processors per
                                  chip
                                 Running uClinux with
                                  TCP/IP network
                                 Software routers on chip
                                 Sample distributed
                                  applications (Chord)
                          Final platform - Research
                           Accelerator for
                           MultiProcessing (RAMP)
                                 Scale up to 16 RISC (?)
                                  processors per chip x 4 = 64
                                  cores
              CS268 Spring 2006                                  3
Previous Status
                       Single processor
                        running uClinux 2.4 on
                        XUP board
                          Running   @ 100 Mhz,
                           256M DDR
                          Zebra open source
                           router (not fully tested)
                          Virtual NIC and point-to-
                           point communication
                           between processors
                                   Support polling and
                                    interrupt
                                   Firmware has been
                                    developed

            CS268 Spring 2006                             4
Current Status Overview
   Multiple processors on one chip
       6 cores on XUP running uClinux
       Multiprocessor debugging and bootstrapping
       XUP Clusters!
   Network Support
       Ethernet Linux driver and TCP/IP network (Socket interface)
       Network Interface Card (NIC)
   TCP/IP interconnect network
       Transparent internal inter-core network and external inter-XUP
        network
   Demo application porting and performance evaluation
       TTCP
       Chord / i3 (after porting)
       Web server w/CGI support



                               CS268 Spring 2006                         5
                       System Architecture                                                                                                                                                                                                              30B 31B 32B 33B 34B 35B 36C 37C 38D 39E




                                                                                                                                                                                                                                                                        bram_block
                                                                                                                                                                                                                                                                        lmb_bram_1

                                                                                                                                                                                                                                                                          PORTA
                                                                                                                                                                                                                                                                      PORTB
                                                                                                                                                                                                                                                              SLMBBRAM BRAMSLMB
                                                                                                                                                                                                    bram_block                            dlmb_1                lmb_bram_if_cntlr
                                                                                                                                                                                                                                                                             lmb_bram_if_cntlr        ilmb_1                                  bram_block                                                        bram_block                                                     bram_block                                                       bram_block
                                                                                                                                                                                                    lmb_bram_3                                                  dlmb_cntlr_1 ilmb_cntlr_1                                                     lmb_bram_0                                                        lmb_bram_2                                                     lmb_bram_4                                                       lmb_bram_5

                                                                                                                                                                                                      PORTA
                                                                                                                                                                                                  PORTB                                                                                                                                        PORTA
                                                                                                                                                                                                                                                                                                                                           PORTB                                                              PORTB
                                                                                                                                                                                                                                                                                                                                                                                                                  PORTA                                                          PORTA
                                                                                                                                                                                                                                                                                                                                                                                                                                                                             PORTB                                                               PORTA
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             PORTB
                                                                                                                                                                                                                                                                 PROCESSOR
                                                                                                                                                                                          SLMBBRAM BRAMSLMB                                                           microblaze                                                     SLMBBRAM BRAMSLMB                                                SLMBBRAM BRAMSLMB                                              SLMBBRAM BRAMSLMB                                               SLMBBRAM BRAMSLMB
                                                                                                                                                                           dlmb_3                                                ilmb_3                                     0
                                                                                                                                                                                                                                                                      microblaze_1                                 dlmb_0                                                  ilmb_0    dlmb_2                                                  ilmb_2   dlmb_4                                                ilmb_4   dlmb_5                                                       ilmb_5
                                                                                                                                                                                                         lmb_bram_if_cntlr
                                                                                                                                                                                            lmb_bram_if_cntlr                                                                                                                                       lmb_bram_if_cntlr
                                                                                                                                                                                                                                                                                                                                       lmb_bram_if_cntlr                                                lmb_bram_if_cntlr
                                                                                                                                                                                                                                                                                                                                                                                                                     lmb_bram_if_cntlr                                              lmb_bram_if_cntlr
                                                                                                                                                                                                                                                                                                                                                                                                                                                                       lmb_bram_if_cntlr                                                            lmb_bram_if_cntlr
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       lmb_bram_if_cntlr
                                                                                                                                                                                            dlmb_cntlr_3 ilmb_cntlr_3                                                                                                                  dlmb_cntlr_0 ilmb_cntlr_0                                        dlmb_cntlr_2 ilmb_cntlr_2                                      dlmb_cntlr_4 ilmb_cntlr_4                                       dlmb_cntlr_5 ilmb_cntlr_5
                                                                                                                                                                                                                                                                      DLMBILMB
                                                                                                                                                                                             PROCESSOR                                        NIC_010                 SFSL0
                                                                                                                                                                                                                                                                          MFSL2      NIC_13                                            PROCESSOR                                                         PROCESSOR                                                      PROCESSOR                                                       PROCESSOR
                                                                                                                                                                                                  microblaze                                                              MFSL1
                                                                                                                                                                                                                                                                      SFSL1                                                                 microblaze                                                        microblaze                                                     microblaze                                                      microblaze
                                                                                                                                                                                                        2
                                                                                                                                                                                                  microblaze_3                                NIC_210                              0 NIC_12                                                       1
                                                                                                                                                                                                                                                                                                                                            microblaze_0                                                      microblaze_2
                                                                                                                                                                                                                                                                                                                                                                                                                    3                                                              5
                                                                                                                                                                                                                                                                                                                                                                                                                                                                             microblaze_4                                                          4
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             microblaze_5
                                                                                                                                                       MASTER_SLAVE                                                                           NIC_410                     MFSL4
                                                                                                                                                                                                                                                                      SFSL3        2 NIC_15
                                                                                                                                                            opb_ethernet                          DLMBILMB                                    NIC_510                     MFSL3
                                                                                                                                                                                                                                                                      SFSL4        1 NIC_14                                                DLMBILMB                                                           DLMBILMB                                                       DLMBILMB                                                        DLMBILMB
                                                                                                                                                            Ethernet_MAC                                    NIC_31                                NIC_31                        NIC_10                                      NIC_10
                                                                                                                                                            B                  NIC_130                MFSL0
                                                                                                                                                                                                  SFSL0                                                                   MFSL0
                                                                                                                                                                                                                                                                      SFSL2                                                                SFSL0
                                                                                                                                                                                                                                                                                                                                               MFSL0         NIC_01                         NIC_120               MFSL0
                                                                                                                                                                                                                                                                                                                                                                                                              SFSL0            NIC_21                     NIC_141            SFSL0
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 MFSL0        NIC_41                     NIC_152             SFSL0
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 MFSL0             NIC_51
                                                                                                                                                            MSOPB                                 DOPBIOPB                                                            DOPBIOPB                                                             DOPBIOPB                                                           DOPBIOPB                                                       DOPBIOPB                                                        DOPBIOPB



          plb
                              opb2plb_bridge        opb2plb_bridge      opb2plb_bridge      opb2plb_bridge      opb2plb_bridge      opb2plb_bridge
                              opb2plb_bridge_0      opb2plb_bridge_1    opb2plb_bridge_2    opb2plb_bridge_3    opb2plb_bridge_4    opb2plb_bridge_5

                           MPLBSOPB                MPLBSOPB            MPLBSOPB            MPLBSOPB            MPLBSOPB            MPLBSOPB

          mb_opb_0




          mb_opb_1




          mb_opb_2
0GLB                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           20A

1GLB                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           21A

2GLB                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           22A

3GLB      mb_opb_3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             23B

4GLB                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           24B

5GLB                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           25B

6GLB                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           26B
          mb_opb_4
7GLB                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           27B

8GLB                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           28B

9GLB                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           29B
          mb_opb_5




                                                                                                                                                                           SLAVES OF mb_opb_3                                             SLAVES OF mb_opb_1                                                                         SLAVES OF plb                                       SLAVES OF mb_opb_2                                           SLAVES OF mb_opb_4                                             SLAVES OF mb_opb_5

                                                                                                                                                                                        12               02                                                 60               00                                                                                                                     13               03                                            15               05                                            14                 04
                                                                                                                                                                                    SOPB            SOPB            SOPB 2                              SOPB            SOPB 0
                                                                                                                                                                                                                                                                             1          SOPB 0                                                SPLB                                              SOPB            SOPB            SOPB 3                         SOPB            SOPB            SOPB 5                         SOPB              SOPB            SOPB 4
                                                                                                                                                                                    opb_uartlite    MyNIC           opb_intc                            opb_uartlite         2
                                                                                                                                                                                                                                                                        MyNIC0          opb_intc                                              plb_ddr                                           opb_uartlite    MyNIC           opb_intc                       opb_uartlite    MyNIC           opb_intc                       opb_uartlite      MyNIC           opb_intc
                                                                                                                                                                                    RS232_Uart_3                    opb_intc_3
                                                                                                                                                                                                    microblaze_3_NIC31                                  RS232_Uart_1    microblaze_1_NIC1023
                                                                                                                                                                                                                                                                             30         opb_intc_1                                            DDR_256MB_32MX64_rank1_row13_col10_cl2_5          RS232_Uart_2                    opb_intc_2
                                                                                                                                                                                                                                                                                                                                                                                                                microblaze_2_NIC21                             RS232_Uart_4    microblaze_4_NIC41
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               opb_intc_4                     RS232_Uart_5                      opb_intc_5
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                microblaze_5_NIC51
                                                                                                                                                                                                                                                        C                    40                                                               A

                                                                                                                                                                                        22                                                                  70                                                                                                                                      23                                                             25                                                             24
                                                                                                                                                                                    SOPB                                                                SOPB                                                                                                                                    SOPB                                                           SOPB                                                           SOPB
                                                                                                                                                                                    opb_timer                                                           opb_timer                                                                                                                               opb_timer                                                      opb_timer                                                      opb_timer
                                                                                                                                                                                    opb_timer_3                                                         opb_timer_1                                                                                                                             opb_timer_2                                                    opb_timer_4                                                    opb_timer_5



                                                                                                                                                                                                                                                                                                                   SLAVES OF mb_opb_0

                                                                                                                                                                                                                                                                                                                                11                 01
                                                                                                                                                                                                                                                                                                                            SOPB              SOPB            SOPB 1
                                                                                                                                                                                                                                                                                                                            opb_uartlite      MyNIC           opb_intc
                                                                                                                                                                                                                                                                                                                            RS232_Uart_0                      opb_intc_0
                                                                                                                                                                                                                                                                                                                                              microblaze_0_NIC01


                                                                                                                                                                                                                                                                                                                                21
                                                                                                                                                                                                                                                                                                                            SOPB
                                                                                                                                                                                                                                                                                                                            opb_timer
                                                                                                                                                                                                                                                                                                                            opb_timer_0




                                                                                                                                                                                                                                                   IP


                                                                                                                                                                                                                                                              util_vector_logic opb_mdm        util_vector_logic
                                                                                                                                                                                                                                                              sysclk_inv        debug_module   ddr_clk90_inv




                                                                                                                                                                                                                                                              dcm_module      dcm_module       util_vector_logic
                                                                                                                                                                                                                                                              dcm_1           dcm_0            clk90_inv
                                                                                                                                                                                                                                                              E               D




                                                                                                                                                                                                                                                           11GLB 13A
                                                                                                                                                                                                                                                        10GLB 12GLB 14A 15A 16A 17A 18A 19A
                      SPECS                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                KEY
   EDK VERSION                            8.1.01                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           SYMBOLS
   ARCH                                virtex2p                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Bus connections            External Ports
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   bus interface
   PART                       xc2vp30ff896-7                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             master or initiator       input
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   shared bus
   GENERATED         Sat Apr 29 05:55:11 2006                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            slave or target           output
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   inout
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         master slave
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         monitor

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           COLORS
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   Bus Standards

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       DCR         FSL        OPB           SOCM         GEN. P2P, USER, etc
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       FCB         LMB        PLB           XIL (prefix) P2P




                                                                                                                                                                                                                        CS268 Spring 2006                                                                                                                                                                                                                                                                                                                                             6
A XUP cluster
                     TO EXTERNAL
                      NETWORK




 OTHER                                                OTHER
DEVICES              ROUTER                          DEVICES




          uB                                    uB
  uB            uB                        uB           uB
          uB                                    uB
   uB          uB                          uB        uB

                          uB
               uB                          uB
                          uB
                uB                        uB

                      CS268 Spring 2006                        7
i3 / Chord / DHT




             CS268 Spring 2006   8
Available Applications / Utilities
chmod df         du    find grep       ifconfig iptables-restore lsmod
    netstat pwd route telnetd traceroute vi            zcat
agetty cmp       dhclient echo flatfsd gunzip ifdown iptables-
    save mkdir nslookup rdate rsyslogd test true                 vmstat
arp     cp     dhcpd egrep free gzip             ifup    kill
    modprobe null     reboot sed      tftp ttcp         wc
arping crond dhcrelay env ftp            hd        inetd killall
    more     passwd reset sh         thttpd umount         wget
basename crontab dmesg expand ftpd head                    init  ln
    mount ping       rm     tail  time uname            which
busybox date dpkg          false ftpget hostname insmod login
    msh     portmap rmdir tcpdump top           uptime       whoami
cat     dd     dpkg-deb fgrep ftpput i3serverd iptables ls
    mv      ps     rmmod telnet touch version xargs Python


                              CS268 Spring 2006                           9
Challenges
   Porting apps is a painstaking job
          Configuration/Makefile problem
          Cross-compiling (Dynamic linking library, CPU type and etc.)
          Fork (using pthread library might be an alternative)
   More cores?
     4-core   version is quite stable now
     6-core version (near 100% BRAM and Slices
      utilization)
          Hitting the power-wall without heat sink
          100 Mhz is difficult
          Building and debugging is much more difficult
          Place and route time is unbearable! (floor plan tools required)

                               CS268 Spring 2006                             10
Future Work
 8 cores?
  Migrating design to RAMP (some
   dependencies)
  And more…




                 CS268 Spring 2006   11

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:58
posted:6/10/2011
language:English
pages:11