CSCE 612_ VLSI System Design

Document Sample
CSCE 612_ VLSI System Design Powered By Docstoc
					  CSCE 313: Embedded Systems

Video Out and Image Transformation




      Instructor: Jason D. Bakos
                  Video on DE2 Board
• DE2 has a VGA DAC and VGA output
• VGA controller (in SOPC Builder) sends timed values to the
  DAC
   – Natively set to 640x480 resolution


• Images are transmitted to VGA controller as row-major
  array of RGB pixels
   – 10 bits of RED, GREEN, BLUE intensities
   – All 0’s is black, all 1’s is white

             bits 29:20       bits 19:10         bits 9:0
            RED INTENSITY   GREEN INTENSITY   BLUE INTENSITY




                                                               CSCE 313 2
                        Video on DE2 Board
• Frame layout:




• X-Y addressing:

• Consecutive addressing:
   – Row-major 2D array, flattened as 1D array:

  row 0, 640 pixels     row 1, 640 pixels     row 2, 640 pixels    …     row 479, 640 pixels

     e.g. pixel at row=34, col=123 would be at offset = 34*(640*30)+123*30 = 656490



                                                                                      CSCE 313 3
                   Video on DE2 Board
• A 640x480x30 image would require 1.1 MB to store
• We will use the on-board SRAM as our VGA frame buffer (“pixel
  memory”)
• The DE2 SRAM is only 512 KB
• Need to scale down the image to:
   – 320 x 240 x 16 bits = 150 KB


• The Altera University Program contains cores to perform color-
  space and resolution re-sampling (scaling/conversion) in hardware

• SOPC Builder:
   – First task: edit your clocks component to add vga_click to your design




                                                                    CSCE 313 4
                                    System Design for Video

      CPU                                                                               simple mode    Video                     sys_clk   dual-clock
                                                                                                       alpha                                 FIFO
                                                                                                      blender                              sys->vga
           data master                                                    Enable
                                                                          transparency




                                                                                                       foreground


                                                                                                                    background
                                                                                                                                                 stream
                                                                             Video                                                               30 bit
                                                          data/control     character
                                                                          buffer with                                                            color
                    consecutive                                              DMA                                                                 640x480
            data         mode     control                                                                                                        vga_clk
                            DMA                   RGB                        Video
       SRAM                                                                                                                                  VGA
                          controller           Resampler                     scaler
     interface                        stream                    stream                     stream                                          controller
                   data   for video             16 -> 30                    (x2,x2)
                                      16 bit                    30 bit                     30 bit
                   From Univ. Program color   10 bits/channel   color                      color
on-chip            16 bits            320x240 3 channels        320x240                    640x480                                                      on-chip
                                      sys_clk                   sys_clk                    sys_clk
off-chip                                                                                                                                                off-chip

       SRAM
                                                                                                                                           VGA DAC
        chip
                                   Use Pixel Buffer
    320x240x16
                                   DMA
      image
                                                                                                                                            DE-15F
                                                                                                                                            D-sub




                                                                                                                                           CSCE 313 5
                                      VHDL Modifications
• Add to component declaration for nios_system:
-- 1) global signals:
             signal clk_0 : IN STD_LOGIC;
             signal reset_n : IN STD_LOGIC;
             signal sdram_clk : OUT STD_LOGIC;
             signal sys_clk : OUT STD_LOGIC;
             signal vga_clk : OUT STD_LOGIC;

 -- the_pixel_buffer
           signal   SRAM_ADDR_from_the_sram_0 : OUT STD_LOGIC_VECTOR (17 DOWNTO 0);
           signal   SRAM_CE_N_from_the_sram_0 : OUT STD_LOGIC;
           signal   SRAM_DQ_to_and_from_the_sram_0 : INOUT STD_LOGIC_VECTOR (15 DOWNTO 0);
           signal   SRAM_LB_N_from_the_sram_0 : OUT STD_LOGIC;
           signal   SRAM_OE_N_from_the_sram_0 : OUT STD_LOGIC;
           signal   SRAM_UB_N_from_the_sram_0 : OUT STD_LOGIC;
           signal   SRAM_WE_N_from_the_sram_0 : OUT STD_LOGIC;

-- the_video_vga_controller_0
           signal VGA_BLANK_from_the_video_vga_controller_0 : OUT STD_LOGIC;
           signal VGA_B_from_the_video_vga_controller_0 : OUT STD_LOGIC_VECTOR (9 DOWNTO 0);
           signal VGA_CLK_from_the_video_vga_controller_0 : OUT STD_LOGIC;
           signal VGA_G_from_the_video_vga_controller_0 : OUT STD_LOGIC_VECTOR (9 DOWNTO 0);
           signal VGA_HS_from_the_video_vga_controller_0 : OUT STD_LOGIC;
           signal VGA_R_from_the_video_vga_controller_0 : OUT STD_LOGIC_VECTOR (9 DOWNTO 0);
           signal VGA_SYNC_from_the_video_vga_controller_0 : OUT STD_LOGIC;
           signal VGA_VS_from_the_video_vga_controller_0 : OUT STD_LOGIC




                                                                                               CSCE 313 6
                              VHDL Modifications
• Add to your top-level entity statement:
  SRAM_ADDR : OUT STD_LOGIC_VECTOR(17 downto 0);
  SRAM_DQ : INOUT STD_LOGIC_VECTOR(15 downto 0);
  SRAM_WE_N : OUT STD_LOGIC;
  SRAM_OE_N : OUT STD_LOGIC;
  SRAM_UB_N : OUT STD_LOGIC;
  SRAM_LB_N : OUT STD_LOGIC;
  SRAM_CE_N : OUT STD_LOGIC;

  VGA_R : OUT STD_LOGIC_VECTOR(9 downto 0);
  VGA_G : OUT STD_LOGIC_VECTOR(9 downto 0);
  VGA_B : OUT STD_LOGIC_VECTOR(9 downto 0);
  VGA_CLK : OUT STD_LOGIC;
  VGA_BLANK : OUT STD_LOGIC;
  VGA_HS : OUT STD_LOGIC;
  VGA_VS : OUT STD_LOGIC;
  VGA_SYNC : OUT STD_LOGIC




                                                   CSCE 313 7
                               VHDL Modifications
• Add to your port map:
  VGA_BLANK_from_the_video_vga_controller_0   =>   VGA_BLANK,
  VGA_B_from_the_video_vga_controller_0       =>   VGA_B,
  VGA_CLK_from_the_video_vga_controller_0     =>   VGA_CLK,
  VGA_G_from_the_video_vga_controller_0       =>   VGA_G,
  VGA_HS_from_the_video_vga_controller_0      =>   VGA_HS,
  VGA_R_from_the_video_vga_controller_0       =>   VGA_R,
  VGA_SYNC_from_the_video_vga_controller_0    =>   VGA_SYNC,
  VGA_VS_from_the_video_vga_controller_0      =>   VGA_VS,

  SRAM_ADDR_from_the_sram_0                   =>   SRAM_ADDR(17 downto 0),
  SRAM_CE_N_from_the_sram_0                   =>   SRAM_CE_N,
  SRAM_DQ_to_and_from_the_sram_0              =>   SRAM_DQ,
  SRAM_LB_N_from_the_sram_0                   =>   SRAM_LB_N,
  SRAM_OE_N_from_the_sram_0                   =>   SRAM_OE_N,
  SRAM_UB_N_from_the_sram_0                   =>   SRAM_UB_N,
  SRAM_WE_N_from_the_sram_0                   =>   SRAM_WE_N




                                                                             CSCE 313 8
                Software: Writing to SRAM
• To load an image into the DE2, I have written a MATLAB script
  that can:
   –   read image file with size 320x240 or smaller
   –   add a black border around the image if smaller than 320x240
   –   write the image in 16-bit color into a text file bitmap
   –   displays the original and bordered images


• To use it:
   –   download it from the course webpage
   –   open MATLAB (command: “matlab”)
   –   change current folder to where you downloaded it
   –   type: convert_image_to_data_file(‘<filename>');
        • You may use my image, lumcat.jpg, if you’d like (or use your own)
   – this will generate myfile.dat and myfile.zip




                                                                              CSCE 313 9
Storing and Accessing an Image on the DE2
•   Altera has designed a read-only flash-based file system that we can use to
    store data files

•   Instead of a traditional file system (i.e. NTFS, FAT32, ext3, reiserfs),
    Altera uses the internal structure of an uncompressed ZIP file to store one
    or more files

•   To use it, you need to add an interface for the 4 MB CFI Flash memory to
    your system design, along with an Avalon tri-state bridge so the flash can
    be initialized externally


                                CPU
                                    data master
                                      slave                        addr=22 bits
                   registered   Avalon-                            data=8 bits
                                   MM         master   CFI Flash   setup 0 ns,
                                tristate               Interface   wait 100 ns,
                                 bridge                            hold 0 ns



                                                                                  CSCE 313 10
                         VHDL Modifications
• Add to component declaration for nios_system:

-- the_tri_state_bridge_0_avalon_slave
signal address_to_the_cfi_flash_0 : OUT STD_LOGIC_VECTOR (21 DOWNTO 0);
signal data_to_and_from_the_cfi_flash_0 : INOUT STD_LOGIC_VECTOR (7 DOWNTO 0);
signal read_n_to_the_cfi_flash_0 : OUT STD_LOGIC;
signal select_n_to_the_cfi_flash_0 : OUT STD_LOGIC;
signal write_n_to_the_cfi_flash_0 : OUT STD_LOGIC


• Add to your top-level entity statement:

  FL_ADDR                    :   OUT STD_LOGIC_VECTOR(21 downto 0);
  FL_CE_N                    :   OUT STD_LOGIC;
  FL_DQ                      :   INOUT STD_LOGIC_VECTOR(7 downto 0);
  FL_OE_N                    :   OUT STD_LOGIC;
  FL_RST_N                   :   OUT STD_LOGIC;
  FL_WE_N                    :   OUT STD_LOGIC




                                                                                 CSCE 313 11
                        VHDL Modifications
• Add to your port map:

 address_to_the_cfi_flash_0         =>   FL_ADDR,
 data_to_and_from_the_cfi_flash_0   =>   FL_DQ,
 read_n_to_the_cfi_flash_0          =>   FL_OE_N,
 select_n_to_the_cfi_flash_0        =>   FL_CE_N,
 write_n_to_the_cfi_flash_0         =>   FL_WE_N


• Add to the top-level architecture:
 FL_RST_N <= '1';




                                                    CSCE 313 12
                  BSP Modifications
• In the BSP Editor, make the following changes:




                                         this must match base
                                         address in SOPC builder
                                          point to your .ZIP file
                                          from MATLAB

                                          make this 0 to use all of
                                          Flash memory




                                                          CSCE 313 13
           Programming the Flash Memory
• To program Flash memory,
                                   Make sure this matches BSP
  prior to running your
  program, in Eclipse, go to
  Nios II | Flash Programmer

• Then, do File | New
   – Get settings from BSP
     settings file
   – Browse for your BSP
     settings file
       • Under <project
         name>/software/<eclipse
         project>_bsp
   – Add myfile.zip and click
     Start




                                                      CSCE 313 14
    Accessing the RO File System from SW
• Declare C-style file pointer:
FILE *myfile;


• Open the file:
myfile=fopen(“my_fs/myfile.dat","rb");
if (myfile==NULL) perror ("error opening datafile");


• Note: path above must match one in BSP!

• Allocate memory and read the image data:
my_image=(alt_u16 *)malloc(320*240*sizeof(alt_u16));
fread(my_image,sizeof(alt_u16),320*240,myfile);




                                                       CSCE 313 15
                                New Header Files
• Add:

  #include   <altera_up_avalon_video_character_buffer_with_dma.h> // to write characters to video
  #include   <altera_up_avalon_video_pixel_buffer_dma.h> // to swap front and back buffer
  #include   <io.h>     // for writing into pixel memory
  #include   <math.h> // for trigonometry functions
  #include   <stdlib.h> // for file I/O




                                                                                             CSCE 313 16
                 Using the SRAM Pixel Memory
• We’re using consecutive mode for the pixel memory, so
  pixels are stored consecutively

    row 0, 320 pixels        row 1, 320 pixels      row 2, 320 pixels         …        row 239, 320 pixels

low addresses                                                                                   high addresses



• Each pixel is a 16-bit value:

                                RED              GREEN                 BLUE
                        15             11 10               5     4                 0



• To write to pixel memory:
    for (i=0;i<240;i++) for (j=0;j<320;j++) {
            IOWR_16DIRECT(SRAM_0_BASE,(i*320+j)*2,my_image[(i*320+j)]);
    }




                                                               Refer to system.h
                                                                                                   CSCE 313 17
                             The Back Buffer
•   The 320x240x2 image requires 153600 bytes, or 150 KB

•   Since this is less than half the SRAM, we’ll also store a second frame at
    address 0x25800
     – Make this setting in the SOPC Builder


•   Two buffers:
     –   FRONT BUFFER: the frame being displayed (streamed/DMA’ed to VGA)
     –   BACK BUFFER: the frame being “drawn” by the CPU
     –   This allows one buffer to be updated while the other buffer is being displayed
     –   After updating, the buffers can be swapped:
     –   Front buffer  back buffer


•   SOPC Builder:
     – Add the base address of the SRAM to both front and back buffer addresses
     – e.g. 0x1400000 and 0x1425800




                                                                                 CSCE 313 18
                       The Back Buffer
• To use:
   – Declare global variable:
       alt_up_pixel_buffer_dma_dev *my_pixel_buffer;


   – Assign it:
       my_pixel_buffer=
       alt_up_pixel_buffer_dma_open_dev("/dev/video_pixel_buffer_dma_0");


   – To swap buffers (update frame):
       alt_up_pixel_buffer_dma_swap_buffers(my_pixel_buffer);


   – To draw pixel in back buffer:
       alt_up_pixel_buffer_dma_draw(my_pixel_buffer,my_image[(i*320+j)],j,i);




                                                                 CSCE 313 19
                             The Back Buffer




     – Faster method (2X speed improvement):
          IOWR_16DIRECT(*(((alt_u32 *)VIDEO_PIXEL_BUFFER_DMA_0_BASE+1)),
          (i*320+j)<<1,my_image[(i*320+j)]);
     – Faster yet (2X additional improvement):
          for (i=0;i<76800;i++)
            IOWR_16DIRECT(*(((alt_u32 *)VIDEO_PIXEL_BUFFER_DMA_0_BASE+1)),
            i<<1,my_image[i]);

•   You may also use:
     –   alt_up_pixel_buffer_dma_clear_screen(my_pixel_buffer,0);    clear front
     –   alt_up_pixel_buffer_dma_clear_screen(my_pixel_buffer,1);    clear back




                                                                                    CSCE 313 20
                  Using the Character Buffer
• Use:

   alt_up_char_buffer_dev *my_char_buffer;
   …
   my_char_buffer=alt_up_char_buffer_open_dev("/dev/video_character_buffer_with_dma_0");
   if (!my_char_buffer) printf ("error opening character buffer\n");
   alt_up_char_buffer_clear(my_char_buffer);
   alt_up_char_buffer_string(my_char_buffer,"Video works!",0,60);




• Allows you to superimpose text on the screen at (col,row)
   – 80 cols x 60 rows




                                                                                           CSCE 313 21
         Image Transformation Matrices
• Simple image transformation matrix can be used to…
   – rotate, scale, shear, reflect, and orthogonal projection


• The matrices we use are 2x2 and used to determine how to
  move each pixel from the original image to the new image
  in order to perform the transformation

• For Lab 2, we want to perform rotation and scaling




                                                                CSCE 313 22
         Image Transformation Matrices
• Clockwise rotation: row'  cos         sin   row
                                 
                         col '   sin  cos   col 
                                                   
                        row'  row  cos  col  sin 
                        col '   row  sin   col  cos


                        row'  s x 0  row
• Scaling (factor s):    col '    0 s   col 
                                      y      
                        row'  row  s x
                        col '  col  s y




                                                             CSCE 313 23
                           Issues to Resolve
• Using these algorithms directly, the rotation and scaling
  occur about the origin (0,0)

           50                                      50




          100                                      100




          150                                      150




          200                                      200




                50   100   150   200   250   300         50   100   150   200   250   300




           50                                      50




          100                                      100




          150                                      150




          200                                      200




                50   100   150   200   250   300         50   100   150   200   250   300




                                                                                            CSCE 313 24
                            Issues to Resolve
• We want it to occur about the center of the image

          50                                        50




         100                                       100




         150                                       150




         200                                       200




                                                         50   100   150   200   250   300
                50   100   150   200   250   300




          50                                       50




          100                                      100




          150                                      150




          200                                      200




                50   100   150   200   250   300         50   100   150   200   250   300




                                                                                            CSCE 313 25
                   Issues to Resolve
• To fix this, you need to:
   – subtract 320/2 from the column
   – subtract 240/2 from the row
• …before you multiply against the transformation matrix

• Then add these values back after your multiply




                                                      CSCE 313 26
                   Issues to Resolve
• Make sure you…
   – use rounding and type casting for the transformation matrix
     (float and alt_u16)
   – disregard output coordinates that fall outside the frame
   – always transform against the original image
   – initialize the output image to black before transforming
   – always transform against the original image




                                                           CSCE 313 27
Affine Transformations




                         CSCE 313 28

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:4/17/2013
language:English
pages:28