Docstoc

SC461_ManishGupta

Document Sample
SC461_ManishGupta Powered By Docstoc
					Jan 2012 SC461 Lecture 1.0                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
                             SC 461 Coding Theory and Applications:
                                  The Heart of Information and
                                Communication Technology (ICT)




                                     Image Credit: Nick Papadakis, Copyright P.W.K.R. and N.P.
                                                                               http://www.dna.caltech.edu/~pwkr/



                                                  Prof. Manish K. Gupta
                                 Laboratory of Natural Information Processing


Jan 2012 SC461 Lecture 1.0                                                                                http://www.guptalab.org
Coding Theory and App           Laboratory of Natural Information Processing
                               Outline of Presentation
    •   Admin Details
    •   Overview and Subject Classification
    •   Historical Introduction & Motivation




                                                                                          2004 @ iGEM: Students of
                                                                                          Boston University, Caltech,
                                                                                          MIT, Princeton and U of Texas




                             http://en.wikipedia.org/wiki/File:UT_HelloWorld.jpg

                                  http://partsregistry.org/cgi/htdocs/SBC04/index.cgi

Jan 2012 SC461 Lecture 1.0                                                                                 http://www.guptalab.org
Coding Theory and App                      Laboratory of Natural Information Processing
                             Administrative Details SC461




Jan 2012 SC461 Lecture 1.0                                                       http://www.guptalab.org
Coding Theory and App             Laboratory of Natural Information Processing
                                Instructor
   •   Manish K Gupta (www.mankg.com)
   •   Office: Room 2209 Faculty Block 2
   •   Office Hours: Friday 2:00 to 3:00 pm every week
   •   Email: mankg@daiict.ac.in
   •   Phone: 91-79-30510549
   •   Course Page:
       http://www.guptalab.org/mankg/public_html//WWW/courses/ctasp12/coursein
       fo.shtml

   Submission Page on Moodle http://courses.daiict.ac.in//course/view.php?id=234




Jan 2012 SC461 Lecture 1.0                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
                                          Tutors
  • None

                             Teaching Assistants
  • None

                              Guest Lectures
  • There could be some guest lectures.




Jan 2012 SC461 Lecture 1.0                                                    http://www.guptalab.org
Coding Theory and App          Laboratory of Natural Information Processing
                             Class Timings

                                  Lectures
  Monday, Tuesday and Thursday 12:00 noon (CEP 105)




Jan 2012 SC461 Lecture 1.0                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
                             Attendance Policy
  • Each of you must attend each lecture as I
    usually give a few questions (called type-2
    questions) that you need to solve by that
    week itself and clear your doubts about it.
    Note that I usually ask them in the exams or
    quizzes. There could be a surprise quiz at
    any time in Lectures and sometime I may
    change the % of quiz for final grade to quite
    a lot.
  • If for some reason beyond your control (for
    example you are sick) you are about to miss
    a lecture please send an email in advance to
    me (subject line: your roll number, name
    and SC 461 Spring 2012) that you will not
    be able to attend the lecture that day.

Jan 2012 SC461 Lecture 1.0                                                    http://www.guptalab.org
Coding Theory and App          Laboratory of Natural Information Processing
                       Semester Break
             March 5 to March 11 (Holiday Week)




Jan 2012 SC461 Lecture 1.0                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
                             Holidays Spring 2012

  1         Republic Day                       26 January 2012                Thursday
  2         Mahashivratri                      20 February 2012               Monday
  3         Holi (2nd day)                     8 March 2012                   Thursday
  4         Mahavir Jayanti                    5 April 2012                   Thursday
  5         Good Friday                        6 April 2012                   Friday




Jan 2012 SC461 Lecture 1.0                                                       http://www.guptalab.org
Coding Theory and App          Laboratory of Natural Information Processing
       Marks Distribution (Tentative) /
               Grading Policy
            Assignments                                   10%

            Mid Term Test 1                               20%

            Projects                                      30%

            Final                                        40%




Jan 2012 SC461 Lecture 1.0                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
                             Project Policy

  • Project has 3 Reporting Stages
                    (Software or Study)
  • Part-1: Submission of (1 page) abstract via
    Moodle consisting of problem formulation and
    key references. It is due in approximately 4
    weeks after the start of the course. 10%
  • Part-2: Mid progress report (1 page) in another
    4 weeks after part-1. 10%
  • Part-3: Final report and presentation /demo of
    the s/w towards the 16th week. 20%

Jan 2012 SC461 Lecture 1.0                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
                         Some Suggested Topics for Projects

  Applications

  Data Storage (Hard Disk, Cloud Data Storage, CD/DVD)
  Data Communications (point 2 point, MIMO)
  Networks (Sensor Networks, Wireless networks, General Networks)
  Security (Cloud Security, Cryptography)
  Deep Space Communications
  Power-line Communications
  Connections to Combinatorial Designs
  Connections to Graph Theory
  Connections to Algebraic Geometry
  Concatenated Codes
  Iterative Decoding
  Optimal DNA codes
  Quantum Codes
  Compressed Sensing Codes
  Art of Decoding
  Network Coding
  QR (Quick Response) Codes for Mobiles


Jan 2012 SC461 Lecture 1.0                                                     http://www.guptalab.org
Coding Theory and App           Laboratory of Natural Information Processing
                             Course Web Page

  • Course Web Page:
  http://courses.daiict.ac.in//course/vie
    w.php?id=234
  • Create your login with your daiict
    email address as and when you get it
    at http://courses.daiict.ac.in
  • Register for the course SC 461 at
    Moodle


Jan 2012 SC461 Lecture 1.0                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
                      Text Book
Coding Theory:




Information Theory:




                                  15
                              Introduction

Men are from Mars, women are from Venus and computers are from hell
thats what we say once they makes an error . . . , IEEE Computer Magazine,
1990’s…




Jan 2012 SC461 Lecture 1.0                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
                           Outline of Story
   Science is the catalyst,
   Technology is the application,
   Business is the opportunity ……




   »ICT is everywhere!
   » Philosophy of Information
     Processing
   » Coding Theory is everywhere !
Do you know what is ICT?
Manish K Gupta, PhD                                                       http://www.guptalab.org
                           Laboratory of Natural Information Processing
              ICT-Information & Communication
                        Technology
  • IT-Information Technology --Applications
    (Mathematical Root- Computer Science –Turing
    Machine)
  • CT-Communication Technology -- Applications
    (Mathematical Root-Theory of Communication-
    Information and Coding Theory)
  • Realization via Logic Gates and Circuits-
    (Computation by Circuits)-Hardware, Software,
    Electronics and VLSI-Semiconductor Technology

Do you know what is ICT?
Manish K Gupta, PhD                                                       http://www.guptalab.org
                           Laboratory of Natural Information Processing
              Overview and Subject Classification
  This exciting course is foundation to ICT.
  • Basics of Coding Theory and Briefs of Information
    Theory
  • Constructions of Codes, Algorithms and Decoding
  • Applications -Cloud Computing, Cryptography
    (Security), Wireless Communications, Quantum
    Computing and DNA Computing
“In science if you know what you are doing you should
not be doing it.
In engineering if you do not know what you are doing
you should not be doing it.”
                         Richard W. Hamming


Jan 2012 SC461 Lecture 1.0                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
                             What is a Computer?

  •   A device (machine) which process Information
  •   Processing -- Storing and Manipulating
  •   Representation of Information in classical computers (classical physics) -
      Binary


  •   Representation of Information in quantum computers (quantum physics) -
      Qubits




Jan 2012 SC461 Lecture 1.0                                                    http://www.guptalab.org
Coding Theory and App          Laboratory of Natural Information Processing
               What is the purpose of Computing?
  “The purpose of computing is insight, not
    numbers. The purpose of computing numbers is
    not yet in sight” Richard W. Hamming




    RW Hamming (1971). Introduction to Applied Numerical Analysis. McGraw Hill.


Jan 2012 SC461 Lecture 1.0                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
                         What is a Communication?
  • Sending /Receiving Information from here to there
  • This will require Information Processing -- Storing and
    Manipulating (So you require a computer !)
  • Storing- Sending Information from now to then
  • Retrieving - Receiving Information from then to now




Jan 2012 SC461 Lecture 1.0                                                   http://www.guptalab.org
Coding Theory and App         Laboratory of Natural Information Processing
                                What is Information?
      Representation of Information is different in different information
      processing process /machine

                             Digital Information Processing
                             Quantum Information Processing
                             Chemical Information Processing
                             Biological Information Processing
                             Structural Information Processing
                             Social Information Processing
                             Natural Information Processing




Jan 2012 SC461 Lecture 1.0                                                          http://www.guptalab.org
Coding Theory and App                Laboratory of Natural Information Processing
                               Information Processing
                         Information and Communication Technology (ICT)


      •   Mathematics of Information Processing
      •   Computing
      •   Algorithms
      •   Communication
      •   Signal Processing
      •   Electronics - Logic Gates and Circuits and VLSI
      •   Programming Languages
      •   Networks
      •   Storage
      •   Logic
      •   Robotics
      •   Security and Cryptography
      •   Coding and Information Theory


      Course: SC 461: Coding Theory and Applications (Spring 2012)
              CT 512: Information and Coding Theory (Spring 2012)




Jan 2012 SC461 Lecture 1.0                                                              http://www.guptalab.org
Coding Theory and App                    Laboratory of Natural Information Processing
               Motivation and Historical Introduction

   As the births of living creatures, at first,
   are ill-shapen: so are all Innovations, which are
     the births of time. . . .
                         Francis Bacon (1561-1626)



Jan 2012 SC461 Lecture 1.0                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
Coding Theory Everywhere…



  Error control is everywhere in
   engineering
      Computer Science and
      Communications (ICT)
• Any Information processing device either
  sends data from here to there
  (communication) or sends data from now to
  then (storage)
• Examples: Phone, Computers, Internet,
  voiceband modems, GSM mobile
  phones
• To process the data reliably they use coding
  theory
• Redundancy is very basic error correction
                                                           http://en.wikipedia.org/wiki/File:Timeline_of_communication_tools.jpg


Jan 2012 SC461 Lecture 1.0                                                                                              http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
   Francis Bacon (1605)
  Advancement of Learning
Biliterarie Cipher Omnia per Omnia
Bacon's Cipher (5-bit Code)
                               Joseph Marie Jacquard
                                    1752-1834
                                       http://upload.wikimedia.org/wikipedia/commons/2/29/Joseph_Marie_Jacquard.jpg



  • Binary Coded Punched Cards
    for Operating Looms (1805)




http://upload.wikimedia.org/wikipedia/commons/5/5f/Jacquard.loom.full.view.jpg
                                                                     http://upload.wikimedia.org/wikipedia/commons/e/e1/Jacquard_Joseph_Marie_woven_silk.jpg
     George Boole (1815-1864)
• Algebra of Propositional Calculus (1854)
(Motivated by Indian Logic)
• Could optimize the design of
systems of electromechanical
relays, then used in
telephone routing switches
(Shannon’s Master Thesis, 1937)

Theory of Electrical Switches
(Vicktor Shestakov, 1935)
Foundations of Digital Age
                          Earliest digital electronic
                          communications medium




                                                                                  Printing Telegraph 1846
                                                                                  Royal Earl House
                                                                                  http://upload.wikimedia.org/wikipedia/commons/2/24/Printing_Telegraph.jpg
http://upload.wikimedia.org/wikipedia/commons/1/13/Women_in_Waldorf-Astoria.jpg
Ticker Tape (1870-1970)To Communicate Stock Prices
                               Two Popular Telegraph
                                     Systems




 http://upload.wikimedia.org/wikipedia/commons/7/7d/Hughes_telegraph.jpg
                                                                           http://upload.wikimedia.org/wikipedia/commons/e/ed/Edison_Stock_Telegraph_Ticker.jpg


 A First Telegraph Printing Text                                           Gold and Stock Ticker
 David E. Hughes 1856                                                      Thomas Alva Edison 1869

The first stock ticker system using a telegraphic printer, Edward A. Calahan in 1863
                                                      Morse Code




                                                                                                  http://upload.wikimedia.org/wikipedia/commons/9/9c/J38TelegraphKey.jpg


                                                                                                              Telegraph Key WW-II



http://upload.wikimedia.org/wikipedia/en/thumb/5/5a/Morse_comparison.svg/1000px-Morse_comparison.svg.png
                                           Morse Code
                                                                                  Manual Telegraph Code




http://upload.wikimedia.org/wikipedia/commons/1/1f/International_Morse_Code.PNG
 Teleprinters (1902-1979)


                                                                                                                 http://upload.wikimedia.org/wikipedia/commons/7/76/ASR-33_1.jpg


                                                                                      A Teletype Model 33 ASR teleprinter 1963



    http://upload.wikimedia.org/wikipedia/commons/8/89/WACsOperateTeletype.jpg

Teletype Machines WWAR II 1939-1945




                                                                                                        http://upload.wikimedia.org/wikipedia/en/6/65/Goodtyping.jpg



                                            Model 15 (1930) Printing News
A British Creed & Company Teleprinter No. 7 in 1930
 http://upload.wikimedia.org/wikipedia/commons/5/50/Bundesarchiv_Bild_183-2008-0516-500%2C_Fernschreibmaschine_mit_Telefonanschluss.jpg
          Emile Baudot (1805-1903)

   Multiplexed printing telegraph system                                  1870,1874
The first electromagnetic telegraph 1833

by Carl Friedrich Gauss and Wilhelm Weber

They used 5 unit code




                                                                            http://upload.wikimedia.org/wikipedia/commons/7/79/Clavier_Baudot.jpg
                  http://www.measurement-valley.de/Gauss-Weber-Telegraf
Baudot Code-International
Telegraph Alphabet No. 1
US-ASCII code chart (1968)
                   Frank Gray Code (1953)




•   http://upload.wikimedia.org/wikipedia/commons/1/14/US02632058_Gray.png
          Major Telegraph Lines 1891




http://upload.wikimedia.org/wikipedia/commons/9/9a/1891_Telegraph_Lines.jpg
                         Major Communication Systems Before 1948


                     •   Telegraph (Morse, 1830’s)
                     •   Telephone (Bell, 1876)
                     •   Wireless Telegraph (Marconi, 1887)
                     •   AM Radio (early 1900’s)
                     •   Single-Sideband Modulation (Carson, 1922)
                     •   Television (1925-1927)
                     •   Teletype (1931)
                     •   Frequency Modulation (Armstrong, 1936)
                     •   Pulse-Code Modulation (PCM)(Reeves, 1937-1939)
                     •   Vocoder (Dudley, 1939)
                     •   Spread Spectrum (1940’s)

    The Morse Code
    Transmission bandwidth (FM,PCM & SS)
    PCM -first digital communication system -analog continuous-time signal
    Bandwidth used by Vocoder was less than the message bandwidth

1924, H. Nyquist- Transmission rate is proportional to log (# of signal levels)

    Jan 2012 SC461 Lecture 1.0                                                     http://www.guptalab.org
    Coding Theory and App           Laboratory of Natural Information Processing
                                              Bell Labs, NJ




                       http://upload.wikimedia.org/wikipedia/commons/8/87/Lucent_HQ.gif




Jan 2012 SC461 Lecture 1.0                                                                http://www.guptalab.org
Coding Theory and App                     Laboratory of Natural Information Processing
   Richard W. Hamming (1947)
• Model V - Bell Laboratories General Purpose Relay
  Calculator




     Model I
• When a machine can detect an error why it can not
  correct it ?
   Claude E. Shannon (1948)
• A Mathematical Theory of Communication

• Birth of Information Theory

• Information Theory
Answers Two Fundamental Questions !
Sets Bounds on …
Information Storage (Now to Then)
Information Communication (Here to There)
The Birth of Data Compression
• How much can you compress the digital
  digital information?
     (Answer: The Entropy H)
The Birth of Channel Capacity
• What is the ultimate transmission rate of
  digital communication ?
(Answer: The Capacity C)
• Fundamental Theorem: Given a noisy
  channel with channel capacity C and
  information transmitted at a rate R, then if
  R < C there exist codes that allow the
  probability of error at the receiver to be
  made arbitrarily small.
     Coding Theory -Types
• Source Coding (Data Compression)
• Channel Coding (Error Correction)




• Joint Source and Channel Coding
               Some Applications -- Coding Theory

    Space and Satellite Communications
    Data Transmission
    Data Storage
    Digital Audio/Video Transmission
    Mobile Communications
    Cloud Computing
Jan 2012 SC461 Lecture 1.0                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
                             Deep Space Communications




Jan 2012 SC461 Lecture 1.0                                                      http://www.guptalab.org
Coding Theory and App            Laboratory of Natural Information Processing
  Deep Space Communications
      and Coding Theory
• 1965 Mariener 4 spacecraft took first time 22 pictures of
  Mars
• 200 x 200
• 000000 white
• 111111 black
• 8-1/3 bits per sec.
• 8 hours 2 transmit
Deep Space Communications (2)
• Mariner 6,7 and 9 (1969-1972) Improved
  pictures of Mars obtained
• 700 x 832
• RM(1,5)
• (32,64,16)
• 6--> 32
• 26 xtra bits
• 16200 bits
 Message Received from Outer
        Space: Aliens
• 00110000011000111111110110010011001
  00110010111100010010001001000100100
  1100110
• Why we can say that these human like
  aliens have one arm twice as long as other?
        Power-line Communications
            IEEE 1901 (2010)
• Internet via Power-Plug at Home/Office
• HomePlug Powerline Alliance,
  Universal Powerline Association
• Smart Grid




http://upload.wikimedia.org/wikipedia/commons/e/ef/Stopcontact-modified.jpg
               Object Tracking and Physical Internet




Jan 2012 SC461 Lecture 1.0                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
                                          Bar Codes



http://upload.wikimedia.org/wikipedia/commons/2/28/EAN-13-ISBN-13.svg
Credit Cards and Coding Theory

                                                                          Error Detecting Codes




 http://upload.wikimedia.org/wikipedia/commons/7/70/Smartcard2.png

                                                                     Last Digit is check digit
                  (0)(124875)(36)(9)
                              Data Storage




Jan 2012 SC461 Lecture 1.0                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
   RAID and Coding Theory
• Redundant array of inexpensive
  disks (provides reliable data storage)
• RAID technology combines two or more
  physical hard disks into a single logical
  unit using special hardware or software
• Raid 2 uses Hamming codes
• Raid-n (1 < =n <=6)
CD/DVD, HD-DVD
    Other Interesting Applications
•   Coding Theory for Software Radio
•   Software Testing
•   Solving Many Mathematical Puzzles
•   Security / Ciphers / Cryptography
•   Cloud Computing
•   Networking
•   Natural Computing
•   Cell Phones
            LDPC Codes Applications
• G.hn/G.9960 (ITU-T Standard for
  networking over power lines, phone
  lines and coaxial cable)
• 802.3an (10 Giga-bit/s Ethernet over
  Twisted pair)
• CMMB(China Multimedia Mobile
  Broadcasting)
• DVB-S2 / DVB-T2 / DVB-C2 (Digital
  video broadcasting, 2nd Generation)
• DMB-T/H (Digital video broadcasting)
• WiMAX (IEEE 802.16e standard for
  microwave communications)
• IEEE 802.11n-2009 (Wi-Fi standard)
http://en.wikipedia.org/wiki/Low-density_parity-check_code#Applications_2
Coding Theory and Cooperative
    Game: The Hat Puzzle
  • Todd Ebert, 1998 What Color is my Hat ?




http://plus.maths.org/content/mathematical-mysteries-what-colour-my-hat
           How to Construct Codes?
    •   Combinatorial Mathematics
    •   Algebra
    •   Algorithms
    •   Applications
An ounce of algebra is worth of a ton of verbal argument !

                                                      J.B.S. Halden (1892-1964)
     Use redundancy in a clever way !
                                             http://upload.wikimedia.org/wikipedia/commons/3/3b/J._B._S._Haldane.jpg
                             Opportunities Available

  • Theoretical
  • Software Development




Jan 2012 SC461 Lecture 1.0                                                      http://www.guptalab.org
Coding Theory and App            Laboratory of Natural Information Processing
      Course Outline-1 (Total Lectures =15-1=14)
  Week                                                   Tentative Topics
  1 (Jan 2)                                              Historical Introduction and
                                                         Motivation of Coding and
                                                         Information Theory
  2 (Jan 9)                                              Basic Review of Finite Fields and
                                                         Finite Rings


  3 (Jan 16)                                             Introduction to Algebraic Coding
                                                         Theory, Codes over finite fields and
                                                         finite rings
  4 (Jan 23) Jan 26 Holiday                              Linear and Non-linear Codes,
                                                         Hamming Codes, Optimal Codes,
              Homework 1                                 Bounds on Codes


  5 (Jan 30)                                             Golay Codes, Cyclic, Quasi-cyclic


  6 (Feb 6)                                              Test 1 (Feb 6 to Feb 8)

Jan 2012 SC461 Lecture 1.0                                                         http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
            Course Outline-2 (Total Lectures =11)
 Week                                                    Tentative Topics

 7 (Feb 13)                                              Reed Muller, Reed Solomon Codes

 8 (Feb 20) Feb 20 Holiday                               BCH Codes, Quadratic Residue
                                                         Codes, Art of Decoding

 9 (Feb 27) Homework 2                                   Convolutional Codes, Turbo Codes

 10 (March 5) Holiday Week                               Holiday No Classes


 11 (March 12)                                           Low density parity check (LDPC)
                                                         codes


 12 (March 19) No Exam for us!                           Projects Presentations

 Test 2 Week (March 22 to
 March 24):
Jan 2012 SC461 Lecture 1.0                                                        http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
     Course Outline-3 (Total Lectures =12-1=11)
 Week                                                       Tentative Topics
 13 (March 26)                                              Applications of Coding Theory to
                                                            Networks (Cloud Computing:
                                                            Cloud Data Storage and Cloud
                                                            Security)
 14 (April 2) April 5 & 6 Holiday                           Cryptography



 15 (April 9) Homework 3                                    Wireless Communications

 Course Evaluation Week
 16 (April 16)                                              Quantum Computing, and DNA
                                                            computing

 17 (April 23)                                              Test 2 (April 23 to April 27)




Jan 2012 SC461 Lecture 1.0                                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing
                             Acknowledgments
  •   Microsoft Research India
  •   DA-IICT, Gandhinagar
  •   My undergraduate students
  •   Instead of redrawing some graphics used in the
      slides have been taken from public domain.
      Thanks to those who created it.




Jan 2012 SC461 Lecture 1.0                                                   http://www.guptalab.org
Coding Theory and App         Laboratory of Natural Information Processing
    “A message with content and clarity
     has gotten to be quite a rarity.
    To combat the terror of serious error, S.W. Golomb (1967)
    Use bits of appropriate parity.”

                             Any Questions ?



Jan 2012 SC461 Lecture 1.0                                                  http://www.guptalab.org
Coding Theory and App        Laboratory of Natural Information Processing

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:1
posted:11/17/2012
language:English
pages:74