Docstoc

File systems

Document Sample
File systems Powered By Docstoc
					File systems




               Introduction to file systems
               A brief review of the physical disk

                    Disk drive performance




                    D. Heffernan (c) 2009 University of   1
                                Limerick
Simplified diagram of the common moving-head magnetic disk drive




              D. Heffernan (c) 2009 University of                  2
                          Limerick
                                                platter
                                                the physical disk


                                                     track
                                                     a full circle of sectors



                                                      sector
                                                      an individual sector




A plan view of the disk to show the sector concept




          D. Heffernan (c) 2009 University of                                   3
                      Limerick
                            HEAD          HEAD ARM



                             Seeking




                                                 DISK PLATTER

                                        TRACK (cylinder)


                               BLOCK (sectors)




     Sector:     512 bytes (typical)

A little more realistic plan view of the disk



     D. Heffernan (c) 2009 University of                        4
                 Limerick
                                        track no. 0
                    track no.   5,000

      track no. 25,000




                                                      cylinder no. 0




Sector:       512 bytes (typical)
Track:        A ring of sectors
Cylinder:     A set of the same track numbers



            D. Heffernan (c) 2009 University of                        5
                        Limerick
Low-level format: manufacturer defines
High-level format: file system specific

                                                                                                 track




                                                                                           Show a track in a linear format



   Track
                             BOT                                                                                                              EOT
            Track


                                            sector        sector            sector   . . . . .                 sector    sector   sector



                    10     2           11                                                                                                        56
                                     GAP1
                    SYNC

                           IAM




                                                                                                                                               GAP4
                                                                                                                                           56 bytes of 00h




                                                                                                                                  All field sizes are
                                                                                                                                  shown in bytes.
  Sector
                 Sector          7          10     2      4    4     5        10     2        512        4        15
                                                                                     DAM
                                                               ECC




                                                                                                         ECC
                                 SPD




                                                                              SYNC
                                                          ID


                                                                     GAP2




                                                                                                                  GAP3
                                                   IDAM
                                            SYNC




                                                                                            DATA



                                       address region                                    data region



                                                     LOW LEVEL FORMAT

                                              D. Heffernan (c) 2009 University of                                                                            6
                                                          Limerick
  Low-level format          manufacturer defines
  High-level format         file system specific

Field   Bytes   Description
SYNC    10      10 bytes of 00h to allow PLL to lock on for read/write clock
IAM     2       Index Address Mark – flags beginning of a track
GAP1    11      Gap to allow for disk speed variations
SPD     7
SYNC    10      10 bytes of 00h to allow PLL to lock on for read/write clock
IDAM    2       ID Address Mark – marks start of address field
ID      4       Sector ID – CYL, HD, SECTOR (also flags a bad sector)
ECC     4       ECC for error control on the sector address field
GAP2    5       Gap to allow for disk speed variations
SYNC    10      10 bytes of 00h to allow PLL to lock on for read/write clock
DAM     2       Data Address Mark –marks start of sector data field
DATA    512     512 bytes of data
ECC     4       ECC for error control on the sector data field
GAP3    15      Gap to allow for disk speed variations
GAP4    56      EOT – End of Track mark




                LOW LEVEL FORMAT



                 D. Heffernan (c) 2009 University of                           7
                             Limerick
Formatted capacity - Sector example


 63 overhead bytes
 unformatted sector size here is (512 + 63) bytes
 formatted capacity of the sector is 512 bytes
 11% (63/575) of the sector cannot be used


Overhead for the total disk drive is greater with space used for
gaps, control data and bad sectors.



 Bad sector mapping

 Microscopic defects can mapped out as bad sectors




                   D. Heffernan (c) 2009 University of             8
                               Limerick
What are blocks and clusters?
Most file systems refer to this file allocation unit as a block
Microsoft prefers to use the term cluster.
A block (or cluster) is a fixed-sized unit of disk storage, whereby a
file consists of a fixed number of blocks.
A block’s sectors will be contiguous so that time is not wasted in
locating a block’s sectors.




      Example – 4 sector block 2kBytes

      sector         sector         sector        sector
     512 bytes      512 bytes      512 bytes     512 bytes



                    A single 2 KByte block




           D. Heffernan (c) 2009 University of                          9
                       Limerick
What size should the block (cluster) be?
Argument for a large block size
Reduces number of block searches, thus will optimise the speed performance


Argument for a small block size
Will optimise the available space on the disk drive, since on average half of the
final block is wasted.
This wasted of space within a block is referred to as internal fragmentation.




                          Example of wasted space

                                                                0.1 kB wasted
          file_X        block           block
       (3.9 kBytes)   2 kBytes        2 kBytes
                                                                 1.9 kB wasted

           file_Y       block           block          block
       (4.1 kBytes)   2 kBytes        2 kBytes       2 kBytes




                             D. Heffernan (c) 2009 University of                    10
                                         Limerick
                       Typical block sizes


Block size   No. of      Comment
             sectors
512 bytes        1       Considered small, wastes access time

  1 kBytes       2       A common size

  2 kBytes       4       A common size

  4 kBytes       8       A common size

  8 kBytes      16       A usable size, but on the large side.

 16 kBytes      32       Too large, wastes storage space

 32 kBytes      64       Far too large – found in some FAT file systems




                        D. Heffernan (c) 2009 University of               11
                                    Limerick
             Zone bit recording (ZBR)

             number of sectors per track is not a fixed number
             outer tracks on the disk are longer than the inner tracks
             zones towards the centre have less sectors per track



                                                    Zone 200 sectors per track


                                                      Zone 120 sectors per track


                                                        Zone 63 sectors per track




ZBR - conceptual representation, a real disk drive would have many more zones.

                      D. Heffernan (c) 2009 University of                           12
                                  Limerick
               Disk Drive Performance

MECHANICAL LATENCIES


Seek access latency (average seek access time)
Time it takes to position the read-write heads on the correct cylinder.


Rotational latency (average rotational latency)
This is half the time that it takes for a single disk revolution.


Assume the following performance specification for a disk drive:
Spindle rotational speed:                10,000 rpm
Average seek access time (ts ):          8 millisecs.




                             D. Heffernan (c) 2009 University of          13
                                         Limerick
Rotational latency and seek time latency



                                    SEEK TIME LATENCY




                                    HEAD          HEAD ARM



                                     Seeking




                                                         DISK PLATTER

                                                TRACK (cylinder)


                                       BLOCK (sectors)

           ROTATIONAL LATENCY




           D. Heffernan (c) 2009 University of                          14
                       Limerick
Latency calculation example:


Rotational delay, tr, is 6 millisecs., calculated as follows:
       10,000 rpm = (10,000 /60 ) rps (revs per second)
Time for a single rotation is thus: ( 1 / (10,000/60) ) = 6 millisecs.
Average rotational delay is half of one revolution, tr/2, or 3 millisecs.


Average latency (delay) to access a block is the sum of the seek average access time (ts ) and the average
rotational delay time (tr/2 ), thus: (ts + tr/2) = 11 millisecs.


Assume that there are 240 kBytes of data on one track and that the block size is 2 kBytes. Then the time
taken to transfer one block of data is approximately:

( 2k / 240k ) x 6 msecs. = 50 microsecs, or 0.05 milliseconds.




      11 millisecs. is time spent finding the block (latency)

                                                                                 0.05 msecs spent
                                                                                in transferring data




                                  D. Heffernan (c) 2009 University of                                        15
                                              Limerick
                        Example question


Spindle rotational speed of 10,000 rpm.


Track contains 1024 sectors, each sector is 512 bytes.


Calculate the raw read/write data transfer rate for a track.




                    D. Heffernan (c) 2009 University of        16
                                Limerick
Answer

Rotational delay, tr, is 6 millisecs., calculated as follows:

10,000 rpm = (10,000 /60 ) rps (revs per second)
Time for a single rotation is thus: ( 1 / (10,000/60) ) = 6 millisecs.

So – to read (or write) a single track takes 6 msecs.

A single track of 1024 sectors contains the following amount of data:

1024 x 512 bytes
= 210 x 29      = 219 bytes
= 219 x 23 bits         = 222 bits = 4 Mbits



So the raw read/write transfer data rate is:

4Mbits per revolution, or 4Mbits every 6 milliseconds.

There are 1000/6 revolutions per second = 166.66 rps.

The data rate is thus:   (166.66 x 4 Mbits) / sec.

666.7 Mbits/sec.




                                D. Heffernan (c) 2009 University of      17
                                            Limerick
                      Disk fragmentation




Non-fragmented file                                   Fragmented file
  (read in 6 msecs)                       (read in many hundreds of milliseconds)



                       D. Heffernan (c) 2009 University of                          18
                                   Limerick
                Disk head scheduling
     Various processes


       request (dest)
         Access disk data               Disk head
       release ()                       scheduler

                                            queue
                                                           DISK
request (dest)
  Access disk data
release ()



       request (dest)
         Access disk data
       release ()




                     D. Heffernan (c) 2009 University of          19
                                 Limerick
                     * thousands of cylinders
                       on a real hard disk



                        Cylinder                           disk
                      SEEK range*                          head
                                                                    Head
                                                                   actuator
                 0........................29




                                                                  FCFS:




                                     23
                                     24




                                                           29
                 10
                 11

                 13
                 14
                                     15
                                     16
                                     17
                                     18
                                     19
                                     20
                                     21




                                                           26
                                                           27
                                                           28
                 12




                                     22



                                                           25
0
1
2

4

6
7
8
9
3

5




                                                                  15 + 22 + 19 + 17 + 7
                                                  15
            22                                                    = 80 cylinders
                                             19
                              17
        7


                                                                  SSF:
                                     23
                                     24




                                                           29
                 10
                 11

                 13
                 14
                                     15
                                     16
                                     17
                                     18
                                     19
                                     20
                                     21




                                                           26
                                                           27
                                                           28
                 12




                                     22



                                                           25
0
1
2

4

6
7
8
9
3

5




                                                                  2 + 7 + 2 + 19 + 3
                 2
        7                                                         = 33 cylinders
2
                                19                     3




                                                                  Elevator algorithm:
                                     23
                                     24




                                                           29
                 10
                 11

                 13
                 14
                                     15
                                     16
                                     17
                                     18
                                     19
                                     20
                                     21




                                                           26
                                                           27
                                                           28
                 12




                                     22



                                                           25
0
1
2

4

6
7
8
9
3

5




                                                                  5 + 2 + 9 + 10 + 3
    5
2                                                                 = 29 cylinders
        9
                                       10
                                                       3



                                     D. Heffernan (c) 2009 University of                  20
                                                 Limerick
            Methods for allocation of disk space

Assume a very simplistic disk drive as shown where the disk contains just 20 blocks.


 On a real disk drive there could be hundreds of millions of blocks,
 e.g. a TeraByte disk drive could have approximately 250 million 4kByte blocks


 210 = 1024 bytes = 1kBytes (kilo)


 210 x 210 = 220 = 1MBytes (Mega)


 210 x 210 x 210 = 230 = 1GBytes (Giga)


 210 x 210 x 210 x 210 = 240 = 1TBytes (Tera)


 Q) How many sectors on a 1 TByte disk drive? How many 4kByte blocks?


 A)             (210 / 29 ) x ( 210 x 210 x 210 ) = 2 x 230 = 2 Giga sectors
                240 / 212 = 228 = (28 x 220 ) = 256 Mega blocks.



                                     D. Heffernan (c) 2009 University of               21
                                                 Limerick
Example 1: Contiguous allocation of blocks (clusters)




                                                        FILE         START   SIZE
         File_A
     0            1   2    3                            File_A         0      3
                                                        File_B         3      5
         File_B
                                                        File_C         8      3
     4            5   6    7


         File_C
     8            9   10   11



    12        13      14   15



    16        17      18   19




                               D. Heffernan (c) 2009 University of                  22
                                           Limerick
Example 2: Linked-list allocation of blocks (clusters)

                                                 File_C

                                                                       FILE    START
                                                 File_A               File_A     3
         0          1      2      3                                   File_B    11
                                                  File_B              File_C     1
         4          5      6      7



         8          9      10     11



        12         13      14     15



        16         17      18     19



       For File_A
     Directory says
     start at Block 3

                        Block 3        Block 9            Block 16

                                                               last block
                                                                (special
                                                                symbol)




                          D. Heffernan (c) 2009 University of                          23
                                      Limerick
                                                                FILE     START
          Example 3: FAT (File allocation table)
                                                                File_A     3
                                         Block    Pointer       File_B    11
                                                                File_C     1
                                         0        Free
                         File_C
                                         1        4
                                         2        Free
                         File_A          3        9
                                         4        12
0    1       2    3
                                         5        8
                          File_B
                                         6        Free
4    5       6    7
                                         7        Free
                                         8        14
8    9       10   11                     9        16
                                         10       Free

12   13      14   15
                                         11       5
                                         12       Last

                                         13       Bad
16   17      18   19
                                         14       17
                                         15       Free
                                         16       Last

                                         17       Last

                                         18       Free
                                         19       Free




                          D. Heffernan (c) 2009 University of                    24
                                      Limerick
         Microsoft FAT-16 File System



               A Linearised View of the Disk Partition




 Boot    1st FAT       2nd FAT           Root Dir               Data Area
sector
  1       A fixed      A fixed       A fixed number of   A large number of sectors
sector   number of    number of            sectors       – all the remaining space
          sectors      sectors                                on the partition.




                         D. Heffernan (c) 2009 University of                         25
                                     Limerick
     FAT - MS-DOS Directory entry structure


     32 byte structure
     Later added further 32-byte entries to support long names.


File name                                   8 bytes
Extension                                   3 bytes
Attributes                                  1 byte
Reserved                                    10 bytes
Time: creation or last change               2 bytes
Date: creation or last change               2 bytes
Start cluster for this file                 2 bytes
File size                                   4 bytes




                                D. Heffernan (c) 2009 University of   26
                                            Limerick
FAT - MS-DOS Directory entry structure


Maximum Disk Partition Size for a FAT-16


216 (which is 65,536 in decimal) entries .. or 64k entries.


For a 4kB cluster size:           64k times 4kByte = 256MBytes


For a 32kB cluster size:          64k times 32kByte = 2 GBytes


For a 64kB cluster size:          64k times 64kByte = 4 GBytes


Microsoft moved to the FAT-32 to allow more clusters!!




                      D. Heffernan (c) 2009 University of        27
                                  Limerick
Example 4: Indexed allocation of disk blocks

                                                            File_A
                                                               3
                                                              9
                                                             16
                                File_A
0    1      2     3                                         File_B
                                                             11
                               File_C                         5
4    5      6     7
                                                              8

                                File_B                       14
8    9     10    11
                                                             17

12   13    14    15
                                                            File_C
                                                              1
16   17    18    19
                                                              4
                                                             12




          The tables become too large very quickly!




                      D. Heffernan (c) 2009 University of            28
                                  Limerick
Example 5: Direct and Indirect indexed allocation of disk blocks
                                                            Directory entry
                                                            File_X        inode no.

                                                                i-nodes’s pointers
                                                                 Direct         3

                                        File_X                   Direct         5
    0    1     2     3
                                                                 Direct         6
                                       Indirect
                                                                 Direct         13
    4    5     6     7
                                                                 Indirect       15


    8    9     10    11                                         Block15 contains:
                                                                 Direct         8
   12    13    14    15                                          Direct         16

                                                                 Direct         17
   16    17    18    19




                          D. Heffernan (c) 2009 University of                         29
                                      Limerick
        Introduction to the i-node file system – classical UNIX!

Each and every file has an i-node (index node)
Each i-node has an unique number for identification
                                                                               mode
                                                                                 uid
                                                                                 gid




                                                        Attribute record
                                                                               atime
                Example i-node
                                                                               ctime
                structure                                                      mtime
                                                                               dtime
                                                                               nlinks
                                                                                 size
                                                                                flags
                                                                                 gen
                                                                            12 x direct




                                                        Block pointers
                                                                              pointers
                                                                                   .
                                                                                   .
                                                                                   .
                                                                           1 x si pointer
                                                                           1 x di pointer
                                                                           1 x ti pointer


                                D. Heffernan (c) 2009 University of                         30
                                            Limerick
Example descriptions of each i-node field. NOTE: this structure is 96 bytes long

         Field           bytes     Description

         mode            2         File type and the protection bits
          uid            2         User id of the file owner

          gid            2         Group id of the file owner

         atime           4         Time of last access to the file

         ctime           4         Time the file was last changed (inode modified)

        mtime            4         Time the file was last modified

         dtime           4         Time file was deleted

         nlinks          2         Number of links to this inode

          size           4         File size

         flags           4         Extended attributes

          gen            4         Generation number ( inode can be reused)

      12 x direct        48        12 x direct block pointers

  1 x single indirect    4         1 x indirect block pointer

  1 x doubleindirect     4         1 x double indirect block pointer

   1 x triple indirect   4         1 x triple indirect block pointer


                                 D. Heffernan (c) 2009 University of                 31
                                             Limerick
Example – pointing to disk blocks
                                                             block size is 1 KByte and pointer size is 32 bits (4 bytes).
                                 inode
                                       mode                   Thus each disk block can contain 256 (1024/4 ) pointers.
                                         uid
                                         gid                      d
                                       times




                                                                  ...
                                       nlinks                                 d
                                         size
                                                                  d
                                        flags




                                                                              ...
                                                                                                d
                                     12 x direct




                                                                                                ...
                                      pointers                                d
            Block pointers




                                           .
                                           .                                                    d
                                                                  256
                                           .
                                   Single indirect                            256

                                   Double indirect                                              d




                                                                                  ...
                                   Triple indirect
                                                                  256




                                                                                                    ...
                                                                               256
                                                                                                    d
                                                                                                              d




                                                                                                              ...
                                                                                                        256
                                                                  256             256
                                                                                                              d
                             KEY




                                                                                                        ...
                                                                                   ...
                             d      data block                                                                d
                                                                                                        256
                                                                                  256




                                                                                                               ...
                                    pointer
                                     block                                               etc.
                                    on disk
                                                                                                                  d



                                                 D. Heffernan (c) 2009 University of                                  32
                                                             Limerick
Question: A Unix file system is implemented using 1kByte (1024) disk
blocks and 32-bit size block pointers. The inode holds 12 direct block
pointers, one single-indirect block pointer, one double-indirect block
pointer, and one triple-indirect block pointer. Showing your
calculations:


What is the maximum file size for such a file system?
What is the maximum file system size?




                           D. Heffernan (c) 2009 University of           33
                                       Limerick
ANSWER:
What is the maximum file size?
Each block can hold (1k/4), i.e. 1024/4 pointers, i.e. 256 pointers.


For direct pointers, size pointed to is:             12 x 1kB = 12kB
For single indirect pointer, size pointed to is:     256 x 1kB = 256 kB
For double indirect pointer, size pointed to is:     28 x 28 x 1kB = 64 MB


For triple indirect pointer, size pointed to is:     28 x 28 x 28 x 1kB = 16 GB


So, max file size is: ( 16GB + 64MB + 256KB + 12kB) bytes.
Or, nominally 16GBytes


What is the maximum file system size?
For 32 bit pointers, the maximum number of pointers is:
232 = 4, 294,967,296
We will call this 4G, so the theoretical maximum file system size is:
4G x 1 kByte = 4 Terabytes
                                  D. Heffernan (c) 2009 University of             34
                                              Limerick
  Consider that the same file system formatted with 4kByte blocks


What is the maximum file size?
Each block can hold (4k/4), i.e. 4096/4 pointers, i.e. 1024 pointers.


For direct pointers, size pointed to is:              12 x 4kB = 48kB
For single indirect pointer, size pointed to is:      1024 x 4kB = 4MB
For double indirect pointer, size pointed to is:      210 x 210 x 4kB = 4GB
For triple indirect pointer, size pointed to is:      210 x 210 x 210 x 4kB = 4TB


So, max file size is: ( 4TB + 4GB + 4MB + 48kB) bytes. Or, nominally 4TBytes


What is the maximum file system size?
As in the earlier example, for 32 bit pointers, the maximum number of pointers is:
232 = 4, 294,967,296
We will call this 4G, so the theoretical maximum file system size is:
4G x 4kBytes = 16 Terabytes



                                           D. Heffernan (c) 2009 University of       35
                                                       Limerick
The UNIX file system – traditional!

Disk structure

                          i-node list                               Data blocks
   Boot    Super
   block   block   123




                    i-node


                    Attributes
                                   Points to specific
                                     data blocks
                    Pointers




                                                     i-node
                                                    number
                                                    (2 bytes)
           Directory entry
                                                                File name (14 bytes)


                                                                16 bytes




                                 D. Heffernan (c) 2009 University of                   36
                                             Limerick
                                                                                                              Data blocks
                                                          i-node list
Example file look up           Boot Super
                               block block 1 2 3

                                                                                        1         2          3                       n

/usr/roger/exam.doc                            root dir
                                                i-node

                                              Attributes
                                                                 Points to specific
                                                                data blocks starting
                                                                   at block 147
                                               Pointers




                                         Block 147
                                     start of root dir file
                                               /
                                     2       .
                                     2       ..
                                    24       bin                                                       i-node 6
                                    40       dev                                                      represents
                                                                                                         /user
                                    51       lib
                                    16       etc
                                                                                                   attributes
                                     6       usr
                                                                                                        d
                                                          Block 205
                                                        start of dir file                          205
                                                             /user                                 pointers
                                                    6         .
                                                    2         ..
                                                   26         edward                                            i-node 49
                                                   49         roger                                            represents
                                                                                                              /user/roger
                                                   62         john
                                                   94         don
                                                                                                            attributes
                                                                                                                 d
                                                                   Block 702
                                                                 start of dir file
                                                                  /user/roger
                                                                                                            702
                                                                                                            pointers
                                                                 49         .
                                                                  6         ..
                                                                 15         accounts
                                                                                                                       i-node 77
                                                                 21         students                                   represents
                                                                 32         lecturers                                  regular file
                                                                                                                       /user/roger/exam.doc
                                                                 77         exam.doc

                                                                                                                     attributes
                                                                                                                          -
                                                                                            Points to the blocks
                                                                                             of the desired file     987
                                                                exam.doc                                             pointers




                       D. Heffernan (c) 2009 University of                                                                                    37
                                   Limerick
                The Berkeley Fast File System

Basis for the well-known ext2 and ext3 and the UFS


Improvements over the traditional UNIX file system as follows:


• Long file names of variable length are supported
• Division of the disk into cylinder groups to improve performance
• Inclusion of redundant structures to enhance reliability
• Uses two block sizes to optimise space allocation and performance


-




                    D. Heffernan (c) 2009 University of               38
                                Limerick
Concept of Cylinder Groups

                                          platter
                                          the physical disk




                        7 6 5 4 3 2 1 0
                                          Cylinders are
                                           grouped in
                                           groups 0..7




    D. Heffernan (c) 2009 University of                       39
                Limerick
                           The ext2 file system

The ext2 is one of the most popular Linux files systems
Based on the Berkeley Fast File System.
The disk is divided into block groups, not cylinder groups
Each block groups has a redundant copy of the superblock and other structures




                           The ext3 file system

The ext3 file system supports the journaling feature and directory hash tree sorting;
otherwise it is similar to the ext2.




                             D. Heffernan (c) 2009 University of                        40
                                         Limerick
The ext2 layout of block groups


   First
data block
(reserved)



         Block group 0    Block group 1   Block group 2    Block group 3   etc ....




                         D. Heffernan (c) 2009 University of                          41
                                     Limerick
   The ext2 – structure of an individual block group


 Block group     Block group   Block group    Block group
                                                            etc ....
      0               1             2              3




Backup   Group     Block   i-node      i-node list
 Super   Descr    Bitmap   Bitmap                                      Data blocks
 Block   Table




                               D. Heffernan (c) 2009 University of                   42
                                           Limerick
The ext2 – fields within a block group

                                  Superblock
  The superblock is similar to the traditional UNIX superblock.
  Describes:
  • number of inodes,
  • number of disk blocks,
  • block size,
  • number of free blocks,
  • number of free inodes,
  • number of first inode etc.


  Other information is contained in the superblock:
  • volume name,
  • the latest write time,
  • last mount time etc.


                             D. Heffernan (c) 2009 University of   43
                                         Limerick
            A Group Descriptor


A structure used to define an individual disk block

Group descriptor – 32 bytes in size.
 Starting block address for the block bitmap     (4 bytes)

 Starting block address for the inode bitmap     (4 bytes)

 Starting block address for the inode table      (4 bytes)

 Number of unallocated blocks in the block group (2 bytes)

 Number of unallocated inodes in the block group (2 bytes)

 Number of directories in the block group        (2 bytes)

 Not used                                        (14 bytes)




                   D. Heffernan (c) 2009 University of        44
                               Limerick
          A Group Descriptor Table


Each block group has a group descriptor
A group descriptor table is a list of all the group descriptors
Each block group contains a group descriptor table
All group descriptor tables contain current up-to-date information
If a group descriptor becomes corrupted for a block group -
there are many spare copies available.




                D. Heffernan (c) 2009 University of                  45
                            Limerick
A block bitmap
Single block in size - used to map the free blocks
Each bit in the block bitmap represents a block within the block group
The bit status indicates whether or not that block is allocated




A inode bitmap
Single block in size - used to map the free inodes
Each bit in the block bitmap represents an inode within the block group
The bit status indicates whether or not that block is allocated




                           D. Heffernan (c) 2009 University of            46
                                       Limerick
Question


 An ext2 file system is formatted with a 4kB block size. What is the maximum
size of a single block group for that file system?
 Answer: Since the block bitmap is a single block in size, such a bitmap can hold
(4k x 8) bits, i.e. 32k bits. Since each bit represents a 4kB block, then the block
group size can be a maximum size of: 32k x 4 kB = 128 MB.




                          D. Heffernan (c) 2009 University of                         47
                                      Limerick
Directory entries
The Berkeley FFS introduced long file names, of variable size.
A name length up to 255 characters was required.
Directories could be read without knowledge of their internal structure.
Used system calls opendir, closedir, readdir and rewinddir.
The scheme was adopted by POSIX.



          The ext2 / ext3 directory entry structure

              i-node                    (4 bytes)
              Size of this entry size   (2 bytes)
              Length of file name       (1 byte)
              File type                 (1 byte)
              Actual file name      (variable bytes)




                               D. Heffernan (c) 2009 University of         48
                                           Limerick
Example directory entries




                                              file name length
                       entry size




                                                                                         file name
                                                                      file type
     i-node



           125                                   4                     f           test                  97        8    f   Limerick   119           7    f     Shannon



                         directory entry                                                                       directory entry                directory entry



Example directory entries – with one entry deleted
                                    file name length
                 entry size




                                                                                    file name
                                                                 file type
  i-node




    125                                4 f                                        test                       NOT NOW USED              119         7 f         Shannon



                                                                                           directory entry                                   directory entry




                                                                                                     D. Heffernan (c) 2009 University of                                  49
                                                                                                                 Limerick
File sharing
                                            /



                  proc                    home                 sbin           X11R6




                      ann        bob                     don




                         docs                   work           play


                         link


   Links (hard links)
   Number of links to a file is registered in the file’s inode’s link count field, nlinks
   There is a single inode for the file work, there is always only a single inode for a file.
   Symbolic links
   . special file of type link is created. Contains only the path name of the target file.
   A
   Here, only the file’s owner has a pointer to the file’s inode.


                                D. Heffernan (c) 2009 University of                             50
                                            Limerick
Open files on a file system

 A process’ file descriptor is used to identify an open file.


 A copy of the file’s inode is placed in the active inode table.


 Each process has a single file descriptor table to list its open files.


 An entry in the file descriptor table points to a structure in the open file table,
 which in turn points to the file’s inode in the active inode table.


 The open file table’s entry holds status information for the file, relating to the process
 which opened the file: file position, the read/write permissions, etc.


 If two separate processes open the same file, each process will have its own
 structure in the open file table.




                                D. Heffernan (c) 2009 University of                           51
                                            Limerick
     File descriptors showing two processes acting on the same open file
     Process_A creates a child Process_B, which inherits the open file status of its parent.


                  File                    Open
               descriptor                                        Active i-node                  Physical
                                        file table                   table                        disk
                 tables




Process_A's    0                                                                         of
    file                            position (offset)                                copy 0      i-node
               1
                                                                                    inode 44      table
 descriptor                            r/w status
               2
   table                             inode pointer
               3                                                     Copy of
               4                                                   inode #440



                                                                                      p
                                     position (offset)                              to oint
                                                                                      blo er
                                        r/w status                                       ck s
 Process_B's
                                                                                            s     disk
               0                      inode pointer
     file      1                                                                                  data
  descriptor                                                                                     blocks
               2
    table
               3
               4




                                        D. Heffernan (c) 2009 University of                                52
                                                    Limerick
        File descriptors showing a child process inheriting a file
        Process_A creates a child Process_X, which inherits the open file status of its parent.
                  File                     Open
               descriptor                                         Active i-node                   Physical
                                         file table                   table                         disk
                 tables




Process_A's    0                                                                           of
    file                             position (offset)                                copy    0
                                                                                                   i-node
               1
                                                                                     inode 44       table
 descriptor                             r/w status
               2
   table                              inode pointer
               3                                                      Copy of
               4                                                    inode #440



                                                                                        p
Process_X's    0                      position (offset)                               to oint
                                                                                        blo ers
    file       1                         r/w status                                        ck
                                                                                              s     disk
 descriptor    2                       inode pointer
   table                                                                                            data
  child of     3                                                                                   blocks
Process_A      4




 Process_B's   0
     file      1
  descriptor   2
    table
               3
               4




                                         D. Heffernan (c) 2009 University of                                 53
                                                     Limerick
Virtual File System - VFS

Process_The Linux VFS supports many different types of file systems including:
ext2, ext3, ext4,
ReiserFS,
UFS, UFS2,
MINIX,
MSDOS, FAT, FAT-32, NTFS,
HFS, HFS+ (Apple Mac),
ISO9660 (CD-ROM),
UDF (DVD).


Network file systems are also supported e.g.:
NFS, AFS etc.
Some special file systems are also supported e.g.:
Linux /proc file system, fuse (file system in user space) and the sysfs.




                               D. Heffernan (c) 2009 University of               54
                                           Limerick
Virtual File System - VFS

A virtual file system for UNIX was introduced in 1985 by Sun Microsystems,
others adopted the concept – including Linux community.


Abstraction layer allows access to different types of file systems


VFS supports file systems, such as ext3, NTFS, FAT-32, MINIX, MSDOS etc.


A common interface integrates the different file systems into a common file system.


VFS architecture (‘switch’) emulates a conventional UNIX file system, using a superblock
and a file is described by an inode etc.


Each type of file system has a dependent translator within the kernel - each read, write, or other
function, is substituted by an equivalent function to support the target native file system.




                               D. Heffernan (c) 2009 University of                                   55
                                           Limerick
                     Overview of the VFS architecture

For simplicity, the diagram above shows a single file system on each physical disk drive



                                 File Manager System Call
                                         Interface
         VFS:
         Common                        Translators                                       inode cache
         file system
                                                                                          directory
              ext3         NTFS            Reiser      FAT-32        ISO9660
                                                                                            cache




                                       Buffer cache


     Device             Device             Device           Device             Device
     driver             driver             driver           driver             driver




      ext3               NTFS              Reiser           FAT-32             ISO9660
       file               file               file             file                file
     system             system             system           system              system


                                         D. Heffernan (c) 2009 University of                           56
                                                     Limerick
VFS mount example




                                               /
                ext3
            file system
                                 a


                                 x                           b


                                                         y       z
             NTFS
         file system
                                             CD-ROM
                                             ISO9660
                       c                   file system
                                                                     d




                           D. Heffernan (c) 2009 University of           57
                                       Limerick
VFS mount example


                                                 /
                  ext3
              file system
                                   a


                                   x                         b


                                                         y         z
               NTFS
           file system

                                             CD-ROM
                                             ISO9660
                         c                 file system                 d




                             D. Heffernan (c) 2009 University of           58
                                         Limerick
Three types of file systems blend into a single common file system

  VFS - a common file system


                                     ext3
                                 file system
        Mounted                                   /
          NTFS
      file system
                                  c



                                                                d




                                        Mounted
                                        ISO9660
                                      file system




                          D. Heffernan (c) 2009 University of        59
                                      Limerick
      VFS objects for an open file, showing two processes accessing separate files


                                  Open                                    Active i-node
              File
           descriptor           file table                                    table
             tables
                                                           dentry
                                   file                                                    inode
                                                           objects
         These are entries       objects                                                  objects
          in the process'
                                                        (struct dentry)
            task_struct        (struct file)                                          (struct inode)
              structure
                                                          d_inode                         i_dentry
Process_A's                                              d_parent                          i_count
    file    0                      f_dentry
                                                        d_child (list)                     i_atime
 descriptor 1                f_mode (permissions)
                                                           d_op                           i_mtime
   table    2                   f_pos (position)
                                                                                             i_op
                                    f_count
            3                                                                               i_nlink
                                      f_op
            4

                                                           d_inode                        i_dentry
                                                          d_parent                         i_count
Process_Z's                                              d_child (list)
                                   f_dentry                                                i_atime
    file    0                                               d_op
 descriptor 1                f_mode (permissions)                                         i_mtime
   table                        f_pos (position)                                             i_op
            2                       f_count                                                 i_nlink
           3                          f_op
           4




                             D. Heffernan (c) 2009 University of                                       60
                                         Limerick
     VFS objects for an open file, showing two processes accessing the same file

                                   Open                                    Active i-node
              File
           descriptor            file table                                    table
             tables
                                                             dentry
                                   file                                                     inode
                                                             objects
         These are entries       objects                                                   objects
          in the process'
                                                         (struct dentry)
            task_struct        (struct file)                                           (struct inode)
              structure
                                                            d_inode                        i_dentry
Process_A's                                                d_parent                         i_count
    file    0                      f_dentry
                                                          d_child (list)                    i_atime
 descriptor 1                f_mode (permissions)
                                                             d_op                          i_mtime
   table    2                   f_pos (position)
                                                                                              i_op
                                    f_count
            3                                                                                i_nlink
                                      f_op
            4

                                                            d_inode
                                                           d_parent
Process_Z's                                               d_child (list)
                                   f_dentry
    file    0                                                d_op
 descriptor 1                f_mode (permissions)
   table                        f_pos (position)
            2                       f_count
           3                          f_op
           4




                             D. Heffernan (c) 2009 University of                                        61
                                         Limerick
           Summary of the key structures in the VFS


suberblock object,
inode object,
file object,
dentry object.


Structures have the characteristics of objects as they enclose a set of functions
within the structure.
Linux documentation refers to such structures as objects, and refers to the
functions as methods.




                         D. Heffernan (c) 2009 University of                        62
                                     Limerick
The superblock, struct super_block, is represented as follows:


struct super_block {
              ……
              struct file_system_type       *s_type;
              struct super_operations       *s_op;
              ……
};


Methods: the super_operations structure (s_op) contains operations that operate on the superblock
and inodes, as follows:
struct super_operations {
              void (*read_inode) (struct inode *);
              int (*notify_change) (struct inode *, struct iattr *);
              void (*write_inode) (struct inode *);
              void (*put_inode) (struct inode *);
              void (*put_super) (struct super_block *);
              void (*write_super) (struct super_block *);
              void (*statfs) (struct super_block *, struct statfs *, int);
              int (*remount_fs) (struct super_block *, int *, char *);
};
                               D. Heffernan (c) 2009 University of                                  63
                                           Limerick

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:7
posted:11/1/2011
language:English
pages:63