LINUX File System • Layers in the file system • Basic principles • Representation of file systems in the kernel • Proc file system • Ext2 file system File System The file system organizes the data into directories and files. It also provides file i/o services to the application. It also handles the file protection when multiple users have access to files. We store information on several different storage media, such as magnetic disks, magnetic tapes and optical disks. The operating system provides a uniform logical view of information storage. Hard Disk Layout: The following diagram shows the physical hard disk layout. The first sector contains the MBR (Master Boot record). The MBR contains the boot code and partition information. There could be maximum four partitions. Example hard disk layout Boot code 1 2 Partition table 3 4 Partitions within an extended partition Boot Sector Partition 1 Partition 2 Partition 3 Partition 4 (Extended) The DOS file system is known as FAT file system. A FAT volume is divided into several regions, as shown below. The file allocation table, which gives the FAT file system format its name, has one entry for each cluster on a volume. Because the file allocation table is critical, it maintains a duplicate copy of FAT. Entries in the file allocation table define file allocation chains for files and directories, where the links in the chain are indexes to the next cluster of a file’s data. A file’s directory entry stores the starting cluster of the file. The last entry of the file’s allocation chain is the reserved value of 0xFFFF for FAT16 and 0xFFF for FAT12. The FAT entries for unused clusters have a value of 0. Boot File Allocation File Allocation Root Other directories Sector Table 1 Table 2 Directory and all files (Duplicate) FAT12’s 12-bit cluster identifier limits its partition to storing a maximum of 2**12 (4096) clusters. Windows uses cluster sizes from 512 bytes to 8KB, which limits a volume to 32MB. FAT16 with a 16bit cluster identifier, can address 2**16 (65536) clusters. The cluster sizes range from 512 bytes to 64KB, which limits volume to 4 GB. FAT32 uses 32 bit cluster identifiers. The windows limits volumes to a maximum of 32GB. Virtual File System There is large number of file systems supported by LINUX. The range of file systems supported is made possible by the unified interface to the LINUX kernel. This is the Virtual File System Switch (VFS). The Virtual file system supplies the applications with the system calls for the management, maintains internal structures and passes tasks onto the appropriate actual file system. The layers in the file system Process Process Process 1 2 … n User mode System mode Virtual File System ext2 msdos minix … proc Buffer cache File system Device drivers … Basic Principles Purposeful structuring of data along with two factors: speed of access and a facility for random access. In unix, the data are stored in a hierarchical file system containing files of different types. These comprise not only regular files and directories but also device files, FIFOs (named pipes), symbolic links and sockets. The management information contained includes access times, access rights and the allocation of data to blocks on the physical media. The inode contains a few block numbers to ensure efficient access to small files. Access to large files via indirect blocks, which also contain block numbers. Thus each node has a unique number. Directories allow the file system to be given a hierarchical structure. These are also implemented as files, but the kernel assumes them to contain pairs consisting of a filename and its node number Structure of a UNIX inode Access rights Data Owner Data Size Times Data … Data Data Data Direct references Data Data to data blocks Data Indirect block Data Two- step indirect reference Data Three-step indirect reference File Access Permissions Every file has the following attributes: Owner’s user ID, Owner’s group ID user-read permission, user-write permission, user-execute permission group-read permission, group-write permission, group-execute permission other-read permission, other-write permission, other-execute permission The representation of file system in the kernel Mounting: Before a file can be accessed, the file system containing the file must be mounted. This can be done using the system command mount. Superblock operations: The superblock structure provides, functions for accessing the file system. Inode: The inode structure holds the information on the file and remainder contains management information and the file system dependent union. File operations: The file_operations structure is the general interface for work on files, and contains the functions to open, close, read and write files. Schematic structure of a UNIX file system Boot block Superblock Inode blocks Data blocks 0 1 2… Ext2 File System • Ext2 file system added by Remy Card in 1993 • data held in files is kept in data blocks • block size is 1024 bytes • structure of Ex2 file system: block groups, data blocks close to their inodes and file nodes close to their directory node • Block Allocation: Target oriented allocation, and pre-allocation Structure of the Ext2 file system Boot block Block group 0 Block group 1 … Block group n Super Group Block Inode Inode block descriptors bitmap bitmap table Data blocks 1 Blk 1Blk 1Blk 1Blk n Blks n Blks The superblock in the ext2 filesystem Number of inodes, number of blocks Number of reserved blocks, number of free blocks Number of free inodes, first data block Block size Blocks per group Inodes per group Time of mounting Ext2 signature The block group descriptors in the Ext2 filesystem Block bitmap, inode bitmap Inode table, No.of free blocks/No. of free inodes No. of directories The inode in the Ext2 filesystem Type/Permissions, User(UID), File size Access time, time of creation Time of modification, time of deletion Group (GID), link counter, no. of blocks File Attributes, Direct blocks, one/two/three stage indirect blocks File ACL, Directory ACL Ext2 File Types File_type Description 0 Unknown 1 Regular file 2 Directory 3 Character Device 4 Block device 5 Named pipe 6 Socket 7 Symbolic link Creating the Filesystem Creating a filesystem means setting up the structures. Ext2 filesystems are created by mke2fs utility. Default options are: Block size: 1024 bytes Fragment size: block size Number of allocated inodes: one for each group of 4096 bytes Percentage of reserved blocks: 5% 1. Initializes the superblock and group descriptors 2. For each bock group, reserves all the disk blocks needed to store the superblock, group descriptors, inode table, bitmaps etc. 3. Initializes the inode and data bitmap 4. Initializes the inode table of each block group 5. Creates the / root directory 6. Creates the lost+found directory, which is used by e2fsck to link the lost and defective blocks. Ext3 Filesystem The enhanced filesystem evolved from Ext2, so is compatible with Ext2. The most significant features are: •Block fragmentation •Access Control Lists (ACL) •Handling of transparently compressed and encrypted files •Logical deletion •Journaling The Ext3 Journaling Filesystem The goal of a journaling filesystem is to avoid running time- consuming consistency checks on the whole filesystem by looking instead in a special disk area that contains the most recent disk w operations named journal. The Ext3 journaling is to perform any high-level change to the filesystem in two steps. First, a copy of the blocks to be written is stored in the journal; then, when the I/O data transfer to the journal is completed (in sh data is committed to the journal), the blocks are written in the filesystem. Proc file system • Provides info on the current status of Linux kernel and running processes • Each currently running process is assigned a directory /proc/pid • This dir contains the files holding info on characteristics of the process • The /proc file system holds the kernel variables for tuning or optimization Some of the frequently used files in /proc file system includes: /proc/modules For a list of modules /proc/meminfo For memory usage statistics /proc/pci For list of PCI devices detected and their configuration status /proc/ioports For I/O Port information /proc/iomem For information regarding Memory Ports /proc/interrupts For information related to interrupts I/O system calls • System calls for I/O are: open, read, write, close and fcntl. Include file is <fcntl.h> • int open(char *pathname, int oflag, [,int mode]); • int read(int filedes, char *buff, unsigned int nbytes); • int write(int filedes, char *buff, unsigned int nbytes); • int fcntl(int fieldes, int cmd, int arg); open System call A file is opened by #include <fcntl.h> int open(char *pathname, int oflag, [ , int mode ]); oflag argument can be O_RDONLY Open for reading only O_WRONLY Open for writing only O_RDWR Open for reading and writing O_NDELAY Do not block on open or read or write O_APPEND Append to end of file on each write O_CREAT Creat the file if it doesn't exist O_TRUNC If the file exists, truncate its length to zero O_EXCL Error if O_CREAT and the file already exists mode argument is optional and is required only if O_CREAT is specified. open returns a file descriptor if successful, otherwise -1 is returned. close System call An open file is closed by int close(int filedes); When process terminates, all open files are automatically closed by the kernel. read System call Data is read from an open file using int read(int filedes, char *buff, unsigned int nbytes); If the read is successful, the number of bytes read is returned. This can be less than nbytes that was requested. If the end of file is encountered, zero is returned. If an error is encountered, -1 is returned. write System call Data is written to an open file using int write(int filedes, char *buff, unsigned int nbytes); The actual number of bytes written is returned by the system call. This is usually equal to the nbytes argument. If an error occurs, -1 is returned. fcntl System call The fcntl system call is used to change the properties of a file that is already open. #include <fcntl.h> int fcntl(int fieldes, int cmd, int arg); The cmd argument must be one of the following: F_DUPED: Duplicate the file descripter. F_SETFD: Set the close-on-exec for the file to the low order bit arg. F_GETFD: Return the close_on_exec flag for the file as the value of the system call. F_SETFL: Set status flags for this file to the value of arg. F_GETFL: Return the status flags for this file as the value of the system call. F_GETOWN: Return the value of the process Id or the process group Id. F_SETOWN: Set the process Id or group Id. ioctl System call The ioctl system call is also used to change the behaviour of an open file. #include <sys/ioctl.h> int ioctl(int filedes, unsigned long request, char *arg); This system call performs a variety of control functions on terminals, devices, sockets and stream. The main difference between fcntl and ioctl is that former is intended for any open file, while the latter is intended for device specific operations.