diff options
Diffstat (limited to 'Documentation/filesystems')
| -rw-r--r-- | Documentation/filesystems/files.txt | 123 | ||||
| -rw-r--r-- | Documentation/filesystems/fuse.txt | 315 | ||||
| -rw-r--r-- | Documentation/filesystems/proc.txt | 42 | ||||
| -rw-r--r-- | Documentation/filesystems/v9fs.txt | 95 | ||||
| -rw-r--r-- | Documentation/filesystems/vfs.txt | 435 |
5 files changed, 888 insertions, 122 deletions
diff --git a/Documentation/filesystems/files.txt b/Documentation/filesystems/files.txt new file mode 100644 index 00000000000..8c206f4e025 --- /dev/null +++ b/Documentation/filesystems/files.txt @@ -0,0 +1,123 @@ +File management in the Linux kernel +----------------------------------- + +This document describes how locking for files (struct file) +and file descriptor table (struct files) works. + +Up until 2.6.12, the file descriptor table has been protected +with a lock (files->file_lock) and reference count (files->count). +->file_lock protected accesses to all the file related fields +of the table. ->count was used for sharing the file descriptor +table between tasks cloned with CLONE_FILES flag. Typically +this would be the case for posix threads. As with the common +refcounting model in the kernel, the last task doing +a put_files_struct() frees the file descriptor (fd) table. +The files (struct file) themselves are protected using +reference count (->f_count). + +In the new lock-free model of file descriptor management, +the reference counting is similar, but the locking is +based on RCU. The file descriptor table contains multiple +elements - the fd sets (open_fds and close_on_exec, the +array of file pointers, the sizes of the sets and the array +etc.). In order for the updates to appear atomic to +a lock-free reader, all the elements of the file descriptor +table are in a separate structure - struct fdtable. +files_struct contains a pointer to struct fdtable through +which the actual fd table is accessed. Initially the +fdtable is embedded in files_struct itself. On a subsequent +expansion of fdtable, a new fdtable structure is allocated +and files->fdtab points to the new structure. The fdtable +structure is freed with RCU and lock-free readers either +see the old fdtable or the new fdtable making the update +appear atomic. Here are the locking rules for +the fdtable structure - + +1. All references to the fdtable must be done through + the files_fdtable() macro : + + struct fdtable *fdt; + + rcu_read_lock(); + + fdt = files_fdtable(files); + .... + if (n <= fdt->max_fds) + .... + ... + rcu_read_unlock(); + + files_fdtable() uses rcu_dereference() macro which takes care of + the memory barrier requirements for lock-free dereference. + The fdtable pointer must be read within the read-side + critical section. + +2. Reading of the fdtable as described above must be protected + by rcu_read_lock()/rcu_read_unlock(). + +3. For any update to the the fd table, files->file_lock must + be held. + +4. To look up the file structure given an fd, a reader + must use either fcheck() or fcheck_files() APIs. These + take care of barrier requirements due to lock-free lookup. + An example : + + struct file *file; + + rcu_read_lock(); + file = fcheck(fd); + if (file) { + ... + } + .... + rcu_read_unlock(); + +5. Handling of the file structures is special. Since the look-up + of the fd (fget()/fget_light()) are lock-free, it is possible + that look-up may race with the last put() operation on the + file structure. This is avoided using the rcuref APIs + on ->f_count : + + rcu_read_lock(); + file = fcheck_files(files, fd); + if (file) { + if (rcuref_inc_lf(&file->f_count)) + *fput_needed = 1; + else + /* Didn't get the reference, someone's freed */ + file = NULL; + } + rcu_read_unlock(); + .... + return file; + + rcuref_inc_lf() detects if refcounts is already zero or + goes to zero during increment. If it does, we fail + fget()/fget_light(). + +6. Since both fdtable and file structures can be looked up + lock-free, they must be installed using rcu_assign_pointer() + API. If they are looked up lock-free, rcu_dereference() + must be used. However it is advisable to use files_fdtable() + and fcheck()/fcheck_files() which take care of these issues. + +7. While updating, the fdtable pointer must be looked up while + holding files->file_lock. If ->file_lock is dropped, then + another thread expand the files thereby creating a new + fdtable and making the earlier fdtable pointer stale. + For example : + + spin_lock(&files->file_lock); + fd = locate_fd(files, file, start); + if (fd >= 0) { + /* locate_fd() may have expanded fdtable, load the ptr */ + fdt = files_fdtable(files); + FD_SET(fd, fdt->open_fds); + FD_CLR(fd, fdt->close_on_exec); + spin_unlock(&files->file_lock); + ..... + + Since locate_fd() can drop ->file_lock (and reacquire ->file_lock), + the fdtable pointer (fdt) must be loaded after locate_fd(). + diff --git a/Documentation/filesystems/fuse.txt b/Documentation/filesystems/fuse.txt new file mode 100644 index 00000000000..6b5741e651a --- /dev/null +++ b/Documentation/filesystems/fuse.txt @@ -0,0 +1,315 @@ +Definitions +~~~~~~~~~~~ + +Userspace filesystem: + + A filesystem in which data and metadata are provided by an ordinary + userspace process. The filesystem can be accessed normally through + the kernel interface. + +Filesystem daemon: + + The process(es) providing the data and metadata of the filesystem. + +Non-privileged mount (or user mount): + + A userspace filesystem mounted by a non-privileged (non-root) user. + The filesystem daemon is running with the privileges of the mounting + user. NOTE: this is not the same as mounts allowed with the "user" + option in /etc/fstab, which is not discussed here. + +Mount owner: + + The user who does the mounting. + +User: + + The user who is performing filesystem operations. + +What is FUSE? +~~~~~~~~~~~~~ + +FUSE is a userspace filesystem framework. It consists of a kernel +module (fuse.ko), a userspace library (libfuse.*) and a mount utility +(fusermount). + +One of the most important features of FUSE is allowing secure, +non-privileged mounts. This opens up new possibilities for the use of +filesystems. A good example is sshfs: a secure network filesystem +using the sftp protocol. + +The userspace library and utilities are available from the FUSE +homepage: + + http://fuse.sourceforge.net/ + +Mount options +~~~~~~~~~~~~~ + +'fd=N' + + The file descriptor to use for communication between the userspace + filesystem and the kernel. The file descriptor must have been + obtained by opening the FUSE device ('/dev/fuse'). + +'rootmode=M' + + The file mode of the filesystem's root in octal representation. + +'user_id=N' + + The numeric user id of the mount owner. + +'group_id=N' + + The numeric group id of the mount owner. + +'default_permissions' + + By default FUSE doesn't check file access permissions, the + filesystem is free to implement it's access policy or leave it to + the underlying file access mechanism (e.g. in case of network + filesystems). This option enables permission checking, restricting + access based on file mode. This is option is usually useful + together with the 'allow_other' mount option. + +'allow_other' + + This option overrides the security measure restricting file access + to the user mounting the filesystem. This option is by default only + allowed to root, but this restriction can be removed with a + (userspace) configuration option. + +'max_read=N' + + With this option the maximum size of read operations can be set. + The default is infinite. Note that the size of read requests is + limited anyway to 32 pages (which is 128kbyte on i386). + +How do non-privileged mounts work? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Since the mount() system call is a privileged operation, a helper +program (fusermount) is needed, which is installed setuid root. + +The implication of providing non-privileged mounts is that the mount +owner must not be able to use this capability to compromise the +system. Obvious requirements arising from this are: + + A) mount owner should not be able to get elevated privileges with the + help of the mounted filesystem + + B) mount owner should not get illegitimate access to information from + other users' and the super user's processes + + C) mount owner should not be able to induce undesired behavior in + other users' or the super user's processes + +How are requirements fulfilled? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + A) The mount owner could gain elevated privileges by either: + + 1) creating a filesystem containing a device file, then opening + this device + + 2) creating a filesystem containing a suid or sgid application, + then executing this application + + The solution is not to allow opening device files and ignore + setuid and setgid bits when executing programs. To ensure this + fusermount always adds "nosuid" and "nodev" to the mount options + for non-privileged mounts. + + B) If another user is accessing files or directories in the + filesystem, the filesystem daemon serving requests can record the + exact sequence and timing of operations performed. This + information is otherwise inaccessible to the mount owner, so this + counts as an information leak. + + The solution to this problem will be presented in point 2) of C). + + C) There are several ways in which the mount owner can induce + undesired behavior in other users' processes, such as: + + 1) mounting a filesystem over a file or directory which the mount + owner could otherwise not be able to modify (or could only + make limited modifications). + + This is solved in fusermount, by checking the access + permissions on the mountpoint and only allowing the mount if + the mount owner can do unlimited modification (has write + access to the mountpoint, and mountpoint is not a "sticky" + directory) + + 2) Even if 1) is solved the mount owner can change the behavior + of other users' processes. + + i) It can slow down or indefinitely delay the execution of a + filesystem operation creating a DoS against the user or the + whole system. For example a suid application locking a + system file, and then accessing a file on the mount owner's + filesystem could be stopped, and thus causing the system + file to be locked forever. + + ii) It can present files or directories of unlimited length, or + directory structures of unlimited depth, possibly causing a + system process to eat up diskspace, memory or other + resources, again causing DoS. + + The solution to this as well as B) is not to allow processes + to access the filesystem, which could otherwise not be + monitored or manipulated by the mount owner. Since if the + mount owner can ptrace a process, it can do all of the above + without using a FUSE mount, the same criteria as used in + ptrace can be used to check if a process is allowed to access + the filesystem or not. + + Note that the ptrace check is not strictly necessary to + prevent B/2/i, it is enough to check if mount owner has enough + privilege to send signal to the process accessing the + filesystem, since SIGSTOP can be used to get a similar effect. + +I think these limitations are unacceptable? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If a sysadmin trusts the users enough, or can ensure through other +measures, that system processes will never enter non-privileged +mounts, it can relax the last limitation with a "user_allow_other" +config option. If this config option is set, the mounting user can +add the "allow_other" mount option which disables the check for other +users' processes. + +Kernel - userspace interface +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following diagram shows how a filesystem operation (in this +example unlink) is performed in FUSE. + +NOTE: everything in this description is greatly simplified + + | "rm /mnt/fuse/file" | FUSE filesystem daemon + | | + | | >sys_read() + | | >fuse_dev_read() + | | >request_wait() + | | [sleep on fc->waitq] + | | + | >sys_unlink() | + | >fuse_unlink() | + | [get request from | + | fc->unused_list] | + | >request_send() | + | [queue req on fc->pending] | + | [wake up fc->waitq] | [woken up] + | >request_wait_answer() | + | [sleep on req->waitq] | + | | <request_wait() + | | [remove req from fc->pending] + | | [copy req to read buffer] + | | [add req to fc->processing] + | | <fuse_dev_read() + | | <sys_read() + | | + | | [perform unlink] + | | + | | >sys_write() + | | >fuse_dev_write() + | | [look up req in fc->processing] + | | [remove from fc->processing] + | | [copy write buffer to req] + | [woken up] | [wake up req->waitq] + | | <fuse_dev_write() + | | <sys_write() + | <request_wait_answer() | + | <request_send() | + | [add request to | + | fc->unused_list] | + | <fuse_unlink() | + | <sys_unlink() | + +There are a couple of ways in which to deadlock a FUSE filesystem. +Since we are talking about unprivileged userspace programs, +something must be done about these. + +Scenario 1 - Simple deadlock +----------------------------- + + | "rm /mnt/fuse/file" | FUSE filesystem daemon + | | + | >sys_unlink("/mnt/fuse/file") | + | [acquire inode semaphore | + | for "file"] | + | >fuse_unlink() | + | [sleep on req->waitq] | + | | <sys_read() + | | >sys_unlink("/mnt/fuse/file") + | | [acquire inode semaphore + | | for "file"] + | | *DEADLOCK* + +The solution for this is to allow requests to be interrupted while +they are in userspace: + + | [interrupted by signal] | + | <fuse_unlink() | + | [release semaphore] | [semaphore acquired] + | <sys_unlink() | + | | >fuse_unlink() + | | [queue req on fc->pending] + | | [wake up fc->waitq] + | | [sleep on req->waitq] + +If the filesystem daemon was single threaded, this will stop here, +since there's no other thread to dequeue and execute the request. +In this case the solution is to kill the FUSE daemon as well. If +there are multiple serving threads, you just have to kill them as +long as any remain. + +Moral: a filesystem which deadlocks, can soon find itself dead. + +Scenario 2 - Tricky deadlock +---------------------------- + +This one needs a carefully crafted filesystem. It's a variation on +the above, only the call back to the filesystem is not explicit, +but is caused by a pagefault. + + | Kamikaze filesystem thread 1 | Kamikaze filesystem thread 2 + | | + | [fd = open("/mnt/fuse/file")] | [request served normally] + | [mmap fd to 'addr'] | + | [close fd] | [FLUSH triggers 'magic' flag] + | [read a byte from addr] | + | >do_page_fault() | + | [find or create page] | + | [lock page] | + | >fuse_readpage() | + | [queue READ request] | + | [sleep on req->waitq] | + | | [read request to buffer] + | | [create reply header before addr] + | | >sys_write(addr - headerlength) + | | >fuse_dev_write() + | | [look up req in fc->processing] + | | [remove from fc->processing] + | | [copy write buffer to req] + | | >do_page_fault() + | | [find or create page] + | | [lock page] + | | * DEADLOCK * + +Solution is again to let the the request be interrupted (not +elaborated further). + +An additional problem is that while the write buffer is being +copied to the request, the request must not be interrupted. This +is because the destination address of the copy may not be valid +after the request is interrupted. + +This is solved with doing the copy atomically, and allowing +interruption while the page(s) belonging to the write buffer are +faulted with get_user_pages(). The 'req->locked' flag indicates +when the copy is taking place, and interruption is delayed until +this flag is unset. + diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt index 5024ba7a592..d4773565ea2 100644 --- a/Documentation/filesystems/proc.txt +++ b/Documentation/filesystems/proc.txt @@ -1241,16 +1241,38 @@ swap-intensive. overcommit_memory ----------------- -This file contains one value. The following algorithm is used to decide if -there's enough memory: if the value of overcommit_memory is positive, then -there's always enough memory. This is a useful feature, since programs often -malloc() huge amounts of memory 'just in case', while they only use a small -part of it. Leaving this value at 0 will lead to the failure of such a huge -malloc(), when in fact the system has enough memory for the program to run. - -On the other hand, enabling this feature can cause you to run out of memory -and thrash the system to death, so large and/or important servers will want to -set this value to 0. +Controls overcommit of system memory, possibly allowing processes +to allocate (but not use) more memory than is actually available. + + +0 - Heuristic overcommit handling. Obvious overcommits of + address space are refused. Used for a typical system. It + ensures a seriously wild allocation fails while allowing + overcommit to reduce swap usage. root is allowed to + allocate slighly more memory in this mode. This is the + default. + +1 - Always overcommit. Appropriate for some scientific + applications. + +2 - Don't overcommit. The total address space commit + for the system is not permitted to exceed swap plus a + configurable percentage (default is 50) of physical RAM. + Depending on the percentage you use, in most situations + this means a process will not be killed while attempting + to use already-allocated memory but will receive errors + on memory allocation as appropriate. + +overcommit_ratio +---------------- + +Percentage of physical memory size to include in overcommit calculations +(see above.) + +Memory allocation limit = swapspace + physmem * (overcommit_ratio / 100) + + swapspace = total size of all swap areas + physmem = size of physical memory in system nr_hugepages and hugetlb_shm_group ---------------------------------- diff --git a/Documentation/filesystems/v9fs.txt b/Documentation/filesystems/v9fs.txt new file mode 100644 index 00000000000..4e92feb6b50 --- /dev/null +++ b/Documentation/filesystems/v9fs.txt @@ -0,0 +1,95 @@ + V9FS: 9P2000 for Linux + ====================== + +ABOUT +===== + +v9fs is a Unix implementation of the Plan 9 9p remote filesystem protocol. + +This software was originally developed by Ron Minnich <rminnich@lanl.gov> +and Maya Gokhale <maya@lanl.gov>. Additional development by Greg Watson +<gwatson@lanl.gov> and most recently Eric Van Hensbergen +<ericvh@gmail.com> and Latchesar Ionkov <lucho@ionkov.net>. + +USAGE +===== + +For remote file server: + + mount -t 9P 10.10.1.2 /mnt/9 + +For Plan 9 From User Space applications (http://swtch.com/plan9) + + mount -t 9P `namespace`/acme /mnt/9 -o proto=unix,name=$USER + +OPTIONS +======= + + proto=name select an alternative transport. Valid options are + currently: + unix - specifying a named pipe mount point + tcp - specifying a normal TCP/IP connection + fd - used passed file descriptors for connection + (see rfdno and wfdno) + + name=name user name to attempt mount as on the remote server. The + server may override or ignore this value. Certain user + names may require authentication. + + aname=name aname specifies the file tree to access when the server is + offering several exported file systems. + + debug=n specifies debug level. The debug level is a bitmask. + 0x01 = display verbose error messages + 0x02 = developer debug (DEBUG_CURRENT) + 0x04 = display 9P trace + 0x08 = display VFS trace + 0x10 = display Marshalling debug + 0x20 = display RPC debug + 0x40 = display transport debug + 0x80 = display allocation debug + + rfdno=n the file descriptor for reading with proto=fd + + wfdno=n the file descriptor for writing with proto=fd + + maxdata=n the number of bytes to use for 9P packet payload (msize) + + port=n port to connect to on the remote server + + timeout=n request timeouts (in ms) (default 60000ms) + + noextend force legacy mode (no 9P2000.u semantics) + + uid attempt to mount as a particular uid + + gid attempt to mount with a particular gid + + afid security channel - used by Plan 9 authentication protocols + + nodevmap do not map special files - represent them as normal files. + This can be used to share devices/named pipes/sockets between + hosts. This functionality will be expanded in later versions. + +RESOURCES +========= + +The Linux version of the 9P server, along with some client-side utilities +can be found at http://v9fs.sf.net (along with a CVS repository of the +development branch of this module). There are user and developer mailing +lists here, as well as a bug-tracker. + +For more information on the Plan 9 Operating System check out +http://plan9.bell-labs.com/plan9 + +For information on Plan 9 from User Space (Plan 9 applications and libraries +ported to Linux/BSD/OSX/etc) check out http://swtch.com/plan9 + + +STATUS +====== + +The 2.6 kernel support is working on PPC and x86. + +PLEASE USE THE SOURCEFORGE BUG-TRACKER TO REPORT PROBLEMS. + diff --git a/Documentation/filesystems/vfs.txt b/Documentation/filesystems/vfs.txt index 3f318dd44c7..f042c12e0ed 100644 --- a/Documentation/filesystems/vfs.txt +++ b/Documentation/filesystems/vfs.txt @@ -1,35 +1,27 @@ -/* -*- auto-fill -*- */ - Overview of the Virtual File System + Overview of the Linux Virtual File System - Richard Gooch <rgooch@atnf.csiro.au> + Original author: Richard Gooch <rgooch@atnf.csiro.au> - 5-JUL-1999 + Last updated on August 25, 2005 + Copyright (C) 1999 Richard Gooch + Copyright (C) 2005 Pekka Enberg -Conventions used in this document <section> -================================= + This file is released under the GPLv2. -Each section in this document will have the string "<section>" at the -right-hand side of the section title. Each subsection will have -"<subsection>" at the right-hand side. These strings are meant to make -it easier to search through the document. -NOTE that the master copy of this document is available online at: -http://www.atnf.csiro.au/~rgooch/linux/docs/vfs.txt - - -What is it? <section> +What is it? =========== The Virtual File System (otherwise known as the Virtual Filesystem Switch) is the software layer in the kernel that provides the filesystem interface to userspace programs. It also provides an abstraction within the kernel which allows different filesystem -implementations to co-exist. +implementations to coexist. -A Quick Look At How It Works <section> +A Quick Look At How It Works ============================ In this section I'll briefly describe how things work, before @@ -38,7 +30,8 @@ when user programs open and manipulate files, and then look from the other view which is how a filesystem is supported and subsequently mounted. -Opening a File <subsection> + +Opening a File -------------- The VFS implements the open(2), stat(2), chmod(2) and similar system @@ -77,7 +70,7 @@ back to userspace. Opening a file requires another operation: allocation of a file structure (this is the kernel-side implementation of file -descriptors). The freshly allocated file structure is initialised with +descriptors). The freshly allocated file structure is initialized with a pointer to the dentry and a set of file operation member functions. These are taken from the inode data. The open() file method is then called so the specific filesystem implementation can do it's work. You @@ -102,7 +95,8 @@ filesystem or driver code at the same time, on different processors. You should ensure that access to shared resources is protected by appropriate locks. -Registering and Mounting a Filesystem <subsection> + +Registering and Mounting a Filesystem ------------------------------------- If you want to support a new kind of filesystem in the kernel, all you @@ -123,17 +117,21 @@ updated to point to the root inode for the new filesystem. It's now time to look at things in more detail. -struct file_system_type <section> +struct file_system_type ======================= -This describes the filesystem. As of kernel 2.1.99, the following +This describes the filesystem. As of kernel 2.6.13, the following members are defined: struct file_system_type { const char *name; int fs_flags; - struct super_block *(*read_super) (struct super_block *, void *, int); - struct file_system_type * next; + struct super_block *(*get_sb) (struct file_system_type *, int, + const char *, void *); + void (*kill_sb) (struct super_block *); + struct module *owner; + struct file_system_type * next; + struct list_head fs_supers; }; name: the name of the filesystem type, such as "ext2", "iso9660", @@ -141,51 +139,97 @@ struct file_system_type { fs_flags: various flags (i.e. FS_REQUIRES_DEV, FS_NO_DCACHE, etc.) - read_super: the method to call when a new instance of this + get_sb: the method to call when a new instance of this filesystem should be mounted - next: for internal VFS use: you should initialise this to NULL + kill_sb: the method to call when an instance of this filesystem + should be unmounted + + owner: for internal VFS use: you should initialize this to THIS_MODULE in + most cases. -The read_super() method has the following arguments: + next: for internal VFS use: you should initialize this to NULL + +The get_sb() method has the following arguments: struct super_block *sb: the superblock structure. This is partially - initialised by the VFS and the rest must be initialised by the - read_super() method + initialized by the VFS and the rest must be initialized by the + get_sb() method + + int flags: mount flags + + const char *dev_name: the device name we are mounting. void *data: arbitrary mount options, usually comes as an ASCII string int silent: whether or not to be silent on error -The read_super() method must determine if the block device specified +The get_sb() method must determine if the block device specified in the superblock contains a filesystem of the type the method supports. On success the method returns the superblock pointer, on failure it returns NULL. The most interesting member of the superblock structure that the -read_super() method fills in is the "s_op" field. This is a pointer to +get_sb() method fills in is the "s_op" field. This is a pointer to a "struct super_operations" which describes the next level of the filesystem implementation. +Usually, a filesystem uses generic one of the generic get_sb() +implementations and provides a fill_super() method instead. The +generic methods are: + + get_sb_bdev: mount a filesystem residing on a block device -struct super_operations <section> + get_sb_nodev: mount a filesystem that is not backed by a device + + get_sb_single: mount a filesystem which shares the instance between + all mounts + +A fill_super() method implementation has the following arguments: + + struct super_block *sb: the superblock structure. The method fill_super() + must initialize this properly. + + void *data: arbitrary mount options, usually comes as an ASCII + string + + int silent: whether or not to be silent on error + + +struct super_operations ======================= This describes how the VFS can manipulate the superblock of your -filesystem. As of kernel 2.1.99, the following members are defined: +filesystem. As of kernel 2.6.13, the following members are defined: struct super_operations { - void (*read_inode) (struct inode *); - int (*write_inode) (struct inode *, int); - void (*put_inode) (struct inode *); - void (*drop_inode) (struct inode *); - void (*delete_inode) (struct inode *); - int (*notify_change) (struct dentry *, struct iattr *); - void (*put_super) (struct super_block *); - void (*write_super) (struct super_block *); - int (*statfs) (struct super_block *, struct statfs *, int); - int (*remount_fs) (struct super_block *, int *, char *); - void (*clear_inode) (struct inode *); + struct inode *(*alloc_inode)(struct super_block *sb); + void (*destroy_inode)(struct inode *); + + void (*read_inode) (struct inode *); + + void (*dirty_inode) (struct inode *); + int (*write_inode) (struct inode *, int); + void (*put_inode) (struct inode *); + void (*drop_inode) (struct inode *); + void (*delete_inode) (struct inode *); + void (*put_super) (struct super_block *); + void (*write_super) (struct super_block *); + int (*sync_fs)(struct super_block *sb, int wait); + void (*write_super_lockfs) (struct super_block *); + void (*unlockfs) (struct super_block *); + int (*statfs) (struct super_block *, struct kstatfs *); + int (*remount_fs) (struct super_block *, int *, char *); + void (*clear_inode) (struct inode *); + void (*umount_begin) (struct super_block *); + + void (*sync_inodes) (struct super_block *sb, + struct writeback_control *wbc); + int (*show_options)(struct seq_file *, struct vfsmount *); + + ssize_t (*quota_read)(struct super_block *, int, char *, size_t, loff_t); + ssize_t (*quota_write)(struct super_block *, int, const char *, size_t, loff_t); }; All methods are called without any locks being held, unless otherwise @@ -193,43 +237,62 @@ noted. This means that most methods can block safely. All methods are only called from a process context (i.e. not from an interrupt handler or bottom half). + alloc_inode: this method is called by inode_alloc() to allocate memory + for struct inode and initialize it. + + destroy_inode: this method is called by destroy_inode() to release + resources allocated for struct inode. + read_inode: this method is called to read a specific inode from the - mounted filesystem. The "i_ino" member in the "struct inode" - will be initialised by the VFS to indicate which inode to - read. Other members are filled in by this method + mounted filesystem. The i_ino member in the struct inode is + initialized by the VFS to indicate which inode to read. Other + members are filled in by this method. + + You can set this to NULL and use iget5_locked() instead of iget() + to read inodes. This is necessary for filesystems for which the + inode number is not sufficient to identify an inode. + + dirty_inode: this method is called by the VFS to mark an inode dirty. write_inode: this method is called when the VFS needs to write an inode to disc. The second parameter indicates whether the write should be synchronous or not, not all filesystems check this flag. put_inode: called when the VFS inode is removed from the inode - cache. This method is optional + cache. drop_inode: called when the last access to the inode is dropped, with the inode_lock spinlock held. - This method should be either NULL (normal unix filesystem + This method should be either NULL (normal UNIX filesystem semantics) or "generic_delete_inode" (for filesystems that do not want to cache inodes - causing "delete_inode" to always be called regardless of the value of i_nlink) - The "generic_delete_inode()" behaviour is equivalent to the + The "generic_delete_inode()" behavior is equivalent to the old practice of using "force_delete" in the put_inode() case, but does not have the races that the "force_delete()" approach had. delete_inode: called when the VFS wants to delete an inode - notify_change: called when VFS inode attributes are changed. If this - is NULL the VFS falls back to the write_inode() method. This - is called with the kernel lock held - put_super: called when the VFS wishes to free the superblock (i.e. unmount). This is called with the superblock lock held write_super: called when the VFS superblock needs to be written to disc. This method is optional + sync_fs: called when VFS is writing out all dirty data associated with + a superblock. The second parameter indicates whether the method + should wait until the write out has been completed. Optional. + + write_super_lockfs: called when VFS is locking a filesystem and forcing + it into a consistent state. This function is currently used by the + Logical Volume Manager (LVM). + + unlockfs: called when VFS is unlocking a filesystem and making it writable + again. + statfs: called when the VFS needs to get filesystem statistics. This is called with the kernel lock held @@ -238,21 +301,31 @@ or bottom half). clear_inode: called then the VFS clears the inode. Optional + umount_begin: called when the VFS is unmounting a filesystem. + + sync_inodes: called when the VFS is writing out dirty data associated with + a superblock. + + show_options: called by the VFS to show mount options for /proc/<pid>/mounts. + + quota_read: called by the VFS to read from filesystem quota file. + + quota_write: called by the VFS to write to filesystem quota file. + The read_inode() method is responsible for filling in the "i_op" field. This is a pointer to a "struct inode_operations" which describes the methods that can be performed on individual inodes. -struct inode_operations <section> +struct inode_operations ======================= This describes how the VFS can manipulate an inode in your -filesystem. As of kernel 2.1.99, the following members are defined: +filesystem. As of kernel 2.6.13, the following members are defined: struct inode_operations { - struct file_operations * default_file_ops; - int (*create) (struct inode *,struct dentry *,int); - int (*lookup) (struct inode *,struct dentry *); + int (*create) (struct inode *,struct dentry *,int, struct nameidata *); + struct dentry * (*lookup) (struct inode *,struct dentry *, struct nameidata *); int (*link) (struct dentry *,struct inode *,struct dentry *); int (*unlink) (struct inode *,struct dentry *); int (*symlink) (struct inode *,struct dentry *,const char *); @@ -261,25 +334,22 @@ struct inode_operations { int (*mknod) (struct inode *,struct dentry *,int,dev_t); int (*rename) (struct inode *, struct dentry *, struct inode *, struct dentry *); - int (*readlink) (struct dentry *, char *,int); - struct dentry * (*follow_link) (struct dentry *, struct dentry *); - int (*readpage) (struct file *, struct page *); - int (*writepage) (struct page *page, struct writeback_control *wbc); - int (*bmap) (struct inode *,int); + int (*readlink) (struct dentry *, char __user *,int); + void * (*follow_link) (struct dentry *, struct nameidata *); + void (*put_link) (struct dentry *, struct nameidata *, void *); void (*truncate) (struct inode *); - int (*permission) (struct inode *, int); - int (*smap) (struct inode *,int); - int (*updatepage) (struct file *, struct page *, const char *, - unsigned long, unsigned int, int); - int (*revalidate) (struct dentry *); + int (*permission) (struct inode *, int, struct nameidata *); + int (*setattr) (struct dentry *, struct iattr *); + int (*getattr) (struct vfsmount *mnt, struct dentry *, struct kstat *); + int (*setxattr) (struct dentry *, const char *,const void *,size_t,int); + ssize_t (*getxattr) (struct dentry *, const char *, void *, size_t); + ssize_t (*listxattr) (struct dentry *, char *, size_t); + int (*removexattr) (struct dentry *, const char *); }; Again, all methods are called without any locks being held, unless otherwise noted. - default_file_ops: this is a pointer to a "struct file_operations" - which describes how to open and then manipulate open files - create: called by the open(2) and creat(2) system calls. Only required if you want to support regular files. The dentry you get should not have an inode (i.e. it should be a negative @@ -328,31 +398,143 @@ otherwise noted. you want to support reading symbolic links follow_link: called by the VFS to follow a symbolic link to the - inode it points to. Only required if you want to support - symbolic links + inode it points to. Only required if you want to support + symbolic links. This function returns a void pointer cookie + that is passed to put_link(). + + put_link: called by the VFS to release resources allocated by + follow_link(). The cookie returned by follow_link() is passed to + to this function as the last parameter. It is used by filesystems + such as NFS where page cache is not stable (i.e. page that was + installed when the symbolic link walk started might not be in the + page cache at the end of the walk). + + truncate: called by the VFS to change the size of a file. The i_size + field of the inode is set to the desired size by the VFS before + this function is called. This function is called by the truncate(2) + system call and related functionality. + + permission: called by the VFS to check for access rights on a POSIX-like + filesystem. + + setattr: called by the VFS to set attributes for a file. This function is + called by chmod(2) and related system calls. + + getattr: called by the VFS to get attributes of a file. This function is + called by stat(2) and related system calls. + + setxattr: called by the VFS to set an extended attribute for a file. + Extended attribute is a name:value pair associated with an inode. This + function is called by setxattr(2) system call. + + getxattr: called by the VFS to retrieve the value of an extended attribute + name. This function is called by getxattr(2) function call. + + listxattr: called by the VFS to list all extended attributes for a given + file. This function is called by listxattr(2) system call. + + removexattr: called by the VFS to remove an extended attribute from a file. + This function is called by removexattr(2) system call. + + +struct address_space_operations +=============================== + +This describes how the VFS can manipulate mapping of a file to page cache in +your filesystem. As of kernel 2.6.13, the following members are defined: + +struct address_space_operations { + int (*writepage)(struct page *page, struct writeback_control *wbc); + int (*readpage)(struct file *, struct page *); + int (*sync_page)(struct page *); + int (*writepages)(struct address_space *, struct writeback_control *); + int (*set_page_dirty)(struct page *page); + int (*readpages)(struct file *filp, struct address_space *mapping, + struct list_head *pages, unsigned nr_pages); + int (*prepare_write)(struct file *, struct page *, unsigned, unsigned); + int (*commit_write)(struct file *, struct page *, unsigned, unsigned); + sector_t (*bmap)(struct address_space *, sector_t); + int (*invalidatepage) (struct page *, unsigned long); + int (*releasepage) (struct page *, int); + ssize_t (*direct_IO)(int, struct kiocb *, const struct iovec *iov, + loff_t offset, unsigned long nr_segs); + struct page* (*get_xip_page)(struct address_space *, sector_t, + int); +}; + + writepage: called by the VM write a dirty page to backing store. + + readpage: called by the VM to read a page from backing store. + + sync_page: called by the VM to notify the backing store to perform all + queued I/O operations for a page. I/O operations for other pages + associated with this address_space object may also be performed. + + writepages: called by the VM to write out pages associated with the + address_space object. + + set_page_dirty: called by the VM to set a page dirty. + + readpages: called by the VM to read pages associated with the address_space + object. + prepare_write: called by the generic write path in VM to set up a write + request for a page. -struct file_operations <section> + commit_write: called by the generic write path in VM to write page to + its backing store. + + bmap: called by the VFS to map a logical block offset within object to + physical block number. This method is use by for the legacy FIBMAP + ioctl. Other uses are discouraged. + + invalidatepage: called by the VM on truncate to disassociate a page from its + address_space mapping. + + releasepage: called by the VFS to release filesystem specific metadata from + a page. + + direct_IO: called by the VM for direct I/O writes and reads. + + get_xip_page: called by the VM to translate a block number to a page. + The page is valid until the corresponding filesystem is unmounted. + Filesystems that want to use execute-in-place (XIP) need to implement + it. An example implementation can be found in fs/ext2/xip.c. + + +struct file_operations ====================== This describes how the VFS can manipulate an open file. As of kernel -2.1.99, the following members are defined: +2.6.13, the following members are defined: struct file_operations { loff_t (*llseek) (struct file *, loff_t, int); - ssize_t (*read) (struct file *, char *, size_t, loff_t *); - ssize_t (*write) (struct file *, const char *, size_t, loff_t *); + ssize_t (*read) (struct file *, char __user *, size_t, loff_t *); + ssize_t (*aio_read) (struct kiocb *, char __user *, size_t, loff_t); + ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *); + ssize_t (*aio_write) (struct kiocb *, const char __user *, size_t, loff_t); int (*readdir) (struct file *, void *, filldir_t); unsigned int (*poll) (struct file *, struct poll_table_struct *); int (*ioctl) (struct inode *, struct file *, unsigned int, unsigned long); + long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long); + long (*compat_ioctl) (struct file *, unsigned int, unsigned long); int (*mmap) (struct file *, struct vm_area_struct *); int (*open) (struct inode *, struct file *); + int (*flush) (struct file *); int (*release) (struct inode *, struct file *); - int (*fsync) (struct file *, struct dentry *); - int (*fasync) (struct file *, int); - int (*check_media_change) (kdev_t dev); - int (*revalidate) (kdev_t dev); + int (*fsync) (struct file *, struct dentry *, int datasync); + int (*aio_fsync) (struct kiocb *, int datasync); + int (*fasync) (int, struct file *, int); int (*lock) (struct file *, int, struct file_lock *); + ssize_t (*readv) (struct file *, const struct iovec *, unsigned long, loff_t *); + ssize_t (*writev) (struct file *, const struct iovec *, unsigned long, loff_t *); + ssize_t (*sendfile) (struct file *, loff_t *, size_t, read_actor_t, void *); + ssize_t (*sendpage) (struct file *, struct page *, int, size_t, loff_t *, int); + unsigned long (*get_unmapped_area)(struct file *, unsigned long, unsigned long, unsigned long, unsigned long); + int (*check_flags)(int); + int (*dir_notify)(struct file *filp, unsigned long arg); + int (*flock) (struct file *, int, struct file_lock *); }; Again, all methods are called without any locks being held, unless @@ -362,8 +544,12 @@ otherwise noted. read: called by read(2) and related system calls + aio_read: called by io_submit(2) and other asynchronous I/O operations + write: called by write(2) and related system calls + aio_write: called by io_submit(2) and other asynchronous I/O operations + readdir: called when the VFS needs to read the directory contents poll: called by the VFS when a process wants to check if there is @@ -372,18 +558,25 @@ otherwise noted. ioctl: called by the ioctl(2) system call + unlocked_ioctl: called by the ioctl(2) system call. Filesystems that do not + require the BKL should use this method instead of the ioctl() above. + + compat_ioctl: called by the ioctl(2) system call when 32 bit system calls + are used on 64 bit kernels. + mmap: called by the mmap(2) system call open: called by the VFS when an inode should be opened. When the VFS - opens a file, it creates a new "struct file" and initialises - the "f_op" file operations member with the "default_file_ops" - field in the inode structure. It then calls the open method - for the newly allocated file structure. You might think that - the open method really belongs in "struct inode_operations", - and you may be right. I think it's done the way it is because - it makes filesystems simpler to implement. The open() method - is a good place to initialise the "private_data" member in the - file structure if you want to point to a device structure + opens a file, it creates a new "struct file". It then calls the + open method for the newly allocated file structure. You might + think that the open method really belongs in + "struct inode_operations", and you may be right. I think it's + done the way it is because it makes filesystems simpler to + implement. The open() method is a good place to initialize the + "private_data" member in the file structure if you want to point + to a device structure + + flush: called by the close(2) system call to flush a file release: called when the last reference to an open file is closed @@ -392,6 +585,23 @@ otherwise noted. fasync: called by the fcntl(2) system call when asynchronous (non-blocking) mode is enabled for a file + lock: called by the fcntl(2) system call for F_GETLK, F_SETLK, and F_SETLKW + commands + + readv: called by the readv(2) system call + + writev: called by the writev(2) system call + + sendfile: called by the sendfile(2) system call + + get_unmapped_area: called by the mmap(2) system call + + check_flags: called by the fcntl(2) system call for F_SETFL command + + dir_notify: called by the fcntl(2) system call for F_NOTIFY command + + flock: called by the flock(2) system call + Note that the file operations are implemented by the specific filesystem in which the inode resides. When opening a device node (character or block special) most filesystems will call special @@ -400,29 +610,28 @@ driver information. These support routines replace the filesystem file operations with those for the device driver, and then proceed to call the new open() method for the file. This is how opening a device file in the filesystem eventually ends up calling the device driver open() -method. Note the devfs (the Device FileSystem) has a more direct path -from device node to device driver (this is an unofficial kernel -patch). +method. -Directory Entry Cache (dcache) <section> ------------------------------- +Directory Entry Cache (dcache) +============================== + struct dentry_operations -======================== +------------------------ This describes how a filesystem can overload the standard dentry operations. Dentries and the dcache are the domain of the VFS and the individual filesystem implementations. Device drivers have no business here. These methods may be set to NULL, as they are either optional or -the VFS uses a default. As of kernel 2.1.99, the following members are +the VFS uses a default. As of kernel 2.6.13, the following members are defined: struct dentry_operations { - int (*d_revalidate)(struct dentry *); + int (*d_revalidate)(struct dentry *, struct nameidata *); int (*d_hash) (struct dentry *, struct qstr *); int (*d_compare) (struct dentry *, struct qstr *, struct qstr *); - void (*d_delete)(struct dentry *); + int (*d_delete)(struct dentry *); void (*d_release)(struct dentry *); void (*d_iput)(struct dentry *, struct inode *); }; @@ -451,6 +660,7 @@ Each dentry has a pointer to its parent dentry, as well as a hash list of child dentries. Child dentries are basically like files in a directory. + Directory Entry Cache APIs -------------------------- @@ -471,7 +681,7 @@ manipulate dentries: "d_delete" method is called d_drop: this unhashes a dentry from its parents hash list. A - subsequent call to dput() will dellocate the dentry if its + subsequent call to dput() will deallocate the dentry if its usage count drops to 0 d_delete: delete a dentry. If there are no other open references to @@ -507,16 +717,16 @@ up by walking the tree starting with the first component of the pathname and using that dentry along with the next component to look up the next level and so on. Since it is a frequent operation for workloads like multiuser -environments and webservers, it is important to optimize +environments and web servers, it is important to optimize this path. Prior to 2.5.10, dcache_lock was acquired in d_lookup and thus in every component during path look-up. Since 2.5.10 onwards, -fastwalk algorithm changed this by holding the dcache_lock +fast-walk algorithm changed this by holding the dcache_lock at the beginning and walking as many cached path component -dentries as possible. This signficantly decreases the number +dentries as possible. This significantly decreases the number of acquisition of dcache_lock. However it also increases the -lock hold time signficantly and affects performance in large +lock hold time significantly and affects performance in large SMP machines. Since 2.5.62 kernel, dcache has been using a new locking model that uses RCU to make dcache look-up lock-free. @@ -527,7 +737,7 @@ protected the hash chain, d_child, d_alias, d_lru lists as well as d_inode and several other things like mount look-up. RCU-based changes affect only the way the hash chain is protected. For everything else the dcache_lock must be taken for both traversing as well as -updating. The hash chain updations too take the dcache_lock. +updating. The hash chain updates too take the dcache_lock. The significant change is the way d_lookup traverses the hash chain, it doesn't acquire the dcache_lock for this and rely on RCU to ensure that the dentry has not been *freed*. @@ -535,14 +745,15 @@ ensure that the dentry has not been *freed*. Dcache locking details ---------------------- + For many multi-user workloads, open() and stat() on files are very frequently occurring operations. Both involve walking of path names to find the dentry corresponding to the concerned file. In 2.4 kernel, dcache_lock was held during look-up of each path component. Contention and -cacheline bouncing of this global lock caused significant +cache-line bouncing of this global lock caused significant scalability problems. With the introduction of RCU -in linux kernel, this was worked around by making +in Linux kernel, this was worked around by making the look-up of path components during path walking lock-free. @@ -562,7 +773,7 @@ Some of the important changes are : 2. Insertion of a dentry into the hash table is done using hlist_add_head_rcu() which take care of ordering the writes - the writes to the dentry must be visible before the dentry - is inserted. This works in conjuction with hlist_for_each_rcu() + is inserted. This works in conjunction with hlist_for_each_rcu() while walking the hash chain. The only requirement is that all initialization to the dentry must be done before hlist_add_head_rcu() since we don't have dcache_lock protection while traversing @@ -584,7 +795,7 @@ Some of the important changes are : the same. In some sense, dcache_rcu path walking looks like the pre-2.5.10 version. -5. All dentry hash chain updations must take the dcache_lock as well as +5. All dentry hash chain updates must take the dcache_lock as well as the per-dentry lock in that order. dput() does this to ensure that a dentry that has just been looked up in another CPU doesn't get deleted before dget() can be done on it. @@ -640,10 +851,10 @@ handled as described below : Since we redo the d_parent check and compare name while holding d_lock, lock-free look-up will not race against d_move(). -4. There can be a theoritical race when a dentry keeps coming back +4. There can be a theoretical race when a dentry keeps coming back to original bucket due to double moves. Due to this look-up may consider that it has never moved and can end up in a infinite loop. - But this is not any worse that theoritical livelocks we already + But this is not any worse that theoretical livelocks we already have in the kernel. |