Guidelines for Setting Up Linux Filesystems

Submitted by Syscrusher on Sun, 2005/06/05 - 22:33.

Linux when it boots has only ONE filesystem available, and that is the root filesystem mounted on the / directory. You can mount additional filesystems on any point beneath /. A mount point is created simply by making an empty directory (using the standard mkdir command) and then using the mount command to logically attach your device or partition to that empty directory.

This is an elegant way to view disk partitions and devices, because to a Linux application or to the user every file on every drive simply appears to reside in a subdirectory of /. There are no clumsy drive letters or volume names to worry about. The system administrator can move everything under /usr/apps/wordperfect off to a separate disk drive (to free up space or improve performance), or even to an NFS-mounted network service, and it STILL appears in /usr/apps/wordperfect to the end user and to the programs themselves.

Two fairly subtle points often are overlooked by those who are new to Linux.

  • First, mount points don't have to be at the top level directory. You can make /home have its own filesystem by using /home as a mount point on a second disk, but you could also put /var/log (for example) on its own filesystem as well. The only requirement for a mount point is that it be an empty directory.
  • Second, mount points can nest within mount points IF you take care to put them in the correct order within your /etc/fstab file. For instance, let's say you have 200 users on your system, and one of them (user 'sqlwiz') creates HUGE test datasets for some complex SQL system. So, for good administrative practice or performance you'd like to have /home on its own filesystem, but you'd also like to have /home/sqlwiz on still another filesystem, allocating an entire drive to that one user for her test data. This is easily accomplished:
    1. Migrate /home itself, excluding /home/sqlwiz, onto its own filesystem. Establish an /etc/fstab entry so that this filesystem mounts on /home at boot time.
    2. Now make the filesystem for /home/sqlwiz. After mounting /home in the usual way, make the empty /home/sqlwiz directory and mount this new filesystem there. Establish an /etc/fstab entry so that the /home/sqlwiz filesystem mounts AFTER /home is mounted. Why? Because before /home is mounted, the empty directory /home/sqlwiz will not exist as a mount point, and the mount will fail.
    There is a pair of numbers at the end of a normal /etc/fstab entry, and the second number of the pair sets the order in which fsck is run to check file structures during boot. This does NOT set the order of the mount processing, though -- you have to control that with the physical order of lines in the /etc/fstab file.

One more comment should be made about system administration in Linux, and here I will point out that this is my own opinion, not gospel. This viewpoint is shared by many system administrators, but every admin has his or her own style and you'll want to think through your own system configuration based upon your needs.

The issue at hand is WHAT directories, of the standard ones that are built by the o.s., should go on separate filesystems?

I recommend that the root filesystem (/) be kept as small as possible and that, to the extent possible, it contains only the operating system itself and those things required to boot. The reasons for this are:

  1. Disaster recovery is easier in the event of a partition corruption problem on the root disk. Since Linux has no Windows-style registry, reinstalling the o.s. does NOT mean you automatically have to reload all of your applications. On my own Linux PC, I have on several occasions reloaded the o.s. on a freshly-formatted root partition, without reinstalling my apps and development tools. What takes me days to do on NT takes a couple of hours on Linux. (By the way, I've done this during upgrades, not for disaster recovery. So far -- knock on silicon -- I've had no disasters on my Linux disks.)
  2. Linux doesn't lend itself well to upgrades, in my opinion. Even the mainstream distributions that nominally have an upgrade script really don't do a good job of it. The system is just too customizable for a script to understand every nuance of your networking, printer, security, and application environments. Fortunately, it doesn't have to work that way. Just isolate the system from your customized configuration, and make the o.s. paritition quite literally a throw-away! This strategy also pays off if you want to beta-test a new release of the o.s., by booting it from a different root partition, but still (selectively) run your applications under it.

With these factors in mind, I recommend the following strategy for filesystem layout:

  • /home should be on its own filesystem, and all user accounts should be in that directory. When you reinstall the operating system, don't have /home mounted at all -- that way, your users' data is completely safe unless you manage to zap the disk partition directly, which is much harder to do than just overwriting or deleting some files.
  • /usr/local should be on its own filesystem, and your customized shell scripts should live in /usr/local/bin or /usr/local/sbin. There should also be a /usr/local/etc directory, which is a great place to make copies of your critical configuration files from /etc itself. Keep your downloaded source tarballs in /usr/local/src and extract them there as well. Now when you update the o.s., you don't lose all those huge tarballs of Samba, Apache, KSH, and so on. Just rebuild them and re-run their 'make install' and you're up and running (this works for most 'autoconf' style source). By the way, keep a copy of your /usr/src/linux/.config file somewhere in /usr/local, as well. That's the kernel configuration file that could save your virtual skin someday when you have to rebuild. Critical files from /etc would include (among others): /etc/hosts, /etc/httpd (directory), /etc/samba.d (directory), /etc/passwd, /etc/shadow, /etc/sysconfig (directory...only if you have customized it), /etc/group, /etc/hosts.allow, /etc/hosts.deny, /etc/resolv.conf, /etc/services, /etc/inetd.conf, /etc/profile, /etc/fstab, /etc/exports, /etc/modules (directory), and /etc/rc.d/rc.local. You may not want to blindly restore all these after a system upgrade, but even so it's very helpful to have them around to look at.
  • You may want to put /usr on its own filesystem, but even if you do, keep /usr/local separate from the rest of /usr. Many things in /usr are really 'owned' by the operating system and are -- and should be -- replaced when you upgrade the operating system.
  • Consider putting /var, or possibly /var/log, on a separate filesystem. This protects your root filesystem from overflowing in the case of a runaway log. An overflowed root filesystem is a Bad Thing.
  • For the same reason, consider putting /tmp on a separate filesystem. Any user can write to this directory, and you could suffer a denial-of-service attack if someone creates huge files there, overflowing the root filesystem. Overflowing /tmp is much less of a problem, and will not likely crash your operating system.
  • If you are running web or FTP services at the system level (as opposed to user homepages only) you may want to consider creating /pub to hold all your "published" files, and putting it on a separate filesystem. This is useful for security management, as you can often say "anyone in the world can read anything under /pub". Perhaps your secured documents can reside elsewhere, or perhaps you are lucky enough not to need a secured portion of your web or FTP site.
  • Finally, it may be useful to put database files (such as those from DB2 or MySQL) on their own partition, mostly for performance reasons if it is on a different spindle from the rest of your system's filesystems. If you are REALLY doing serious database work, you may want to have the log areas separate from the data areas. If your needs are that elaborate, then you are probably experienced enough not to need the advice from this writer!

Remember that there are as many "correct" ways to build a Linux filesystem layout as there are system administrators. Think through the kinds of things you want your machine to do, then make your own choices. The nice thing about the way Linux and UNIX systems work is that it's very easy to change your mind later, as your needs grow.

( categories: )