CSCI 333

Storage Systems

Home | Schedule | Labs | Williams CS | Piazza

Lab 2a: FUSE FAT, part I

Objectives

You have become (relatively) familiar with FUSE though your barebones "Hello FUSE" implementation of a minimal file system containing a single pseudo-file. Now we will extend our FS behavior to support the creation, deletion, and modification of files and directories. Through this lab, we will explore the challenges of managing persistent data in a file system.

Overview

Developing a working file system is very hard. For that reason, this assignment is divided into two parts. In part I (this part), you will develop a lot of scaffolding and enough code for your filesystem to do something testable. In part II (the next part), you will complete the filesystem.

The Assignment

Your assignment is to develop a FAT-like filesystem that supports the following features by the end of part I:

• The general structure of the filesystem is similar to the Microsoft FAT design (see "FAT Design") below.
• The filesystem supports the following operations at a minimum: getattr, access, readdir, and mkdir. (Note that at this point it is not necessary to support file I/O, or even regular files.)
• Your mkdir operation must allocate space from the free list.
• The filesystem is backed by a SINGLE preallocated 10-MB file. That file should have a fixed name and exist on a non-fuse file system. For example, you may have a file named "fat_disk" in your project directory (i.e., it lives outside your FUSE file system). The size of the file should be a #defined constant (this way you can change it later). Remember to watch out for the working-directory gotcha.
• When the filesystem is invoked, if the backing file doesn't exist, it is created and initialized. However, if it does exist, the backing file should be attached and its previous contents should be visible.
• When a mutating operation occurs, its effects must be immediately visible in the backing file. (This means that you can't do everything in memory and then wait until exit time to write things out. I will test this feature by killing your process with SIGKILL. O_DSYNC isn't necessary, because the operating system will make sure your data gets to stable storage unless the entire OS crashes—which isn't part of the testing plan!)
• Subdirectories must be supported.
• Your directories may be fixed-size; it is not necessary to be able to create an arbitrary number of entries in a directory. You should document the limits somewhere.
• Directory entries may also be fixed-size, as long as the name length is moderately reasonable. (Nothing under 16 characters is "reasonable" in my book; my minimal implementation compromises with a limit of 32.) Recall that FAT uses the "8.3" naming scheme with a special long-file-name scheme.
• If you choose, file sizes may be limited to either 2³² or 2⁶⁴ bytes.
• The acid test of your filesystem should be that is possible to create directories, list them (with ls -la returning reasonable results including "." and ".."), and cd into them.
• Other operations are up to you. We will be extending the filesystem to support files, rmdir, etc. in the next assignment, so you are welcome to implement those things. However, they will not be tested in the current assignment.

Why this particular set of features? It's the minimum necessary to have a filesystem where you can do something visible: create and list directories. You'll find that you need to create quite a bit of scaffolding to get that far (in particular, the code that creates an initialized FAT filesystem from scratch).

FAT Design

When I refer to a "FAT-like" filesystem, I mean the following:

• Allocation is managed by an in-memory table with one entry per filesystem block (cluster in FAT terminology). Each table entry contains either 0 or the number of another block. In toto, the table constitutes a set of linked lists of blocks.
• The free list is a linked list (held in the in-memory table) reached from the superblock. (An alternative would be to use the (undesirable) Microsoft FAT design, which marks free blocks with a special code and requires scanning the FAT to find free blocks.)
• The on-disk copy of the block table is read at mount (filesystem initialization) time and is updated at your discretion (but note that your process might be killed at any time!).
• All file metadata is kept in the directory entry. At a minimum, this should include the file type (directory or file), the size (in bytes), the name, and the number of the first block (cluster in FAT terminology). (Subsequent clusters are located via the block table.) Other metadata, such as ownership, permissions, and timestamps, are up to you but are not required.
• The block size is up to you, but it must be at least 512. (I recommend 4096, just to keep up with the modern world.)
• Like any other file system, the on-disk data structures are stored in a single file (pseudo-disk) and are kept in binary.

For reference, my minimal implementation used a block size of 512 bytes, had six fields in the superblock (including a magic number), and had four fields in the directory entry. To make it easy to store the superblock in a filesystem block, I used the following union:

        union {
            struct fat_superblock s;
            char		pad[512];
        }  superblock;

(Note that the superblock should be only 512 bytes, even if you use a different block size for your filesystem. That design makes it possible to read the superblock without knowing the block size, which is a useful feature. If the filesystem uses blocks larger than 512 bytes, the remaining space in the larger "block" is simply wasted.)

I also found it useful to create a few macros to do things like seeking to a particular block, converting back and forth between byte offsets and block numbers, etc.

Important Notes

Note: You are supposed to be writing a real filesystem. The only differences from a true implementation of FAT should be:

• Your implementation is backed by a plain file in the filesystem, rather than an actual disk, and
• Your data structures are not required to be compatible with other FAT implementations (i.e., you don't have to be able to create or mount MS-DOS FAT disks).

In particular, this means not taking easy shortcuts. Like any filesystem, your implementation must satisfy the following criteria:

• All access to the "disk" must be in multiples of the block size, which must be a power of 2 and must be 512 or greater.
• Changes to files and directories must be reflected on disk immediately. No fair saving things in memory and then writing them out when you unmount.
• Information must persist in the backing store (your persistent file) after unmount.

Submission

Submit your code (it should be inside a single file named fat.c) to your git repository. If you implement any additional features, describe them prominently in your README.md file so that you receive credit.

I would like everyone to use a "new" feature when submitting part 2a: git tags. A "tag" is essentially a label for a specific commit. You can create a tag from the command line (git manual on tags), or directly through the github web interface by creating a "release".

When you have completed your Lab 2a, please do two things:

1. Create a tag named "v2a" (this does not follow the semantic versioning spec...)
2. Send an email to let me know that you have finished.

Since you will continue to work on code in the same repository for part 2b, the tag will make sure that I test and give feedback on the correct version of your lab.

This lab borrows heavily from an assignment created by Geoff Kuenning, with only slight modificatons.