Assignment 3b: Files

Learning Objectives

This lab is an extesion of Lab 3a, and it shares the same learning objectives and logistics.

After completing this lab (parts A and B), you will:

• be able to describe the on-disk format for a functional file system
• be able to map logical file system abstractions onto a physical address space
• create C structures that represent the key components of file system metadata, including the superblock, inode, and allocation structures
• implement pathname resolution
• implement logic to enforce file permissions
• implement logic to list the contents of directories
• implement logic to create and delete directories

Overview

In Lab 3b, you will:

1. Finish the prototype filesystem you began in Lab 3a, and
2. Add enough functionality to your prototype filesystem that it can be used to store actual data in mutable files.

Assignment Spec

Your finished filesystem should implement the following functionality:

• It should implement all of the required operations from Lab 3a, which include:
  • getattr(),
  • access(),
  • readdir(), and
  • mkdir()
• In addition, you are should implement the following additional operations:
  • rmdir(),
  • chmod() (this is most easily implemented by adding a mode_t mode field to your inode),
  • close() (i.e., release()),
  • create() (can likely call a do_mknod() helper function),
  • fgetattr() (can likely call a do_getattr() helper function),
  • mknod() (only for plain files; you can assume that the argument mode will always be S_IFREG, and you can completely ignore dev),
  • open(),
  • read(),
  • truncate(),
  • unlink(), and
  • write().

Descriptions of each method can be found in the HMC documentation, and suggestions/clarifications will be provided below as they come up.

Note that you are not required to support:

• readlink(),
• symlink(),
• chown(),
• fsync(),
• link(),
• statfs(),
• rename(), or
• utimens().

Many of these optional operations would require additional filesystem design (e.g., chown() requires that you track ownership).

Important Notes

• Your write() operation should support full Unix semantics. Specifically, it should be possible to:
  • extend a file by writing at the end;
  • rewrite arbitrary bytes in the middle of a file; and
  • extend a file by a large amount by seeking far past the end and writing a single byte.

• You are not required to support sparse files (a sparse file is a large file that does not explicitly represent all of the “holes”; instead it creates “0”s on demand).

• Directories should be variable-sized and should grow as files are created within them. However, you may choose to simplify the design:
  • it is not necessary to support “indirect blocks” for your directories. If this means that you have a limit to the number of “children” that a a directory may have, note what this limit is.
  • It is not necessary to shrink the size of directories when files are removed. In other words, as long as you mark the deleted contents of the directory invalid and ensure that deleted files are logically inaccessible, you do not need to reclaim fragmented space within the directory’s data blocks.

• Your implementation need not prioritize speed, but it should still be reasonably efficient (no unnecessary work or obviously wasteful big-O time or space). It should be possible to create a 1-MiB file in well under a second.

• You should not waste memory. It’s OK to store the bitmaps and the entire inode table in memory. But you may not store a copy of the entire disk in memory, nor may you store entire files or directories in memory. An obvious exception is 1-block files, which of course may be assumed to fit… This just means you should bring blocks into memory on demand, and free memory when you are no longer using it.

• Note that if you do decide to implement symbolic links, your path-evaluation code doesn’t need to do anything special for symbolic links; FUSE will do that for you.

• Note that if you do decide to implement the statfs() operation, it should correctly report the amount of free space in the data region of the file system. In other words, if it reports that N blocks are left, it should be possible to write N additional blocks of data to an existing file, and impossible to write N+1 blocks. (You may wish to add a counter somewhere rather than manually count your bitmap’s bits each time statfs() is called.)

Testing ReFS

Many of your features can be tested using standard Unix utilities inside scripts like we did for Lab 1. The dd utility is especially useful (see man dd, or “Google” dd for example usage; looking up how to use this tool for testing is not considered an honor code violation.).

Unlike the Part A, I recommend that you write test code/scripts. I provide some test recommendations below, but I also encourage you to share any tests ideas or strategies that you have with the group on Slack.

A good strategy is, as soon as you add functionality, create a small test. Then run all of your existing tests to verify your new “feature” didn’t break anything!

Test Ideas

Here are some things that I expect to work, so you are strongly encouraged to create tests for (at least) these features:

• Subdirectories (and sub-sub, and…)
• Directories with many entries (enough that the directory requires more than one disk block to store the (inode number -> path) mappings)
• ls -la, including for files of various sizes, and including . and ..
• Zero-length files
• Very large files (e.g., a file that uses an indirect block)
• Extending files (e.g., with echo xxx >> foo)
• Extending a file while overwriting a few bytes at the end
• Truncating files to zero length
• Truncating files to nonzero length
• Using truncate to extend files by adding zeros
• Changing a few bytes in the middle of a file (hint: invoking dd with the conv=notrunc switch is one way to do this, as is our write_test.c from Hello Fuse)

Submission

Submit your code (it should be inside a single file named refs.c) to your git repository. If you implement any additional features, be sure to mention them (prominently?) in your README.md file so that I see them.

When you have completed your Lab 3b, please do two things:

• create a git tag for the commit that “completes” your submission
• Send an email to let me know that you have finished.

This lab borrows from an assignment created by Geoff Kuenning.