JSON Dir

2/23/2021 CS 265 — Assignment – JSON Dir

https://www.cs.drexel.edu/~kschmidt/CS265/Assignments/Bash-Dir2Json/ 1/4

Assignment — JSON Dir CS 265 Advanced Programming Techniques

Introduction

The purpose of this assignment is for students to become writing Bash scripts, traversing the filesystem, and using Unix filters, pipes, and redirection to solve a text processing problem. The student will also write custom filters in Awk.

Problem

For this assignment, you will write a Bash script that visits every subdirectory in a folder (provided via command line argument). In each folder (including the top-level folder), you will create a JSON file called dir.json that contains some data extracted from a README file that may or may not be present in that directory. You will use Awk to parse this README file.

Input — Sample Directory

See tux:/home/kschmidt/public_html/Files/Dir2 . Here is what a sample directory might look like:

README

The README may have two entries, each on its own line. Fields are separated by colons. Neither entry is required. The two possible entries are:

index: A single file, the top-level HTML page for this directory. required; A list of files that should be in this directory.

You must use Awk to parse this file.

dir.json

In each subdirectory (including the top-level directory on which the script was called), you will create a file called dir.json. If it already exists, simply overwrite it. Do not include it as one of the files listed in the keys described below.

dir.json should contain three top-level keys:

A key called index that contains a single filename representing the top-level HTML page for this directory. This will be extracted from the README (if present). If no index entry is found in the README, this should be an empty string. A key called required that contains an object with two keys. The first key is files, which contains a (possibly empty) list of all files marked as required in the README. The second key is directories, which contains a (possibly empty) list of all directories marked as required in the README. A key called other that has the same format as required, only it will contain a list of all files and a list of all directories that are not in the index or required keys.

Here is an example JSON file:

2/23/2021 CS 265 — Assignment – JSON Dir

https://www.cs.drexel.edu/~kschmidt/CS265/Assignments/Bash-Dir2Json/ 2/4

dir.json

{ "index" : "index.html", "required" : { "files": [ "file2.1", "file2.2" ], "directories" : [] }, "other" : { "files": [ "other1", "other4", "other5", "README" ], "directories" : [ "Data" ] } }

Other Requirements

You must use Awk to parse the README file. The rest of your script must be written in Bash.

You may use common utilities we've already covered (e.g., ls) In particular, you may not use jq

Each dir.json file your script creates must be valid JSON. See the JSON section below if you've never worked with JSON. Your top-level script must be named a1-top (that's a one, not an "ell"). Your top-level script must take an optional command line argument of the directory we want to process. If no directory is provided, your script should use the current directory. We'll run your top-level script directly, so get the sha-bangs right.

Otherwise, the details are left to you.

JSON Description

If you're unfamiliar with the JSON file format, see this Mozilla Developer Network tutorial about it. Your dir.json files must be valid JSON. For example:

All strings must be surrounded in double quotes, not single quotes. Keys are strings and thus must also be quoted. Trailing commas are not allowed. Whitespace is not required.

Example

Say a folder named myfolder contained this directory structure:

Your script will create a dir.json file inside every subdirectory. If myfolder/folder2/folder3/README looked like this:

index:top.html required:test1:testa

Then myfolder/folder2/folder3/dir.json> should contain:

{ "index": "top.html",

2/23/2021 CS 265 — Assignment – JSON Dir

https://www.cs.drexel.edu/~kschmidt/CS265/Assignments/Bash-Dir2Json/ 3/4

myfolder/ ├── folder1 │ ├── a │ ├── b │ ├── c │ ├── index.html │ └── README ├── folder2 │ ├── emptyfolder │ ├── file1 │ ├── file2 │ ├── file3 │ ├── folder3 │ │ ├── README │ │ ├── test1 │ │ ├── test2 │ │ ├── testa │ │ ├── testb │ │ └── top.html │ └── README └── folder4 ├── afile └── afolder └── onefile

"required": { "files": ["test1", "testa"], "directories": [] }, "other": { "files": ["test2", "testb"], "directories": [] } }

If

myfolder/folder2/README looked like this:

required:file2

Then myfolder/folder2/dir.json should contain these contents:

{ "index": "", "required": { "files": ["file2"], "directories": [] }, "other": { "files": ["file1", "file3"], "directories": ["folder3", "emptyfolder"] } }

Since myfolder/folder4 doesn't have a README, its dir.json should contain:

{ "index": "", "required": { "files": [], "directories": [] }, "other": { "files": ["afile"], "directories": ["afolder"] } }

Tips

2/23/2021 CS 265 — Assignment – JSON Dir

https://www.cs.drexel.edu/~kschmidt/CS265/Assignments/Bash-Dir2Json/ 4/4

Make sure you understand the problem before you begin to code. It's very difficult to solve a problem that you don't even understand. Plan out your code before you write it. For example, if you plan on cding into every directory, upon reflection, you'll realize that you'll need to figure out how to cd back out to the directory you were in previously to keep exploring. (Hint: use pushd/popd or don't cd at all). Create your own test directory that you can test your script on as you write it. Your test should contain all of the edge cases you can think of (missing READMEs, missing index/required entries, empty folders, etc.). Write a little bit at a time, then test, then write a little bit more. As a rough guide, you shouldn't write more than three lines of code without stopping to test what you've written. If you write 20 lines of code and then run your script for the first time, it'll contain a dozen errors and you'll have no idea where they are.

So, process a single directory Then, simply verify that you visit each directory. Maybe ls Now put them together

Separate your code into functions to make it easier to test. See ~kschmidt/public_html/CS265/Labs/Bash/func.bash for some usage examples. You may want to use Bash arrays to create the lists of files/directories in the other key; see ~kschmidt/public_html/CS265/Labs/Bash/arrays.bash for some usage examples.

Submission

Do not submit temporary files, nor your test data. Only submit the required scripts, and any helper files you wrote.

What to hand in

a1-top — your top-level script, the entry point, the script we will call with the directory to be processed All the rest of your source code README (optional) — anything you want to tell us before we grade.