Scripting is a great way to automate otherwise manual and time consuming tasks. The objective for this project is to give you another opportunity to play with your ever-expanding UNIX and command-line skills, and assist you with developing an even finer relationship between process and data: by having you calculate your grade in the class based on your evaluation data.

Throughout the semester your participation, weekly journal entries, and projects have been gradually tallying up and made available to you via the super-nifty status tool.

In order to pull up your information, you merely need to invoke the command with the pertinent class:

lab46:~$ status unix

And you will be greeted with the summary data related to your standing in the class (you can also append the detail option to get even more of an information line-by-line breakdown).

But, throughout all this, some very obvious data points are lacking: your actual, calculated results. All I am giving you is the raw data, and while I have scripts that can quickly give me an overview of your progress, I've left that same role up to you to fill.

And since you're all unprocrastinating, highly motivated and impassioned learners, I figured I'd give you a project that you've probably finished weeks ago in your individual quests for perfection: to write a script that will collect, process, and output that very information (you're welcome).

You'll recall from the syllabus that the grading policy for the class is divided into three areas:

• participation (your “presence”, hopefully consistently of the mental variety, but I lower the bar so as not to overly ding anyone on an off day), which is worth a total of 28% of the final grade.
• journal/blog/opus (your personal reflections / informational composition of your experience in the course, which some have already been reaping great value from), worth a total of 36% of the final grade.
• projects (the assigned activities you've been enjoying each week), which rounds out the remaining 36% of your final grade.
  • the majority of reported bonus points are earned in relation to the projects, and are therefore applied exclusively to the projects component of your grade.

Each component needs to be calculated separately to gain the total number of points, and then those points summed together to get your final result.

Of course, we're not done yet, so new data is being added all the time (with each passing week, journal entry, and evaluated assignment), which means whatever value you may come up with today would potentially be obsolete come tomorrow.

The algorithms and the process for determining your grade based on the available data, however, remain the same.

Which is why this makes for such a great scripting project.

As previously mentioned, throughout the semester you've been able to see the processed results of your various efforts in the class (projects, journal, participation). That information was made available to you via the handy status tool (expanded some more with the detail argument given).

The participation data is in more of a binary representation (it is either there for a particular class day, or not; historically, I had 'X''s appear, but due to a logic bug in one of my scripts, it now just omits missed values– same difference difficulty-wise, and doesn't impact the correctness of your data at all).

    [UNIX] 019 021 026 028 033 035 040 042 054 056 061 063 068 070 075 077 082 084 096 098 
[username] 019 021 026 033 035 040 042 054 063 068 070 075 077 084 096 098

You'll see you have a list of values containing the master list of class days (absolute from start of year), and then your days. From that information, you can perform the necessary calculations to get the data you need.

The opus/blog/journal data is recorded, currently, in two categories per week:

• did you make an entry, and is it in accordance with general posting criteria (I've been somewhat softer on the evaluation than the posted requirements otherwise indicate, but I do occasionally crack down on prolific abusers).
• is your entry relevant: is the content in some substantive way related to class, your journey into computing, etc.

You won't be disqualified by incorporating prose, song lyrics, pictures of cats, etc., provided they enhance the necessary core content. If they are there as zero-avoidance filler. well…

Data can look something like this:

    1:opus:week1entry
    1:opus:week1relevant
    0:opus:week2entry
    0:opus:week2relevant
    1:opus:week3entry
    0:opus:week3relevant
    1:opus:week4entry
    1:opus:week4relevant
    1:opus:week5entry
    0:opus:week5relevant
    1:opus:week6entry
    1:opus:week6relevant
    1:opus:week7entry
    0:opus:week7relevant
    0:opus:week8entry
    0:opus:week8relevant
    1:opus:week9entry
    1:opus:week9relevant

That data, specifically for projects, is of the format (sample follows):

4:uxi0:final tally of results (4/5)
*:uxi0:cloned lab46 mercurial repository [1/1]
*:uxi0:subscribed lab46 address to class mailing list [1/1]
*:uxi0:subscribed to class mailing list [0/1]
*:uxi0:customized opus intro [1/1]
*:uxi0:customized opus title [1/1]
16:pbx0:final tally of results (16/16)
*:pbx0:lab46 submission successful [8/8]
*:pbx0:email submission successful [8/8]
16:pbx1:final tally of results (16/16)
*:pbx1:content correct [8/8]
*:pbx1:lab46 submission successful [8/8]
3:bonus:final tally of results (3/0)
*:bonus:uxi0 early submission bonus (2 days) [2/0]
*:bonus:pbx1 early submission bonus (1 day) [1/0]

You should see 2 types of lines. The first is the “final tally” line:

YOUR_SCORE:PROJECT_NAME:final tally of results (YOUR_SCORE/TOTAL_POSSIBLE_SCORE)

The data lines all start with an '*', and merely provide a deeper level of detail (how we arrived at that grand total for the project). It is of the format:

*:PROJECT_NAME:description [YOUR_POINT/TOTAL_POINT]

Each line item's point value (against its total) is presented at the very end of the line in square brackets (as shown).

Finally, to handle bonus points, notice what happens: the “total” values are all out of 0… so it ONLY counts for you, should we get any bonus points, and doesn't count against you should you not gain any.

Each project is clumped together, its “final tally” line leading the stanza, with the follow-up line items beneath it (note: NOT an average, but a tally).

Should there be any points deducted for lateness, they should be on the second line of the stanza (immediately following the “final tally” line). And would have the following format:

*:cus0:LATE penalty (1 day) [-26]

On the command-line, how would you:

• Capture JUST the PROJECTS category as produced by the status tool?
• Determine the total number of projects evaluated?
• Figure out which project(s) was/were worth the most points?
• Figure out which project(s) was/were worth the least amount of points?
• Figure out the total number of project points available?
• Determine the total number of points you received on projects?

Your ultimate task for this project is to write a script that scrapes the data generated by the status tool and computes your overall grade in the class.

I'd like your script to conform to a particular output format, described below.

Here is an example of the sort of information I'd like to see your script output:

lab46:~/src/unix/gfo0$ ./mygrade.sh
===============================================================================
UNIX      uxi0 pbx0 cus0 pbx1 arc0 wpa0 udr0 sfp0 bns prjct+ptcpn+jrnal= result
===============================================================================
username: 100  86   91   79   99   0    0    83   3   23/36+28/28+31/36= 82/100
===============================================================================
lab46:~/src/unix/gfo0$ 

NOTE that in the above, “username” should actually display YOUR USERNAME.

And note that the actual scores are left justified underneath the column heading listing the project name.

As a bonus opportunity, I offer you the following:

Despite how much automation I may have on the backend of things to generate all this data, there is still a considerable amount of manual interaction on my part (as much as I'd love to entirely script the process, evaluation still very much requires that human element), and that means there's always the chance of errors creeping in.

I will consider for bonus points the implementation of a “check” argument, which if provided on your script's command-line, will perform necessary tallys and sanity checks (for example, if a project is out of 39 points, do all your values check out; similarly, do all the line-by-line point items add up to the reported total you have?), and nicely displays any inconsistencies (so that you can report them to me for potential correction).

By no means an exhaustive nor implied list, you may want to ensure familiarity with the following tools/concepts when performing this project:

• cat(1)
• bc(1)
• printf(1)
• echo(1)
• grep(1)
• head(1)
• tail(1)
• sed(1)
• cut(1)
• tr(1)
• wc(1)
• I/O redirection (found in bash(1))
• variables/variable expansion (found in bash(1))
• command expansion (found in bash(1))
• selection structures and evaluation facilities
  • if statements (found in bash(1))
  • [(1)
• loops (found in bash(1))
• arrays (found in bash(1))

Often times related and potentially useful commands you may want to know about can be found in the “SEE ALSO” section near the bottom of many manual pages.

There are some sample scripts available for viewing in the scripts/ subdirectory of the class public directory.

If you add a -x argument to bash (in the shabang, or on the command-line if running your script that way), it will be immensely more verbose, reporting more precisely on its actions. This can prove most useful when trying to figure out various problems, be it syntax errors in your script or spurious logic errors.

Successful completion will result in the following criteria being met:

• When all is said and done, you will submit your script, which:
  • runs the status tool and creates a snapshot of its output. Have your script process this snapshot vs. running status 20 million times during the run of your script (the data may change over time, but it isn't going to change that fast- cache the output to improve performance).
  • should be an organized presentation of the requested information as specified above.
    • spacing is important. I want to see column justification just like in my examples above.
  • is functional, displaying no errors of any kind.
  • utilizes shell logic prodominantly
    • for those who can, do not write a C program that does it all; that doesn't mean you cannot write small programs to assist with pieces of the process though
    • use the various pattern matchers, filters, and text manipulation tools we've been playing with all semester (as well as new ones you find as you look for ways of performing this task).
  • is WELL and CONSISTENTLY COMMENTED.
    • and I don't just want to see “this is an if statement”-style comments. I want to see explanations of why you are doing things, especially the central processes.
  • is CONSISTENTLY INDENTED according to logic level (don't have everything riding the left margin, indent code inside if statments, loops, functions, etc.)
    • see how this bulleted list is consistent and indented?
  • no line exceeds 80 characters in width.
  • the script should entirely automate the process; all you should have to do is run the script, and it will do all the work to output the necessary information.

Please submit as follows:

lab46:~/src/unix/gfo0$ submit unix gfo0 mygrade.sh
Submitting unix project "gfo0":
    -> mygrade.sh(OK)

SUCCESSFULLY SUBMITTED
lab46:~/src/unix/gfo0$