Differences

This shows you the differences between two versions of the page.

--- haas:spring2014:unix:projects:dataproc [2014/03/23 21:06] – [Task 1: Obtain irc bot] wedge
+++ haas:spring2014:unix:projects:dataproc [2014/03/23 21:29] (current) – [Submission] wedge
@@ Line 12: / Line 12: @@
 =====Background=====
-To most of us, computers are a frequently used interactive tool for accomplishing work. With our recent explorations into the realm of shell scripting, concepts of automation are starting to enter our peripherary.
+Often times, we will find ourselves encountering data in a slightly one-off format- not quite meeting some requirement we need for further processing.
-With automation, comes the need to do things outside that interactive environment, and run at a designated time or in reaction to a particular event.
+Luckily, the UNIX environment provides many facilities for filtering and manipulating data so that we can "reformat" it to meet expectations.
-We will be scheduling tasks with respect to time in this project, to get better acquainted with the functionality and capabilities of non-interactive yet automated tasks.
+This activity has you dabbling in one such scenario: a program that generates "raw" data (simulated from a scientific/industrial instrument). This "raw" data needs to be sanitized and reformatted (to perhaps be further analyzed by other tools upstream).
-=====cron=====
+=====Task 0: Post/respond to a question=====
-From the wikipedia article on [[wp>Cron]]:
+  * Because the class mailing list has been rather quiet of late, and we've got a break coming up, I would like each person to post at least 1 focused question regarding this project to the class mailing list.
+    * Please do not give away any answers to the actions requested by this project in doing so.
-"Cron is a time-based job scheduler in Unix-like computer operating systems. The name cron comes from the word "chronos", Greek for "time". Cron enables users to schedule jobs (commands or shell scripts) to run periodically at certain times or dates. It is commonly used to automate system maintenance or administration, though its general-purpose nature means that it can be used for other purposes, such as connecting to the Internet and downloading email."
+    * Be sure to identify which "task" or aspect of the project you are asking about
+  * Respond to at least 1 question, not by giving an explicit answer, but by asking further questions, or giving a pointer to a resource that may contain additional information (i.e. see **cut(1)** manual page)
-Be sure to check the manual page for **cron**(**8**), and the corresponding manual pages for **crontab**(**1**) and **crontab**(**5**). When you are familiar with where pertinent information can be found regarding cron, proceed to the question below.
+    * To get credit, your response can**not** be to one of your own questions.
+  * Put a URL to the mailing list post of your question asked in a file called: **task0.question**
+    * See http://lab46.corning-cc.edu/mailman/listinfo/unix to access the archives
+  * Put a URL to the mailing list post of your response in a file called: **task0.response**
+    * See http://lab46.corning-cc.edu/mailman/listinfo/unix to access the archives
+    * A question may receive multiple answers.
 =====Task 1: Obtain source code=====
@@ Line 116: / Line 121: @@
 </code>
+NOTE: Copying/pasting this code into a file to do the project will not earn you credit for task 1. You MUST copy the file from the specified location.
 =====Task 3: Execute your program=====
@@ Line 130: / Line 136: @@
   * Ignoring the index values in the left-most column, determine which numerical codes occur more than once by concocting a command-line incantation or script that appropriately filters and processes the output.
   * Also display with a count of the total number of lines in the output, along with the total number of lines with valid numeric values (ignore "blank" lines and lines with error codes). Finally, display the total count of lines that have duplicates.
-  * Put your resulting command-line(s) or script in a file called **task5.txt**
+  * Put your resulting command-line(s) or script in a file called **task5.sh**
   * Put the output (result) of your command-line(s) or script in a file called **task5.out**
@@ Line 166: / Line 172: @@
   * Sorts the resulting numeric data according to the value to the left of the dash.
   * Re-indexes the data to create a new, more refined, data file. Have a single tab separate the index value from the data value on each line.
-  * Put your logic in a file called **task6.txt**
+  * Put your logic in a file called **task6.sh**
   * Put your output in a file called **task6.out**
-=====Task 2: Configure irc bot=====
-An irc bot, being a network-aware piece of software, needs sufficient configuration in order to operate properly. While it is up to you to derive a working configuration, you'll want to keep in mind the following information:
-  * irc handles are limited to 9 characters max (and they may not like starting with numbers or having spaces in them)
-  * the irc server you want to connect to is: **irc.offbyone.lan** (port 6667 if it matters)
-  * you may want to initially configure your bot to join a secluded channel so you can test it. For the project, it will ultimately need to join: **#botchan**
-  * you should configure both yourself and me (username **wedge**) as administrators for the bot.
-Verify you can successfully start the bot and that it connects to the intended server and channel. You may want to run it in a sub-console in your screen session, so that you can keep an eye on any messages it generates.
-=====Task 3: Enhance the bot with modules/plugins=====
-In addition to core usability, I'd like you to enable additional functionality through the use of modules. A few modules come with the stock Phenny software distribution, and there appear to be a few third-party sources, such as:
-  * phenny-games
-  * oblique
-Install and enable modules for your bot, and verify some form of functionality.
-**NOTE:** Due to changes in the service it uses, the Phenny **weather** module is beyond broken. Trying to use it will result in an error being displayed. If you are skilled with Python and can craft a solution, that can certainly count toward completing this task of the project.
-=====Task 4: Script to check bot status=====
-Using tools and concepts we learned previously, especially:
-  * ps
-  * grep
-  * pgrep
-Write a script that checks for a currently running Phenny instance (run by you). If not instance is found, launch a new instance. If an instance IS running, do nothing.
-Be sure to make use of **absolute paths**.
-=====Task 5: Script to kill running bot=====
-Using tools and concepts we learned previously, especially:
-  * ps
-  * grep
-  * pgrep
-  * kill
-  * pkill
-Create a script that will check for a currently running Phenny instance (run by you). If an instance is running, kill it.
-If no instance is running, do nothing.
-=====Task 6: Create a cron job=====
-Reading up on cron and figuring out how to add an entry to your user's personal crontab, add a job that runs your check script (Task 4 script) every 10 minutes.
-You can verify successful cron job deployment by ensuring your bot is not running and waiting for the next 10 minute marker and see if it starts.
-=====Task 7: Create an at job=====
-While cron is used for long-term scheduled jobs, **at** is useful for short term ones.
-Read up on **at**, and deploy an **at** job that will run your **botkill** script (task 5) a couple minutes before a 10 minute marker.
-Use this to test both your script and correct usages of **at**, as well as the **cron**-related activities.
 =====Submission=====
@@ Line 228: / Line 179: @@
   * All criteria indicated above
-  * To signal completion, submit an archive containing the following files:
+  * To signal completion, submit an archive containing all the files generated in each task above.
-    * botcheck script (call it **botcheck.sh**)
+    * Task 0: **task0.question** and **task0.response**
-    * botkill script  (call it **botkill.sh**)
+    * Task 1: **task1.txt**
-    * a text file called (**at.txt**) containing syntax used to create/deploy **at** job
+    * Task 2: **task2.txt**
-    * a listing of your crontab entries (**crontab.txt**), showing the correctly scheduled deployment of your script
+    * Task 3: **task3.txt**
-    * a copy of your configured bot's config file (**config.py**)
+    * Task 4: **task4.txt**
-    * a text file containing the URL you downloaded your bot, along with any URLs you obtained modules/plugins used (**botdata.txt**)
+    * Task 5: **task5.sh** and **task5.out**
-    * Put all these files in a **tar** archive
+    * Task 6: **task6.sh** and **task6.out**
-    * Compress it with max compression using **bzip2**
+    * Put all these files in a **tar** archive called **dataproc.tar**
-    * Name the archive **ircbot.tar.bz2**
+    * Compress it with max compression using **gzip**
-  * After you've recorded the crontab entry with your botcheck script, remove the entry from your crontab (so we don't end up with a bunch of phantom bots nobody cares about).
+    * The resulting archive should be named: **dataproc.tar.gz**
-  * When you're done with this project, kill your bot. Do not leave it running.
-**NOTE:** If this activity fascinates you and you'd like to keep playing with your bot, do so using a different config/handle, so it will not be confused with your project submission. If you want to schedule it, do so using a differently named script at a different time interval).
 To submit this project to me using the **submit** tool, run the following command at your lab46 prompt:
 <cli>
-$ submit unix dataproc ircbot.tar.bz2
+$ submit unix dataproc dataproc.tar.gz
 Submitting unix project "dataproc":
-    -> ircbot.tar.bz2(OK)
+    -> dataproc.tar.gz(OK)
 SUCCESSFULLY SUBMITTED

Lab46 Wiki

User Tools

Site Tools

Differences

Page Tools