======UNIX-y tasks====== Things you should do before too much time passes: =====November 28th, 2012===== * If I had a dime for everytime someone said they could use a refresher in basic programming / problem solving / computer science concepts... and then this bumps into me: [[https://www.openlearning.com/courses/unsw/computing1|UNSW Computing 1 - The Art of Programming]] 12 week free course starts December 3rd. =====November 9th, 2012===== * Talked out the following commands: * head * tail * paste * join * diff * patch * comm * Among others. Using them in the filtering of data. * [[(UNIX-O07-20121116235959)|OPUS Part 3 Keyword]] is due before it is no longer November 16th. =====November 7th, 2012===== * [[(UNIX-P04-20121114235959)|Super Puzzle Box 2 Turbo]], due November 14th. * [[(UNIX-O06-20121110235959)|OPUS Part 3 Keyword Selection]] select a keyword and get started. =====November 2nd, 2012===== * Change of pace! The X Window System (or X, X11, X11R6/7) * client/server model designed to operate in a networked environment * X clients talk to an X server-- they do NOT have to be on the same machine * The thin clients in the LAIR take advantage of this capability * DISPLAY environment variable * your X location string (host + X screen) * redirected X client output (xeyes, xlogo) to other screens * played on the LAIRwall =====October 31st, 2012===== * Continued to play with RegEx class listing example. I assigned its completion for Friday (e-mail me you final command-line to produce the filtered output). =====October 26th, 2012===== * More fun with Regular Expressions, started an in-class exploration doing some hands-on learning with RegEx on the CCC Spring course schedule. =====October 24th, 2012===== * Regular Expressions are upon us! =====October 19th, 2012===== * Some more process fun: * The **init** process * Hierarchy of processes / connectivity to terminals (PTYs) * The existence of **zombies** * How parents should always prefer their children to die once they complete their usefulness, else they'll become a zombie * Played with **grep(1)**, **sed(1)**, and **cut(1)** to tweak some **ps(1)** output * Used a Regular Expression, something we will talk more about this coming week. =====October 17th, 2012===== * [[(UNIX-P03-20121024235959)|Project 0x03- Puzzle Box 2]] is now available, and is due on the 24th. Different than the first, I would **not** wait until the last minute to complete this one. * We talked about processes (programs in action) * **ps(1)** - process status * **top(1)** - show currently active processes * We talked about signals (the **kill(1)** command), and killed many cats in various odd yet specific ways. * We wrote a [[stdin_redirection|small C program]] that: * took as input 4 integers from STDIN * calculated an displayed their average * Used STDIN redirection to feed the program its data from an external file =====October 12th, 2012===== * Ventured into C programming for a bit * Covered some of the recognized programming paradigms: * structured (most languages we are familiar with) * functional (LISP) * logical (Prolog, Haskell) * object-oriented (Eiffel, Smalltalk) * Wrote and more-closely analyzed "Hello, World" * Implemented an equivalent command-line C program as we did in bash on Wednesday. =====October 10th, 2012===== * More scripting: * "Self-aware" script through the use of file access * "Intruder" detection with an elif and command-line arguments * More extensive command-line arguments, including usage with a list-based for loop. =====October 5th, 2012===== * Due to the impending break, I declared the day a work day, so everyone could get caught up on work and ask questions. * gimmeh is mangling some of the "Unconventional Naming" responses. I am aware of the problem- just run with whatever it decides to store. Any responses that come up blank just e-mail me what you put. =====October 3rd, 2012===== * [[(UNIX-O06-20121016235959)|OPUS Part 2 Keyword]] is due October 16th. * Starting this month, I will cease accepting assignments submitted past their due date (unless there are extenuating circumstances). Many people have been getting assignments done on time, others are procrastinating until the last day, and have trouble as they are rushing and don't have much time left. Please don't wait until the last minute, procrastination isn't the way. * Please look through this wiki page I put together on the [[guess_a_number_bash|"6 tries to guess a number"]] script we started writing in class on Friday. There are some additional variations from what we did. Please play with the code and figure out what is going on. And be sure to ask questions on anything that isn't making sense. * Some insightful commentary on the UNIX Command Line: [[http://geekblog.oneandoneis2.org/index.php/2012/09/30/to-understand-the-command-line|To Understand the Command Line]]. Well worth giving a read. =====September 28th, 2012===== * [[(UNIX-P02-20121005235959)|Project #2: Unconventional Naming]] by October 5th. * Don't forget! The full Part 1 of the OPUS is due once it becomes October. * We explored more with shell scripting, including: * Using **echo** to output information to the user (STDOUT) * Using **read** to input information from the user into a variable (STDIN) * The **$RANDOM** environment variable, which gives us obscenely ranged random whole numbers * **if** statements, enabling us to make boolean choices (exactly 1 per if block) * the closing **fi** statement to end an "if block" (1, at the very end of all the if/elif/else action) * We can have an **else** statement to contain the countering action (0 or 1 in total) * if there is more than one condition we wish to check for, else if (**elif**) statements can be strung along (0 or more in total). * The **[** and **]** commands, giving us some more capable evaluative functionality, including our 6 relational operators: * **-eq** is equal * **-ne** is NOT equal * **-gt** is greater than * **-ge** is greater than or equal * **-lt** is less than * **-le** is less than or equal * The use of the **exit** command to force termination of the script * The good habit of placing a "**exit 0**" at the end of your script * We ended up writing one of those [[guess_a_number_bash|"6 tries to guess a number"]] games, a perfect utilization of I/O, variables, and decisions using an if block. * Customization of the range of random values **$RANDOM** will give us with a little **bc**, **here string**, and **command expansion** magic: * **value=`bc -q <<< "($RANDOM % 10)+1"`** (for a value in the range of 1-10, leave off the +1 and it is 0-9, by the mathematical properties of the modulus division (%)). * Finally, a quick **for** loop was thrown in doing a quick count from 0-5 (6 total iterations) giving the user their 6 choices. =====September 26th, 2012===== * [[(UNIX-O04-20120930235959)|OPUS Part 1 to be completed]] by October 1st. * [[(UNIX-O05-20120930235959)|OPUS Part 2 Keyword Selection]] by October 1st. * Useful commands to know: **head(1)**, **tail(1)** * [[http://www.youtube.com/watch?v=6c3z2ZGvShM&feature=player_embedded|This]] is a great example of when consumers become the [[http://www.shapeways.com/model/683524/super_mario_mobius_strip.html|producers]]. * For those that enabled workdue notifications in **gimmeh**, you may have started receiving daily e-mails (depending on your settings). Remember, it is a double opt-in system- you have to enable **workdue-status**, AND **email-workdue**, otherwise it'll skip you by. * We played a lot more with wildcards, posing some puzzles to craft patterns for (in **/usr/bin**) * All the files 4 chars in length: **ls -d ???? | wc -l** * All the files 2 or more chars in length: **ls -d ??* | wc -l** * All the files that end with a lowercase vowel: **ls -d *[aeiou] | wc -l** * All the files that do not start with an uppercase vowel and whose 3rd letter is a 'd': **ls -d [^AEIOU]?d* | wc -l** * All the files that do NOT start NOR end with a lowercase 'm', but DOES contain elsewhere in its name a lowercase 'm': **ls -d [^m]*m*[^m] | wc -l** * If you are ever in need of an "emergency" ls, **echo *** will work in a pinch, but won't be pretty. * We discovered that the default behavior of **ls(1)** when it encounters a directory is to expand is list that directory's contents. This might seem to "break" our wildcard until we see that it is in fact successfully matching the directory's name, then just brings whatever contents along for the ride. We can fix this by issuing the **-d** argument to ls, which will cause it to treat directories like regular files (ie just list them, and not descend into them). * When you see mention of **command(#)**, it is referring to a particular section of the manual pages for a command (**file(1)**, **cp(1)**, **printf(1)**, **printf(3)**). * To look up a specific section of the manual, insert it as an argument before the command; for example: * **printf(3)**: man 3 printf * **cp(1)**: man 1 cp * By default, without a section number, the first available manual page in the lowest numbered section will be called up. * You can search through the available manual pages with **apropos(1)** =====September 21st, 2012===== * Last full day of summer in the Northern Hemisphere. * [[(UNIX-P01-20120928235959)|Project #1: The Puzzle Box]] by September 28th. * Control Characters were covered (CTRL-a through CTRL-z). * TIP: Do NOT try and issue Control Characters while not on the command line! Programs like VI and SCREEN capture control sequences and can apply different functionality. * Some vi power tips given * recording mode explained and demonstrated * CTRL-a on a number in vim will do something amazing * The opposite is CTRL-x (apparently) * How to unfreeze your terminal (CTRL-q, it is what started us on control characters). * Wildcards were introduced: * * - match 0 or more of any character * ? - match 1 of any character * [ ] - character class; match 1 of any of the enclosed characters * [^ ] - inverted character class; do **NOT** match 1 of any of the enclosed characters * Examples: * ls *.c -- match all files ending with ".c" * ls g?? -- match all 3 letter files starting with a lowercase 'g' * ls ??* -- match all files that are 2 or more characters in length * ls *[rstlne] -- match all files ending with lowercase r, s, t, l, n, e * ls *[0-9] -- match all files ending a number (0-9) * Common mistakes made with the character class: * you don't need to separate values with commas. Putting a comma in a character class will cause the comma to be a character searched for to match that particular pattern. * you don't need a character class if you know the specific character you are looking to match. [e] is more typing than **e**. * Character classes support some ranges: * Single digit numbers: [0-9] * Lowercase alphabet: [a-z] * Uppercase alphabet: [A-Z] * Any letter, regardless of case: [A-Za-z] * Partial ranges: [a-f] * Multiple partial ranges: [b-gm-q] =====September 19th, 2012===== * [[(UNIX-P00-20120926235959)|Project #0: Archive Handling]] by September 26th. * [[(UNIX-O03-20120924235959)|Be sure to evaluate your received OPUS Part 1 Keyword]] by September 24th. Then get started on the demonstration before the month is up. * X11, the traditional and standard UNIX Graphics environment, [[http://www.phoronix.com/scan.php?page=article&item=x11_25_years&num=1|turned 25 on Saturday]]. * I introduced the **gimmeh** tool. Please run it and customize your settings, and tend to any course-related content. * Opus Part 1 keywords have been swapped. You have until the end of the month to wrap up the demonstrations of the received keyword. * We reviewed pipes, which lead into a great introduction to shell scripting. * We saw one-liners like: **last | less** * and: **last | grep $USER | wc -l** * which evolved into: **echo "You logged in `last | grep $USER | wc -l` times this month."** * plus the amazing concept connection to put that line in a file: **echo 'echo "You logged in `last | grep $USER | wc -l` times this month."' > logincnt.sh** * The most frequent mistake is typing the wrong quotes. Double check those things! * grep returns matching lines (by default) based on a provided search pattern (**grep $USER** will return all lines that contain your username, for example). It stands for **g**lobally search for **r**egular **e**xpression and **p**rint (GREP). We will be doing a lot more with grep in the coming weeks. * A script is just a text file with valid command-line combinations set with the execute bit and then run like any other command (**./logincnt.sh**, or invoke through a shell via: **bash ./logincnt.sh**). * Proper shell scripts should have a **shabang** line to start them off- the one most valid for our explorations this semester will be: **#!/bin/bash** * The interpreter specified via absolute path in the shabang line should conform to the syntax present within the script. We'll be learning the **bash** shell ([[http://en.wikipedia.org/wiki/Bash_%28Unix_shell%29|bourne again shell]]) and syntax, so if you're doing stuff for class, that'll probably be the one you want. * On the other hand, if you are exploring something like awk, perl, python, or even some other shell, and using its syntax, putting the appropriate shabang line will yield you the most success. * a script is generally called such as it is typically high level and can be executed as-is. * a programming language is generally called such as its syntax needs to be converted into a more machine-readable form (compiler, assembler) to be run. The distinction is not always that cut and dry, but could be a good rule of thumb. * Mail forwarding is accomplished by sending a copy of any incoming messages to all addresses contained within the **~/.forward** file, located in the base of your home directory. This file must **not** be world writable; if it is, this functionality will be disabled. =====September 14th, 2012===== * Remember that your keyword for the Opus is due to be completed this weekend. * We [[fall2012/common/vim_customize|customized our vim sessions]] by editing/creating (with vim) our **~/.vimrc** file. * We learned how to create files with **touch(1)** * Some explanations of STDIN, STDOUT, and STDERR were in order (remember, everything is a file). * I/O Redirection was then explored (<, >, >>, 1>, 1>>, 2>, 2>>). * I also mentioned the [[http://en.wikipedia.org/wiki/Here_document|here string]] << and we used it with the STDIN redirector to force command-line level input as file-based STDIN redirection: <<< "desired input" * For example, with **bc**: **bc <<< "2+2"** * Pipes were then built (using the **|** symbol) for some awesome incantations. * Variables and the variable expansion operator ($) * Quotes were covered (half- "", full- '', and command expansion `` (backticks)). * File permissions were covered, and accomplished using the **chmod** command. * We looked at what happens when you have read-only and write-only files. * [[(UNIX-O02-20120916235959)|Remember that your OPUS Part 1 Keyword]] is due by September 16th. We'll be doing a swap shortly thereafter. =====September 12th, 2012===== * We finished covering the basic vi commands (prefix + [modifier] + command) * for navigation * by character: h, j, k, l (and cursor keys) * by word: w/W, b/B * by line: ^, $, #G * for insertion: i, I, a, A, o, O * for getting back to command mode: ESC * for modification: * copy/yank: prefix + y + navigation command * cut/delete: prefix + d + navigation command, x/X * paste: prefix + p * change: prefix + c + navagation command * toggle (alter case): prefix + ~ (tilde) * substitute: prefix + s * join lines: prefix + J * undo: u * for searching: * initiate a search: /thing you are looking for * jump to next match: n * jump to previous match: N * We also looked at a few of the extended vi command mode commands (hit ":" to enter extended command mode): * move: //address//**m**//address// * copy: //address//**co**//address// * search/replace: //address//**s**///pattern/////replacement///g * save (current file): **w** * save (as file): **w** //filename// * quit (if saved): **q** * quit (without saving): **q!** * save and quit: **wq** * An //address// can be one of the following (you can mix and match them as is appropriate): * an absolute line number: **7** (7th line in the file) * a relative line number: **.** (current line), **-2** (two lines prior) * an absolute range: **1,3** (lines 1-3) * a relative range: **.,+5** (current line through the following 5 lines) * for search/replace: **%** (entire file) * Be sure to take some time to play in **vi/vim**. We covered some of the basic commands last week (**i**nsert mode, **ESC** for command-mode, character (arrow keys/h, j, k, l), word (w, b), and line (^, $) navigation). * In addition to playing in vi, be sure to keep playing on the command-line with the tools we've learned so far. * There will likely be a project appearing some time this week or next. =====September 7th, 2012===== * By this day, be subscribed to the class mailing list, and have chosen a keyword. I will count the successful completion of these two things as a project. * Please make some time to play- experiment with the commands we've learned; explore the filesystem. * To prepare for what is soon to come, please [[http://thomer.com/vi/vi.html#why|glance at this]] and start getting ready to change the way you think about editing files. * If you haven't set up your data directory yet, please type this: **ln -sf /usr/local/etc/data/$USER ~/data** * If after doing this, you see a red "data" symlink in your home directory, you typed that wrong. * Please be mindful of others- at times it has been approaching too noisy with conversations during class. I encourage communication, but not at the expense of other's learning- learn the fine art of soft speaking or whispering. * UNIX Historical Video: [[https://www.youtube.com/watch?v=tc4ROCJYbm0|AT&T Archives: The UNIX Operating System]] =====September 5th, 2012===== * My assertion from the first day of class is that Joe O's preference for Internet Explorer, despite the fact that he knows better, is an example of [[http://en.wikipedia.org/wiki/Akrasia|akrasia]] (the state of acting against one's better judgement). * For your consideration, [[http://en.wikipedia.org/wiki/Clarke%27s_three_laws|Arthur C. Clarke's Three Laws]]: - When a distinguished but elderly scientist states that something is possible, she/he is almost certainly right. When they state that something is impossible, they are very probably wrong. - The only way of discovering the limits of the possible is to venture a little way past them into the impossible. - Any sufficiently advanced technology is indistinguishable from magic. * Please to be learning and committing to memory the three [[http://en.wiktionary.org/wiki/tenet|tenets]] of the UNIX philosophy (then sample some [[http://en.wikipedia.org/wiki/UNIX_philosophy|other views of UNIX Philosophy]] and [[http://www.catb.org/esr/writings/unix-koans/|UNIX Koans]]): - Small is beautiful. - Do one thing, and do that one thing well. - [[http://en.wikipedia.org/wiki/Everything_is_a_file|Everything is a file.]] (well, [[http://www.informit.com/articles/article.aspx?p=424451|99% of the time]]) * Please do yourself a favor and read [[http://unixmages.com/ufbm.pdf|UNIX for the Beginning Mage]]. * Although interpretations vary, I prefer to believe there are [[http://pic.dhe.ibm.com/infocenter/aix/v6r1/index.jsp?topic=%2Fcom.ibm.aix.baseadmn%2Fdoc%2Fbaseadmndita%2Ffiletypes.htm|three types of files]] (please familiarize yourself with them and how they differ): - regular - directory - special * Also, begin acclimating yourself to the notion of the standard system [[http://en.wikipedia.org/wiki/Standard_streams|data streams]]: * 0. Standard Input (STDIN) * 1. Standard Output (STDOUT) * 2. Standard Error (STDERR) * Become comfortable with navigating the filesystem, for we will soon be exploring some sights, and the more familiar you are with **cd** and **ls**, the better. * If you enjoy contemplating incomprehensible objects and seeing their adoption into culture, check out the [[http://www.catb.org/jargon/html/B/blivet.html|blivet]]. * You have an e-mail account on Lab46. Your address is **lab46username@lab46.corning-cc.edu**. You can check your mail with the **alpine** command. =====August 31th, 2012===== * I had a request for documentation on setting up the screen/irssi session for class chat. [[fall2012:common:class_chat|Here it is]]. * Clone and start using your personal [[fall2012/common/repo|Lab46 Mercurial Repository]] * [[(unix-O01-20120906235959)|Select your Part 1 Opus keyword]] BEFORE September 7th. * Start working on said keyword, for you need to have the definition part done by mid-September. * Before leaving, detach from any screen sessions (**CTRL-a d**), log out of any terminals (**exit** command), close any web browsers or other open applications on your desktop, and **right click -> exit** to log off. You do NOT want to leave while still being logged in! * familiarize yourself with some commands we have learned so far: ls, hg, man, wtf, pom, cd =====August 29th, 2012===== * [[(unix-O00-20120906235959)|Edit/customize at least the Title and Introduction sections of your Opus]] BEFORE September 7th. * Subscribe a frequently checked (or pushed) e-mail address to the [[http://lab46.corning-cc.edu/mailman/listinfo/unix|class mailing list]] * When you come to class, remember to open a Lab46 terminal, which will be used for attendance/participation purposes. * Make sure you can access the [[http://lab46.corning-cc.edu/haas/fall2012/unix|UNIX]] course homepage from home (you may want to bookmark it). * Make sure you can log into Lab46 from home with your username/password (using PuTTY if on Windows, or some other SSH client). * Get on the class chat, and familiarize yourself with basic screen usage (attaching/detaching). * * familiarize yourself with some commands we have learned so far: who, man, screen, irssi, exit/logout