Kate Schermerhorn's Fall 2012 Opus
A new chapter.. for a brand new book…
After 16 years in an administrative / management position, I recently decided that I was ready for a career change. My previous career had afforded me the opportunity to dabble in the computer/network/system administrative realm just log enough to peak a keen interest into the “unknown”. I am currently pursuing a computer science degree, with hopes to transfer to Binghamton University in the Spring of 2014. At the present, outside of classes, my life primarily revolves around spending time with my husband and two boys. However, I love to travel and look forward to having more opportunities to explore the world.
I just wanted to record how I felt on the very first day of class…. Clueless! Totally Clueless! Before today, I would have described myself as computer literate and quick to catch on. However, today… I feel like I just stepped off the plane in a foreign country. Everyone is speaking a mile a minute, and I am lucky if I can catch a single word that I understand. Ahh… a challenge. Ok, time to pull out the pocket translator and get to work. Heck, I wouldn't be here if I already knew this stuff…. right…RIGHT.
Thus far, we have been working in a non GUI environment which is definitely a little intimidating. I feel a little confused…as in my C++ program class we are also using Unix (I think), but that version has a GUI interface. Today in class we set up Mecurial on our accounts. Mecurial is one type of version control software that we will be using to track changes to our projects. I am a little confused as to how and when we will be using Mecurial…. as there doesn't seem to be a user interface. We just type commands when we want to save, update and commit?
We also explored many different commands today: rm, cd, ls/dir, mkdir, rmdir, pwd, passwd, clear, man, cp, mv, wtf, pom who/ whoami, screen, irssi, exit/logout/CTRL +D
We explored the difference between absolute and relative paths. An absolute path gives you all the information needed to get you to a specific location. For example: cd/home/user/src Whereas a relative path only gives you directions to a specific location starting from your current location. Example: cd src
Remember on the first day of class, when it was highly suggested that we should read Unix for the Beginning Mage _? Well, like the “good” student I am.. I started to read it the first day of class… but then I put it down after the first couple of pages…. as the viewing screen on my phone… just wasn't cutting it. I will be honest, I didn't pick it back up until after I missed a couple of classes. WOW! It took about 5 hours to read and work through the entire book and exercises.. but let me tell you, I wish I had done this during week 1. With the background from the book, I was able to read through the missed class notes, and actually feel like I had at least a good basic working knowledge of the topics covered. To anyone who has yet to read it, and to all future class attendees…. read ASAP!
Today, I think I figured out the concept of the screen sessions within a terminal. Up until this point, I had been under the impression that every time I logged into lab 46, I had to reattach to a screen session, and then alternate between the bash sessions.. i.e. chat and command line. It was until we started playing with Ctrl + options in class today, did I realize the error of my ways. The Ctrl + options do not have the same command effects in a screen session that they do in the home terminal session. Therefore, for future use… screen sessions should be used for purposes of chat, or if there comes a time when it is necessary to leave the computer for a period of time (leave class), and then come back (perhaps from home) and pickup exactly where I left off.
File Archival
File archival, sometimes referred to as concatenating files, is used to join multiple files and data regarding those files into a single file which can then be compressed to reduce the amount of required storage space needed for files. Each computer platform or operating system has their own formats for concatenating files. Tar, ar, and shar are the file formats used by Unix operating system to join the files and gzip is the format of the compressed file. Each format is distinct in the treatment of the files. For example, the tar format is used for the creation of tape archives, and has replaced the ar format for most files other than static libraries. The shar format is for shell script archives.
List any sites, books, or sources utilized when researching information on this topic. (Remove any filler text).
Data Streams (STDIN, STDOUT, STDERR)
Data Streams: Incoming and outgoing information between commands, files, programs and the computer.
Standard Input (STDIN)
Data being accepted as incoming information into another command, file or program.
Standard Output (STDOUT)
The information displayed or redirected from a command, file or program.
Standard Error (STDERR)
The message displayed when a problem occurs with a command, file or program. This message can be redirected like standard output as long as the file descriptor "2" is used before the redirect indicator.
1. Unix for the Beginning Mage A Tutorial by Joe Topjian
Demonstration of Data Streams.
In the example below, the STDOUT of the who command is being piped as STDIN to the wc-l command. The STDOUT of the wc -l command is the number 9, which in this example is simply displayed on screen.
lab46:~$ who | wc -l 9
In the first example below, the STDERR message is displayed on screen from a bad command. The second example shows the STDERR message being redirected to a file, and the cat command used to display contents of file.
lab46:~$ ls files ls: cannot access files: No such file or directory
lab46:~$ ls files 2> error.txt lab46:~$ cat error.txt ls: cannot access files: No such file or directory
Is it possible to import a google address book into the Alpine mail program?
http://mailman2.u.washington.edu/pipermail/alpine-info/2010-July/003413.html
http://www.wijeyesakere.com/tech/alpine/
http://linux.die.net/man/1/alpine
It appears that there are a few different approaches for importing an existing email address book into the Alpine email client. The address book appears to work with the CSV format. Many have used a script to sync Google email address books with the Alpine address books while others have copied and pasted addresses from google into the .addressbook or appended a spreadsheet of data into alpine file. Upon further reading, it appears that it may even be possible to configure an outside email account to open within the Alpine environment instead of having to go out to a web browser.
Based upon the reading I believe that it is most definitely possible to import an outside email address book from Google into the Alpine client, as there have been others who have achieved success.
I am going to try copying the contacts from my Google account and pasting into my Alpine address book. Currently I do not have any contacts in my Alpine address book, so if my hypothesis is correct, after I finish my experiment, I should be able to open my address book with in Alpine and have contacts available.
Yes, my hypothesis was correct. You can import addresses from an outside account into the Alpine account. The downside of the way I imported the addressees is that it is a manual action. I would need to do this action occasionally in order to keep the accounts synced.
While exploring the possibility of being able to get outside contacts into Alpine, without having to manually re-type them, I learned more about the functionality of Alpine, and started to realize its functionality as a front end for other email accounts.
Today we learned about fixing file names that have spaces.
In Unix white space is an argument separator. Therefore a file named: the best file.txt will cause issues when you try to run a command on it.
There are three main ways to fix or run command on the file. 1. Use Quotes. Put the file name inside quotes. Example: “the best file.txt” 2. Use the Wild Card Example: cat the* 3. Use the whack (\). It is an escape character that disables the whitespace or character following the whack. Example: cat the\best\file.txt
Other uses for whack (\):
\n new line character
\b backspace
\t tab
Today we were given Tools to Fix Problems, i.e we learned how to kill processes!
By the act of running a program, you create a process or an instance (copy of program in memory that is running) of that program. You can see what processes you have running by using the ps command.
lab46:~$ ps
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND ks010285 6609 0.0 0.1 13636 2024 pts/32 SNs 13:53 0:00 -bash ks010285 6684 0.0 0.0 10360 992 pts/32 SN+ 14:28 0:00 less ks010285 6918 0.0 0.0 13664 348 pts/17 SNs+ Sep21 0:00 /bin/bash ks010285 7321 0.1 0.1 13628 1952 pts/33 SNs 15:21 0:00 -bash ks010285 7324 0.0 0.0 8588 984 pts/33 RN+ 15:21 0:00 ps u ks010285 11118 0.0 0.0 13596 20 pts/52 SNs Aug29 0:00 /bin/bash ks010285 11233 0.0 0.1 42436 1704 pts/52 SN+ Aug29 8:08 irssi
ps aux will give you all processes running on a system regardless of user.
lab46:~$ ps aux
The top command will show the most active processes running on a system.
lab46:~$ top
Once you have identified a process by its PID, you can run any number of commands on that PID including the kill command which will terminate processes in different manners.
kill -1 will terminate a process nicely, closing any files if necessary kill -2 will interrupt a process. Similar to Control C kill -9 should be used sparingly as a last resort when process isn't responding to other commands.
Todays class lecture was very helpful as we explored the use of regular expressions, and my confusion as to when to use regular expressions and when to use wildcards was resolved.
Wildcards are used for pattern matching outside of a file. Regular expressions are used for pattern matching within the file.
Basic Regular Expressions
Important! Put regular expression pattern in ' '.
(grep, vi, sed and regular expressions go together.
^ - match beginning of line
$ - match end of line
\< - match beginning of word
\> - match end of word
. - match any single character
%%*%% - match 0 or more of the previous
[ ] - character class - match any of enclosed
[^] - inverted character class - do not match any of enclosed
Examples:
'and' : all lines that have this pattern somewhere in word
'^or' : all lines that start with or
'^[Oo}r : match beginning of line that has either upper O or lower o
'^[Oo]r.*e$ : match beginning of line w/ upper or lower Or, 0 or more of any character, and ends with lowercase e.
'^[^aeiouy].*g.*[es]$' : do not start with a lowercase vowel, have a 'g' somewhere and end with a 's' or 'e'.
'...k' : at least 4 characters end with a k.
Extended Regular Expressions
In order to use extend regular expressions, you need to use the egrep command.
() - grouping with see command, there are \( \) there are backslashes around parentheses.
| - or
+ - match 1 or more of the previous
Examples:
lines ending in ing or ed: egrep'(ing|ed)$' or '(ing$|ed$)' or '^.*(ing|ed)$' or 'ing$|ed$'
Patterns: Its important to be observant of one word per line patterns.
Common Debugging Strategies
The following are variables:
{0}
{0#**/}
If you want to find out what is in a variable you can echo the variable in your script. i.e.
echo ${0}
Today we put into practice the regular expressions that we learned on Wednesday. We created a command line script to grab course information from a previously saved 2013 course html file from the CCC website.
The first step was to view the html file, therefore we performed the cat command on the file.
lab46:~$ cat 2012courselist | less
By recognizing patterns within the html code, we were able to identify key pieces of the code that were necessary for pattern identification for our next command. As the line that contained the course information began with <TH CLASSS=“ddtitle”, we used the grep command and the regular expression ^ (line begins with identifier) to grab all of the lines that contained course information. In order to view the information in a page like environment, we piped the “grepped” file to less.
lab46:~$ cat 2012courselist | grep '^<TH CLASS="ddtitle"' | less
This produced all the html lines that began with <TH CLASS=“ddtitle”. As this list was still messy, we cleaned up and formatted the list by using the sed (string editor) command.
A few key points about the sed command:
As we were only interested in the course numbers and course names from these html lines, we used the sed command to get rid of all the information before the course name, by identifying the portion of the line that came before the course name with following regular expressions:
lab46:~$ cat 2012courselist | grep '^<TH CLASS="ddtitle"' | sed 's/^.*crn_in=.....">//g'
We cleaned up the list further, by deleting the “garbage characters” at the end of the line with the following regular expressions:
lab46:~$ cat 2012courselist | grep '^<TH CLASS="ddtitle"' | sed 's/^.*crn_in=.....">//g'| sed 's/<\/A><\/TH>$//g' | less
Finally, in order to change the order of the information on the line, the following command line argument was written. By identifying groups between the dashes within the original pattern, the groups were able to be replaced once again by using the sed command.
\1 \2 \3 \4
Title -CRN -CRSE NUMBER -SEC
CRSE NUMBER-SEC:CRN:Title
lab46:~$ cat 2012courselist | grep '^<TH CLASS="ddtitle"' | sed 's/^.*crn_in=.....">//g' | sed 's/<\/A><\/TH>$//g' | sed 's/^\(.*\) - \([0-9][0-9][0-9][0-9][0-9]\) - \(.*\) - \([0-9]*\)$/\3-\4:\2:\1/g' | less
shell
A shell acts as the interface between a user and an operating system in which the user can launch other programs or access directories. There are two types of shells, command line or graphical. A windows operating system uses the graphical interface Windows Explorer to allow users to access programs and view files, where as a Unix systems tend to use the command line interfaces wherein text is typed at the command line prompt in order to launch programs and enter commands to view directories.
here string
The symbol for a here string is «< . A here string puts whatever input you want into a command. Like bc «< 2+2 sends 2+2 into the calculator and it will come back as 4. So it does the operation in one step instead of going into the calculator and typing 2+2 or whatever you want to solve.
* Notes from when we talked about here strings and the binary calculator.
Demonstration of here string
lab46:~$ bc <<< 86+24 110
Is it possible to make a script executable when you create it (at inception) rather than having to change the permissions after you write it?
I discussed the possibilities with Professor Haas, and we worked to create a script that will set the permissions of the file at the creation of the script.
I would have to surmise that it is most definitely possible to create a script executable upon initial creation.
Despite the fact that I was told that it was possible by a very reliable source, it was my gut instinct that with the endless possibilities involving scripts and command line arguments, that there must be some way to set permissions.
I am going to write a script that includes the steps to create and set permissions of a file. After I have written the script, I am going to run the script and test out the results.
Step 1: Create a script named mkscript.sh
#!/bin/bash # #set the permissions of an executable script if [ ! -z "$1" ]; then touch $1 chmod 700 $1 vi $1 else echo "Enter name of file to be created with mkscript" fi
Step 2: Set the permissions on the mkscript.sh to executable
Step 3: Executed mkscript.sh and enter a file name to be created.
The if statement of the script tests to make sure that when the mkscript.sh is executed, that a file name is given for the new file. If not, it prompts the user to enter a file name when executing the mkscript.sh. Once a file name is entered, script creates the file, changes the permissions to executable, and opens the file in vi for editing.
1. Ran script without entering a file name
lab46:~$ ./mkscript.sh Enter name of file to be created with mkscript
2. Ran script including a file name
lab46:~$ ./mkscript.sh testfile
3. vi opened automatically. Wrote a short script in the file and then saved and closed.
#!/bin/bash # #testfile from mkscript echo This is a test to see if my experiment worked!!!!
4. Ran the file command on the testfile
lab46:~$ file testfile testfile: Bourne-Again shell script text executable
5. Executed the testfile
lab46:~$ ./testfile This is a test to see if my experiment worked!!!!
Based on the data that I collected, my hypothesis is correct. It is possible to set the permissions of a file at inception through the use of another script file.
Through the use of scripts, you can streamline any repetitious task.
Today we experimented with running something on one system but displaying it on another system. We did this by using variations of the xterm command. For example we ran xeyes, on our own terminal but would send the output to a different terminal with the following command line:
lab46:~$ xeyes -display flake01.2
When you run a process like this in the foreground, you cannot work on anything else until you Ctrl C out of the process. However, there is an option to run processes in the background. If you use the & at the end of a command line, this will put the process in the background so that you can work on other items.
lab46:~$ xeyes -display flake01.2 &
It is important to remember, that you can't Ctrl C out of a a process that is running in the background. You have to kill the process in the command line.
lab46:~$ kill xeyes
Today we discussed the tail and head commands. The tail command allows you to grab the last lines of a file or a previous command output. The default number of lines that the tail command grabs is ten, however you can specify how many of the end lines you want to grab by adding -n to the tail command.
Example
lab46:~$ last | cat -n | tail -5 906 lalexan3 pts/33 10.100.60.118 Thu Nov 1 11:03 - 12:06 (01:03) 907 jpettie pts/29 rrcs-184-74-43-2 Thu Nov 1 09:59 - 12:21 (02:21) 908 wedge pts/19 10.80.81.2 Thu Nov 1 07:22 - 11:10 (03:47) 909 910 wtmp begins Thu Nov 1 07:22:52 2012
The head command does just the opposite. It grabs the first lines.
Example
lab46:~$ last | cat -n | head -5 1 csimon1 pts/83 grrasp.offbyone. Wed Nov 14 15:46 still logged in 2 wedge pts/79 grrasp.offbyone. Wed Nov 14 15:42 still logged in 3 jcavalu3 pts/77 caprisun.offbyon Wed Nov 14 15:39 still logged in 4 dsherbur pts/33 grrasp.offbyone. Wed Nov 14 15:37 still logged in 5 wedge pts/76 grrasp.offbyone. Wed Nov 14 15:36 still logged in
As seen above, the cat -n command, numbers the lines of output. This is extremely useful in order to search and grab lines.
We were also given a list of other helpful commands to look up and research.
Today we discussed networking, with networking being defined as communication among devices between a common medium using common protocols.
A computer needs consistency and therefore needs to adhere to protocols.
Every computer on a network has a unique IP address.
Example: 10.80.2.83
With this being the last official class for November, I leave this class with just the final project left to accomplish. I definitely feel like I have gained a vast amount of information about the Unix system. I feel that I am well equipped to continue experimenting with the Unix system, however there are still some areas in which I struggle. I tend to make things more complicated then they are, so with the simplistic philosophy of the Unix system, I feel that there must be more efficient ways to accomplish tasks then the way that I am trying to do them. I still need to learn the most simplistic ways to combine commands in order to accomplish a task. I guess with time and practice, I will get the hang of it.
Tail
The tail command allows you to grab the last line(s) from the output of a previous command or from a file. Unless you specify the number of lines that you would like the command to grab, it will default to return the last ten lines. In order to specify the the number of lines, add -# (# being the actual number if lines you want to grab) to the command.
List any sites, books, or sources utilized when researching information on this topic. (Remove any filler text).
mesg/write/wall
Definition (in your own words) of the chosen keyword.
List any sites, books, or sources utilized when researching information on this topic. (Remove any filler text).
Demonstration of mesg
To check current mesg access on terminal:
lab46:~$ mesg is n
To turn mesg access on and check that it is turned on:
lab46:~$ mesg y lab46:~$ mesg is y
To turn mesg access off and check that it is turned off:
lab46:~$ mesg n lab46:~$ mesg is n
Demonstration of write
I opened two terminals for this demonstration. One at pts/49 and the other at pts/63.
From my terminal on pts/63
lab46:~$ write ks010285 pts/49 HI Kate on pts 49, this is just a test to show how the write command works! Pretty neat. I will hit the CTRL and D character after this line in order to get the EOF symbol. -o lab46:~$
On the pts/49 terminal, I received this message:
Message from ks010285@lab46 on pts/63 at 14:52 ... HI Kate on pts 49, this is just a test to show how the write command works! Pretty neat. I will hit the CTRL and D character after this line in order to get the EOF symbol. -o EOF
Demonstration of wall
lab46:~$ wall testwall -bash: /usr/bin/wall: Permission denied
How does the tail command with the -f option work?
Unix Man Page : The -f option allows you to follow a file and receive updated output as the file gets bigger.
Harley Hahn's Guide to Unix and Linux: You can use the -f option of tail either in the foreground or background. If you run in the foreground, you cannot enter anymore commands in the command line until the tail command is halted. An option here is to run the tail command in the foreground of a separate terminal window and Ctrl C when you wish to end the command. If you run in the background with the &… you can monitor a growing file while you work on something else. You have to use the kill command to end the tail process.
Based on what I have read, the tail -f command monitors a file for changes in the number of lines in the file, and outputs changes as lines are added to a file, therefore, I believe that when I run the command on a file that I create for monitoring, it will initially spit out the last lines of the file. When I update the file, it will display the new lines that I add to the file.
How are you going to test your hypothesis? What is the structure of your experiment?
lab46:~$ cat > finalExp
lab46:~$ tail -f lastExp
* I will then update the lastExp file in the first terminal window, and watch the second terminal window for updates. * When I have finished updating the lastExp file, I will Ctrl D out of the first terminal window, and Ctrl C to end the tail -f process in the second terminal window.
First terminal session
lab46:~$ cat > finalExp This is just a test file to test the tail -f command and follow option. I will be running the tail -f command in the foreground, so I will have two terminal windows open. I hope that this works the way I am hoping. lab46:~$ cat >> finalExp Okay, now I am adding additional lines to my file. Hopefully this line will appear on the other terminal screen. It worked! Everything that I am typinng on this screen is appearing on the other terminal screen! Yes! Experiment Completed for tonight! ^D
Second terminal session
lab46:~$ tail -f finalExp This is just a test file to test the tail -f command and follow option. I will be running the tail -f command in the foreground, so I will have two terminal windows open. I hope that this works the way I am hoping. Okay, now I am adding additional lines to my file. Hopefully this line will appear on the other terminal screen. It worked! Everything that I am typinng on this screen is appearing on the other terminal screen! Yes! Experiment Completed for tonight! ^C
Based on the data collected:
The tail -f option could be a useful tool for monitoring logins and updates to files. However, as it appears to only give additional lines to a file, it would only be useful for monitoring appends or additions to files, and not for monitoring internal changes to existing information.