August 26, 2014

First day of class. Time to become a mystical powerful hacker through the use of a black background coloured terminal. Yay for listening to the amazing speeches of Matt all class and going over the syllabus.

August 28, 2014

Second day of class. Had fun using wall on everyone to mask myself as a hacker. Also did some advertising of CIST club. Matt went over some very basic commands and then talked about the unix philosophy.

The Unix Philosophy

1. Ordered List ItemSmall is beautiful
2. Ordered List ItemDo one thing, and do that one thing extremely well.
3. Ordered List ItemEverything is a file (almost).

3 Types of Files

1. Regular/Ordinary
2. Directory(links) - metadata
3. Special

3 Tiers of Ownership

1. user
2. group
3. other (world)

Permissions

• - means no permissions
• x means execute or search
• w means write
• r means read

September 2, 2014

Third day to the most amazing class ever! I get really excited to come to this class for some reason.

Started out class talking about projects. There is a new notes project where Matt will pick 3 people to put class notes on an official notes page for each class. There will be an author, designer, and editor. I remember that Matt had talked about doing this last semester, but it never actually happened. I am very interested to try it out this semester. I'm not sure how this will be better than just putting our own notes on our opus. We will still have to take personal notes down, which I do on my opus, to have a record for the main notes page. It just seems a bit redundant, but I see why he would want it for if someone is absent or for “the web” to google.

Next we went over Alpine. This is the email service that Matt uses for lab46. To be honest, I don't expect myself to really ever use it. I would much rather just use my college gmail.

We also spent a large amount of class time allowing people to set up their mercurial repositories. I got a good refresher on how repository cloning works with mercurial. To this point in day, I'm not sure if I like git or mercurial better. I tend to like git more, because github is a wonderful web-side application for git repository hosting.

September 2, 2014

All directories in unix/linux are denoted with a forward slash “/”

Relative Path - a path that works off of our current location

Absolute Path - a path that works globally

A wild baconfish has appeared! I choose you Pikachu!

0 STDIN 1 STDOUT 2 STDERR

“>” redirects STDOUT to file (writes) “»” redirects STDOUT to file (append) “2>” redirects STDERR to file (writes) “2»” redirects STDERR to file (append)

tail command views end of a file and adding -f means don't end - usefull for viewing live changes to a file or for log files

cat command is useful to view file contents in terminal view

here is an interesting command: “cat file 2> ~/err” this makes the STDERR go to a file in my directory named err

another interesting command: “cat file 2> /dev/null” this sends the STDERR to a null place lel

September 2, 2014

Started out class learning how to check our current status in the course :)

 lab46:~$ status unix
ATTENDANCE
          [UNIX] 238 240 245 247
         [cclay] 238 240 245 247

OPUS
    1:opus:week1entry
    1:opus:week2entry

PROJECTS
    1:titlecustomize:customized Opus title section

lab46:~$

Learned about the super nifty calendar with the cal command.

lab46:~$ cal 04 2000
     April 2000
Su Mo Tu We Th Fr Sa
                   1
 2  3  4  5  6  7  8
 9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30
lab46:~$

It is a little strange that the only holiday available is easter lel. Also learned about a cool command called date

lab46:~$ date -d 20140803 +"%j"
215

The above code shows the amount of seconds since time began until 2014-08-03 Lastly, the command pom can be used to find the moon cycles :D

lab46:~$ pom
The Moon is Waning Gibbous (99% of Full)

Cool little command is tty, which shows my current terminal number the :/dev/pts folder shows all of the terminals

When using the who command, + shows who has messages on and a . means they have been active in the last minute.

rhenggel + pts/48       2014-09-09 16:04   .         21292 (pod01.offbyone.lan)
cclay    - pts/49       2014-09-09 16:05   .         21281 (172.16.200.195)

You can view the specific libraries that need to be in a specific place for a specific command with the ldd command:

lab46:/lib$ ldd /bin/ls
        linux-vdso.so.1 (0x00007fffad5ab000)
        libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f90ede17000)
        libacl.so.1 => /lib/x86_64-linux-gnu/libacl.so.1 (0x00007f90edc0e000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f90ed865000)
        libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f90ed5fc000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f90ed3f8000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f90ee046000)
        libattr.so.1 => /lib/x86_64-linux-gnu/libattr.so.1 (0x00007f90ed1f2000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f90ecfd5000)

ls -l for file/directory information :D

LELELELELELEL

lab46:/usr/local/bin$ ptt "sup noob"
      _
  .-./*)
_/___/   `.
  U U      'sup noob'

Finished class today with some awesome games

September 11, 2014

Going over some stuff about archives for the archive project. Mostly stuff I already know. One thing I didn't know that I learned is that Unix/Linux does not register files extensions. It is just part of the name of the file (some times you might see 2 extension like parts in the file name). Some programs though may use parts of the file name.

Started learning VI (were using VIM) (ed is command line based line editor - cant really “see” the text) the idea of insert text and commands such as save being split up (idea created by Bill Jay)

Insert Mode:
i,I
a,A
o,O

esc = command mode
Navigation - h=left,j=down,k=up,l=left (you can prefix these commands with a number such as 80l - will move 80 spaces to left)
word - w=forward by word, b=backwards by word (W,B treat things based on only spaces while lowercase treat other types of grammar)
line - ^=beginning of line, $=end of line, G=go t a certain line like 3G - goes to line 3
x=delete
X=delete backwards
p,P=paste
dw,db,dW,dB - delete by word
d^, d$, dd -del by line
s=substitue character
cw,cb,cW,cB - change word
y = yank ex. yw, yb, yW, yB, y^, y$
u,U = undo
r = redo
/ = search - n = next match, N = prev match ex. /hi 3n
:w filename = save as
:w = save
:q = quit
:q! quit without saving
:wq = save and quit

September 16, 2014

Went over the new puzzles project. Then started a wrap up of vim.

More commands:
:means current line
:co is copy
:2,4co9 copys lines 2-4 to line 9 (and moves down line 9)
:3m7 moves line 3 to line 7
:r filename - reads in a file
:w filename - writes to a file
:x saves and quits
:3,9s/lab46/LAB46 - searches for lab46 in lines 3-9 and will replace with LAB46 - adding /g on   the end will replace every occurence on each line, without it will only replace once per line
:s/[a-z]$/Z/g -will add to the end any lines that end with a lowercase letter a Z
:%s/./DEADBABY/g - will replace everything with dead babies (dead baby joke lel)
:set number - gives you line numbers (finally)
:set tabstop=4 - sets tab to only go 4 spaces instead of the default 8
set shiftwidth=4
syntax on
set cursorline
set smartindent

editing the .exrc file you can add config to set things like tab number every time the vim specific config file is .vimrc make sure these files are in the home directory

September 16, 2014

Doing the archive project. I started out making some directories for my unix/linux projects and then for the archive project. I copied both archive files into

 lab46:~/src/unix_linux/projects/archives$

Had to read up on the command “file” It's pretty cool that it puts the file against 3 different tests (filesystem tests, magic tests, and language tests). I got these results back when I ran file on each archive:

archive1.zip: Zip archive data, at least v2.0 to extract
lab46:~/src/unix_linux/projects/archives/archive1$

archive2.tar.bz2: bzip2 compressed data, block size = 900k
lab46:~/src/unix_linux/projects/archives/archive2$

I figured out that I can use unzip for the first archive. Unzip has a lot of cool modifiers. One example is unzip data1 -x joe ⇒ extract all files except joe from zipfile data1.zip When I ran unzip on archive one, I notices that some things were inflated, while others were extracted. This is interesting.

Archive:  archive1.zip
  inflating: image1.jpg
 extracting: image2.gif
  inflating: image3.png
 extracting: image4.txt

I figured out that the ones that were inflated were compressed, hence they were originally deflated, while the extracted files were simple just put into the archive and you are taking them back out. Pretty cool :D

Next I figured out that I can use the command tar to unzip the bzip2 files. This explains why there was bzip2 and tar in the name of the archive. I have to use the x modifier of tar to unzip. Oh wait… tar x filename didn't work… Looks like I actually have to use tar xjf filename. x for extract, j for bzip2, and f to use the archive. I can also add the modifier v to display the names of files as they are extracted.

Next I connected to lab46 via FTP and downloaded the pictures to see if any were corrupt when I viewed them (too lazy to put them up on my lab46 website :P)

So it looks like in archive1, image2 and image4 files are empty. I deleted them using rm. Image 1 is a bunny and image3 is a whale. In Archive2, image1 is empty. I also noticed that the whale picture was either corrupted with black, or that was supposed to be water. Then I remembered there was a complete whale picture in archive1, so I assumed it was corrupted and deleted it. Lastly, even though it was a txt file, it still had ascii art of a horse in it so I kept it (plus I knew I needed 4 files for naming purposes as said in the directions :P)

Next, I assumed the directions meant to rename the files based on the size of the animals in the picture :P bunny is smaller than a horse is smaller than an elephant is smaller than a whale. I figured out that the command I used to move files (mv) is also used to rename files. Interesting.

Now I made an archive called myarchive.tar for all of the good files from both archives and put them in the archive.I used the command tar cfv myarchive filename filename filename filename to do this. C to create an archive, f to use the archive, and v to output the file names when each file is added. Of course I forgot to put the file name in the first time I did it so I broke my smallest picture. I fixed it the the second time.

lab46:~/src/unix_linux/projects/archives$ tar cfv myarchive.tar smallest.jpg small.txt big.gif biggest.png
smallest.jpg
small.txt
big.gif
biggest.png
lab46:~/src/unix_linux/projects/archives$ ls
archive1  big.gif      myarchive.tar  smallest.jpg
archive2  biggest.png  small.txt

I used tar vtf myarchive.tar to make sure everything was in there (also shows file permissions, sweet).

lab46:~/src/unix_linux/projects/archives$ tar tvf myarchive.tar
-rw-r----- cclay/lab46  132216 2014-09-17 21:52 smallest.jpg
-rw-r----- cclay/lab46    2579 2014-01-26 11:01 small.txt
-rw-r----- cclay/lab46    4895 2014-01-26 10:54 big.gif
-rw-r----- cclay/lab46   52592 2014-01-01 16:17 biggest.png

Looks good so now it is time to compress myarchive.tar. I read the man page for gzip and saw that 9 is the best compression level, so I want to use -8 (the default is 6 and lowest/fastest is 1).

lab46:~/src/unix_linux/projects/archives$ gzip -n -8 myarchive.tar
lab46:~/src/unix_linux/projects/archives$ ls
archive1  big.gif      myarchive.tar.gz  smallest.jpg
archive2  biggest.png  small.txt

Now all I need to do is submit this! WOot!

Reflection:
1. I suppose tar works the way it does for maximum optimization.
2. The benefit of separating archival and compression functionality is that you can then create 
an archive for transporting a lot of files without compressing them. This would be good if you 
are worried that compressing the files might wreck them. Also it allows you to easily compress 
1 file vs. having to compress a whole archive to compress something. 

September 18, 2014 Started clas today going over some cool math tricks for parties lel.

Cat analyzes data of a file and splits it up into 8 bits (for ascii) wont work for binaries because characters in binaries cause weird sequences to occur. Cat will try to read the binaries as ascii files.

cat -n filename will add line numbers
cat -e filename will show $ at end of all lines ($ means end of line)
od filename will take an octal dumb of the motd file
od -x filename gives hexadecimal dump

Endianess
-little endian
-big endian
Intel is little endian (every 2 bytes are flipped)

grep - globally search for regular expression and print by default, grep prints matching lines to a provided pattern

ex. grep pattern filename
grep "the" /etc/motd
grep '\<the\>' /etc/motd 
\< start of word
\> end of word
-i takes away case sensitivity 
-o only displays the matching text
-n of course will display line numbers in the output

cat ouputs, while grep inputs for processing can use od and grep with a pipe

ex od -x /etc/motd | grep '0a'
pipe wc for word count
od -x /etc/motd | grep -o '0a' | wc -l //will give the account of new line characters

September 20, 2014

Starting the puzzlebox project. Not exactly sure what to expect.

I copied the puzzle.txt file into my src. I used cat to view its contents and it was a binary file. I then used file to see what kind of file it was. Looks like it was an archive, so I used the tar utility to extract it. After doing this, I found a “making.waves” file. I looked in the file with vim, and it says begin 644 leaf_on_the_wind.wav and then a bunch of gibberish. Not exactly sure what to do at this point, but I know a .wav file is a sound/music file.

I then decided to check the file type of the making.waves file. That came back with: uuencoded or xxencoded, ASCII text I googled uuencoded and xxencoded, and it seems that they are used for binary-text encoding and are used for email operations. The “begin 644 leaf_on_the_wind.wav” means that 644 are the permissions and the file that being encoded is leaf_on_the_wind.wav I think that I am going to have to use a decoder now to see what the other characters are.

I found a uuencode tool. I read up on the man, and I decided to try uuencode making.waves. Didn't seem to do much.

Now I feel stupid, I decided to try uudecode to see if it was an actual tool and it is. This time I tried uudecode making.waves. This created a leaf_on_the_wind.wav file. Looks like we are getting somewhere, which is good. When I look inside that file, it seems like more binary stuff, but I do see one line that says dinosaurs_with_hats.mp4. Not sure what that is about. I'm just going to check the filetype again of the .wav file.

Looks like the filetype is a zip archive. When I unzip it, it inflates dinosaurs_with_hats.mp4. I looked into that file, and it is another uuencoded file. You have got to be kidding me. I immediately imagine the scenario of infinite going through this cycle. I really hope I am not missing the big picture. I am going to try decoding this now.

Now I have a file named results. I look it says I unravelled the puzzle box. Woot lol. I thought there was going to be something more than this that would make it really complicated O.o Now I just need to do the steps for submitting it. Yay!

Remember: submit classname checks projects that were submitted and such

September 23, 2014

lab46:~/src$ cat /etc/motd|grep "the" | wc -l
3

lab46:~/src$ who > who.dat
lab46:~/src$ cat who.dat
NAME       LINE         TIME             IDLE          PID COMMENT
smeas    + pts/3        2014-08-29 11:52 02:31        1921 (cpe-67-251-64-18:S.0)
dsaunde6 + pts/9        2014-09-19 15:59   .         30058 (pod01:S.1)
mgardne8 + pts/11       2014-09-10 19:38  old        19834 (mosh [19834])
dsaunde6 + pts/14       2014-09-19 16:03 16:01       30107 (pod01:S.2)

Find the number of lines in who.dat

lab46:~/src$ cat who.dat | wc -l
57

break the file into 3 sections

lab46:~/src$ head -19 who.dat > section1.dat
lab46:~/src$ tail -19 who.dat > section3.dat
lab46:~/src$ head -38 who.dat | tail -19 > section2.dat
lab46:~/src$ cat section2.dat

putting all the files together and sorting them alphabetically

lab46:~/src/unix_linux/classwork/9_23_14$ cat section2.dat section1.dat section3.dat > who.all
lab46:~/src/unix_linux/classwork/9_23_14$ cat section2.dat section1.dat section3.dat | sort | less

lets find my username bin for my password use getent to display entries in a database - queries the system for everything it has access too (local and remote), because our user accounts are not defined on lab46, but on an LDAP system.

lab46:~/src$ getent passwd | grep "$USER"
cclay:x:5798:5000:Casper Clay:/home/cclay:/bin/bash

passwords now are in etc/shadow instead of the password file. This is because people could get the encrypted passwords and crack them. Now only the system has access to them.

bc -l opens a handy dandy calculator
change base to hexadecimal obase=16

use the groups command to see what group you are in (as you can see im in 3 groups)

lab46:~/src/unix_linux/classwork/9_23_14$ groups
lab46 unix cclay

see everyone who is in the unix group

lab46:~/src/unix_linux/classwork/9_23_14$ getent group | grep 'unix'
unix:*:1730:abuck4,acarson1,ahoover3,avolino,cclay,ddewert1,dsaunde6,dshadeck,dsoutha3,jcliteur,jjacobs7,mgleaso7,mp010784,mquesad1,nsano,nvitull1,rhenggel,skramer1,ssmit133,stiwari1,tarnold4,tmosgrov,vgarfiel,wedge

September 25, 2014

Starting class today talking about how some kid who fork bombed lab46 un-intentionally. Then we started talking about the recent bash vulnerability that has surfaced.

Just a reminder that ps creates a list of processes.

Flashback to the hilarious Vogue website easter egg :P

Continuation of the fun with getent, pipes, grep, tail, head, and now cut :D Cut allows us to remove sections from each line of files :D

teames3:x:5944:5000:Timothy S. Eames:/home/teames3:/bin/bash
jfenchel:x:5945:5000:Jonathan South:/home/jfenchel:/bin/bash
lab46:~/src/unix_linux/classwork$ getent passwd | cut -d':' -f1,7
root:/bin/bash
daemon:/usr/sbin/nologin

filtering out the above to things with 'th' in them, and then taking the last 16 results. In this case we will see there are not actually 16 results to be displayed.

lab46:~/src/unix_linux/classwork$ getent passwd | cut -d':' -f1,7 | grep 'th' | tail -n 16
appelthp:/bin/bash
ssmith85:/bin/bash
rthatch2:/bin/bash
thakes3:/bin/bash
mhether2:/bin/bash
gerth:/bin/bash
bkathan:/bin/bash
dsoutha3:/bin/bash
athoma13:/bin/bash
lab46:~/src/unix_linux/classwork$

We can count 9 names, or count the number of matches with word count.

lab46:~/src/unix_linux/classwork$ getent passwd | cut -d':' -f1,7 | grep 'th' | tail -n 16 | wc -l
9

Quotes:

Quote examples:

lab46:~/src/unix_linux/classwork$ echo $USER
cclay
lab46:~/src/unix_linux/classwork$ echo "$USER"
cclay
lab46:~/src/unix_linux/classwork$ echo '$USER'
$USER
lab46:~/src/unix_linux/classwork$ echo "${USER}, you are logged in `who|grep "$USER"|wc -l` times"
cclay, you are logged in 1 times
lab46:~/src/unix_linux/classwork$
lab46:~/src/unix_linux/classwork$ uol=$(/usr/bin/who|wc -l)
lab46:~/src/unix_linux/classwork$ echo $uol
37
lab46:~/src/unix_linux/classwork$ t=`echo "There are $uol users on the system"` lab46:~/src/unix_linux/classwork$ echo $t
There are 37 users on the system

Semi colons aka ; will separate commands on the same line

September 30, 2014

Started class out today talking about puzzlebox2 and the new project. Then had a lecture on problem solving. If you are stuck, manipulate the unix world in a way where it works for what you need it for.

Need to look-up the command uniq for the new project. Also will need the command tr, which will get rid of specified characters. Example:

lab46:~/src/unix_linux/projects/puzzlebox2$ printf "\tHello\tthere\n" | tr '\t' ' '
 Hello there

This got rid of the spaces.

You can use man man to view the different sections of the man file.

Use man ascii to see a complete ascii table.

Wildcards are like a watered down version of regular expressions. Regular expressions are used within files, while wildcards are used in the command line.

Wildcards:

Examples:

ls ???????
ls l*
ls *.zip
ls *.tar*
ls ?[aeiou]?
ls c[aeiou]*t
ls c[^aeiou]*t

October 2, 2014

Need to make sure I'm subscribed to the class mailing list.

Practicing commands today with wildcards. Went into usr/bin to count files :D

Shows the number of files with 5 or more characters in the name:

lab46:/usr/bin$ ls -d ?????* | wc -l
678

shows the number of files that have at least 4 or more characters, and whose first, second, and last character is oneof the jeopardy characters.

lab46:/usr/bin$ ls -d [rstlne][rstlne]?*[rstlne] |  wc -l 
43

All files whos names do not have vowels in them.

lab46:/usr/bin$ ls -d [^aeiouy][^aeiouy][^aeiouy] |  wc -l 
27

Looking for the number of all directories and symbolic links

lab46:/usr/bin$ ls -l|egrep '^(l|d)' | wc -l
203

October 7, 2014

Started out class talking about the dataproc assignment. Due date changing to the 22nd; however, if we want a bonus point, we can still turn it in by the original due date.

A process is a “program in action.”

Each process has a PID or ProcessID.

Running the command ps lists the current processes running and their PID's, as-well as other information regarding each process running.

In this example, we can see that when we run the command cat and let it sit, it can then be viewed by running the process ps in a different terminal.

We can kill processes with the kill command. Here are the different kill signals available.

lab46:~$ kill -l
 1) SIGHUP       2) SIGINT       3) SIGQUIT      4) SIGILL       5) SIGTRAP
 6) SIGABRT      7) SIGBUS       8) SIGFPE       9) SIGKILL     10) SIGUSR1
11) SIGSEGV     12) SIGUSR2     13) SIGPIPE     14) SIGALRM     15) SIGTERM
16) SIGSTKFLT   17) SIGCHLD     18) SIGCONT     19) SIGSTOP     20) SIGTSTP
21) SIGTTIN     22) SIGTTOU     23) SIGURG      24) SIGXCPU     25) SIGXFSZ
26) SIGVTALRM   27) SIGPROF     28) SIGWINCH    29) SIGIO       30) SIGPWR
31) SIGSYS      34) SIGRTMIN    35) SIGRTMIN+1  36) SIGRTMIN+2  37) SIGRTMIN+3
38) SIGRTMIN+4  39) SIGRTMIN+5  40) SIGRTMIN+6  41) SIGRTMIN+7  42) SIGRTMIN+8
43) SIGRTMIN+9  44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+13
48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-12
53) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9  56) SIGRTMAX-8  57) SIGRTMAX-7
58) SIGRTMAX-6  59) SIGRTMAX-5  60) SIGRTMAX-4  61) SIGRTMAX-3  62) SIGRTMAX-2
63) SIGRTMAX-1  64) SIGRTMAX

The following can be used to close the process cat with the appropriate PID:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
cclay    24925  0.0  0.1  14048  2048 pts/10   Ss   16:46   0:00 -bash
cclay    24933  0.4  0.1  14048  2044 pts/107  Ss   16:46   0:00 -bash
cclay    24941  1.0  0.0   6424   348 pts/10   S+   16:46   0:00 cat -
cclay    24942  0.0  0.0  11332  1076 pts/107  R+   16:46   0:00 ps u
lab46:~$ kill -1 24941
lab46:~$ kill hup 24941
lab46:~$ kill sigup 24941

If we kill a process by simply saying “kill PID”, the kill process automatically uses signal “15” or “SIGTERM”

In the kill command, signal “9” or “SIGKILL” is the most powerful signal and will close anything. It should be used sparingly, as it can cause problems if not fully understood.

Running “ps aux”, we can view every process running on the system.

Here is an example of viewing all of a single persons processes on the system.

lab46:~$ ps aux | grep $USER
root     24922  0.0  0.2  99900  4520 ?        Ss   16:46   0:00 sshd: cclay [priv]
cclay    24924  0.0  0.1  99900  2152 ?        S    16:46   0:00 sshd: cclay@pts/10
cclay    24925  0.0  0.1  14056  2068 pts/10   Ss   16:46   0:00 -bash
root     24930  0.0  0.2  99900  4524 ?        Ss   16:46   0:00 sshd: cclay [priv]
cclay    24932  0.0  0.1  99900  2152 ?        S    16:46   0:00 sshd: cclay@pts/107
cclay    24933  0.0  0.1  14056  2096 pts/107  Ss   16:46   0:00 -bash
cclay    24953  0.0  0.0   6424   344 pts/10   S+   16:47   0:00 cat -
cclay    26488  0.0  0.0  11332  1064 pts/107  R+   16:57   0:00 ps u aux
cclay    26489  0.0  0.0  15228   948 pts/107  S+   16:57   0:00 grep cclay
lab46:~$

Another interesting command is top, which will show realtime system information such as process and resource usage.

Example(Only the first 12 lines of output):

top - 16:58:09 up 12 days, 17:52, 49 users,  load average: 1.39, 0.57, 0.33
Tasks: 408 total,   4 running, 399 sleeping,   5 stopped,   0 zombie
%Cpu(s):  2.1 us,  7.9 sy,  0.0 ni, 75.6 id,  0.0 wa,  0.0 hi,  0.3 si, 14.1 st
KiB Mem:   1535680 total,  1499612 used,    36068 free,   398056 buffers
KiB Swap:   131068 total,    37104 used,    93964 free.   524684 cached Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
31518 thakes3   20   0  120988   2484   1364 S  23.3  0.2   3:07.53 irssi
28580 root      20   0       0      0      0 S   4.3  0.0   2:08.08 kworker/0:1
    7 root      20   0       0      0      0 R   3.6  0.0  33:35.86 rcu_sched
26493 jjacobs7  20   0   16044   1700   1052 S   2.6  0.1   0:00.52 top
26549 wedge     20   0    9908   1740   1348 S   2.3  0.1   0:00.07 bash

You can customize the top process by toggling keys such as “z” for color, or 1 for specific cpu core usage.

Example:

In unix we use shells. The default shell that we use is Bash.

(View the latest bash vulnerability here: http://en.wikipedia.org/wiki/Shellshock_(software_bug)

AT&T UNIX VS. BSD UNIX SHELLS

When running a process in a terminal, we can save it rather than killing it by using CTRL Z

Example:

lab46:~$ cat -
^Z
[1]+  Stopped                 cat -
lab46:~$

We can view these stopped or active process with the command jobs. We can use the command bg to run processes in the background and the command fg to run processes in the foreground.

Example of freezing, viewing, and re-running processes:

lab46:~$ sleep 3600
(sleep 180;echo hi)
^Z
[2]+  Stopped                 sleep 3600
lab46:~$ jobs
[1]-  Stopped                 cat -
[2]+  Stopped                 sleep 3600
lab46:~$ bg 1
[1]- cat - &
lab46:~$ jobs
[1]+  Stopped                 cat -
[2]-  Stopped                 sleep 3600
lab46:~$
bg 2
[2]- sleep 3600 &
lab46:~$ jobs
[1]+  Stopped                 cat -
[2]-  Running                 sleep 3600 &
lab46:~$ fg 2
sleep 3600

October 9, 2014

We can view hidden files by issuing ls -a , which will show all files. Lots of these hidden files such as .bashrc are configuration files.

A useful command we can use is alias. Here is an example so that when we type bob, we get bob printed on our terminal.

lab46:~$ alias bob='echo bob'
lab46:~$ bob
bob

An alias will stay until you end your session. You can take aliases away with the unalias command. If you want permanent aliases, you can add them in .bashrc in our home directory.

The environment variable $PATH shows all of the paths that the system will look for commands. If we type jim, it will look for a command called jim in all of the following places:

lab46:~$ $PATH
-bash: /usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games: No such file or directory

Started going over scripting!

Shell Scripts

we can save shell scripts in files ending in .sh or .bash Here is an example of a simple script called myscript.sh 3 line script:

df
who
ls

Now lets run this script (not all output included)!

lab46:~/src/unix_linux/classwork/10_9_14$ bash myscript.sh
Filesystem      1K-blocks       Used  Available Use% Mounted on
/dev/xvda1        3997376    1784952    1986328  48% /
udev                10240          0      10240   0% /dev
tmpfs              153568        160     153408   1% /run

We are not able to run this script with ./ because we don't have the proper permissions to execute it, which is why we used bash. If we changed the permissions with chmod, we will be able to then execute the script normally. Example (not all output included):

lab46:~/src/unix_linux/classwork/10_9_14$ chmod 700 myscript.sh
lab46:~/src/unix_linux/classwork/10_9_14$ ./myscript.sh
Filesystem      1K-blocks       Used  Available Use% Mounted on
/dev/xvda1        3997376    1784968    1986312  48% /
udev                10240          0      10240   0% /dev
tmpfs              153568        160     153408   1% /run

To see our environment, which is bash: #!/bin/bash

Now lets make another cool script called nametest.sh to create a mini program that asks for your name and spits it back out :D Here is the script:

echo -n "What is your name? "
read name
echo "Hello, $name, nice to meet you! "
exit 0

Now lets run it!

lab46:~/src/unix_linux/classwork/10_9_14$ ./nametest.sh
What is your name? Casper
Hello, Casper, nice to meet you!
lab46:~/src/unix_linux/classwork/10_9_14$

Bash is a higher level among programming languages. We don't have to worry about things such as data types.

We can use tab completion (AKA push tab after typing in an in-complete name) to complete unique names such as file names without typing it all out. If have two files with similar names, tab completion will complete the name up until the point at which it is still unique. If you have two files with similar names such as hhhhhs and hhhs, you can push tab twice to get information on the files, and see why it wont complete the entire name. Example:

lab46:~/src/unix_linux/classwork/10_9_14$ ls
hasdkhfakjhfafqhfiufubflks  hasdkhfakjhfafqhfiufubflksajfasjfdsafjffaoiehfiquhfahf  myscript.sh  nametest.sh
lab46:~/src/unix_linux/classwork/10_9_14$ hash
hasdkhfakjhfafqhfiufubflks/
hasdkhfakjhfafqhfiufubflksajfasjfdsafjffaoiehfiquhfahf/
myscript.sh
nametest.sh
lab46:~/src/unix_linux/classwork/10_9_14$ hash

We can use the rm command to remove files and directories. Using rm -f deletes a director and everything in it.

Now lets test checking conditions inside our scripts and using loops. It is important to remember that things must be properly spaced such as 5 = 0. If it was 5=0, the script will try to assign 5 to 0 instead of checking the condition.

Here is a script called numbertest.sh (we used -eq for = and -gt for > to avoid string vs number comparison problems):

Now lets run this amazing game we just made!

lab46:~/src/unix_linux/classwork/10_9_14$ ./numbertest.sh
Pick a number: 1
too low
Do you want to play a again?y
Pick a number: 7
too high
Do you want to play a again?y
Pick a number: 5
correct
Do you want to play a again?n
lab46:~/src/unix_linux/classwork/10_9_14$

October 21, 2014

Starting class today talking about random numbers to improve our number guessing game! Going to have to leave early today for a Dentist appointment :(

We can print a random number with the command echo $RANDOM

lab46:~/public_html/Lab09$ echo $RANDOM
2944
lab46:~/public_html/Lab09$ echo $RANDOM
15863
lab46:~/public_html/Lab09$ echo $RANDOM
9721

% modulus / division

34%5 = 4 34/5 = 6

October 23, 2014

We can use bc to easily do math in scripts.

lab46:~/src/unix_linux/classwork/10_23_14$ echo "2+2" | bc
4

Went over loops.

Now need to make a grade calculator for the course for our next project. Started working on that in class.

October 23, 2014

We can use floating points in bash with the -l argument of bc. If we tried to compute 6/7, we would get 0. Instead, we can try the following command:

lab46:~$ echo "(6/7)*100" | bc -l
85.71428571428571428500

To get only an integer, we can do the following with what we already know:

lab46:~$  echo "(6/7)*100" | bc -l | cut -d'.' -f1
85

To get the first two decimals, the following can be done:

lab46:~$ echo "(6/7)*100" | bc -l | cut -c1-5
85.71

To finally add the percent sign, we can do this:

lab46:~$ echo "$(echo "(6/7)*100" | bc -l | cut -c1-5)%"
85.71%

For the grade calculator project, we can calculate the attendance by splitting up the dates. Currently, they are all on one line. Lets split them up onto their own lines using translate, and white space as a delimiter.

lab46:~$ status unix | grep $USER | tr " " "\n"

[cclay]
238
240
245
247
252
254
259
261
266
268
273
275
280
282
294
296

October 30, 2014

Knowledge Assessment! Wasn't so bad!

November 4, 2014

Today we venture off into regular expressions!

*Remember - Wildcards are for files, regular expressions are for text *

Regular Expressions are used to describe patters, patterns in text that we are looking for.

Now, lets try some stuff out.

To find the number of words that are exactly 4 characters long, the following can be done:

lab46:/usr/share/dict$ cat words | grep '^....$' | wc
   3346    3346   16730

Therefore, there are 3346 words that are exactly 4 letters long.

To find the amount of words that are at least 4 letters long and end with a lower case g, the following can be used:

lab46:/usr/share/dict$ cat words | grep '...g$' | wc
   6999    6999   70411

This results in 6999 words.

You can also use this command to do the same thing. The reason the previous command worked was because the beginning of the line did not matter.

lab46:/usr/share/dict$ cat words | grep '^....*g$' | wc
   6999    6999   70411

This also results in 6999 words.

We can find the words with at-least 3 characters and are all lowercase with the following:

lab46:/usr/share/dict$ cat words | grep '^[a-z][a-z][a-z][a-z]*$' | wc
  62797   62797  584774

We get back 62797 words.

To find any words that have at least 3 or more vowels, we can use the following:

lab46:/usr/share/dict$ cat words | grep '^.*[aeiouy].*[aeiouy].*[aeiouy].*[aeiouy]*' | wc
  64422   64422  682102

This results in 64422 words.

We can enable extended regular expressions mode by using **egrep** instead of **grep**.
**fgrep** is used to find only literal strings. This can be faster if you are trying to only  
find literal strings.

To find any words that end in 'ed' or 'ing', we can use the following:

lab46:/usr/share/dict$ cat words | egrep '*(ed)$|(ing)$' | wc
  13412   13412  130200

This results in 13412 words.

Next class, we will blow our minds once again by learning about sed :D

November 6, 2014 The grade calculator is now an official project. Today we are continuing regular expressions and will start sed.

If we want to find all of the even weeks in the opus grades, we can use the following:

lab46:~$ status unix | grep 'opus' | egrep 'week([02468]|[0-9][02468])'
    1:opus:week2entry
    1:opus:week4entry
    1:opus:week6entry
    1:opus:week8entry

Here is a cool script using sed to re-arrange the opus grades. I don't completely understand it, but here is the code!

lab46:~$ status unix | grep 'opus' | sed 's/^.*\([01]\):\([a-z][a-z]*\):\(week\)\([0-9]\)\(.*\)$/\5 for \3 \4 \2 [\1]/g'
entry for week 1 opus [1]
entry for week 2 opus [1]
entry for week 3 opus [1]
entry for week 4 opus [1]
entry for week 5 opus [1]
entry for week 6 opus [1]
entry for week 7 opus [1]
entry for week 8 opus [1]
entry for week 9 opus [1]

November 11, 2014

./myscript a b c

$# - number of arguments $* - all your arguments “a b c” $0 - script name $1 - “a” $2 - “b” $3 - “c”

Determine where a script is located.

lab46:~$ whereis status
status: /usr/local/bin/status /usr/local/bin/status.logic

commands at and cron / crontab are good for automation of things at certain times.

November 13, 2014

Going over more automation stuff.

We can use the command at to specify a time to run a command at. Example:

at 16:16
ls > out
ctrl-d

Cron jobs are useful for automating tasks.

Crontab -e will open crontab configuration to create new tasks. An example of a task is:

*/4 * * * * /usr/bin/who

This task will run the who command ever 4 minutes of every minute, hour, day, week, year

November 18, 2014

Today we got a new project. It is the timeline project. Basically, we need to calculate the amount of time spent on the system overall and per month for a specified user. Something interesting that we will do in this project is create a histogram.

Two useful commands: last and lastlog. Last will show all of the logins and information about those logins of all of the users in the last year. Lastlog will show the most recent last login of each user.

I think I can either search against the home directory to see if a username exists, or simply say there is no data/ the user does not exist by searching through login data. It would be nice to distinguish between the two.

November 20, 2014

Today was a work on stuff day. New project to do, the timeonline project. Last night I finished the grade calculator project. It is quite nice. I have a nice gui, along with a debug mode.

December 2, 2014

Had issues editing my opus, they are fixed now though (at least when I am at the lair). Time for the end of course experience :D Still need to do the time online project.