Sidebar
UNIX/Linux Fundamentals Journal
MONTH Day, YEAR
August 26, 2014
First day of class. Time to become a mystical powerful hacker through the use of a black background coloured terminal.
Yay for listening to the amazing speeches of Matt all class and going over the syllabus.
August 28, 2014
Second day of class. Had fun using wall on everyone to mask myself as a hacker. Also did some advertising of CIST club. Matt went over some very basic commands and then talked about the unix philosophy.
The Unix Philosophy
- Ordered List ItemSmall is beautiful
- Ordered List ItemDo one thing, and do that one thing extremely well.
- Ordered List ItemEverything is a file (almost).
3 Types of Files
- Regular/Ordinary
- Directory(links) - metadata
- Special
3 Tiers of Ownership
- user
- group
- other (world)
Permissions
- - means no permissions
- x means execute or search
- w means write
- r means read
September 2, 2014
Third day to the most amazing class ever! I get really excited to come to this class for some reason.
Started out class talking about projects. There is a new notes project where Matt will pick 3 people to put class notes on an official notes page for each class. There will be an author, designer, and editor. I remember that Matt had talked about doing this last semester, but it never actually happened. I am very interested to try it out this semester. I'm not sure how this will be better than just putting our own notes on our opus. We will still have to take personal notes down, which I do on my opus, to have a record for the main notes page. It just seems a bit redundant, but I see why he would want it for if someone is absent or for “the web” to google.
Next we went over Alpine. This is the email service that Matt uses for lab46. To be honest, I don't expect myself to really ever use it. I would much rather just use my college gmail.
We also spent a large amount of class time allowing people to set up their mercurial repositories. I got a good refresher on how repository cloning works with mercurial. To this point in day, I'm not sure if I like git or mercurial better. I tend to like git more, because github is a wonderful web-side application for git repository hosting.
September 2, 2014
All directories in unix/linux are denoted with a forward slash “/”
Relative Path - a path that works off of our current location
Absolute Path - a path that works globally
A wild baconfish has appeared! I choose you Pikachu!
0 STDIN 1 STDOUT 2 STDERR
“>” redirects STDOUT to file (writes) “»” redirects STDOUT to file (append) “2>” redirects STDERR to file (writes) “2»” redirects STDERR to file (append)
tail command views end of a file and adding -f means don't end - usefull for viewing live changes to a file or for log files
cat command is useful to view file contents in terminal view
here is an interesting command: “cat file 2> ~/err” this makes the STDERR go to a file in my directory named err
another interesting command: “cat file 2> /dev/null” this sends the STDERR to a null place lel
September 2, 2014
Started out class learning how to check our current status in the course :)
lab46:~$ status unix ATTENDANCE [UNIX] 238 240 245 247 [cclay] 238 240 245 247 OPUS 1:opus:week1entry 1:opus:week2entry PROJECTS 1:titlecustomize:customized Opus title section lab46:~$
Learned about the super nifty calendar with the cal command.
lab46:~$ cal 04 2000 April 2000 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 lab46:~$
It is a little strange that the only holiday available is easter lel. Also learned about a cool command called date
lab46:~$ date -d 20140803 +"%j" 215
The above code shows the amount of seconds since time began until 2014-08-03 Lastly, the command pom can be used to find the moon cycles :D
lab46:~$ pom The Moon is Waning Gibbous (99% of Full)
Cool little command is tty, which shows my current terminal number the :/dev/pts folder shows all of the terminals
When using the who command, + shows who has messages on and a . means they have been active in the last minute.
rhenggel + pts/48 2014-09-09 16:04 . 21292 (pod01.offbyone.lan) cclay - pts/49 2014-09-09 16:05 . 21281 (172.16.200.195)
You can view the specific libraries that need to be in a specific place for a specific command with the ldd command:
lab46:/lib$ ldd /bin/ls linux-vdso.so.1 (0x00007fffad5ab000) libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f90ede17000) libacl.so.1 => /lib/x86_64-linux-gnu/libacl.so.1 (0x00007f90edc0e000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f90ed865000) libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f90ed5fc000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f90ed3f8000) /lib64/ld-linux-x86-64.so.2 (0x00007f90ee046000) libattr.so.1 => /lib/x86_64-linux-gnu/libattr.so.1 (0x00007f90ed1f2000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f90ecfd5000)
ls -l for file/directory information :D
LELELELELELEL
lab46:/usr/local/bin$ ptt "sup noob" _ .-./*) _/___/ `. U U 'sup noob'
Finished class today with some awesome games
September 11, 2014
Going over some stuff about archives for the archive project. Mostly stuff I already know. One thing I didn't know that I learned is that Unix/Linux does not register files extensions. It is just part of the name of the file (some times you might see 2 extension like parts in the file name). Some programs though may use parts of the file name.
Started learning VI (were using VIM) (ed is command line based line editor - cant really “see” the text) the idea of insert text and commands such as save being split up (idea created by Bill Jay)
Insert Mode: i,I a,A o,O esc = command mode Navigation - h=left,j=down,k=up,l=left (you can prefix these commands with a number such as 80l - will move 80 spaces to left) word - w=forward by word, b=backwards by word (W,B treat things based on only spaces while lowercase treat other types of grammar) line - ^=beginning of line, $=end of line, G=go t a certain line like 3G - goes to line 3 x=delete X=delete backwards p,P=paste dw,db,dW,dB - delete by word d^, d$, dd -del by line s=substitue character cw,cb,cW,cB - change word y = yank ex. yw, yb, yW, yB, y^, y$ u,U = undo r = redo / = search - n = next match, N = prev match ex. /hi 3n :w filename = save as :w = save :q = quit :q! quit without saving :wq = save and quit
September 16, 2014
Went over the new puzzles project. Then started a wrap up of vim.
More commands: :means current line :co is copy :2,4co9 copys lines 2-4 to line 9 (and moves down line 9) :3m7 moves line 3 to line 7 :r filename - reads in a file :w filename - writes to a file :x saves and quits :3,9s/lab46/LAB46 - searches for lab46 in lines 3-9 and will replace with LAB46 - adding /g on the end will replace every occurence on each line, without it will only replace once per line :s/[a-z]$/Z/g -will add to the end any lines that end with a lowercase letter a Z :%s/./DEADBABY/g - will replace everything with dead babies (dead baby joke lel) :set number - gives you line numbers (finally) :set tabstop=4 - sets tab to only go 4 spaces instead of the default 8 set shiftwidth=4 syntax on set cursorline set smartindent
editing the .exrc file you can add config to set things like tab number every time the vim specific config file is .vimrc make sure these files are in the home directory
September 16, 2014
Doing the archive project. I started out making some directories for my unix/linux projects and then for the archive project. I copied both archive files into
lab46:~/src/unix_linux/projects/archives$Had to read up on the command “file” It's pretty cool that it puts the file against 3 different tests (filesystem tests, magic tests, and language tests). I got these results back when I ran file on each archive:
archive1.zip: Zip archive data, at least v2.0 to extract lab46:~/src/unix_linux/projects/archives/archive1$ archive2.tar.bz2: bzip2 compressed data, block size = 900k lab46:~/src/unix_linux/projects/archives/archive2$
I figured out that I can use unzip for the first archive. Unzip has a lot of cool modifiers. One example is unzip data1 -x joe ⇒ extract all files except joe from zipfile data1.zip When I ran unzip on archive one, I notices that some things were inflated, while others were extracted. This is interesting.
Archive: archive1.zip inflating: image1.jpg extracting: image2.gif inflating: image3.png extracting: image4.txt
I figured out that the ones that were inflated were compressed, hence they were originally deflated, while the extracted files were simple just put into the archive and you are taking them back out. Pretty cool :D
Next I figured out that I can use the command tar to unzip the bzip2 files. This explains why there was bzip2 and tar in the name of the archive. I have to use the x modifier of tar to unzip. Oh wait… tar x filename didn't work… Looks like I actually have to use tar xjf filename. x for extract, j for bzip2, and f to use the archive. I can also add the modifier v to display the names of files as they are extracted.
Next I connected to lab46 via FTP and downloaded the pictures to see if any were corrupt when I viewed them (too lazy to put them up on my lab46 website :P)
So it looks like in archive1, image2 and image4 files are empty. I deleted them using rm. Image 1 is a bunny and image3 is a whale. In Archive2, image1 is empty. I also noticed that the whale picture was either corrupted with black, or that was supposed to be water. Then I remembered there was a complete whale picture in archive1, so I assumed it was corrupted and deleted it. Lastly, even though it was a txt file, it still had ascii art of a horse in it so I kept it (plus I knew I needed 4 files for naming purposes as said in the directions :P)
Next, I assumed the directions meant to rename the files based on the size of the animals in the picture :P bunny is smaller than a horse is smaller than an elephant is smaller than a whale. I figured out that the command I used to move files (mv) is also used to rename files. Interesting.
Now I made an archive called myarchive.tar for all of the good files from both archives and put them in the archive.I used the command tar cfv myarchive filename filename filename filename to do this. C to create an archive, f to use the archive, and v to output the file names when each file is added. Of course I forgot to put the file name in the first time I did it so I broke my smallest picture. I fixed it the the second time.
lab46:~/src/unix_linux/projects/archives$ tar cfv myarchive.tar smallest.jpg small.txt big.gif biggest.png smallest.jpg small.txt big.gif biggest.png lab46:~/src/unix_linux/projects/archives$ ls archive1 big.gif myarchive.tar smallest.jpg archive2 biggest.png small.txt
I used tar vtf myarchive.tar to make sure everything was in there (also shows file permissions, sweet).
lab46:~/src/unix_linux/projects/archives$ tar tvf myarchive.tar -rw-r----- cclay/lab46 132216 2014-09-17 21:52 smallest.jpg -rw-r----- cclay/lab46 2579 2014-01-26 11:01 small.txt -rw-r----- cclay/lab46 4895 2014-01-26 10:54 big.gif -rw-r----- cclay/lab46 52592 2014-01-01 16:17 biggest.png
Looks good so now it is time to compress myarchive.tar. I read the man page for gzip and saw that 9 is the best compression level, so I want to use -8 (the default is 6 and lowest/fastest is 1).
lab46:~/src/unix_linux/projects/archives$ gzip -n -8 myarchive.tar lab46:~/src/unix_linux/projects/archives$ ls archive1 big.gif myarchive.tar.gz smallest.jpg archive2 biggest.png small.txt
Now all I need to do is submit this! WOot!
Reflection: 1. I suppose tar works the way it does for maximum optimization. 2. The benefit of separating archival and compression functionality is that you can then create an archive for transporting a lot of files without compressing them. This would be good if you are worried that compressing the files might wreck them. Also it allows you to easily compress 1 file vs. having to compress a whole archive to compress something.
September 18, 2014 Started clas today going over some cool math tricks for parties lel.
Cat analyzes data of a file and splits it up into 8 bits (for ascii) wont work for binaries because characters in binaries cause weird sequences to occur. Cat will try to read the binaries as ascii files.
cat -n filename will add line numbers cat -e filename will show $ at end of all lines ($ means end of line) od filename will take an octal dumb of the motd file od -x filename gives hexadecimal dump
Endianess -little endian -big endian Intel is little endian (every 2 bytes are flipped)
grep - globally search for regular expression and print by default, grep prints matching lines to a provided pattern
ex. grep pattern filename grep "the" /etc/motd grep '\<the\>' /etc/motd \< start of word \> end of word -i takes away case sensitivity -o only displays the matching text -n of course will display line numbers in the output
cat ouputs, while grep inputs for processing can use od and grep with a pipe
ex od -x /etc/motd | grep '0a' pipe wc for word count od -x /etc/motd | grep -o '0a' | wc -l //will give the account of new line characters
September 20, 2014
Starting the puzzlebox project. Not exactly sure what to expect.
I copied the puzzle.txt file into my src. I used cat to view its contents and it was a binary file. I then used file to see what kind of file it was. Looks like it was an archive, so I used the tar utility to extract it. After doing this, I found a “making.waves” file. I looked in the file with vim, and it says begin 644 leaf_on_the_wind.wav and then a bunch of gibberish. Not exactly sure what to do at this point, but I know a .wav file is a sound/music file.
I then decided to check the file type of the making.waves file. That came back with: uuencoded or xxencoded, ASCII text I googled uuencoded and xxencoded, and it seems that they are used for binary-text encoding and are used for email operations. The “begin 644 leaf_on_the_wind.wav” means that 644 are the permissions and the file that being encoded is leaf_on_the_wind.wav I think that I am going to have to use a decoder now to see what the other characters are.
I found a uuencode tool. I read up on the man, and I decided to try uuencode making.waves. Didn't seem to do much.
Now I feel stupid, I decided to try uudecode to see if it was an actual tool and it is. This time I tried uudecode making.waves. This created a leaf_on_the_wind.wav file. Looks like we are getting somewhere, which is good. When I look inside that file, it seems like more binary stuff, but I do see one line that says dinosaurs_with_hats.mp4. Not sure what that is about. I'm just going to check the filetype again of the .wav file.
Looks like the filetype is a zip archive. When I unzip it, it inflates dinosaurs_with_hats.mp4. I looked into that file, and it is another uuencoded file. You have got to be kidding me. I immediately imagine the scenario of infinite going through this cycle. I really hope I am not missing the big picture. I am going to try decoding this now.
Now I have a file named results. I look it says I unravelled the puzzle box. Woot lol. I thought there was going to be something more than this that would make it really complicated O.o Now I just need to do the steps for submitting it. Yay!
Remember: submit classname checks projects that were submitted and such
September 23, 2014
lab46:~/src$ cat /etc/motd|grep "the" | wc -l 3
lab46:~/src$ who > who.dat lab46:~/src$ cat who.dat NAME LINE TIME IDLE PID COMMENT smeas + pts/3 2014-08-29 11:52 02:31 1921 (cpe-67-251-64-18:S.0) dsaunde6 + pts/9 2014-09-19 15:59 . 30058 (pod01:S.1) mgardne8 + pts/11 2014-09-10 19:38 old 19834 (mosh [19834]) dsaunde6 + pts/14 2014-09-19 16:03 16:01 30107 (pod01:S.2)
Find the number of lines in who.dat
lab46:~/src$ cat who.dat | wc -l 57
break the file into 3 sections
lab46:~/src$ head -19 who.dat > section1.dat lab46:~/src$ tail -19 who.dat > section3.dat lab46:~/src$ head -38 who.dat | tail -19 > section2.dat lab46:~/src$ cat section2.dat
putting all the files together and sorting them alphabetically
lab46:~/src/unix_linux/classwork/9_23_14$ cat section2.dat section1.dat section3.dat > who.all lab46:~/src/unix_linux/classwork/9_23_14$ cat section2.dat section1.dat section3.dat | sort | less
lets find my username bin for my password use getent to display entries in a database - queries the system for everything it has access too (local and remote), because our user accounts are not defined on lab46, but on an LDAP system.
lab46:~/src$ getent passwd | grep "$USER" cclay:x:5798:5000:Casper Clay:/home/cclay:/bin/bash
passwords now are in etc/shadow instead of the password file. This is because people could get the encrypted passwords and crack them. Now only the system has access to them.
bc -l opens a handy dandy calculator change base to hexadecimal obase=16
use the groups command to see what group you are in (as you can see im in 3 groups)
lab46:~/src/unix_linux/classwork/9_23_14$ groups lab46 unix cclay
see everyone who is in the unix group
lab46:~/src/unix_linux/classwork/9_23_14$ getent group | grep 'unix' unix:*:1730:abuck4,acarson1,ahoover3,avolino,cclay,ddewert1,dsaunde6,dshadeck,dsoutha3,jcliteur,jjacobs7,mgleaso7,mp010784,mquesad1,nsano,nvitull1,rhenggel,skramer1,ssmit133,stiwari1,tarnold4,tmosgrov,vgarfiel,wedge
September 25, 2014
Starting class today talking about how some kid who fork bombed lab46 un-intentionally. Then we started talking about the recent bash vulnerability that has surfaced.
Just a reminder that ps creates a list of processes.
Flashback to the hilarious Vogue website easter egg :P
Continuation of the fun with getent, pipes, grep, tail, head, and now cut :D Cut allows us to remove sections from each line of files :D
teames3:x:5944:5000:Timothy S. Eames:/home/teames3:/bin/bash jfenchel:x:5945:5000:Jonathan South:/home/jfenchel:/bin/bash lab46:~/src/unix_linux/classwork$ getent passwd | cut -d':' -f1,7 root:/bin/bash daemon:/usr/sbin/nologin
filtering out the above to things with 'th' in them, and then taking the last 16 results. In this case we will see there are not actually 16 results to be displayed.
lab46:~/src/unix_linux/classwork$ getent passwd | cut -d':' -f1,7 | grep 'th' | tail -n 16 appelthp:/bin/bash ssmith85:/bin/bash rthatch2:/bin/bash thakes3:/bin/bash mhether2:/bin/bash gerth:/bin/bash bkathan:/bin/bash dsoutha3:/bin/bash athoma13:/bin/bash lab46:~/src/unix_linux/classwork$
We can count 9 names, or count the number of matches with word count.
lab46:~/src/unix_linux/classwork$ getent passwd | cut -d':' -f1,7 | grep 'th' | tail -n 16 | wc -l 9
Quotes:
| Quote Char | Quote Name | Meaning |
|---|---|---|
| ' ex. 'test' | full quote | literal quote |
| “ ex. “test” | half quote | Allow for expansion (variable) |
| ` ex. `test` | backquote, backtick | command expansion |
| ( ex. (test) | backquote, backtick | command expansion |
Quote examples:
lab46:~/src/unix_linux/classwork$ echo $USER cclay lab46:~/src/unix_linux/classwork$ echo "$USER" cclay lab46:~/src/unix_linux/classwork$ echo '$USER' $USER lab46:~/src/unix_linux/classwork$ echo "${USER}, you are logged in `who|grep "$USER"|wc -l` times" cclay, you are logged in 1 times lab46:~/src/unix_linux/classwork$ lab46:~/src/unix_linux/classwork$ uol=$(/usr/bin/who|wc -l) lab46:~/src/unix_linux/classwork$ echo $uol 37 lab46:~/src/unix_linux/classwork$ t=`echo "There are $uol users on the system"` lab46:~/src/unix_linux/classwork$ echo $t There are 37 users on the system
Semi colons aka ; will separate commands on the same line
September 30, 2014
Started class out today talking about puzzlebox2 and the new project. Then had a lecture on problem solving. If you are stuck, manipulate the unix world in a way where it works for what you need it for.
Need to look-up the command uniq for the new project. Also will need the command tr, which will get rid of specified characters. Example:
lab46:~/src/unix_linux/projects/puzzlebox2$ printf "\tHello\tthere\n" | tr '\t' ' ' Hello there
This got rid of the spaces.
You can use man man to view the different sections of the man file.
Use man ascii to see a complete ascii table.
Wildcards are like a watered down version of regular expressions. Regular expressions are used within files, while wildcards are used in the command line.
Wildcards:
| Wildcard Char | Meaning |
|---|---|
| ? | matches any single character |
| * | match 0 or more of any character |
| [] | match any one of enclosed characters |
| [^] | do not match any of enclosed chars |
Examples:
ls ??????? ls l* ls *.zip ls *.tar* ls ?[aeiou]? ls c[aeiou]*t ls c[^aeiou]*t
October 2, 2014
Need to make sure I'm subscribed to the class mailing list.
Practicing commands today with wildcards. Went into usr/bin to count files :D
Shows the number of files with 5 or more characters in the name:
lab46:/usr/bin$ ls -d ?????* | wc -l 678
shows the number of files that have at least 4 or more characters, and whose first, second, and last character is oneof the jeopardy characters.
lab46:/usr/bin$ ls -d [rstlne][rstlne]?*[rstlne] | wc -l 43
All files whos names do not have vowels in them.
lab46:/usr/bin$ ls -d [^aeiouy][^aeiouy][^aeiouy] | wc -l 27
Looking for the number of all directories and symbolic links
lab46:/usr/bin$ ls -l|egrep '^(l|d)' | wc -l 203
October 7, 2014
Started out class talking about the dataproc assignment. Due date changing to the 22nd; however, if we want a bonus point, we can still turn it in by the original due date.
A process is a “program in action.”
Each process has a PID or ProcessID.
Running the command ps lists the current processes running and their PID's, as-well as other information regarding each process running.
In this example, we can see that when we run the command cat and let it sit, it can then be viewed by running the process ps in a different terminal.
We can kill processes with the kill command. Here are the different kill signals available.
lab46:~$ kill -l 1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL 5) SIGTRAP 6) SIGABRT 7) SIGBUS 8) SIGFPE 9) SIGKILL 10) SIGUSR1 11) SIGSEGV 12) SIGUSR2 13) SIGPIPE 14) SIGALRM 15) SIGTERM 16) SIGSTKFLT 17) SIGCHLD 18) SIGCONT 19) SIGSTOP 20) SIGTSTP 21) SIGTTIN 22) SIGTTOU 23) SIGURG 24) SIGXCPU 25) SIGXFSZ 26) SIGVTALRM 27) SIGPROF 28) SIGWINCH 29) SIGIO 30) SIGPWR 31) SIGSYS 34) SIGRTMIN 35) SIGRTMIN+1 36) SIGRTMIN+2 37) SIGRTMIN+3 38) SIGRTMIN+4 39) SIGRTMIN+5 40) SIGRTMIN+6 41) SIGRTMIN+7 42) SIGRTMIN+8 43) SIGRTMIN+9 44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+13 48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-12 53) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9 56) SIGRTMAX-8 57) SIGRTMAX-7 58) SIGRTMAX-6 59) SIGRTMAX-5 60) SIGRTMAX-4 61) SIGRTMAX-3 62) SIGRTMAX-2 63) SIGRTMAX-1 64) SIGRTMAX
The following can be used to close the process cat with the appropriate PID:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND cclay 24925 0.0 0.1 14048 2048 pts/10 Ss 16:46 0:00 -bash cclay 24933 0.4 0.1 14048 2044 pts/107 Ss 16:46 0:00 -bash cclay 24941 1.0 0.0 6424 348 pts/10 S+ 16:46 0:00 cat - cclay 24942 0.0 0.0 11332 1076 pts/107 R+ 16:46 0:00 ps u lab46:~$ kill -1 24941 lab46:~$ kill hup 24941 lab46:~$ kill sigup 24941
If we kill a process by simply saying “kill PID”, the kill process automatically uses signal “15” or “SIGTERM”
In the kill command, signal “9” or “SIGKILL” is the most powerful signal and will close anything. It should be used sparingly, as it can cause problems if not fully understood.
Running “ps aux”, we can view every process running on the system.
Here is an example of viewing all of a single persons processes on the system.
lab46:~$ ps aux | grep $USER root 24922 0.0 0.2 99900 4520 ? Ss 16:46 0:00 sshd: cclay [priv] cclay 24924 0.0 0.1 99900 2152 ? S 16:46 0:00 sshd: cclay@pts/10 cclay 24925 0.0 0.1 14056 2068 pts/10 Ss 16:46 0:00 -bash root 24930 0.0 0.2 99900 4524 ? Ss 16:46 0:00 sshd: cclay [priv] cclay 24932 0.0 0.1 99900 2152 ? S 16:46 0:00 sshd: cclay@pts/107 cclay 24933 0.0 0.1 14056 2096 pts/107 Ss 16:46 0:00 -bash cclay 24953 0.0 0.0 6424 344 pts/10 S+ 16:47 0:00 cat - cclay 26488 0.0 0.0 11332 1064 pts/107 R+ 16:57 0:00 ps u aux cclay 26489 0.0 0.0 15228 948 pts/107 S+ 16:57 0:00 grep cclay lab46:~$
Another interesting command is top, which will show realtime system information such as process and resource usage.
Example(Only the first 12 lines of output):
top - 16:58:09 up 12 days, 17:52, 49 users, load average: 1.39, 0.57, 0.33 Tasks: 408 total, 4 running, 399 sleeping, 5 stopped, 0 zombie %Cpu(s): 2.1 us, 7.9 sy, 0.0 ni, 75.6 id, 0.0 wa, 0.0 hi, 0.3 si, 14.1 st KiB Mem: 1535680 total, 1499612 used, 36068 free, 398056 buffers KiB Swap: 131068 total, 37104 used, 93964 free. 524684 cached Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 31518 thakes3 20 0 120988 2484 1364 S 23.3 0.2 3:07.53 irssi 28580 root 20 0 0 0 0 S 4.3 0.0 2:08.08 kworker/0:1 7 root 20 0 0 0 0 R 3.6 0.0 33:35.86 rcu_sched 26493 jjacobs7 20 0 16044 1700 1052 S 2.6 0.1 0:00.52 top 26549 wedge 20 0 9908 1740 1348 S 2.3 0.1 0:00.07 bash
You can customize the top process by toggling keys such as “z” for color, or 1 for specific cpu core usage.
Example:
In unix we use shells. The default shell that we use is Bash.
(View the latest bash vulnerability here: http://en.wikipedia.org/wiki/Shellshock_(software_bug)
AT&T UNIX VS. BSD UNIX SHELLS
| Shells | Meaning |
|---|---|
| AT&T- sh | bourne shell |
| AT&T- bash | bourne again shell |
| BSD- csh | C shell |
When running a process in a terminal, we can save it rather than killing it by using CTRL Z
Example:
lab46:~$ cat - ^Z [1]+ Stopped cat - lab46:~$
We can view these stopped or active process with the command jobs. We can use the command bg to run processes in the background and the command fg to run processes in the foreground.
Example of freezing, viewing, and re-running processes:
lab46:~$ sleep 3600 (sleep 180;echo hi) ^Z [2]+ Stopped sleep 3600 lab46:~$ jobs [1]- Stopped cat - [2]+ Stopped sleep 3600 lab46:~$ bg 1 [1]- cat - & lab46:~$ jobs [1]+ Stopped cat - [2]- Stopped sleep 3600 lab46:~$ bg 2 [2]- sleep 3600 & lab46:~$ jobs [1]+ Stopped cat - [2]- Running sleep 3600 & lab46:~$ fg 2 sleep 3600
October 9, 2014
We can view hidden files by issuing ls -a , which will show all files. Lots of these hidden files such as .bashrc are configuration files.
A useful command we can use is alias. Here is an example so that when we type bob, we get bob printed on our terminal.
lab46:~$ alias bob='echo bob' lab46:~$ bob bob
An alias will stay until you end your session. You can take aliases away with the unalias command. If you want permanent aliases, you can add them in .bashrc in our home directory.
The environment variable $PATH shows all of the paths that the system will look for commands. If we type jim, it will look for a command called jim in all of the following places:
lab46:~$ $PATH -bash: /usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games: No such file or directory
Started going over scripting!
Shell Scripts
we can save shell scripts in files ending in .sh or .bash Here is an example of a simple script called myscript.sh 3 line script:
df who ls
Now lets run this script (not all output included)!
lab46:~/src/unix_linux/classwork/10_9_14$ bash myscript.sh Filesystem 1K-blocks Used Available Use% Mounted on /dev/xvda1 3997376 1784952 1986328 48% / udev 10240 0 10240 0% /dev tmpfs 153568 160 153408 1% /run
We are not able to run this script with ./ because we don't have the proper permissions to execute it, which is why we used bash. If we changed the permissions with chmod, we will be able to then execute the script normally. Example (not all output included):
lab46:~/src/unix_linux/classwork/10_9_14$ chmod 700 myscript.sh lab46:~/src/unix_linux/classwork/10_9_14$ ./myscript.sh Filesystem 1K-blocks Used Available Use% Mounted on /dev/xvda1 3997376 1784968 1986312 48% / udev 10240 0 10240 0% /dev tmpfs 153568 160 153408 1% /run
To see our environment, which is bash: #!/bin/bash
Now lets make another cool script called nametest.sh to create a mini program that asks for your name and spits it back out :D Here is the script:
echo -n "What is your name? " read name echo "Hello, $name, nice to meet you! " exit 0
Now lets run it!
lab46:~/src/unix_linux/classwork/10_9_14$ ./nametest.sh What is your name? Casper Hello, Casper, nice to meet you! lab46:~/src/unix_linux/classwork/10_9_14$
Bash is a higher level among programming languages. We don't have to worry about things such as data types.
We can use tab completion (AKA push tab after typing in an in-complete name) to complete unique names such as file names without typing it all out. If have two files with similar names, tab completion will complete the name up until the point at which it is still unique. If you have two files with similar names such as hhhhhs and hhhs, you can push tab twice to get information on the files, and see why it wont complete the entire name. Example:
lab46:~/src/unix_linux/classwork/10_9_14$ ls hasdkhfakjhfafqhfiufubflks hasdkhfakjhfafqhfiufubflksajfasjfdsafjffaoiehfiquhfahf myscript.sh nametest.sh lab46:~/src/unix_linux/classwork/10_9_14$ hash hasdkhfakjhfafqhfiufubflks/ hasdkhfakjhfafqhfiufubflksajfasjfdsafjffaoiehfiquhfahf/ myscript.sh nametest.sh lab46:~/src/unix_linux/classwork/10_9_14$ hash
We can use the rm command to remove files and directories. Using rm -f deletes a director and everything in it.
Now lets test checking conditions inside our scripts and using loops. It is important to remember that things must be properly spaced such as 5 = 0. If it was 5=0, the script will try to assign 5 to 0 instead of checking the condition.
Here is a script called numbertest.sh (we used -eq for = and -gt for > to avoid string vs number comparison problems):
Now lets run this amazing game we just made!
lab46:~/src/unix_linux/classwork/10_9_14$ ./numbertest.sh Pick a number: 1 too low Do you want to play a again?y Pick a number: 7 too high Do you want to play a again?y Pick a number: 5 correct Do you want to play a again?n lab46:~/src/unix_linux/classwork/10_9_14$
October 21, 2014
Starting class today talking about random numbers to improve our number guessing game! Going to have to leave early today for a Dentist appointment :(
We can print a random number with the command echo $RANDOM
lab46:~/public_html/Lab09$ echo $RANDOM 2944 lab46:~/public_html/Lab09$ echo $RANDOM 15863 lab46:~/public_html/Lab09$ echo $RANDOM 9721
% modulus / division
34%5 = 4 34/5 = 6
October 23, 2014
We can use bc to easily do math in scripts.
lab46:~/src/unix_linux/classwork/10_23_14$ echo "2+2" | bc 4
Went over loops.
Now need to make a grade calculator for the course for our next project. Started working on that in class.
October 23, 2014
We can use floating points in bash with the -l argument of bc. If we tried to compute 6/7, we would get 0. Instead, we can try the following command:
lab46:~$ echo "(6/7)*100" | bc -l 85.71428571428571428500
To get only an integer, we can do the following with what we already know:
lab46:~$ echo "(6/7)*100" | bc -l | cut -d'.' -f1 85
To get the first two decimals, the following can be done:
lab46:~$ echo "(6/7)*100" | bc -l | cut -c1-5 85.71
To finally add the percent sign, we can do this:
lab46:~$ echo "$(echo "(6/7)*100" | bc -l | cut -c1-5)%" 85.71%
For the grade calculator project, we can calculate the attendance by splitting up the dates. Currently, they are all on one line. Lets split them up onto their own lines using translate, and white space as a delimiter.
lab46:~$ status unix | grep $USER | tr " " "\n" [cclay] 238 240 245 247 252 254 259 261 266 268 273 275 280 282 294 296
October 30, 2014
Knowledge Assessment! Wasn't so bad!
November 4, 2014
Today we venture off into regular expressions!
*Remember - Wildcards are for files, regular expressions are for text *
Regular Expressions are used to describe patters, patterns in text that we are looking for.
| Regular Expressions | Meaning | Extended Expressions (newer)? |
|---|---|---|
| . | matches any single character | No |
| * | match 0 or more of the previous | No |
| [] | match any one of enclosed characters | No |
| [^] | do not match any of enclosed chars | No |
| \< | match start of word | No |
| \> | match end of word | No |
| ^ | match start of line | No |
| $ | match end of line | No |
| | | or | Yes |
| () | grouping → \( \) (sed) | Yes |
| + | match 1 or more of the previous | Yes |
Now, lets try some stuff out.
To find the number of words that are exactly 4 characters long, the following can be done:
lab46:/usr/share/dict$ cat words | grep '^....$' | wc 3346 3346 16730
Therefore, there are 3346 words that are exactly 4 letters long.
To find the amount of words that are at least 4 letters long and end with a lower case g, the following can be used:
lab46:/usr/share/dict$ cat words | grep '...g$' | wc 6999 6999 70411
This results in 6999 words.
You can also use this command to do the same thing. The reason the previous command worked was because the beginning of the line did not matter.
lab46:/usr/share/dict$ cat words | grep '^....*g$' | wc 6999 6999 70411
This also results in 6999 words.
We can find the words with at-least 3 characters and are all lowercase with the following:
lab46:/usr/share/dict$ cat words | grep '^[a-z][a-z][a-z][a-z]*$' | wc 62797 62797 584774
We get back 62797 words.
To find any words that have at least 3 or more vowels, we can use the following:
lab46:/usr/share/dict$ cat words | grep '^.*[aeiouy].*[aeiouy].*[aeiouy].*[aeiouy]*' | wc 64422 64422 682102
This results in 64422 words.
We can enable extended regular expressions mode by using **egrep** instead of **grep**. **fgrep** is used to find only literal strings. This can be faster if you are trying to only find literal strings.
To find any words that end in 'ed' or 'ing', we can use the following:
lab46:/usr/share/dict$ cat words | egrep '*(ed)$|(ing)$' | wc 13412 13412 130200
This results in 13412 words.
Next class, we will blow our minds once again by learning about sed :D
November 6, 2014 The grade calculator is now an official project. Today we are continuing regular expressions and will start sed.
If we want to find all of the even weeks in the opus grades, we can use the following:
lab46:~$ status unix | grep 'opus' | egrep 'week([02468]|[0-9][02468])' 1:opus:week2entry 1:opus:week4entry 1:opus:week6entry 1:opus:week8entry
Here is a cool script using sed to re-arrange the opus grades. I don't completely understand it, but here is the code!
lab46:~$ status unix | grep 'opus' | sed 's/^.*\([01]\):\([a-z][a-z]*\):\(week\)\([0-9]\)\(.*\)$/\5 for \3 \4 \2 [\1]/g' entry for week 1 opus [1] entry for week 2 opus [1] entry for week 3 opus [1] entry for week 4 opus [1] entry for week 5 opus [1] entry for week 6 opus [1] entry for week 7 opus [1] entry for week 8 opus [1] entry for week 9 opus [1]
November 11, 2014
./myscript a b c
$# - number of arguments $* - all your arguments “a b c” $0 - script name $1 - “a” $2 - “b” $3 - “c”
Determine where a script is located.
lab46:~$ whereis status status: /usr/local/bin/status /usr/local/bin/status.logic
commands at and cron / crontab are good for automation of things at certain times.
November 13, 2014
Going over more automation stuff.
We can use the command at to specify a time to run a command at. Example:
at 16:16 ls > out ctrl-d
Cron jobs are useful for automating tasks.
Crontab -e will open crontab configuration to create new tasks. An example of a task is:
*/4 * * * * /usr/bin/who
This task will run the who command ever 4 minutes of every minute, hour, day, week, year
November 18, 2014
Today we got a new project. It is the timeline project. Basically, we need to calculate the amount of time spent on the system overall and per month for a specified user. Something interesting that we will do in this project is create a histogram.
Two useful commands: last and lastlog. Last will show all of the logins and information about those logins of all of the users in the last year. Lastlog will show the most recent last login of each user.
I think I can either search against the home directory to see if a username exists, or simply say there is no data/ the user does not exist by searching through login data. It would be nice to distinguish between the two.
November 20, 2014
Today was a work on stuff day. New project to do, the timeonline project. Last night I finished the grade calculator project. It is quite nice. I have a nice gui, along with a debug mode.
December 2, 2014
Had issues editing my opus, they are fixed now though (at least when I am at the lair). Time for the end of course experience :D Still need to do the time online project.
