~~TOC~~
A Record of Sorts
Ctrl-Alt-wink-wink
… one man's trip down the rabbit hole.
You have stumbled upon Matt Cooper's web log. This page is organized in a manner that makes it easy to stroll through many technical topics of one sort or another, mostly gathered throughout Matt's journey through the universe of computer science and information security. Don't look for topics to bubble up to the top. That's not how this works. This is more of a wiki-log. Use the navigation to the left, or visit The Lab46 Link Garden (highly recommended).
Drawing on my fine command of language, I said nothing.
- Mark Twain
Hi, I'm Matt, small business owner, husband, father and lately, student of computer science. I read somewhere that a computer is to computer science as the telescope is to astronomy, and I agree.
Here are some interesting things (you decide) about me:
In my professional life I develop web applications and business systems for small businesses. (MyPluribus.com). Running a business can be pretty difficult, so I work daily to help small business owners learn to use technology to solve their problems. I spend lots of time reading help files, writing help files and creating widgets and systems that accomplish a variety of goals.
The other side of my life revolves around my family. I have a beautiful wife and a very bright young son. My wife is a great wife and mother. My son is a great kid. When I'm not working, I'm with them. This is my idea of a really good life.
A hastily-thrown-together group of information that may help you memorize some things (as it did me):
In the game Freecell, cards are dealt into columns. Later, interactions take place with the cards that makes the game fun to play … however, more interesting to me is the logic of dealing out the cards.
This first snippet deals out the tableus (that is the columns) by dealing 1 card per column for 8 columns until all 52 cards have been dealt. This snippet uses an arbitrary check to see if the number of cards dealt has reached 52. (Not all that complex, however it takes care of the deal pattern.)
#include <stdio.h> #include <stdlib.h> int main(){ int ctDeal=0; int deal[8][52]; int row, col; // deal the cards for(row = 0; row < 52; row++){ for(col = 0; col < 8; col++){ deal[col][row] = ctDeal; ctDeal++; if(ctDeal > 52){ col = 8; row = 52; } } } // show the deal ctDeal = 0; for(row = 0; row < 52; row++){ for(col = 0; col < 8; col++){ printf("| %d |", deal[col][row]); ctDeal++; if(ctDeal > 52){ row = 52; col = 8; } } printf("\n"); } }
Next, a short snippet that takes care of some random numbering and dealing out 13 cards from each suit:
#include <stdio.h> #include <stdlib.h> #include <time.h> int main(){ int ctSpades = 0; int ctClubs = 0; int ctHearts = 0; int ctDiamonds = 0; int ctDeal = 0; srand(time(NULL)); printf("time is %d \n", time(NULL)); int num, num2, row, col; row = 0; col = 0; while(ctDeal < 52){ num = rand() % 4+1; switch(num){ case 1: if(ctClubs < 13){ printf("case is Clubs ctClubs = %d -- ctDeal = %d \n", ctClubs, ctDeal); ctClubs++; ctDeal++; } break; case 2: if(ctSpades < 13){ printf("case is Spades ctSpades = %d -- ctDeal = %d \n", ctSpades, ctDeal); ctSpades++; ctDeal++; } break; case 3: if(ctHearts < 13){ printf("case is Hearts ctHearts = %d -- ctDeal = %d \n", ctHearts, ctDeal); ctHearts++; ctDeal++; } break; case 4: if(ctDiamonds < 13){ printf("case is Diamonds ctDiamonds = %d -- ctDeal = %d \n", ctDiamonds, ctDeal); ctDiamonds++; ctDeal++; } break; } } }
The process is as follows: Fist, generate a random number between 1 and 4 (I.E. the number of suits in a deck of cards); next, a switch statement is hinged upon this random number and a counter for each suit is incremented. If the suit's counter reaches 12 then that suit is empty (that is, it has been dealt out) and a random number is redrawn. However, note that this script outputs the cards in a highly predicable manner. more to come.
The linked list is a fairly complex idea that consists of a series of simple ideas. The general idea is to implement a system whereby each element has a reference to the next and/or previous elements in a list. The list could be ordered or unordered. The important part is that the list knows the end of one element and/or the beginning of the beginning of the next.
/* * linkedlist4.c - singly linked list node insertion * */ #include <stdio.h> #include <stdlib.h> // The node struct node { int value; struct node *next; }; typedef struct node Node; int main() { Node *start, *tmp, *devnull; int i = 0, input = 0; start = tmp = NULL; do { printf("Enter a value (-1 to quit): "); scanf("%d", &input); if ((start == NULL) && (input != -1)) { start = (Node *) malloc (sizeof(Node)); tmp = start; tmp -> value = input; tmp -> next = NULL; } else if (input != -1) { tmp -> next = (Node *) malloc (sizeof(Node)); tmp = tmp -> next; tmp -> value = input; tmp -> next = NULL; } } while (input != -1); tmp = start; printf("Linked List DISPLAY: "); while (tmp != NULL) { printf ("(%d) %d -> ", i, tmp -> value); tmp = tmp -> next; i++; } printf ("NULL\n"); tmp = start; printf("Select Node # to insert before: "); scanf("%d", &input); for (i = 0; i < (input - 1); i++) { tmp = tmp -> next; } if ((i == 0) && (input == 1)) { i++; } printf("Enter value for new node: "); scanf("%d", &input); // Create new node devnull = (Node *) malloc (sizeof(Node)); devnull -> value = input; if (i != 0) // anything but the first { devnull -> next = tmp -> next; tmp -> next = devnull; } else // the first node { devnull -> next = start; start = devnull; } tmp = start; i = 0; printf("Linked List DISPLAY: "); while (tmp != NULL) { printf ("(%d) %d -> ", i, tmp -> value); tmp = tmp -> next; i++; } printf ("NULL\n"); return(0); }
/* * linkedlist3.c - singly linked list node deletion * */ #include <stdio.h> #include <stdlib.h> // The node struct node { int value; struct node *next; }; typedef struct node Node; int main() { Node *start, *tmp, *devnull; int i = 0, input = 0; start = tmp = NULL; do { printf("Enter a value (-1 to quit): "); scanf("%d", &input); if ((start == NULL) && (input != -1)) { start = (Node *) malloc (sizeof(Node)); tmp = start; tmp -> value = input; tmp -> next = NULL; } else if (input != -1) { tmp -> next = (Node *) malloc (sizeof(Node)); tmp = tmp -> next; tmp -> value = input; tmp -> next = NULL; } } while (input != -1); tmp = start; printf("Linked List DISPLAY: "); while (tmp != NULL) { printf ("(%d) %d -> ", i, tmp -> value); tmp = tmp -> next; i++; } printf ("NULL\n"); tmp = start; printf("Select Node # to delete: "); scanf("%d", &input); for (i = 0; i < (input - 1); i++) { tmp = tmp -> next; } if (i != 0) // anything but the first { devnull = tmp -> next; tmp -> next = devnull -> next; } else // the first node { devnull = start; start = devnull -> next; } devnull -> next = NULL; free(devnull); tmp = start; i = 0; printf("Linked List DISPLAY: "); while (tmp != NULL) { printf ("(%d) %d -> ", i, tmp -> value); tmp = tmp -> next; i++; } printf ("NULL\n"); return(0); }
I asked for, and was awarded with the DokuWiki LaTeX plugin, with hopes that I would be able to find some extra time to explore some mathematic markup and ultimately find a digital den for my math notes. Before I could start, I needed to take a short side-trip through the LaTeX documentation.
Here's an example of a matrix:
<latex> D_z = \left| \begin{array}{ccc} y_1 ... C_1\\ \vdots \ddots \vdots\\ y_2 ... C_2\end{array} \right| -> z = \frac{D_z}{D} => y = \frac{(-4)(-55)-(14)(0)}{(-4)(-12)-(14)(5)} => z = -10 </latex>
This is the output of the above LaTex:
<latex>
D_z
= \left|
\begin{array}{ccc}
y_1 … C_1
\vdots \ddots \vdots
y_2 … C_2\end{array}
\right| → z = \frac{D_z}{D} ⇒ y = \frac{(-4)(-55)-(14)(0)}{(-4)(-12)-(14)(5)} ⇒ z = -10
</latex>
Here's another one showing some simple math layouts
<latex> \begin{array}{3} -5(3x - 3y + 2z = 11)\\ \underline{+ 15x - y - 2z = 0}\\ 14y - 12z = -55\\ .\end{array}\\ </latex>
This is the output of the above LaTeX
<latex>
\begin{array}{3}
-5(3x - 3y + 2z = 11)
\underline{+ 15x - y - 2z = 0}
14y - 12z = -55
.\end{array}
</latex>
As you can see, LaTeX isn't the easiest on the eyes. In fact, it's a little ugly. But it does something that can't be done easily on any platform, so you have to respect that.
Diplomacy is the art of saying “Nice doggie” until you can find a rock.
- Will Rogers
A Metaphor:
Since there are two groups:
Further, since the laws of nature state that possession of said chickens and mules must, at some point, be earned.
Then, I suggest that the Group 1 get together and start earning chickens and mules, and then give them all away to the citizens that they believe are entitled to those chickens and mules. I also suggest that those who belong to Group 2, each earn their own chickens and mules.
Having been asked the question, 'What do you think your grade should be?' by more than one professor, I've cataloged my response here to save time in the future. If you're one of said professors, please find my response below:
Education is relative. Education requires motivation. Education shrugs off the disingenuous, and accepts only those who arrive without gimmick. Education cannot be lured, rather it is a gift to those who find a thrill and sense of accomplishment in the pursuit. Many seek an end to their education, others look for the newest beginning. Yet, learning never stops, and therefore cultivation and innovation remain the accomplishments of the curious. That is to say, the journey is the prize.
The objective of the project is simply an exercise in learning a new (to me) programming language using a simple data set. The following .PHP file reads in a local (to Lab46) file containing a single line which holds the first million digits of pi. The line is split into pages of 100 groups of 64 digits (that is, 6400 digits per page). A pagination script breaks down simple links to nearby pages, allowing the user to investigate all 1 million digits. Enjoy:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Frameset//EN"> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=MacRoman"> <title>1 Million Bits of Pi</title> <link rel="stylesheet" type="text/css" href="pi.css"> </head> <body> <?php $max_rows = 100; // maximum rows of output $chunk = 64; // size of data chunk $nearby = 21; // number of nearby pages /* * number of chars per page == * ($max_rows * $chunk * $page_num > ftell(END) ? ftell(END) - ($max_rows * $chunk * $page_num) : $max_rows * $chunk) * * number of pages == ftell(END) / ( $max_rows * $chunk ) * starting digit of page == $page_num * $max_rows * $chunk * * end digit of page == * ($max_rows * $chunk * $page_num + $max_rows * $chunk > ftell(END) ? * $max_rows * $chunk * $page_num) + ftell(END) - ($max_rows * $chunk * $page_num) * :$max_rows * $chunk * $page_num + $max_rows * $chunk); */ $i = 0; // iterator $odd = 0; // number of odd digits per line $max_odd = 0; $even = 0; $max_even = 0; $sum = 0; $max_sum = 0; $primes = 0; $zeros = 0; $ones = 0; $twos = 0; $threes = 0; $fours = 0; $fives = 0; $sixes = 0; $sevens = 0; $eights = 0; $nines = 0; $file_name = "pi.txt"; $f = fopen($file_name, "rb"); //ECHO TITLE echo "<div id=\"title\">1 Million Bits of Pi</div>"; // BEGIN PAGINATION (minimal use of variables) $p = (int)$_GET['p']; //page sought echo "<div id=\"pagination\">"; // go to the place sought fseek($f, $chunk*$max_rows*$p); echo ($p > 1 ? "<div class=\"start\"><a href=\"?p=0\">Start</a></div>" : ""); echo ($p > 0 ? "<div class=\"prev\"><a href=\"?p=" . ($p-1) . "\">Previous</a></div>" : ""); // loop nearby previous page lines $n = ($p < 1 ? 0 : ($p < $nearby ? 0 : $p-$nearby )); $limit = $p; for($n; $n < $limit; $n++){ echo "<span class=\"link\"><a href=\"?p=" . $n . "\">" . $n . "</a></span>"; } echo "<div class=\"page\">Page : " . $p . "</div>"; // go to end fseek($f, -$chunk*$max_rows, SEEK_END); // loop nearby next page lines $n = $p+1; $limit = ($p+$nearby < round(ftell($f)/($chunk*$max_rows)+0.5) ? $p + $nearby: round(ftell($f)/($chunk*$max_rows) + 0.5)); for($n; $n <= $limit; $n++){ echo "<span class=\"link\"><a href=\"?p=" . $n . "\">" . $n . "</a></span>"; } echo ($p < ftell($f)/($chunk*$max_rows) ? "<div class=\"next\"><a href=\"?p=" . ($p+1) . "\">Next</a></div>" : ""); echo ($p != round(ftell($f)/($chunk*$max_rows)+0.5) ? "<div class=\"end\"><a href=\"?p=" . round(ftell($f)/($chunk*$max_rows) + 0.5) . "\">End</a></div>":""); // back to the place sought fseek($f, $chunk*$max_rows*$p); echo "</div>"; // END PAGINATION echo "<table>"; echo "<colgroup class=\"i\"></colgroup>"; echo "<colgroup class=\"pi\"></colgroup>"; echo "<colgroup class=\"even\"></colgroup>"; echo "<colgroup class=\"odd\"></colgroup>"; echo "<colgroup class=\"primes\"></colgroup>"; echo "<colgroup class=\"sum\"></colgroup>"; echo "<colgroup class=\"zeros\"></colgroup>"; echo "<colgroup class=\"ones\"></colgroup>"; echo "<colgroup class=\"twos\"></colgroup>"; echo "<colgroup class=\"threes\"></colgroup>"; echo "<colgroup class=\"fours\"></colgroup>"; echo "<colgroup class=\"fives\"></colgroup>"; echo "<colgroup class=\"sixes\"></colgroup>"; echo "<colgroup class=\"sevens\"></colgroup>"; echo "<colgroup class=\"eights\"></colgroup>"; echo "<colgroup class=\"nines\"></colgroup>"; echo "<thead><tr>"; echo "<th colspan=\"2\">"; echo "<div class=\"showing\">Showing digits <strong>" . ftell($f) . "</strong> thru <strong>" . (ftell($f) + $chunk*$max_rows -1) . "</strong></div>"; echo "</th><th colspan=\"2\"></th><th></th><th></th><th colspan=\"10\">Occurence of Each Digit</th>"; echo "</tr><tr>"; echo "<th>i</th><th>64 Bits of Pi</th><th>Even</th><th>Odd</th><th>Primes</th><th>Sum</th>"; echo "<th>Zeros</th><th>Ones</th><th>Twos</th><th>Threes</th><th>Fours</th>"; echo "<th>Fives</th><th>Sixes</th><th>Sevens</th><th>Eights</th><th>Nines</th>"; echo "</tr></thead>"; echo "<tbody>"; $i = $p*$max_rows; while (!feof($f) && $i < ($p*$max_rows+$max_rows)) { $d = fread($f, $chunk); $a = str_split($d); foreach($a as $v){ if($v % 2 == 0){ $even++; }else{ $odd++; } $sum += $v; switch($v){ case 0: $zeros++; break; case 1: $ones++; $primes++; break; case 2: $twos++; break; case 3: $threes++; $primes++; break; case 4: $fours++; break; case 5: $fives++; $primes++; break; case 6: $sixes++; break; case 7: $sevens++; $primes++; break; case 8: $eights++; break; case 9: $nines++; break; } } echo "<tr>"; echo "<td class=\"i\">" . $i . "</td>"; echo "<td class=\"pi\">" . $d . "</td>"; echo "<td class=\"even\">" . $even . "</td>"; echo "<td class=\"odd\">" . $odd . "</td>"; echo "<td class=\"primes\">" . $primes . "</td>"; echo "<td class=\"sum\">" . $sum . "</td>"; echo "<td class=\"zeros\">" . $zeros . "</td>"; echo "<td class=\"ones\">" . $ones . "</td>"; echo "<td class=\"twos\">" . $twos . "</td>"; echo "<td class=\"threes\">" . $threes . "</td>"; echo "<td class=\"fours\">" . $fours . "</td>"; echo "<td class=\"fives\">" . $fives . "</td>"; echo "<td class=\"sixes\">" . $sixes . "</td>"; echo "<td class=\"sevens\">" . $sevens . "</td>"; echo "<td class=\"eights\">" . $eights . "</td>"; echo "<td class=\"nines\">" . $nines . "</td>"; echo "</tr>"; $odd = 0; $even = 0; $sum = 0; $zeros = 0; $ones = 0; $twos = 0; $threes = 0; $fours = 0; $fives = 0; $sixes = 0; $sevens = 0; $eights = 0; $nines = 0; $primes = 0; $i++; } echo "</tbody></table>"; echo "<div id=\"footer\">PHP in Vi (<a href=\"http://lab46.corning-cc.edu/~mcooper6\">M</a>)</div>"; fclose($f); ?> </body> </html>
To get a list of all current alias's on the system:
:~$ alias
So, for example to make a change to your ls command, issue the following:
:~$ alias ls='ls -color=auto -a'
The previous command will cause ls to automatically show color and dot files.
Other useful alias commands:
Protect yourself from accidental overwrite when copying files: (-i for interactive)
:~$ alias cp='cp -i'
Do the same when moving files:
:~$ alias mv='mv -i'
And of course for removing files:
:~$ alias rm='rm -i'
Installing a package begins with aptitude search (that is, unless you already know the name). So, for example, if you're looking for the MySQL package, use the following command:
:~$ aptitude search mysql
The above command returns a list of all packages containing the search phrase. Regular expreshnas can be substituted for the search term. Additionally, due to Debian's package manager, getting dependent packages happens automatically.
Installing packages requires the install
option to be passed to the aptitude command…and since the average user doesn't have ROOT privilege, the sudo (super-user do) command is necessary. For example:
:~$ sudo aptitude install less :~$ sudo aptitude install screen
Other packages I've installed:
The previous commands install the less and screen utilities. (which I installed during Linux class earlier this week.)
screen has many more uses than just running irssi. For example, if you have a large file that you're downloading with wget, simply open a new screen session, start the download, detach the from the screen, go home, maybe make a snack, catch a little shut eye … you're screen session will be plugging away, awaiting your return.
I began by checking to see if Apache was already installed on vm34.student.lab. In this case, it wasn't so I search for the latest package with the following command.
:~$ aptitude search apache
At the top of the list is Apache2 “Metapackage”. I chose that and installed it.
:~$ sudo aptitude install apache
After installing Apache, you can check the install using telnet.
:~$ telnet localhost 80
However, since telnet is not yet installed on vm34, issue the following command:
:~$ aptitude install telnet
Then have another go at the previous command.
Uninstalling packages is only slightly different from installing:
:~$ sudo aptitude remove nano
The previous command uninstalls the nano editor. (not sure who ever needed this in the first place … vi)
All you need to create a bash file is a text editor and a bash shell to run it. Start by creating a file with no extension:
:~$ nano ./home/$USER/basherific
OR
:~$ touch ./home/$USER/basherific
OR
:~$ vi ./home/$USER/basherific
After you've created your file, open it with your favorite text editor and get ready to add your bash script. Let's do something simple like create a variable and pass some arguments from the command line.
Begin by adding the following to your bash file:
#! /bin/bash usr=`who | grep $USER` echo $usr
This short script creates a variable (in this case, named who
) and assigns it the value of who | grep $USER
. Notice the $USER
? That's an environment variable that returns your user name. Also note, those aren't single quotes, they're back quotes (A.K.A. back ticks
). The next line echos the variable out to standard output. Save your file when you're finished.
What's #! /bin/bash
?? … known as the sh-bang, this tells the interpreter what shell to use when executing the script. Since the #
symbol acts as a comment in the bash shell, programmers call this the sh
… as in “hush up compiler”, and the exclamation point is known as the bang
as in something exciting that just went down. Further, had this been a Perl script this line would have read #! /bin/perl
… get it?
Now simply change the file permissions and you can call your new bash script from the command line:
:~$ chmod +x ./home/$USER/basherific # ... $USER refers to your linux username.
Now call it:
:~$ ./home/$USER/basherific
This example, while very simple, illustrates one of the many ways that users can use bash scripts to turn Linux into a multi-tasking machine. But, what if we wanted to get a little more involved. Wouldn't it be nice to call the script by file name, like you do any other utility. You can. Simply add the directory containing your script to your $PATH environment variable:
:~$ PATH=$PATH:./home/$USER
The dot above is very important. To make sure you got it right:
:~$ echo $PATH
You should see your updates at the end of the list. Now, you can call your bash script by name, and bash will happily search through all the possible directories listed in $PATH until it finds your new utility.
:~$ basherific
Next, using the redirection operator »
, send the output of this script to a file in the filesystem. Add the following line to your bash script:
echo $usr >> whoami.txt
Save your file and run it.
:~$ ./basherific
Now cat
the new file:
:~$ cat ./whoami.txt
The contents of your new file will be the results of your bash script.
Lets have one more test. Let's have the bash script tell us if we were successful. Add the following at the end of your bash script:
file ./whoami.txt && echo "Success" || echo "Fail"
This is a programmer friendly way check to make sure that the script actually did what it was told to do. However, as usual, there's another way. Try this:
Comment out this line
#file ./whoami.txt && echo "Success" || echo "Fail"
Then add this:
if [ -e ./whoami.txt ]; then echo "Success" else echo "Fail" fi
Save and close your bash script. Don't forget to remove your whoami.txt file, so it can be recreated:
:~$ rm ./whoami.txt :~$ ./basherific
You should receive a message stating that your venture was a success. Now, remove your file again and then comment out this line:
#echo $usr >> whoami.txt
Now, when the script is run, whoami.txt
will not be created and the script returns “Fail”. There you have it, a simple, yet effective way to test different boolean conditions.
The >
>
operator appends to the end of a file, while the >
operator overwrites the contents of the file. Be warned.
## ## Append to a file ## :~$ echo $usr >> whoami.txt ## ## Overwrite a file ## :~$ echo $usr > whoami.txt
while read LINE ; do echo "$LINE" done < fall2010-20100315.html
The above lines simply tell bash to read in another line from the file, as long as there are more lines to read. Once the file runs out of lines, the script is done.
In this case, the actual iterator is LINE, so we can output that by itself to get the actual line of text, however we can't count using this line as an iterator, and wouldn't it be handy to output some line numbers while coding the solution. To do this, add another variable inside the body of the while loop. This variable is incremented each time the loop executes.
while read LINE ; do N=$(($N+1)) echo "$N : $LINE" done < fall2010-20100315.html
Now the code outputs line numbers, which might or might not come in handy.
This script uses a for loop
to gather some numbers from the user. However, you'll notice that I didn't use bc
, in fact I didn't do any multiplication at all. I'll be revisiting bc
this in the coming days.
#!/bin/bash product=1 for((i=0;i< 4; i++)); do num=0 echo -n "Enter number $i: " read num let sum=$sum+$num let product=$(($product*$num)) done echo "sum is $sum" echo "product is $product"
This bash script accepts the user's name and then outputs the name and the number of characters. In this case, it was necessary to subtract 1 from the wc -l
…as wc -l
returns the number of lines +1
#!/bin/bash # # a program that propts for a name, counts the characters # and outputs the results echo -n "Please enter your name " read name; chars=$((`echo $name | wc -c` - 1)) echo "Your name is $name it has $chars characters"
This script generates a random number between 1 and 10. The user is allowed to guess 4 times before the number is given.
#!/bin/bash RANGE=10 rand=$(($RANDOM%$RANGE)) thisFlag=1 tries=1 while [ $thisFlag -eq 1 ] do echo "Guess a number between zero and ten " read num if [ $num -eq $rand ]; then echo "Got it!" thisFlag=0 else if [ $tries -gt 3 ]; then echo "The number is $rand" thisFlag=0 else echo "Sorry. Try again" fi fi tries=$(($tries+1)) done
I used the tr
utility in the script to translate a string to lower case. Simply flip the “[:upper:]”
and “[:lower:]”
to convert the opposite way.
#!/bin/bash echo -n "Enter a string " read theString lower=`echo "$theString" | tr "[:upper:]" "[:lower:]"` echo "You entered $theString" echo "Lower case: $lower"
This script is pretty short, in fact, it was taken from timebandit (see below). This line will provide you with the number of lines output by the last
utility. (Truth be known, it's probably better to subtract one or two from this one too, as last
outputs a couple additional lines)
#!/bin/bash echo "You've been logged in "`last $USER | wc -l`" times"
In this case, I first created a program that creates files. filemaker
is a short script that cuts the output of last
into “fields” based on the position of the colon (:). Using the -f1 option returns field one (left side of the : ); -f2 returns field 2 and so on. I also grepped out (using -v to invert) the “Still logged in” and “no logout” entries. Then, write these to a file by redirecting standard out to a file with the »
operator. (>
overwrites)
After that, filejoin simply puts it all back together using the paste
utility into a third file.
#!/bin/bash paste -d- test_logfile1 test_logfile2 > test_logfile3
#!/bin/bash # # a program that makes 2 files last | grep -v 'still logged in' | grep -v 'gone - no logout' | cut -d':' -f1 >> test_logfile1 last | grep -v 'still logged in' | grep -v 'gone - no logout' | cut -d':' -f2 >> test_logfile2
This script uses wget
to download a web page. After that, the lines of that file (called index.html) are counted and stored in the variable n_count. Then, the lines of a file called index.html.BU are gathered and counted and stored in a variable called o_count. Next, the two values are compared. If the older file (o_count) has more lines than (-gt for greater than) the new file (n_count), it echos “More”; if it has less it echos “Less” otherwise, it must be equal.
After everything's finished, the contents of the new file (index.html) are written over the contents of the old file (index.html.BU) and the new file (index.html) is deleted.
#!/bin/bash wget http://www.yahoo.com n_count=`cat index.html | wc -l` o_count=`cat index.html.BU | wc -l` echo "Counting lines..." if [ $o_count -gt $n_count ]; then echo "The web page has more lines now" elif [ $o_count -lt $n_count ]; then echo "The web page has less lines now" else echo "The web page has the same number of lines" fi echo "Old count is $o_count, the new count is $n_count" echo "Backing up ..." cat index.html > index.html.BU echo "Cleaning up..." rm -rf index.html echo "Done"
This scripts has some interesting points within. This script will output the amount of time each user has been logged into Lab46 for the current month. The script can be called from the command line by itself, or it can be called with arguments, in the form of user names.
@lab46:~$ ./timebandit @lab46:~$ ./timebandit $USER
The logic is as follows:
# A program that uses the last utility to determine the time that # each student is logged on for the current month # This program can be invoked by passing the students username as # an arguement, which returns the logged time for that student; # or the program can be invoked with no arguments to retrieve a # list of all users' logged time # check to see if the program has been invoked with arguements # if yes, use those username, if no use the entire class list if [ -z "$*" ];then users="`last | sort | sed -e 's/^\([a-z][a-z0-9]*\).*$/\1/' | uniq`" else users="$*" fi # loop through the list to retrieve logged times for n in $users; do days="`last | grep $n | grep -v 'still logged in' | grep -v 'gone - no logout' | grep '+' | sed -e 's/^.*(\(.*\)).*$/\1/' | cut -d':' -f1`" hours="`last | grep $n | grep -v 'still logged in' | grep -v 'gone - no logout' | sed -e 's/^.*(\(.*\)).*$/\1/' | cut -d':' -f1`" minutes="`last | grep $n | grep -v 'still logged in' | grep -v 'gone - no logout'| sed -e 's/^.*(\(.*\)).*$/\1/' | cut -d':' -f2`" # loop through the list of hours, removing leading zeros # and adding the remaining numbers together for h in $hours;do # check for '+' in case there's more than 24 hours chk=`echo $h | grep '+' | wc -l` #totaldays=0 #days=0 if [ $chk -eq 0 ]; then # not more than 24 hours h="`echo $h | sed -e 's/^0\([1-9][0-9]*\)/\1/'`" else # get the number of days, then add it up as d days="`echo $h | cut -d '+' -f1 | sed -e 's/^0\([1-9][0-9]*\)/\1/'`" let totaldays=$totaldays+$days # use the logic above to get h h="`echo $h | cut -d '+' -f2 | sed -e 's/^0\([1-9][0-9]*\)/\1/'`" fi # add the hours extracted to totalhrs let totalhrs=$totalhrs+$h let totaldays=$totaldays+0 done # loop through the list of minutes, removing leading zeros # and adding the remaining numbers together for m in $minutes;do m="`echo $m | sed -e 's/^0\([1-9][0-9]*\)/\1/'`" let totalmin=$totalmin+$m done # devide totalminutes by 60 to extract the number of hours # add hours extracted to hrs and leave the remaining minutes in min hrs=$(($totalmin/60)) min=$(($totalmin-$(($hrs*60)))) # add totalhrs to hrs hrs=$(($hrs+$totalhrs)) # devide hrs by 24 to extract days from hrs, then add totaldays to get final number of days days=$(($hrs/24)) hrs=$(($hrs-$(($days*24)))) days=$(($days+$totaldays)) for ((nn=0; nn<$days; nn++ ));do dbar=$dbar"@" done for ((nnn=0; nnn<$hrs; nnn++)); do hbar=$hbar"#" done for ((nnnn=0; nnnn<$min; nnnn=nnnn+10 )); do mbar=$mbar"." done echo "-----------------+------+" bar="$dbar $hbar $mbar" printf " %-8s | %-2s : %-2s : %-2s | %-2s %-2s %-2s \n" $n $days $hrs $min $dbar $hbar $mbar #echo "$n | $days days : $hrs hours : $min min | $dbar $hbar $mbar" #echo "$n | $h" bar="" hbar="" dbar="" mbar="" totalmin=0 totalhrs=0 totaldays=0 days=0 hrs=0 min=0 done
Fun may be had with the ~/.bashrc file in your home directory. Here's my simple, yet helpful .bashrc:
## ## automagically ls files after CD ## cdl() { cd "$@"; ls -Alh --color=auto; } alias cdl=cdl ## ## auto update in 3 chars ... remove the -y if you please ## alias upd='sudo apt-get -y update && sudo apt-get -y upgrade' ## ## move around the file system a bit more swiftly ## alias ..='cd ..' alias .2='cd ../..' alias .3='cd ../../..' alias .4='cd ../../../..' alias .5='cd ../../../../..' ## ## get me home ## alias home='cd ~/' ## ## edit those pesky priviliged files a bit faster ## alias svi='sudo vi' ## ## give me more information ... but only when I want it ## alias lls='ls -lAh --author --color=auto' ## ## show me what's open in a pinch ## alias ports='netstat -tulanp' ## ## edit the hosts file ## alias hosts='sudo vi /etc/hosts'
List all running jobs:
:~$ jobs
List process ID (PID) of all running jobs:
:~$ jobs -p
Kill all running background jobs:
:~$ sudo kill -9 $(jobs -p) ## ## or ## :~$ sudo kill `jobs -p`
Useful network connection Commands
lab46:~ ifconfig
lab46:~ iwlist scan
lab46:~ lshw -C network
lab46:~ lspci -nn
lab46:~ lsusb
lab46:~ lshw -C usb
lab46:~ cat /etc/modprobe.d/blacklist
lab46:~ lsmod
lab46:~ route -n
lab46:~ sudo route add default gw 192.168.1.1
lab46:~ sudo route del default gw 192.168.1.1
lab46:~ sudo modprobe ***** Example usage: lab46:~ sudo modprobe ndiswrapper lab46:~ sudo modprobe r818x lab46:~ sudo modprobe ath_pci
lab46:~ sudo modprobe -r **** Example usage: lab46:~ sudo modprobe -r ndiswrapper
lab46:~ sudo ifup eth0 lab46:~ sudo ifdown eth0
lab46:~ sudo ifconfig eth0 up lab46:~ sudo ifconfig eth0 down
lab46:~ sudo dhclient
lab46:~ sudo dhclient -r
lab46:~ sudo iptables -L
lab46:~ dmesg | more
lab46:~ uname -r
lab46:~ /etc/iftab
lab46:~ cat /etc/resolv.conf
lab46:~ cat /etc/dhcp3/dhclient.conf
While installing a bot on Lab46, I found that I was going to have to kill the bot's processes (over and over again). I learned that a simple ps
command would lead me to the information I needed.
:~$ ps
The ps
command retrieves a list of all the processes currently running under your account. To find all the processes on the system, use the aux
option:
:~$ ps aux
With list in hand (or in terminal) I picked out the process ID (PID) of my unsuspecting bot and pulled the trigger:
:~$ kill <pid>
A few tricks in screen
that make it easier to debug applications. When using multiple screen processes, it's not a bad idea to name each processes. Do this with the following line:
# ...replace <name> with ... well ... a name :~$ screen -S <name>
The above command opens a new screen session containing a second bash shell. Now, with multiple screens open, after you detach using ctrl + a + d
you can list your open screen sessions with the following command:
:~$ screen -ls
Reattach as you normally would, except now give screen the name of the session you seek:
:~$ screen -r <name>
Handy.
After users have been added, it's helpful to give some users a little extra access. This can be done with sudo. To give a user sudo access, edit the sudoers file with the command visudo. visudo is a special utility that allows root to manage the sudoers file with a little bit of back-end syntax checking to make sure that you don't make a mistake. (visudo's default editor is nano, though it can be changed to us vi … as its name would imply.)
:~$ visudo
Adding the following line to the sudoers file will give a user root privileges, except that they cannot change the root password.
user ALL = (ALL) ALL, !/usr/bin/passwd root
Adding users requires the following command: (again with sudo)
:~$ sudo useradd -gadm -gunix -pxxxx -d/home/$USER -m $USER
(where $USER is your desired username)
The above command adds a user to the groups (-g) adm and unix. There is a difference between capital G and lowercase g, as the capital G will not actually add you to the desired group. However, you can manually add yourself to groups by directly editing the file etc/group. However, since most users don't have root access, this again requires sudo:
:~$ sudo vi etc/group
Once open, find the adm
and unix
entries and add your new user's name to the end of the list. (no points for being first)
Using the -p option of the useradd command doesn't encrypt your password in the etc/shadow file. Additionally, if the password is not encrypted, the system doesn't like to let you authenticate.
To see what I mean, issue the following command: (you may want to pipe this to less)
:~$ sudo cat etc/shadow | less
Scroll through the file and see that the password you entered with the -p option of the useradd command is there in plain text for all to see. Whoops. Now, attempt to authenticate with the following command:
:~$ su $USER
You will be prompted for your password. No dice huh? To fix this, as ROOT, issue the following command:
:~$ sudo passwd $USER
Again, substitute $USER with your new user's name. You will then be prompted for the new password, and, once complete, the NEW password will be hashed.
If you've ever locked yourself out of a box, you've probably used single user mode to recover the password. As easy as this is with bare metal, it is less so inside a VMWare virtual machine … particularly if you happen to be running Ubuntu server which speeds happily past the boot menu upon startup. This trick is helpful:
## ## edit the .vmx file for your VM adding the following ## :~$ vi /vmdk/volumes/datastore1/<vm folder>/<vm name>.vmx ## ## add this line to slow down the boot to give you time to hit F2 or whatever needs hitting ## bios.bootDelay = "15000"
Now, hit your F2 and select recovery mode. From here, mount the drive in read/write mode. (this works in all Linux recovery mode situations)
## ## remount the drive in read/write mode ## :~$ mount -o remount,rw / ## ## you are now root ##
You need to enable ssh before using the CLI within ESXi. Currently, the only way I know of doing this remotely is throught the VMWare vSphere client (Windows/Mac OS X). (See vim-cmd hostsvc disable_ssh for a method of turning this off remotely)
From the client: click the host → click the configuration tab → click security profile → under services click properties → highlight ssh → click options → click start.
You can now ssh into the ESXi host.
Begin by listing the running VM processes
:~$ esxcli vm process list ## ## Likely output ... shows 2 VMs running ... note the World ID ## ubuntu-vm World ID: 1361123 Process ID: 0 VMX Cartel ID: 1361122 UUID: 56 4d 20 bb dd 70 c4 80-e0 c6 f7 a6 ee 51 ee 25 Display Name: ubuntu-vm Config File: /vmfs/volumes/509298eb-d48bf52f-d03d-842b2b5c1eb7/ubuntu-vm/ubuntu-vm.vmx debian-vm World ID: 1341691 Process ID: 0 VMX Cartel ID: 1341690 UUID: 56 4d 18 73 98 73 59 c3-3e 71 44 ff 15 58 17 1e Display Name: debian-vm Config File: /vmfs/volumes/509298eb-d48bf52f-d03d-842b2b5c1eb7/debian-vm/debian-vm.vmx
Next, use vim-cmd
to restart any given VM using the path to its .vmx file. (This can also use the VM's world ID)
:~$ vim-cmd vmsvc/power.reboot /vmfs/volumes/datastore1/ubuntu-vm/ubuntu-vm.vmx
:~$ vmkfstools -i /vmfs/old/path.vmdk -d thin /vmfs/new/path.vmdk
Next, to register the VM in the datastore, first copy the .vmx file into the newly created VM directory.
## ## copy the .vmx file ## :~$ cp /vmfs/old/path.vmx /vmfs/new/path.vmx ## ## register the VM ## :~$ vim-cmd solo/registervm /vmfs/new/path.vmx
Given a vm named slugworth, and a missing .vmdk file or corrupted … here's how to regenerate:
First, identify the size of the file slugworth-flat.vmdk
## ## navigate to the directory of ... ## :~$ cd /vmfs/volumes/datastore1/slugworth ## ## determine its size ## :~$ ls -l -rw------- 1 root root 17179869184 Feb 4 14:10 slugworth-flat.vmdk <---- FROM THIS LINE ... 17179869184 -rw------- 1 root root 8684 Feb 4 14:10 slugworth.nvram -rw------- 1 root root 493 Feb 4 14:09 slugworth.vmdk -rw-r--r-- 1 root root 0 Nov 1 19:42 slugworth.vmsd -rwxr-xr-x 1 root root 2746 Feb 4 14:10 slugworth.vmx -rw-r--r-- 1 root root 3423 Nov 5 19:13 slugworth.vmxf -rw-r--r-- 1 root root 124730 Jan 4 13:12 vmware.log
Identify the type of SCSI controller the virtual disk is using by opening the .vmx file …
## ## determine the SCSI controller ... vitrualDev ## :~$ cat slugworth.vmx | grep -i scsi scsi0.present = "TRUE" scsi0.sharedBus = "none" scsi0.virtualDev = "lsilogic" <---- THIS LINE scsi0:0.present = "TRUE" scsi0:0.fileName = "ubuntu.vmdk" scsi0:0.deviceType = "scsi-hardDisk" scsi0.pciSlotNumber = "16" scsi0:0.redo = ""
Use the vmkfstools command to create a new virtual disk:
## ## create a new disk ## :~$ vmkfstools -c 17179869184 -a lsilogic -d thin temp.vmdk ## ## -c size ## ## This is the size of the virtual disk. ## ## -a virtual_controller ## ## Whether the virtual disk was configured to work with BusLogic, LSILogic (for both lsilogic and lsilogic SAS) or IDE. ## ## -d thin ## ## This creates the disk in thin-provisioned format. ##
Delete the un-needed temporary flat.vmdk
:~$ rm -rf temp-flat.vmdk
Edit the new temp.vmdk and change the name of the .flat file to match the orphaned .flat file, and find and remove the line ddb.thinProvisioned = “1” if the original .vmdk was not a thin disk. If it was, retain this line.
:~$ vi temp.vmdk # Disk DescriptorFile version=1 CID=fb183c20 parentCID=ffffffff createType="vmfs" # Extent description RW 8388608 VMFS "temp-flat.vmdk" <--- CHANGE THIS LINE # Extent description RW 8388608 VMFS "slugworth-flat.vmdk" <--- TO THIS LINE # The Disk Data Base #DDB ddb.virtualHWVersion = "4" ddb.geometry.cylinders = "522" ddb.geometry.heads = "255" ddb.geometry.sectors = "63" ddb.adapterType = "lsilogic" ddb.thinProvisioned = "1" <---- REMOVE THIS LINE IF NOT THIN PROVISIONED
Optionally, check the disk consistency
## ## check disk consistency ... esxi 5.0 or later ## :~$ vmkfstools -e filename.vmdk Disk chain is consistent ## ## check disk consistency ... esxi 5.0 or earlier ## :~$ vmkfstools -q test.vmdk test.vmdk is not an rdm
Now start the VM.
Instructions taken from: Recreating a missing virtual machine disk (VMDK) descriptor file
## ## Options available under vim-cmd ## hbrsvc/ internalsvc/ solo/ vmsvc/ hostsvc/ proxysvc/ vimsvc/ help
## ## other options available under vim-cmd vmsvc/ ## acquiremksticket get.snapshotinfo acquireticket get.spaceNeededForConsolidation connect get.summary convert.toTemplate get.tasklist convert.toVm getallvms createdummyvm gethostconstraints destroy login device.connection logout device.connusbdev message device.disconnusbdev power.getstate device.diskadd power.hibernate device.diskaddexisting power.off device.diskremove power.on device.getdevices power.reboot device.toolsSyncSet power.reset device.vmiadd power.shutdown device.vmiremove power.suspend devices.createnic power.suspendResume disconnect queryftcompat get.capability reload get.config setscreenres get.config.cpuidmask snapshot.create get.configoption snapshot.dumpoption get.datastores snapshot.get get.disabledmethods snapshot.remove get.environment snapshot.removeall get.filelayout snapshot.revert get.filelayoutex snapshot.setoption get.guest tools.cancelinstall get.guestheartbeatStatus tools.install get.managedentitystatus tools.upgrade get.networks unregister get.runtime upgrade
## ## Options available under vim-cmd solo/ ## connect environment logout querycfgoptdesc disconnect login querycfgopt registervm
## ## Options available under vim-cmd hbrsvc/ ## vmreplica.abort vmreplica.pause vmreplica.create vmreplica.queryReplicationState vmreplica.disable vmreplica.reconfig vmreplica.diskDisable vmreplica.resume vmreplica.diskEnable vmreplica.startOfflineInstance vmreplica.enable vmreplica.stopOfflineInstance vmreplica.getConfig vmreplica.sync vmreplica.getState
## ## Options available under vim-cmd hostsvc/ ## advopt/ enable_ssh refresh_services autostartmanager/ firewall_disable_ruleset reset_service datastore/ firewall_enable_ruleset runtimeinfo datastorebrowser/ get_service_status set_hostid firmware/ hostconfig standby_mode_enter net/ hosthardware standby_mode_exit rsrc/ hostsummary start_esx_shell storage/ login start_service summary/ logout start_ssh vmotion/ maintenance_mode_enter stop_esx_shell connect maintenance_mode_exit stop_service cpuinfo pci_add stop_ssh disable_esx_shell pci_remove task_list disable_ssh queryconnectioninfo updateSSLThumbprintsInfo disconnect querydisabledmethods enable_esx_shell refresh_firewall
## ## Options available under vim-cmd internalsvc/ ## perfcount/ host_mode_lock refresh set_log_level vprobes/ login refresh_consolenic shutdown access_address loglist refresh_datastores throw_exception cold_quit logout refresh_gateway use_fds connect redirect_stderr refresh_network disconnect redirect_stdout refresh_pnic
## ## Options available under vim-cmd proxysvc/ ## add_np_service disconnect port_info add_tcp_service login remove_service connect logout service_list
## ## Options available under vim-cmd vimsvc/ ## auth/ license property_dump task_info connect login task_cancel task_list disconnect logout task_description