This is a sample format for a dated entry. Please substitute the actual date for “Month Day, Year”, and duplicate the level 4 heading to make additional entries.

As an aid, feel free to use the following questions to help you generate content for your entries:

• What action or concept of significance, as related to the course, did you experience on this date?
• Why was this significant?
• What concepts are you dealing with that may not make perfect sense?
• What challenges are you facing with respect to the course?

This week we continued to become familiar with the use of the UNIX commands and how to access and edit text files containing html and php content. We also dabbled with outputting php to a webstream written within the html markup.

ls – lists files
pwd – print working directory, gives absolute path
mkdir php – makes a dir called php
cd public_html – changes dir to public_html
cp – copy files
mv – moves files
rm – removes files
man ls – shows online manual for ls

Pod colors:

• white – txt files
• red – old
• green – can run exec

Text Editor: nano

To open a file and edit: nano test.html

PHP:

Text files containing php have to be saved with the .php extension
Echo outputs to webstream
\ escapes quotes
; necessary statement ender 

Bacon salt is a thing!

This week we learned how to access Firefox from the terminal with a super easy command, firefox &. We also started loops, more specifically loops within html tables. The if statement is a condition that resolves to true or false. The numeric for, while and do while loops are condition-based, but the list-based for loop is not. It still has stuff to go through.

ctrl K – cuts current line and stores in buffer
ctrl U – uncut from cutbuffer into current line
ctrl C – displays position of cursor
ctrl A – moves to beginning of current line
ctrl E – move to end of current line

Produce a checkerboard table using a combination of PHP, HTML and CSS.

Attempt 1:

Generated a 10px border around a 480px high and wide table. The two for loops added 8 red rows and 8 black rows running side-by-side and half the length of the table. Not exactly sure why that happened but at least there was output.

Multiple Attempts:

Generated a 10px border around a 480px high and wide table with a black square in the bottom right hand corner. I used two for loops to output the table row and table columns. I then added an if, else statement to add the red and black backgrounds. It was supposed to print a checkerboard but only printed a single black square in the bottom right hand corner of the table.

Another Attempt:

Did some research and found that you need to add the rows and columns before you use an if statement to output them in checkerboard fashion.

for ($i = 1; $i <= 8; $i ++)
 for ($j = 1; $j<=8; $j ++)
$rowcol=$i + $j;

This week we ventured into the world of arrays and functions. An array is a container. It stores multiple values in one variable. The values are accessed using an index number. In PHP, arrays have value to key associations.

There are three different types of arrays…

Multidimensional →contains one or more arrays

Indexed →has a numeric index

Syntax:

$item [0] = “mango”;
$item [1] = “banana”;
$item [2] = “apple”;

Associative →has named keys

Syntax:

<?php
$array(
key  => value,
key  => value2,
key  => value3,
… 
);  
?>

The key can be an integer or a string and one array can contain both at the same time. The value can be of any type.

Always use quotes around a string array index. e.g. $item[‘papaya’] Do not put quotes around keys that are constants or variables.

Syntax:

Function functionName( optional: parameters/arguments ) {
 code to be executed;
}

Function names can start with letters or an underscore but NOT a number. Information is passed to the function through comma-separated arguments.

Draw a picture using PHP. See php.net’s image processing and GD documentation for help.

Using the example code done in class (function.php) I attempted to create an abstract image using rectangles and ellipses. The xy coordinates were a little baffling at first but once I confronted the documentation things seemed to make some sense. I was able to create several rectangles of varying widths and lengths and even executed a transparent ellipse using imageallocatealpha over the center of the design.

Pro tip:

• x1 is the upper left x coordinate
• 1 is the upper left y coordinate
• 0,0 is the top left corner of the image
• x2 is the bottom right x coordinate
• y2 is the bottom right y coordinate

Next attempt might be an homage to the square.

This week’s entry, which includes the week before break (Feb 8 – 12) we explored the vexing world of graphs. The first attempt started before break and was hampered possibly by pre-breakitis, or maybe it had something to do with arrays or lack thereof either-way it didn’t have the finesse that this weeks graph code showed.

The graph starts out with a call to the header(), which outputs an image in the form of a png file.

header("Content-type: image/png");

The variable $img stores the container parameters for the graph using php’s imagecreatetruecolor. The container is 800 x 100.

$img = imagecreatetruecolor(800, 100); 

Next, we mixed some colors in our virtual palette using imagecolorallocate, which allocates a color for an image. It must be called in order to create each color used in the image represented by $img. The first call to imagecolorallocate() fills the background color in images created by imagecreate().

Because we can…we shall Foof. It. Up.

$fuschia = imagecolorallocate($img, 0xff, 0x00, 0xff);

The imageline() function does exactly what you think, it draws a line between two given points. Here we established our x-axis and y-axis lines in white on the graph.

imageline($img,   0,  0,  99,  $white);  // y-axis line

The fopen() function in php binds a named resource, specified by a filename, to a stream.

$fh = fopen("/home/username/public_html/php/data.txt", "r");

Here we are opening a stream to our data.txt file that contains a list of numbers used as plot points on our graph. The “r” is a mode parameter that specifies the type of access set to the stream. The “r” tells fopen to open the file with read only access.

Now we tackle the for loop(s).

First, we assign a starting point for each axis.

$x = 10;
$y = 97;

Next we use fscanf() to parse input from the $fh (data.txt) file and interprets the input according to the specified format. A type specifier determines how the argument data is treated. The %d means that the argument is treated as an integer, and presented as a decimal number.

fsanf($fh, "%d", $pt);
imageline($img, $x, $y, $x + 98, 100 - $pt, $blue);
$x = $x + 98;
$y = 100 - $pt;

This loop was created for each of the four plotted lines on the graph.

The second attempt at the vexing world of graphs yielded a cleaner, more efficient piece of code. Arrays were not fought but embraced and implemented within a nested for loop.

A color array was created ($color = array();) and added to our virtual palette.

$color[0] = $blue = imagecolorallocate($img, 0x00, 0x00, 0xff);

The first loop looped through the data series or the four rows and eight columns of numbers in our datafile.txt file.

for($i = 0; $i < 4; $i++)
{
   $dataline = fscan($fh, "%d %d %d %d %d %d %d %d");
}

The nested for loop plots the lines on the graph using imageline and calls our color array() to plot four lines on our graph.

for($x =0; $x < 8; $x++)  // loops each line
{
imageline($img, $px, $y, 100 * $x + 10, 99 -$dataline[$x], $color[$i]);
$px = 100 * $x + 10;
$y = 99 - $dataline[$x];
}

Pro tip – Don’t forget to close your file. fclose($fh);

In observance of today being a leap day (or intercalary day) here’s a fun fact that may prove useful reference in another four years.

Fact: The point of a leap year is to help adjust the Gregorian calendar to the solar calendar. A leap second adjustment is occasionally applied to Coordinated Universal Time to compensate for the fact that adding an extra day every four years doesn’t keep the two calendars aligned.

If the last two digits of the year are divisible by four then it’s a leap year. Century years are the exception to this rule. They must be divisible by 400 to be a leap year. http://www.rd.com/culture/leap-year-facts/

Enough diversions, let's continue with more PHP fun.

This week we started exploring how to use the XML (eXtensible Markup Language) parser. XML is a data format used to store and transport data on the Web. We grabbed an XML parser from the web and used it in our code to pull data from a url on the corning-cc.edu website. The variable $url = “……”; pulls the data from the page and the include(“xml2array.php”) references the parser whose contents are stored in a separate file.

Using variables, we defined the information we wanted to extract and then echoed and printed that information to the web using php.

Challenge: Star fun

Thrown out to us a while ago was a challenge to create a star using PHP. I started tinkering with this shortly after the challenge was laid down and have managed to create a five-pointed star. The bottom two points need some tweaking. Currently they expand to the bottom corners of the container, which makes the lower half of the star look disproportionate to the upper three points. I think moving both points in towards the center will help to create a better balance. Note: make these changes!

Using the imagecreatetruecolor function I created a 400 x 400 container to hold the star.

$img = imagecreatetruecolor(400, 400);

Then set the colors for each line to a different hue using imagecolorallocate.

$blue = imagecolorallocate($img, 0, 255, 153);

I did this so that as I was creating the image I would be able to easily match each line in the code to a line in the image of the star. I found keeping them all one color and bouncing between the image and the code sometimes maddening. Cutting, buffering and pasting multiple versions of the same line at once probably contributed to the confusion. To create the star I used imageline and started plotting the points by referencing the x, y coordinates of the four corners on the container. For reference the coordinates are as follows:

• Upper left: 0, 0
• Upper right: 400, 0
• Lower left: 0, 400
• Lower right: 400, 400

I also stumbled upon a line(x1, y1, x2, y2) processor from the KhanAcademy that I found useful in tinkering or fine tuning some lines.

https://www.khanacademy.org/computer-programming/linex1-y1-x2-y2-processingjs/827916099

At this point, I want to explore thickening the line weight and maybe playing with the color(s). Note: make these changes!

This week's exploration into PHP landed us in the world of drawing apparatus'. We cooked up a clever html/php concoction that allowed the user to choose certain specs from a given selection. Based on those chosen specs the output is displayed to the user in a web page.

Using html we created a table within a submission form. On submit the form data is sent to a file called draw.php using the post method. The draw.php file processes the data from the drawerer.html file and displays the output to the user in a png image. It does this by storing all the “posted” values into an associative array called $_POST.

$width = $_POST["width"];

A post request never caches, doesn't remain in the browser history, has no restrictions on data length and cannot be bookmarked. Hitting the back button requires re-submitting the information typically after alerting the user about the data re-submission.

The html input options for the user explored the use of radio buttons, check boxes and submit buttons. Using the “checked” option at the end of the input type allows the checkbox or radio button to automatically load with the specified value selected.

<input type="radio" name="bcolor" value="teal" checked /> Teal
<input type="checkbox" name="fill" value="fill" checked /> 

The draw.php file starts out with a call to the header(), which outputs an image in the form of a png file.

header("Content-type: image/png");

$_POST is an associative array of variables passed to the current script. The isset() function checks to see whether a variable is set and is not null. If a variable is already unset with the unset() function, it will no longer be set and isset() will return FALSE if the testing variable contains a NULL value.

if  (isset($_POST[“fill”]))
$fill  =  $_POST[“fill”];
else
$fill  = “no”;

Next, we mix up some colors in our virtual palette using the imagecolorallocate () function and then we start running through our loops to check which values the user selected. Parameters are passed depending on which selections were chosen.

This week we tinkered with various things and added to our collection of example codes. Using a native php extension called Imagick we created and modified an image. The ImageMagick API is extensive and capable of creating, editing, and composing bitmap images as well as reading, converting, and writing images in over 100 formats.

The magick.php file holds the all the image trickery discussed in the following ramblings. The first few lines of our code after the header creates a thumbnail by “newing up” the Imagick object.

$image = new Imagick('mudkip.jpg');
$image -> thumbnailImage(320, null);

The next little ditty of code creates a reflection of the mudkip by flipping it and overlaying a gradient on top of the image. Then both images (the original and reflection) are overlayed onto a canvas.

This line of code creates a white border around the image.

$img -> borderImage(new ImagickPixel("white"), 5, 5);

It might be beneficial at this point to stop and attempt to explain the arrow in the line of code above. It's called an arrow operator (→) and is used in object scope to access the members and properties of an object directly from its address. The content on the right of the operator is a member of the “newed up” or instantiated object passed to the variable on the left side of the arrow operator. A more familiar approach to this is to use a dot (“.”) operator to access the methods and properties of instantiated objects. In php the dot operator is used to concatenate strings.

The next step is to clone the image and flip it. This is accomplished in the next two lines using the clone() and flipImage() functions. The clone() function creates an object copy by copying all of the object’s properties. The flipImage() function creates a vertical mirror image of the object by reflecting the pixels around the center x-axis.

$reflection = $img -> clone();
$reflection -> flipImage();

It is now time to create the gradient that will be overlayed on the reflection

$gradient = new Imagick();

Creates a gradient large enough to encompass the image and the borders.

$gradient -> newPseudoImage($reflection -> getImageWidth() + 10, $reflection -> getImageHeight() + 10,   
"gradient:transparent-black");

Composite the gradient on the reflection by combining the two images.

$reflection -> compositeImage($gradient, imagick::COMPOSITE_OVER, 0, 0);

Here’s another foreign symbol worth mentioning (“::”). The double colon is a scope resolution operator that allows access to static, constant, and overridden properties or methods of a class.

Add some opacity to the image.

$reflection -> setImageOpacity( 0.3 );

Next we create an empty canvas large enough to hold both images and set its format to a png.

$canvas = new Imagick();
$width = $img -> getImageWidth() + 40;
$height = ($img -> getImageHeight() * 2) + 30;

$canvas -> newImage($width, $height, new ImagickPixel("black"));
$canvas -> setImageFormat("png");

Combine the original image and the reflection onto the canvas.

$canvas -> compositeImage($img, imagick::COMPOSITE_OVER, 20, 10);
$canvas -> compositeImage($reflection, imagick::COMPOSITE_OVER, 20, $img-> getImageHeight() + 10);

The last two lines output the image.

header("Content-Type: image/png");
echo $canvas;

This week we welcomed our first official project: Monte Carlo Pi. The concept started as an homage to Pi day and blossomed into a project type thing.

A Little Background Information

The Monte Carlo method is a class of computational algorithms relying on pseudo random samplings to obtain numerical results. The method is useful for obtaining numerical solutions to problems that are too complex to solve otherwise.

We started with a PHP program that calculated Pi by generating pseudo random numbers, which were then counted by how many landed within the circle and how many landed in the square. We then multiplied by four the number of iterations that fell within the circle and divided that outcome by the number of iterations that fell in the square.

The Math Explained

Full disclosure I am “appropriating” this explanation from the interwebs because, well…math.

The area of a circle is equal to:

area = Pi * (radius * radius)

The area of a square is equal to:

area = s₂ or length of one side * length of one side

When you enclose a circle within a square the length of one side of the square is equal to 2 * the radius of the circle.

The area of the square is equal to:

area = (2 * radius) * (2 * radius) = 4 * (radius * radius) 

Dividing the area of the circle by the area of the square yields Pi/4. To get the value of Pi we multiply the area of the circle by four and divide the result by the area of the square surrounding the circle.

http://mytinylab.com/20101020_MonteCarloSimulationToCalculatePiInPHP

The Code

Now for a look at the code that runs all that complicated math.

We start by declaring and defining our variables.

The number of iterations it takes to estimate Pi

$iterations  = 35000;          

The radius of the circle

$radius = 1.00; 

Calculate radius squared

$radiusSquare  = $radius * $radius;            

$count   = 0;
$init	   = 0;

Next up is the for loop.

for ($iters = 0;  $iters < $iterations;  $iters++) 
{
  $x = lcg_value();     -- random x-coordinate 
  $y = lcg_value();     -- random y-coordinate 

The lcg_value() math function in PHP returns a pseudo random float value in the range of (0,1). For both the x and y coordinates a random number from 0 to radius or 1 because we set the value of radius to 1 is chosen.

The coordinates inside the square of the radius

$isin   = ($x * $x) + ($y * $y);    

if ($isin  <= $radiusSquare)      
{
Count the number of points in the circle
  $init = $init + 1;       
 }
}

The number of points inside the square

$pi 	= ($init * 4) / $iterations;    

Originally we output the results to the browser in text format.

# in the square: 35000
# in the circle: 275001
pi = 3.1428685714286

Finding the results visually boring it was decided that we incorporate our results into a visually stimulating graph. Thus a project was crafted. The challenge laid before us was to graphically plot the points of the original text output onto a circle within a square.

Coming off break week, I have nothing exciting to report. We buttoned up our Monte Carlo Pi PHP program before break began, which leaves us open to starting something new this week.

In last week’s OPUS, post I broke down the PHP code used to calculate Monte Carlo Pi and left off with a tantalizing pointer to a project. This week I will break down the code for implementing the Monte Carlo Pi code into a graphical display of goodness involving rectangles, circles, and pixels.

The first step was to figure out how to visually display the graph in a web browser. I started by wrapping everything in HTML tags and then went on to create the graph (rectangle, nested circle, and plotted points) in PHP. It functioned, output a bordered rectangle with a nested circle, and plotted points. I abandoned this method and moved to what is the “final” version of the project after realizing that all the steps could be accomplished using only PHP.

The Code

First up is creating the container for the graph. Leaving an extra 100 pixels on the height allows room to add the string output of the plotted point count at the bottom of the graph.

$img = imagecreatetruecolor(800, 900);

Next step is to mix up our virtual palette of colors. One color example is listed below.

$redish = imagecolorallocate($img, 100, 0, 0);

Now for the graph

Start by creating a rectangle to hold the nested circle and plotted points.

imagefilledrectangle($img, 0, 0, 800, 800);

The allotted x and y coordinates fill the container created above with the exception of 100 pixels at the bottom, which is reserved for output that’ll be explained later.

To nest a circle in the rectangle you can use either the imagefilledellipse() or imagefilledarc() function. I chose the arc function.

imagefilledarc($img, 400, 400, 600, 600, 0, 360, $redish, IMG_ARC_PIE);

imagefilledarc draws a partial arc centered at a specified coordinate with the image. It takes nine parameters listed below.

• $img is the image resource returned by imagecreatetruecolor
• The x-coordinate (400) and y-coordinate (400) of the center
• The width (600) and height (600) of the arc
• The starting (0) and ending angle (360), in degrees of the arc
• A color identifier chosen from our virtual palette ($redish)
• A style (IMG_ARC_PIE), which produces a rounded edge

After the creation of the rectangle and nested circle add the Monte Carlo Pi function explained last week with a few modifications.

The for loop was modified to reflect the multiplication of the rectangle size by the lcg_value().

$x = lcg_value();     *  800;
$y = lcg_value();     *  800;

After the closing of the if statement, but before the closing of the for loop insert the function to display the plotted points using pixels. imagesetPixel draws a pixel at the specified coordinate using an image resource parameter , x-coordinate and y-coordinate and color identifier.

imagesetPixel($img, $x, $y, purple);

The Monte Carlo Pi function now displays in a graphical format. To add the original output text to the bottom of the graph use the imagestring() function. This function draws a horizontal string at given coordinates.

It takes six parameters listed below.

• $img is the image resource returned by imagecreatetruecolor
• Font uses a number between 1-5 for built-in fonts in latin2 encoding
• The x-coordinate of the upper left corner
• The y-coordinate of the upper left corner
• The width (600) and height (600) of the arc
• The string to be written to the specified coordinates
• A color identifier chosen from our virtual palette ($green)

Example:

imagestring($img, 3, 75, 820, “# in the square: $iterations”,  $green); 
imagestring($img, 3, 75, 840, “# in the circle: $init”,  $green); 
imagestring($img, 3, 75, 860, “pi =  $pi”,  $green); 

This concludes the breakdown of the code for Monte Carlo Pi.

Bonus post.

Our next project is Show and Tell. Unfortunately, it does not involve exploding volcano’s, slimy frogs, or baking soda and vinegar rockets. However, it does allow for some creative thinking in PHP. The objective is to pick something interesting in relation to PHP and implement/explore it to gain a deeper understanding of it’s functionality. The final presentation is in a web-accessible format for all to view.

Some ideas I am interested in exploring.

• Create a PHP photo gallery using the exif function
• Databases and PHP
• Create a PHP/MySQL powered forum from scratch
• Management Systems

Not sure what direction I want to take at this point.

For the Show and Tell project I chose to explore photo galleries using PHP. In my explorations I found several ways to implement a photo gallery using strictly PHP, PHP and jQuery, PHP and SQL, and PHP and Ajax. In addition, galleries have multiple storage options. Some galleries are created and stored in databases using MySQL and other galleries are created and stored in folders on the web server. I found a really old but very informative tutorial on creating an automated PHP gallery system using PHP, MySQL, GD and ImageMagick on the SitePoint website. It walks you through the process of setting up the database schema and building a browser-based uploading system, which sets up an index within the tables, stores the files, and automatically builds thumbnails.

Ultimately, I settled on something a little less complicated for a first timer. I chose to create a photo gallery using strictly PHP. I am working through the code to display the images as thumbnails until clicked where they are displayed full size. I need to upload some images to the server to test the code. I anticipate a good amount of troubleshooting to get everything processing correctly. Also, I need to figure out how to create the thumbnails. I have seen code that automatically does this process, which I’ll have to explore further. If I can get the initial photo gallery working I’d like to attempt to add a watermark feature to each image, but that is not a critical component at this time.

The end of the semester is fast approaching as noted by the influx of final papers/projects being deployed. In fact, our very own EoCE has just deployed. Before delving to depths of the abyss with this experience I will take one last post to explain the code associated with the Show and Tell project that I am working on.

First up is the variable declaration. These variables are used to declare the columns displayed, location of the thumbnails, and location of the full size images used in the gallery.

$images = "image_gallery/";  // Location of thumbnails
$big    = "big/";  // Location of full size images
$cols   = 2;  // Number of displayed columns 

$images builds the contact sheet or “thumbnails”.

Next step is to open the directory containing the images and loop through it to add the image files to an array. The first line of the if statement opens the directory where all of the images are stored. The while condition loops over the directory and adds every file it finds to an array called $files[]. The readdir function reads entries from the directory handle and returns the name of the next entry in the directory. Entries are returned in the order they are stored by the filesystem. Any directory entry name that evaluates to FALSE will stop the loop. The nested if statement prevents the . (the current directory), .. (the directory one level up), and the $big directory from being added to the array. If displayed in the browser, they would show as broken links. Any file in the $images directory is added to the array and rendered as an image so be careful what you place here. The closedir function closes the directory handle opened by the opendir function.

if ($handle = opendir($images)) { 
  while (false !== ($file = readdir($handle))) { 
     if ($file != "." && $file != ".." && $file !=       rtrim($big,"/")) { 
         $files[] = $file; 
     } 
 } 
 closedir($handle); 
}

Now it’s time to output the table that displays the images. The goal here is to display the thumbnails as a contact sheet while linking each to its full size image. The first echo statement outputs the first part of the table. The first line in the for loop tests whether $colCtr (initialized to 0 before the loop) is of equal value to $col (number of columns to display). If yes, the table row closes, a new row is opened and an hr line break is used to separate the rows. The next echo statement creates a link to the full size images. The output is an a href attribute that specifies the link’s destination. In this case the link is built using the variables defined earlier in the code, $image, $big, and $file. The img src attribute outputs the image using $image. $file. The last step before closing the table is to increment the $colCtr variable to output the number of columns needed for displaying the images.

$colCtr = 0; 

echo '<table width="100%" cellspacing="3"><tr>'; 

foreach($files as $file) 
{ 
if($colCtr %$cols == 0) 
  echo '</tr><tr><td colspan="2"><hr /></td></tr><tr>'; 
echo '<td align="center"><a href="' . $images . $big .   $file . '">
<img src="' . $images . $file . '" /></a></td>'; 
  $colCtr++; 
} 

echo '</table>' . "\r\n";

In comparing the process of creating a basic photo gallery using PHP and HTML/CSS it appears PHP has the advantage. The code is more streamlined meaning it uses far less steps to accomplish the same end result.

The countdown to the end of the semester is approaching warp speed. The end is nigh. It’s final crunch time for end of semester presentations and projects, which some are scrambling to complete. I am slowly whittling away at the EoCE. I have started 0x0, draw a picture reflecting your interpretation of “April showers bring May flowers.” Conceptually, I have an image floating around in my head and to date the “creation” has closely represented said image. The only constraints I foresee with this task is my indecisive nature steering me off the plank and into the abyss.

I have discovered that the imagefilledpolygon function in PHP has the capability to draw a triangle using an array to define its points. My initial use for the triangle was to add texture to the grass section of the image, but after seeing the output I have discovered other uses for the imagefilledpolygon. Immediately after seeing the sharply pointed, pink object appear on the screen I wanted to use it to represent the sun’s rays. The challenge is figuring out how to wrap it around the outside edge of the imagefilledarc. This is something I need to explore as it may be beyond my skill level.

I have also started thinking about the binary table as I feel this may cause me to bang my head against a wall. I am exploring methods and ideas but haven’t settled on something solid yet.

This entry marks the last week of classes for the semester and therefore the last blog post for PHP. It’s been a semester full of fun and exciting new explorations. I’ve learned that PHP is not just for querying a database, drawing pictures is an excellent way to learn a new programming language, and there is a whole world of memes that lure you into a time suck.

Still working on the EoCE. Progress has slowed as I needed to focus on other courses, but with most other projects behind me I can return to drawing pictures, creating meme’s and binary tables.

This is a sample format for a dated entry. Please substitute the actual date for “Month Day, Year”, and duplicate the level 4 heading to make additional entries.

As an aid, feel free to use the following questions to help you generate content for your entries:

• What action or concept of significance, as related to the course, did you experience on this date?
• Why was this significant?
• What concepts are you dealing with that may not make perfect sense?
• What challenges are you facing with respect to the course?

This week we were introduced to the pod and given directions on how to open a terminal session and ssh into Lab46. To become familiar with editing our Opus project UXIO had us customize our title and subtitle in addition to adding a brief introductory statement. We also subscribed to the class mailing list and sent a brief introductory message to the class. This task familiarized us with maneuvering within the mail program Alpine. Using the IRC or class chat is still somewhat unclear but the setup went smoothly.

• Screen -r –re-attaches a screen session in IRC chat
  
• Press and hold CTRL while hitting the “a” key. Release both and hit the “d” key –detaches screen session in IRC and returns to terminal session.

Don't forget to read the MOTD!
Check your Lab46 email frequently!

This weeks project (arc0) relates to archive handling. Archives are containers that encapsulate things or files. Compression is an action performed on a single file.

The tar (GNU version) command stores and extracts files from a tape or disk archive. It also combines multiple files into a singe file.

xz is a data compression tool. It compresses or decompresses a file.

gzip compresses or expands files.

zip packages and compresses (archive) files.

tac concatenates and prints files in reverse.

rev reverses lines characterwise

To decompress the .tar.xz file use the tar xf filename command in the directory you want to extract the files to. Use the tac (cat backwards) and rev commands to help unscramble the files.

Mars’ two moons are Phobos and Deimos and are named after Greek mythological characters. The Latin translation of each is fear and panic respectively.

This week we learned about pipes, filters and the file system. Actually we learned about multiple pipes but the one most pertinent to us is the | command. Pipes are extremely useful and powerful. They allow the connection of two or more commands, which allows for the output (stdout) of one element to be the input (stdin) of the next. Each pipe tells the system to connect the standard output of the left element to the standard input of the right program. Pipes are unidirectional, meaning that data flows through the pipe from left to right.

Command Expansions: back ticks/back quotes

• ' full quote –literal
• “ half quote –allows for expansion, a grouping
• ` back quote –enables command expansion

The File System:

Dir –points to many files. Home dir mounted from the file server.

1. / or root has grand cosmic powers. Equivalent to admin in Windows
2. home
3. username

. shows current location; doesn’t change
.. parent goes up in dir; takes you to home

Project Notes

This week’s project is the legendary puzzle box (pbx0)! As soon as I solve it, I will update this section with my notes.

Fun with Vi! Enlightenment has finally come albeit with a learning curve. Let’s begin our journey into the key elements of this “cursor less” world.

To enter this world type ‘vi filename’. The screen before you is command mode where letter keys are individual commands. The second mode is insert where most of your text input happens. To enter insert from command hit the lowercase “i” key, to exit insert and enter command hit the “esc” key.

Commands follow a general pattern: [cmd] [number] textobject. Lowercase commands define a word as a contiguous sequence of digits, and underscores. Uppercase commands define a word as a contiguous sequence of non-whitespace characters. Commands can be prefixed with a number, e.g. 24h goes left 24 characters.

There are many commands in Vi that edit, navigate, and manipulate the text file. We will explore them next time.

Pro tip: Do not use cursor keys in classic Vi.

Project Notes

This week’s project is the follow up to the legendary puzzlebox (pbx0) appropriately titled puzzlebox next (pbx1). I am currently hacking my way through this labyrinth. The first challenge was opening the zipped pbx1 file, which was easily accomplished with the unzip tool. This inflated a README file and prompted for a password to an enigmatic.file file. Not having a password to open said file I sheepishly hit the enter key, which skipped the enigmatic file due to incorrect password and inflated an octal2ascii.c file. Deciding it was best to actually read the README I opened it with a text editor after establishing it was an ASCII text file. The file contained the password to the enigmatic file along with a sprinkling of hints and insights to help unravel the puzzle. With the newly acquired information I ran the unzip tool on the pbx1 file again and when prompted input the four character password. This inflated enigmatic but prompted for a password to part1, not having one I hit enter again, and watched as it skipped parts 1-4 due to an incorrect password. Running file on enigmatic revealed it was a uuencoded or xxencoded, ASCII text file. To decode it I ran uudecode and found a new file titled stage2 in my directory. This file is a regular file without any read permissions. To change the file permissions use chmod 640 or 440. After changing the permissions, stage2 is listed as an ASCII text file with no line terminators. Opening the file reveals a single line of two digit numbers and characters. To be continued…

The octal2ascii.c file is c source, ASCII text. Open in a text editor and read the comment section. The c code is a helper program that takes data from STDIN, converts it, and displays it to STDOUT. Follow the directions to compile the code using the correct file names (gcc octal2ascii.c –o octal2ascii). This creates an executable file that can be used like any other tool in your UNIX arsenal. To be continued…

In observance of today being a Leap Day, (or intercalary day) here’s a short fact that may prove useful reference in another four years.

Fun Fact:

February 30th is a thing! In the year 1712 in the countries of Sweden and Finland an extra Leap Day was added to February to align their outdated Julian calendar with the new Gregorian calendar.

Bonus Fun Fact:

February 30th is still celebrated in the mythical world J.R.R. Tolkien created in Lord of the Rings. The fantastical Hobbits observe twelve 30-day months every year, including February. http://www.rd.com/culture/leap-year-facts/

This week we time traveled in shell and learned about the history command. A full featured, useful new command that when typed at the command prompt shows all the commands entered EVER. There were so many delicious new commands that I will give a brief highlight of the more interesting ones.

At prompt:

• Ctrl R –brings up reverse-i-search. Type gcc at prompt to search all
• Ctrl G—beeps the speaker. Useful for annoying your neighbors
• Ctrl K—kills all text from cursor to end
• Ctrl L—refreshes screen , clears garbage
• Ctrl S—generates a transmit of char terminal appears
• Ctrl Q—reanimates a frozen terminal, undoes ctrl S

To access process management type ‘ps’ at the command prompt and all the processes running are displayed to the screen. Task manager in windows is the equivalent.

To run cat or firefox or … in the background type ‘cat &’ at the prompt. To bring it back to the foreground type ‘fg’.

To obliterate PATH:

• echo $PATH displays path * oldpath=$PATH set path to a variable
• echo $PATH displays path * PATH = sets path equal to nothing
• Echo $PATH displays path. Nothing works. * PATH=$oldpath restores path by setting it equal to the variable we created earlier

Project Notes:

Wpa0 or web page adventure is on task for this week’s project. Create an adventure using HTML, file permissions and styling. For the styling in-line or CSS works.

I have started this little ditty and am finding it quite fun to create. Not going to give any spoilers because the project isn’t complete and I’ve hit a creative roadblock. I will update this post as soon as I unblock and finish the adventure.

For those sensitive to the well being of the animal kingdom please refrain from reading the next line. This week we killed cats! Not the fury, hairball coughing kind but the process running kind related to the “cat” command.

Process management or the windows equivalent “task manager” shows every process running on your machine. Every command run has a numeric number or process ID associated with it and when “ps” is typed at the command prompt, a table is displayed listing the various information about each running process.

Killing things is a way of terminating sending signals to processes. The operating system communicates with processes. The default action is to terminate if process does not have a signal.

The command to send a signal to a process is “kill”. The syntax is:

Kill –signal PID

Kill –l shows 64 ways to kill a process. When typed at the command prompt it lists all of the kill commands. Signals are actually not about killing but communicating.

“Killing” cats a brief tutorial.

To “kill” cat.

• Type ‘cat’ in one Lab46 terminal (1)
• Type ‘ps’ in another Lab46 terminal (2)
• Find the process id (PID) number of cat
• In terminal 2 type, “kill -1 ‘cat’s PID’”
• Watch what happens to cat in terminal 1.
• Boom! You just killed a cat!

Useful killing knowledge:

“SIGBUS” A signal is sent to the processor because something is off balance. More specifically, it is when you access mapped memory that doesn’t correspond to a file.

“SIGFPE” is a floating-point exception issued when an illegal math operation is attempted.

“SIGKILL” The ultimate kill. Absolute termination of a process without performing any cleanup operations.

We now transition from the process of killing cats to an undead state. Zombies! Not the supernaturally reanimated ones, these zombies wreak havoc on the process state. A zombie process state occurs when a child process dies before the parent process. Zombies eat random resources, are unpredictable, and when they become dislodged from the operations process can’t be killed. When a child process spawns or forks the process terminates, but creates a return value. If too much time passes before the parent collects its remains, the process becomes a zombie. If this happens try launching a tactical nuke or just use the ultimate kill command, “SIGKILL”.

Kill order:

1. hang-up
2. interrupt
3. hang-up
4. nuke

SIGTERM (kill #15) is the default kill for a process when the command “kill PID” is issued without a kill number following the PID.

Project Notes:

Due to an unforeseen illness, I have zero project notes to date. Will update when completed.

This week's adventure is all about scripting (in bash). Scripts unlock super powers, which is pretty amazing considering they are just files containing ASCII text.

Shell variances:

• Sh—bourne shell
• Bash—bourne again shell

Bash Scripting

Start every bash script with a shabang/shebang, tomehtoe/tomahtoe.

#!/bin/bash
#script2.sh
#

This first line tells the shell what program to use when interpreting the script. The example above uses /bin/bash/. The second line is a comment (#) and ignored by bash since it appears after the # symbol. It is good practice to comment your scripts.

Always end your script with an exit or end. Exit status is a return value issued from a terminating command. The value is an integer between 0 and 255 an indicates the success or failure of the command's execution. A value of 0 is a success and a non-zero is a failure. The true command always executes successfully and the false command always executes unsuccessfully. Set the exit status of your script when it's finished. This causes the script to terminate immediately and sets the exit status to the given value as an argument.

To execute a script, it needs to have the proper file permissions. Quit out of the text editor and at the command prompt type:

Chmod 755 script2.sh

The 755 gives you read, write, and execute permissions.

To run the script, type the following at the command prompt:

./script2.sh

This is a specified path issued on the command line that tells the shell where to find the executable file. To have bash show you the steps it takes when running your script add a ”-x“ to the first line of your script. Bash displays each line as it executes it.

#!/bin/bash -x

Bash has if statements!

The if statement makes a decision based on the exit status of a command. When you have an if statement you must have exactly:

1 if

Syntax:

If  [ “$val” = “hello” ]; then
   commands/do something
fi

0 or 1 else statements (always at the end)

Syntax:

If  [ “$val” = “hello” ]; then
  commands/do something
else
  commands/do something
fi

0 or more elif statements

Syntax:

If  [ “$val” = “hello” ]; then
  commands/do something
elif [  .………….……….  ]; then
  commands/do something
else
  commands/do something
fi

You can also have nested ifs and compound stuff. Compound:

If [ command ] && [ condition 2 ]; then

Command Line Scripting

Anything you type in a command line can be typed in a script. The square brackets command can be used when scripting on the command line.

[ $num -lt $val ]

Quotes make a valid string. Use them around your strings.

[ "$val" = "hello" ] && echo "true" || echo "false"  //Everything is a string.

Protip:

Square brackets don’t like to touch things except the semi-colon.

Binary Control

Binary control or bc is an arbitrary precision calculator language or calculator function run at the command prompt. By default, it is in integer mode. ibase(input base) and obase (output base) are special variables that define the conversion base for input and output numbers. To switch to another base type “obase=2 or ibase=2” at the command prompt. “Bc -l” gives decimals.

Miscellaneous Tips:

• # is the comment symbol for bash scripting
• When scripting in bash assume everything is a string.
• Curly braces isolate variable names. Num6=${num2}1${num4}

Logical operators chain commands together and can be used on the command line. &&–the AND logical operator

Project Notes:

Preliminary investigation yields the following results:

Absolute mindbender.

Endianness and Etymology

The concept of Big-Endian and Little-Endian was first introduced by Danny Cohen in 1980 who derived his byte ordering names from a segment of Johnathan Swift’s “Gulliver’s Travels.”

Endianness Explained

Endianness is the arrangement or storage of bytes (binary data). Big-Endian is where the most significant byte of a multi-byte data field is stored at the lowest memory address. Little-Endian is where the least significant byte of a multi-byte data field is stored at the lowest memory address. Thus creating a Big vs Little feud. Fact: there are 256 unique values in 1 byte.

Big-endian = B0 CF Little-endian = CF B0

Regular Expressions

A regular expression or regex is a sequence of characters that define a search pattern. Most commonly used with strings to perform pattern matching operations. Often considered a jacked up wildcard.

Requirements: Know your data! If you do not know what you want, you have no business poking around the data. Knowing the data allows you to make more intelligent requests.

Regex Symbols

. (dot) 	matches any single character
* 		O or more of the previous; cannot use by itself; pair it with something before it 
[    ]	character class, match any one of the enclosed characters
[^  ]		inverted character class; do not match any one of the enclosed characters
\< 		match start of word
\>		match end of word
^		match start of line
$		match end of line
? 		match the preceding 0 or 1 times. 1 time optionality - It’s either there or not. Pair it.
+ 		1 or more of the previous. Not all tools support this.

The grep command is used to match patterns within a text file.

Example

Wander into the dictionary and start searching.

Find all of the 4 character lines in the dictionary.

cat words | grep  ‘^….$’ | wc -l

Extended Regex Symols

        (   )  	Group together
sed ->  \( \) 	Group together
          | 	        Logical OR

The egrep command is used to match patterns within a text file.

Example

Wander into the dictionary and start searching.

Find all words that start with M followed by ic, ou.

cat words | egrep ‘^m(ou|ic)’ | wc -l

Project Notes:

Preliminary investigation yields the following results:

Absolute mindblender.

It’s break week and I have taken a chunk of time to organize my notes from the last few weeks so this post is going to be a modge podge of things that failed to make it in the previous entries.

Data

All binary is text data, but not all text is binary. Text—vi, cat, head, tail, echo all deal with text. Binary—not encoded in ASCII.

All About the Double 'd'

The 'dd' command is your main tool for the udr0 project and it makes an appearance in udr1 when overlaying the backup onto disk.image. 'dd' (data dump or data duplicator) is cat on steroids. It reads from stdin and writes to stdout unless otherwise specified by the if (input file) and of (output file) options. To use give it a set of flags. A report of the ‘records in’ and ‘records out’ is displayed as an output message after the 'dd' command executes. ‘dd’ likes decimal data (base 10) if not in base 10 convert first.

Syntax of ‘dd’ command:

dd if=<source file name> of=<target file name [Options]

Examples:

dd if=/etc/motd of=newfile bs=512 count=1

The ‘if’ stands for input file, ‘of’ stands for output file, ‘bs’ stands for the block size (number of bytes to be read/write at a time), and ‘count’ refers to the number of input blocks to be copied.

The default value for input and output block sizes is 512 bytes (one sector)

To Use dd as cat:

dd if=/etc/motd of=/dev/tty

To Use dd as copy:

dd if=/etc/motd of=newfile

Project Notes: udr1—Data Recovery

New this year is a mind numbing data recovery experience. This week’s project delves deep into the realm of abstraction and roots around the topic of raw data management and data recovery in relation to hard drives and partition tables.

Useful Information

The Master Boot Record (MBR) is located on the first sector (one sector is equal to 512 bytes) of the hard drive. It contains the Master (or Primary) Partition Table information needed to determine the starting sectors and overall sizes of each partition needing extraction. The primary partition table is 64 bytes of the 512 byte MBR. The Partition Table contains four 16 byte entries and thus can have no more than four Primary partitions.

Restoration:

The dd command has the ability to modify data in a specified place.

Example:

dd if=<source file name> of=<target file name> bs=1 conv=notrunc

This command overwrites the first 512 bytes of a file. The notrunc option means do not truncate the output file. If the target file already exists, replace the specified bytes and do not touch the rest of the target file.

To be continued…

Bonus Post.

Coming off break week, I find myself grappling with the abstract concepts of the udr series of labs. Each progression in the series catapults you further down the rabbit hole. It is clear that my natural sandbox is not in data storage or recovery at the bit level. I am more comfortable dissecting work from Willem de Kooning, Franz Kline or Jackson Pollock during the Abstract Expressionism period.

As I muddle through these labs, I find I am gaining a deeper understanding of how partition tables and data packets work. Two extremely useful commands in the labs’ arsenal are dd and search. These are your friends throughout the duration of this mind numbing, but educational experience. The most difficult concept was finding a starting point. Once found things started to make more sense and flow smoother. If there is a follow-up post then I survived the udr series. Otherwise, I never found my way out of the rabbit hole.

Review week and more bash scripting. We’ve been adding more bash examples to our arsenal of scripting files this week. This should prove useful with our next project on grade calculations. Remember that the proper use of quotes matters and backticks are not single quotes. Single quotes are literal quotes meaning that it searches for what is literally within the ‘quotes.’

Project Notes:

Upr0—UNIX Permission Review

This week’s project schooled procrastinator’s in the art of time management while reviewing file permissions. A concept we explored early in the semester and something we are currently using with bash scripting.

Files in Unix have three permission attributes: owner, group, and other or world. Owner permissions determine what privileges the file owner has on the file. Group permissions determine what privileges a user in a group that a file belongs to has on the file. Other or world permissions determine what privileges all other users have on the file. Each file permission is broken into groups consisting of three characters per group. The three characters are read, write, and execute. The first two are self-explanatory and the execute bit grants the user privileges to run a file as a program. All three are associated with a numerical value—read (4), write (2), execute (1), which is used to change the file permission. A dash (-) in the place of a numerical value represents nothing or (0). Adding up the values in each group results in a value with a range between 0 and 7.

Example:

srwx--x-w- (712)

The ‘s’ is the file type and changes with depending on the file. Watch out for these in the project they can trip you up at first. The ‘rwx’ represents a value of 7 and grants access to all privileges in the owner field. The ‘–x’ grants executable access to members of a group owning the file and ‘-w- ‘gives write privileges only to all other users.

Special file permissions surface about halfway through the project. These permissions allow the user running an executable file to assume the ID of the owner of the file. The setUID permission has a numerical value of 4 and is displayed in owner permissions in the executable field. The setGID has a numerical value of 2 and is displayed in the group permissions in the executable field. The sticky bit has a numerical value of 1 and is displayed in the other or world permissions in the executable field. If an executable bit is set the setUID or setGID represented by an ‘s’ will be lowercase and if no executable fields exists, then the bit is uppercase. The same rule applies to the sticky bit except it is represented by a ‘t’.

Example:

dr-sr-srwt (7557)

The file type field is set to a directory as noted by the 'd' at the beginning of the file permission. The ‘r-s’ represents a value of 5 for the regular permissions and 4 for the special permissions and the ‘rwt’ presents 7 for the regular permissions and 1 for the special permissions. Add up the special permissions and put that value in front of the regular permissions.

The chmod command is used to change the permissions of a file or directory. Specify the numerical permission settings followed by the file or directory name you want to modify.

chmod 6777 file1.txt

This concludes the review session.

For the last two weeks or so we have been exploring the use of regular expressions. Regular expressions are used in conjunction with several different commands such as sed, ed, awk, and grep. We have focused primarily on sed and grep. Here is a brief recap of the commands and their syntax.

The grep command mentioned in an earlier post, but not fully explored searches standard input or text within a file for string patterns and prints any lines matching the specified pattern to stdout (the terminal).

It’s syntax is:

grep [options] PATTERN [FILE…]
grep [options] [-e PATTERN | -f FILE] [FILE…]

The sed command or stream editor is invoked by sending data to it through a pipe. The data sent passes through sed and goes to stdout without changing the input file.

It’s syntax is:

/pattern/action

Where pattern is a regular expression and action is a command given. The slash characters are used as delimiters. Sed is line oriented meaning if you are using the substitute (s) command it will only replace the first instance of the string being replaced. If you have an input line that reads “One fish, Two fish, Red Fish, Blue fish.” and you want to replace “fish” with “FISH” by default sed will only replace the first instance of fish with FISH.

Miscellaneous Notes

Single quotes are literal quotes. Searches for what is literally within the ‘quotes.’

Printf “%&s “ “ “ right justifies by default. “%$s “ ” ” adds padding value. Printf “%-3s ” Negates padding value (left justify)

The end of the semester is upon us and the EoCE for UNIX has been deployed. Gulp! I have started to tackle the first problem, 0x0: Script Analysis, and found that once I created and ran the script I had a much better understanding of what it was doing. With my nerves eased I was able to settle in and begin answering the questions pertaining to the various components of the script. I have my responses captured in screen shots and randomly written on paper I just need to add them to the blog. This brings me to the next problem, 0x1: super puzzlebox 2 turbo, I had a feeling we weren’t done with the puzzlebox yet. I equate these to solving a Rubik’s cube. For some people finding the solution only takes seconds while others may spend minutes, hours or even days twisting the colorful cube to perfection. The last category of people spends many frustrating hours twisting that cube until they have two squares left to align. They are forced with the ultimate decision do I keep twisting knowing that I will destroy all progress or do I carefully peel back the corner of the sticker? I will know my fate once I open and start unraveling the puzzlebox.

This entry marks the last week of classes for the semester and therefore the last blog post for UNIX. It’s been a crazy, whirlwind semester filled with sharp twists and mind bending (numbing) experiences. Entering the course with much trepidation at the suggestion of a higher power I initially expected to flail and run. Over the course of the last 14 weeks I’ve gained what feels like an enormous amount of knowledge about a new OS. My skill level is still firmly planted in beginner mode, but I can at least nod and say I know about the super powers of the dd command.

I am still working on the EoCE. Progress has slowed or come to a screeching halt as I needed to focus on other courses, but with most other projects behind me I can return to puzzle boxes and scripting.