======UNIX/Linux Fundamentals Journal====== ====01 26, 2015==== Over this past week I've been going through the commands in reading (mage book). I've noticed lots of commands that relate to DOS (like ls, cd, mkdir...). I have also noticed during the permission part of files that you can give someone read/write/execute. I'm wondering how someone can write to a file but not have read (if possible). I will be checking into this tomorrow during class if I have time. So far I've been able to create directories and files. I've viewed files/directories via ls -a and looked at the output. I do like the command history so that I can view all the things I've completed. It also helps as a quick lookup if I forget a command. I also enjoyed the command whoami since I currently run Mint on one of my machines at work. I used that command to double check that I was not logged in as root (which I wasn't). I'm interested in piping commands and look forward with learning how to properly pipe commands. I also look forward to learning how to script. I can create bash files easily but I do not know the syntax for Linux/UNIX. ====02 02, 2015==== Today I read some manuals on certain commands. I read about null and how it's a data sink. I did send some data to null last week and today I tried to read null. nothing was returned but I did notice when I pico null and fill the file with "hello world" it saves to a file called null. I tried to read the manual for "<, >, 2>, >>, 2>>" but it came back with a syntax error. So I went to Google instead. I read my notes from last week and thought about everything being a file. Since mouse/keyboard are considered a file I figured the fan/processor/motherboard are all considered files. I googled this and found that it's true (according to some sites). I opened the manual for stdout and found that it pointed to stdio. I opened the manual for stdio and found that it's standard input/output library functions. The manual mentioned streams and how they are external files. This does not mean it's hardware but it's something external from the system. In last weeks class we used apropos without explaining what it does. I ran "man" on apropos and found that it is a search tool for the manual. this has helped me out a lot since I can now search the manual for any command that might relate to my task. I ran apropos edit and found a ton of commands that can edit/create/view files. I had forgotten nano since last week and resorted to pico but now I remember due to the search. ====02 09, 2015==== I ran file on file.txt and it came back saying it's ASCII text. I ran cat on the file and it printed out "This is a simple text file. It contains ASCII text." so I'm assuming file is correct. I compressed file.txt via gzip and it turned file.txt into file.txt.gz. I ran file on file.txt.gz and it came back saying it was compressed by gzip and was file.txt but was changed on todays date. I compressed the file.txt via --fast as an argument and it said the same thing when I ran file. I hope that's what was supposed to return but maybe I'm doing something wrong. I've opened riddle.html and found when I cat the file it returns something un-readable. the file command says it was called shiny.tar and gzip was used. I'm going to try to un-compress the file via gzip but we'll see what happens. gzip does not want to run on the file. I renamed the file to shiny.tar.gz and ran gzip on the file. It ran correctly and the text seems to becoming a little less compressed. After running tar it created the file 'making.waves'. File says it's uuencoded or xxencoded, ASCII text. I ran uudecode on making.waves and it created leaf_on_the_wind.wav. File says that it's a is a gzip file and when I cat the file everything is gibberish. I moved the file so that it has the extension .gz and ran gzip. The file is a little less gibberish but it's still compressed. file doesn't return anything correct but I do think uudecode should fix the issue. That was it! It created a file called results and says I've completed the puzzle box. I ran the two commands and received an email with results as the text. Now that's I've finished the project, I'm looking into the command 'alias'. I've used the command in the past to make things simpler and I'm wondering if I can do it now. I've created cd1 alias so that it takes me to the public directory. I've also created backup1, backup2, backup2.1, and backup3. These aliases did not stick from last week and I'm wondering if it's per session. If yes I will need to find a different way to create aliases or something like it to make things easier. ====02 23, 2015==== Today I started looking into the next project. I copied the zip file into a week 4 directory and started off. I tried to extract the pbx1.zip but it kept on asking for a password. I figured it had to be something I’d know so I tried ccc, week4, pbx1, and unix. Nothing seemed to work and then I thought “maybe something was extracted besides the password protected files”. I was correct and found the README file. From this file I was able to extract the enigmatic.file and octal2ascii.c file. After playing around with the file, reading the contents and using the command file, I noted that it was an uuencoded file. I used the uudecode command and it created a file called stage2. I ran the command file on stage2 and came back with a permission denied error. I checked the permissions and noticed that I had none so I used (chmod) (for some reason when I type that word by itself it breaks this page) 700 to give myself full permissions. Then I read the file and noticed it was all octal. I then used the documentation in ocal2ascii.c to compile the oct2ascii command. I tried piping the contents of stage2 into oct2ascii but it kept on failing. So I tried using “cat stage2 ./oct2ascii” instead. This worked properly (so I think) and the organized numbers switched to a jumbled mess of letters, special characters, and numbers. I’ve tried unzip, tar, and a couple other decompression tools to try and make the text readable. After that I decided to use the command file (should have used it first). File told me that the file (that I named “file”) was an elf file. So I read some of the manual on elf and found that it’s some kind of executable. I’ve tried many different ways of executing the file and I did go into the file and remove the first line of octal numbers from the stage2 file. I’ve had no luck but we’ll see right now if I’m able to crack the code. I ran readelf -a file.elf but the contents didn’t tell me anything useful. I found that I need to run the bash command to run the file but it’s saying an error about it being in binary. After doing some searched via Google and the manual it’s saying I need to convert it to 32bit. I’m not thinking this is correct since it looks like an extremely long process and is most likely wrong. Well, I figured the file needed to be compiled so I renamed it and tried to compile it with the same arguments that ocal2ascii.c had. This did not go over well and the screen started scrolling crazily out of control and I had to ctrl c to cancel the command. So I ran “getcon LONG_BIT and noted that we’re running 64 bit OS. From the output of the file command I also noted that the ELF file is also a 64 bit executable. From that knowledge I know that the file does not need to be compiled and it is supported by the OS. I’ve tried bash with different arguments and it still does not want to run the ELF file. I’m going to wait until tomorrow to finish this lab since it’s late and I’m probably over thinking the problem. ====03 02, 2015==== So, I finally figured out last weeks puzzle box. I had to ask for help but after a hint was given I was able to figure out the rest. I was translating the code from octal to ASCII but it should have been hex to ASCII. After changing this I was able to read the text that gave the password to unzip the next couple of files. The files were out of order and I had to use cat and >> to pipe the contents into one file. Then I moved the lines around via vim (I don't remember the commands). The contents turned into a bat picture and I submitted that as the project. One thing that was noticed when looking at the project that was converted via octal to ASCII was the file created was the same file as octal to ASCII but there was a space at the end of the file that was throwing the whole thing off. So if I had noticed that space at the end of the file, I could have done a compare of the two files and noticed they were exactly the same. I thought this was pretty nifty since the file was a message also and the code to convert itself into the message (if you changed it from octal to hex). I noticed that we're supposed to do backups of our data every week (which I thought I was doing correctly). Now I think I've been doing it wrong because it keeps on saying nothing has changed. So I'm thinking that I have to add to the repository and then do the backup. For some reason I figured it backed up the whole user account. I will be looking into this more tomorrow and ask questions during class if possible. I've been playing around a lot on my own linux mint machine (I have one for work) and it was recommended to me that I learn the "at" and "mail" command. This I found very usefully once the person described it to me. Supposedly the "at" command will allow you to delay a command to run. The "mail" command is exactly what the name says (mail). So first thing I tried to do was run "man mail" and "man all". Neither of these commands came up and it came back with command not found error. I then turned to google and found that I needed to download the "mail" command. I ran the commands that were recommended and then the "mail" command was downloaded. I tried to run the command but it had some issues with smtp and such (can't remember the exact error). I still have not made the command operational but I want it to work for quick reminders and being able to send text messages to my phone. Another command that I'm trying to understand, is the ls $$$ [a-z] special char command/filter. I forget the name of this but we covered it last week and I'm a little confused on what this filter can be applied to. I know cat can have these filters but can it be used inside files? Can it be used for any command? I'm not really sure but I've been playing around with it on my machine and so far have not run into a command that does not take the special characters. One last thing I've been working on is automation. I like how you can script anything into a file (like a bash file in windows) but I'd like to learn how to make this file part of the rest of the normal commands like "ls". I'm not sure how this could be done since I do not have root access to the server and doubt I can add a file to the same location as "ls" but I guess I can try. Maybe there is another way of adding it as a command but only for my user. I'll be asking the tutor tomorrow about this question and I bet he has an answer. ====03 09, 2015==== I ran cat on dectohex.c and memdump.ram (lots of un-readable data). After looking at the manual, I changed the file to octaltohex.c compiled and ran the file. It returned 0. I also ran the dectohex.c and it also returned a 0. I read some forums on recovering a mem dump into a file and none of it seemed to help. So I figured I’d try to answer my question on my previous posts due to having trouble with the udr0 assignment. I’m going to have to ask for help tomorrow when I’m in class. One of the things that I remember trying is setting an alias that will stay instead of a temp alias commands. I tried “man alias” and there was no manual so I turned to google. I found that I needed to add the alias to .bashrc and it is located in my home directory. I created ls=’ls -la’ and ran the file to make the changes (. ~/.bashrc). After that ls now shows all hidden files and lists them in an easier to read format. I’ll be adding to this file so that I do not need to type every command over and over. I’ve had some trouble with the special char commands [cdh]$$$ (stuff like that). I thought that it could be run with every command but I cannot get it to run with cat. I tried [t]$[e] and it didn’t not return only the letters that started with t and end with e. I guess I will need to research the special char more but for now I’ll go on to my next question. When creating the alias in .bashrc I found there were a couple “if” statements. After googling the statements I found that you can use them for alias commands! I did try to read the man for .bashrc but there is no manual. This will come in handy for my backups. I’ll create an if statement to go through the proper steps and if there is already a src folder it will move and rename to something else. Then it will run the normal commands. I’m excited to try this new process but I have to do some server work and cannot play with it tonight. ====03 16, 2015==== Saturday I tried udr1 and became frustrated so I decided to pick it back up on Sunday (which I didn't) and leaves us to today. I grabbed and converted the second part of the file (the small part) and flipped it both ways. I decided to go with the normal file since I didn’t want it anymore confusing and I figured it would be easier but longer. I created two scripts, one to convert the small file so that the last byte turned into the first byte. I basically did the same thing for the second script but I created two so that I could submit either of them depending on what you would like at the end. How did I grab the second file first? Below are the steps that I took. First I ran edithex on the file and noticed the string that said how far down in the file it was located. I ignored this and looked at the text right above the other text and noticed it was word backwards. So I read them and found the second file was 202 bytes above that text. For some reason I thought this would be easier and if I goofed it would not take hours for the file to process. So I moved my to where the bytes all turned to zeros and found the address. I converted the address to decimal and ran dd on the file with the address that I just converted and 202 as the count. I also put in the argument conv=slab (or something like that) so that the bytes would be flipped. Then I ran my script to take the last byte and change it to the first. This created a little image of some kind of creature and I’m guessing that’s what I’m suppose to see. Now for the harder part which I think was suppose to be part 1 but for me is part 2. I noticed I was doing some basic math wrong and started with the wrong address but after I fixed that I used dd to start at that location and go to the end of the file. After going to the end of the file via hexedit and finding the file size in hex (the address), I converted it to binary and added it to my script. Now I’m running the script but it looks like it’s going to take a couple hours to complete so I figure I’ll get some sleep or maybe finish some other work before tomorrow. ====04 06, 2015==== Hello Opus, I know it's been a long time and I apologize for the neglect. Break was great but It's time to get back to the Unix/Linux world. I have not completed udr2 yet but I did read documentation before I start to tackle the project. This is the first time I read the manual before diving in. I figured it might give me a little insight and due to this project looking confusing as heck, I figured it wouldn't hurt. Below is what I've learned on each command and notes to help me complete the project. dd: Copy a file, converting and formatting according to the operands. cbs=BYTES convert BYTES bytes at a time conv=CONVS convert the file as per the comma separated symbol list count=N copy only N input blocks ibs=BYTES read up to BYTES bytes at a time (default: 512) iflag=FLAGS read as per the comma separated symbol list oflag=FLAGS write as per the comma separated symbol list seek=N skip N obs-sized blocks at start of output skip=N skip N ibs-sized blocks at start of input bc: (basically it's for calculations) is a language that supports arbitrary precision numbers with interactive execution of statements. There are some similarities in the syntax to the C programming language. A standard math library is available by command line option. If requested, the math library is defined before processing any files. bc starts by processing code from all the files listed on the command line in the order listed. After all files have been processed, bc reads from the standard input. All code is executed as it is read. (If a file contains a command to halt the processor, bc will never read from the standard input.) This version of bc contains several extensions beyond traditional bc implementations and the POSIX draft standard. Command line options can cause these extensions to print a warning or to be rejected. This document describes the language accepted by this processor. Extensions will be identified as such. od: Looks like it runs files Cat for binary (very useful MAKE sure you use it!) <-- Note to self bvi: A list of filenames. The first one will be the current file and will be read into the buffer. The cursor will be positioned on the first line of the buffer. You can get to the other files with the ":next" command. -R "Readonly": The readonly flag is set for all the files, preventing accidental overwriting with a write command. -b begin causes bvi to load a file not from the start but from offset begin. -e end causes bvi to load a file not till end but till address end. -s size causes bvi not to load the complete file but only size bytes. -c cmd cmd will be executed after the first file has been read. If the cmd contains spaces it must be enclosed in double quotes (this depends on the shell that is used). -f script This command provides a means for collecting a series of "ex" (colon) commands into a script file, then using this file to edit other files. Since there is no binary stream editor "bsed", you can use this option to make several global changes in a binary file. hexedit: I remember this command it is a good way of viewing a binary file and searching grep: search for words/symbols in a file date: Not sure what I'll use this for but it looks like it returns the current date Display the current time in the given FORMAT, or set the system date. Mandatory arguments to long options are mandatory for short options too. -d, --date=STRING display time described by STRING, not 'now' -f, --file=DATEFILE like --date once for each line of DATEFILE -I[TIMESPEC], --iso-8601[=TIMESPEC] output date/time in ISO 8601 format. TIMESPEC='date' for date only (the default), 'hours', 'minutes', 'seconds', or 'ns' for date and time to the indicated precision. -r, --reference=FILE display the last modification time of FILE -R, --rfc-2822 output date and time in RFC 2822 format. Example: Mon, 07 Aug 2006 12:34:56 -0600 --rfc-3339=TIMESPEC output date and time in RFC 3339 format. TIMESPEC='date', 'seconds', or 'ns' for date and time to the indicated preci- sion. Date and time components are separated by a single space: 2006-08-07 12:34:56-06:00 -s, --set=STRING set time described by STRING bgrep: Like grep but for binary use (USE IT!) <-- Note to self ====04 07, 2015==== Notes:\\ \\ Problem:\\ Opus not updating correctly. Fix: use a tool to locally update the Opus (from lab).\\ ssh tunnel\\ - ssh -g -L 8080:www:80 user@lab46.corning-cc.edu\\ 8080 - local port\\ www - destination\\ 80 destination port\\ - http://localhost:8080/opus/spring2015/user/start\\ \\ Process ID: PID\\ \\ ps - process status\\ for grounding\\ stop\\ jobs\\ fg - for ground\\ bg - background\\ kill -l\\ kill -1 PID - hangup\\ kill -sighup PID - hangup\\ kill -9 - kill no matter what\\ \\ ps aux | less\\ top - most active process\\ ====04 13, 2015==== So today I started looking at the EoCE final project. There are many parts to the project but they all seem to relate to the past projects. For about an hour I thought the project was due tomorrow and I started working like crazy on it but I'm glad now that I've got plenty of time to work on it. Time to dive into Linux/Unix. I have linux mint installed on one of my machines at work and today I had to remote a linux server. With the knowledge that I've gained from this class I was able to create an account so that I wasn't always root and find the jsp file so that I could edit it. I do not remember all of the commands that I used but I do remember using "man" a couple times to understand the command so that I wasn't just taking someones word on a blog that I found via Google. This helped make sure the commands were not going to harm the server. Besides that I've been working on my desktop (Mint) and running a couple basic scripts/commands. I created a script to copy a file and rename it. I know this sounds simple and it was but I had to do this command every time I tested my java scripts. So it made it a lot easier for me to run a quick command instead of typing the cp name .\newname every time. I also created a script to stop a program from running (when my machine would bog down with too many programs). This was not hard and was only a couple lines but saves me an extra step. Now if only I could automate this Opus, hmmm... Maybe next time! ====04 28, 2015==== wemux --> command in class to see what Matt is typing 0x97 0xff 0xff 0xff 0x00 0x80 chksum = 0x15 only 1 byte (_ _) echo "ibase=16;obase=10;97+FF+FF+00+80" | bc | head -16 | tail -1 | cut -d':' -f1 OR -f2 = 315 tr ':' ' ' --> replace : with spaces sed 's/usr/parachute/g'