Table of Contents
Part 3
Entries
November 15 2011
Today as a class we've started going over basic UNIX networking tools.
* These tools are invaluable to system administrators because they allow snapshots of the systems network interfaces. * UNIX is designed to be a network oriented operating system. * Networking tools can be incorporated into scripts so system administrators may automate their daily cleanup procedures. * Network interfaces break a holy rule of unix; network interfaces are NOT a file!
NoVember 23, 2011
Today I set up an administrative user account on PogoPlug server. It has come to my understanding that constantly using the root account is not ideal for day to day interaction with your system for a variety of reasons:
- It is not secure; what if your session as root is somehow intercepted? Your system would them be compromised.
- When a user uses 'su' or 'sudo' instead of the root account the commands are logged to syslog (3)
- a user cannot simply use sudo, they must be included in the sudoers file.
- the sudoers file is located here(on my system): /etc/sudoers
November 29, 2011
Today I setup 'htop' on my BlueHost domain server.
- I connect to this server via SSH
- htop is proving useful for monitoring the load limits my server can handle whilst I'm developing CivicBeat
- This plays more into the administration and maintenance portion of UNIX
- I can see that the themes in this course have dramatic real world application.
November 30, 2011
Last night I Noobishly executed a crippling loop script that broke my pipe and has incapacitated me…
- It would be wise to never execute scripts that you don't understand.
- I think it was a good lesson concerning what I had stated above.
- I sure am missing that “are you sure you want to do this?” prompt!
- What challenges are you facing with respect to the course? - I'm still locked out of lab46… so at this juncture; everything.
unix Topics
Kill
The kill command does just that… Kill PIDs of course. Simply run the ps command to see your listed processes, then type: “kill [the PID yo would like to kill]”
[root@alarm ~]# peace -bash: peace: command not found [root@alarm ~]# love -bash: love: command not found [root@alarm ~]# kill You must specify whom to kill [root@alarm ~]# ;)
dstat
dstat is a tool for monitoring system resource statistics
andrew@Dimension-4600:~$ dstat You did not select any stats, using -cdngy by default. ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- usr sys idl wai hiq siq| read writ| recv send| in out | int csw 0 0 99 0 0 0|1734B 2747B| 0 0 | 136B 210B| 24 28 0 1 99 0 0 0| 0 0 | 594B 1766B| 0 0 | 53 61 0 0 100 0 0 0| 0 0 | 264B 1228B| 0 0 | 30 18 1 1 98 0 0 0| 0 0 | 741B 2739B| 0 0 | 55 71 9 0 91 0 0 0| 0 0 | 708B 1928B| 0 0 | 62 40 0 0 93 7 0 0| 0 16k| 701B 1478B| 0 0 | 64 81 1 1 98 0 0 0| 0 0 | 264B 1140B| 0 0 | 44 34 0 0 100 0 0 0| 0 0 | 198B 944B| 0 0 | 34 39 0 0 100 0 0 0| 0 0 | 264B 1140B| 0 0 | 35 30 0 0 100 0 0 0| 0 0 | 264B 944B| 0 0 | 35 40 1 1 98 0 0 0| 0 0 | 414B 1144B| 0 0 | 39 56 1 0 99 0 0 0| 0 4096B| 198B 944B| 0 0 | 41 47 0 0 100 0 0 0| 0 0 | 198B 1078B| 0 0 | 33 39 2 1 97 0 0 0| 0 0 | 893B 2309B| 0 0 | 68 72 8 1 91 0 0 0| 0 0 | 840B 2016B| 0 0 | 59 42 ^C andrew@Dimension-4600:~$
slurm
Slurm is another utility that displays network interface activity with an ascii text graph.
-= slurm 0.3.3 on Dimension-4600 =- x x x x x x x x x x x x x x x x x x x x x x x Current RX Speed: 0.00 KB/s Current TX Speed: 0.00 KB/s Graph Top RX Speed: Graph Top TX Speed:
mtr
mtr is the wonderful love-child of traceroute and ping. <3
[andrew@alarm ~]$ mtr civicbeat.com My traceroute [v0.81] alarm (0.0.0.0) Sun Nov 27 17:44:42 2011 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. 192.168.1.1 0.0% 7 0.5 1.1 0.5 4.6 1.6 2. ??? 3. gi5-2-19-cmts.emirny27-rtr002.c 0.0% 7 9.5 10.3 9.2 13.5 1.5 4. ae15.ithcnycy-rtr001.cny.northe 0.0% 7 10.7 11.1 10.0 15.2 1.8 5. rdc-74-74-108-116.cny.northeast 0.0% 7 13.7 13.9 12.9 18.2 1.9 6. ae-3-0.cr0.chi10.tbone.rr.com 0.0% 7 26.7 27.9 26.3 30.4 1.6 7. ae-0-0.pr0.chi10.tbone.rr.com 0.0% 7 26.4 30.6 26.0 55.7 11.1 8. te0-2-0-7.ccr22.ord03.atlas.cog 0.0% 7 26.8 26.9 26.1 28.0 0.6 9. te0-0-0-1.ccr21.ord01.atlas.cog 0.0% 7 33.6 30.1 27.2 35.7 3.3 10. te0-4-0-3.ccr21.jfk02.atlas.cog 0.0% 7 42.1 42.1 41.1 42.9 0.5 11. te0-2-0-1.ccr22.jfk05.atlas.cog 0.0% 7 42.9 42.9 42.4 43.9 0.6 12. te4-2.mag01.jfk05.atlas.cogentc 0.0% 7 42.0 42.5 41.7 45.3 1.3 13. 38.104.74.118 0.0% 6 42.5 43.2 42.3 45.7 1.3 14. tg1-1.ar02.prov.acedc.net 0.0% 6 102.2 103.6 101.1 107.2 2.5 15. port99.ar02.prov.bluehost.com 0.0% 6 103.6 104.0 101.1 109.8 3.2 16. box259.bluehost.com 80.0% 6 105.7 105.7 105.7 105.7 0.0
powertop
This utility displays system power consumption for each individual component.
andrew@Dimension-4600:~$ sudo powertop PowerTOP 1.97 Overview Idle stats Frequency stats Device stats Tunables Summary: 0.0 wakeups/second, 0.0 GPU ops/second and 0.0 VFS ops/sec Usage Events/s Category Description 7.7 ms/s 0.0 Process powertop 7.4 ms/s 0.0 Process sshd: unknown [priv] 397.4 µs/s 0.0 Process sshd: unknown [net] 262.7 µs/s 0.0 Process /usr/lib/gnome-settings-daemon/gnome-settings-daemon 201.3 µs/s 0.0 Process rsyslogd -c5 157.0 µs/s 0.0 Process /usr/lib/gvfs/gvfs-gdu-volume-monitor 142.2 µs/s 0.0 Process avahi-daemon: running [Dimension-4600.local] 139.5 µs/s 0.0 Process nautilus -n 129.9 µs/s 0.0 Process /usr/lib/gvfs/gvfsd-trash --spawner :1.1 /org/gtk/gvfs/exec_spaw/0 127.5 µs/s 0.0 Process sshd: andrew@pts/1 106.8 µs/s 0.0 Process [migration/0] 101.0 µs/s 0.0 Process /usr/lib/i386-linux-gnu/colord/colord 79.4 µs/s 0.0 Process /usr/bin/python /usr/bin/update-manager --no-focus-on-map 77.3 µs/s 0.0 Interrupt [9] RCU(softirq) 71.9 µs/s 0.0 Timer hrtimer_wakeup 57.6 µs/s 0.0 Process update-notifier 49.6 µs/s 0.0 Process /usr/lib/indicator-datetime/indicator-datetime-service 41.5 µs/s 0.0 Process /usr/lib/accountsservice/accounts-daemon 40.1 µs/s 0.0 Timer tick_sched_timer 35.3 µs/s 0.0 Interrupt [3] net_rx(softirq) 31.6 µs/s 0.0 Process /usr/bin/ssh-agent /usr/bin/dbus-launch --exit-with-session /usr/bin/gnome-session --session=ubuntu 25.4 µs/s 0.0 Process [flush-ecryptfs-] 21.6 µs/s 0.0 Interrupt [1] timer(softirq) 20.8 µs/s 0.0 Process [ksoftirqd/0] 14.7 µs/s 0.0 Interrupt [21] eth1 14.5 µs/s 0.0 Process [kworker/0:0] 10.5 µs/s 0.0 kWork flush_to_ldisc 8.0 µs/s 0.0 Timer clocksource_watchdog
calcurse
ever wonder why software distributors bundle heavyweight, cumbersome and convoluted Calendar programs? Why not go lean with 'calcurse'. Calcurse is a shell based calendar program with a nano'esque feel that keeps the unix philosophy in mind!
andrew@Dimension-4600:~$ calcurse ┌───────────────────────────────────────────────────────────┐┌──────────────────────────────┐ │ Appointments ││ Calendar │ │───────────────────────────────────────────────────────────││──────────────────────────────│ │ November 27, 2011 ││ November 2011 │ │ ││ Mon Tue Wed Thu Fri Sat Sun │ │ ││ 1 2 3 4 5 6 │ │ ││ 7 8 9 10 11 12 13 │ │ ││ 14 15 16 17 18 19 20 │ │ ││ 21 22 23 24 25 26 27 │ │ ││ 28 29 30 │ │ ││ │ │ │└──────────────────────────────┘ │ │┌──────────────────────────────┐ │ ││ ToDo │ │ ││──────────────────────────────│ │ ││1. Finish the Opus │ │ ││ │ │ ││ │ │ ││ │ │ ││ │ │ ││ │ │ ││ │ │ ││ │ │ ││ │ │ ││ │ │ ││ │ └───────────────────────────────────────────────────────────┘└──────────────────────────────┘ ───[ Sun 2011-11-27 | 22:41:47 ]───(apts)──────────────────────────────────────────────────── i Import 0 beg Week ^A Add Appt ^L +1 Day ^J +1 Week ^R Redraw x Export $ end Week ^T Add Todo ^H -1 Day ^K -1 Week o OtherCmd
X11
X11 or “the X window system” makes it possible for UNIX and Linux: both text based operating systems, to run graphical applications. X11 is the graphical core for the NIX based GUI, it takes care of your mouse, keyboard, resolution and screen refresh rates. X11 is void of hardware dependancies and can load window sessions over network connections.
Window Managers
Your window manager is a graphical layer that sits atop of X11. Window managers help keep track of your windows, provide a way to graphically interact/launch applications, manage images and video. Technically the window manager is a program running on top of X11 and it's purpose is to extend the functionality of the X window system.
Desktop Environments
The final layer of our NIX based graphical user interface. The Desktop environment takes care of extended features above the window manager such as additional task and menu bars or an interactive desktop so the user can organize his work. An example of a rather popular desktop environment would be GNOME or “GNU Network Object Model Environment”
Executing processes in the background
Sometimes you need to run a process in the background and work on something else to do this execute your command with an '&' on the end. this will run the process and return the terminal prompt back to the user.
wget
wget is a “non-interactive” utility for downloading files via http, https or ftp protocols. Wget is non-interactive becasue it does not require the user to be logged on to continue downloading. Wget is also robust meaning the downladed files integrity are check for consistency, this is particularly helpful on poor performing network connections.
To use wget: ~$ wget “example address”
ls -u
List files in order of changes made
[root@alarm ~]# ls notes PKGBUILDs projects system vault [root@alarm ~]# ls -u projects system notes PKGBUILDs vault [root@alarm ~]#
unix Objective
Objective
Integrate what I've been learning in my UNIX class into my job @ civicbeat.com
use network monitoring and sftp tools in order utilize our domain server to its full potential.
Measurement
Use filezilla to download and upload files quickly and more efficiently then the GUI that bluehost provides. htop allows me to monitor network traffic and system resources in real time so I can test the integrity of my server.
Analysis
Reflect upon your results of the measurement to ascertain your achievement of the particular course objective.
- Great! my Filezilla has brought up my workflow whilst I'm working at the office. instead of editing the files on the server directly I download the PHP or CSS file onto my machine for manipulation. This means I will always have a copy if I mess something up!
- I'm considering hosting my own server… perhaps some time in HPC next semester will help me make that decision?
- Could the course objective be altered to be more applicable? How would you alter it? yes, how to integrate UNIX networking tools into your work flow.
Experiments
"Cool Script" bro.
Question
Is it a good idea to execute crazy scripts you find on forums?
Resources
04-17-2009 quirkasaurus Registered User Join Date: Jan 2009 Location: canton, michigan Posts: 479 Thanks: 0 Thanked 7 Times in 7 Posts Quote: Originally Posted by Ikon This is an old one. At a job interview this was a script I had to explain. Code: :(){ :|:& };:
Hypothesis
Perhaps I should plug this in to Lab46 and see what it does? Will this script indeed be 'cool'? let us hope so!
Experiment
Script as follows:
#!/bin/bash
:(){ :|:& };:
exit 0
Then chmod 755 test (test was the name of the script) and execute ./test
Data
Uh-oh! we're caught in a pretty nasty loop… lets log out and re-establish our session, perhaps that might be a crude fix? - A wise man named Chuck Testa once said “Nope!” let's see why we can't re-establish our ssh session with an ssh -verbose or “ssh -v asowers@lab46.corning-cc.edu”
Last login: Wed Nov 30 20:33:03 on ttys001 Welcome Andrew! iMac:~ asow123$ ssh -v asowers@lab46.corning-cc.edu OpenSSH_5.6p1, OpenSSL 0.9.8r 8 Feb 2011 debug1: Reading configuration data /etc/ssh_config debug1: Applying options for * debug1: Connecting to lab46.corning-cc.edu [143.66.50.18] port 22. debug1: Connection established. debug1: identity file /Users/asow123/.ssh/id_rsa type -1 debug1: identity file /Users/asow123/.ssh/id_rsa-cert type -1 debug1: identity file /Users/asow123/.ssh/id_dsa type -1 debug1: identity file /Users/asow123/.ssh/id_dsa-cert type -1 debug1: Remote protocol version 2.0, remote software version OpenSSH_5.5p1 Debian-6+squeeze1 debug1: match: OpenSSH_5.5p1 Debian-6+squeeze1 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_5.6 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-ctr hmac-md5 none debug1: kex: client->server aes128-ctr hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Host 'lab46.corning-cc.edu' is known and matches the RSA host key. debug1: Found key in /Users/asow123/.ssh/known_hosts:3 debug1: ssh_rsa_verify: signature correct debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: Roaming not allowed by server debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey,password debug1: Next authentication method: publickey debug1: Trying private key: /Users/asow123/.ssh/id_rsa debug1: Trying private key: /Users/asow123/.ssh/id_dsa debug1: Next authentication method: password asowers@lab46.corning-cc.edu's password: debug1: Authentication succeeded (password). Authenticated to lab46.corning-cc.edu ([143.66.50.18]:22). debug1: channel 0: new [client-session] debug1: Requesting no-more-sessions@openssh.com debug1: Entering interactive session. debug1: channel 0: free: client-session, nchannels 1 Connection to lab46.corning-cc.edu closed by remote host. Connection to lab46.corning-cc.edu closed. Transferred: sent 1440, received 1752 bytes, in 0.0 seconds Bytes per second: sent 117482.9, received 142937.6 debug1: Exit status -1 iMac:~ asow123$
Welp kids, it looks as if we're locked out of the house… it turns out this script wasn't so cool after all.
Analysis
- script was in fact un-cool and essentially crippling.
- what shortcomings might there be in your experiment? - I cannot do anything… :(
Conclusions
Never, EVER execute scripts all nilly-willy… Especially if you find them on some forum.
Retest
If you're doing an experiment instead of a retest, delete this section.
If you've opted to test the experiment of someone else, delete the experiment section and steps above; perform the following steps:
State Experiment
Whose existing experiment are you going to retest? Prove the URL, note the author, and restate their question.
Resources
Evaluate their resources and commentary. Answer the following questions:
- Do you feel the given resources are adequate in providing sufficient background information?
- Are there additional resources you've found that you can add to the resources list?
- Does the original experimenter appear to have obtained a necessary fundamental understanding of the concepts leading up to their stated experiment?
- If you find a deviation in opinion, state why you think this might exist.
Hypothesis
State their experiment's hypothesis. Answer the following questions:
- Do you feel their hypothesis is adequate in capturing the essence of what they're trying to discover?
- What improvements could you make to their hypothesis, if any?
Experiment
Follow the steps given to recreate the original experiment. Answer the following questions:
- Are the instructions correct in successfully achieving the results?
- Is there room for improvement in the experiment instructions/description? What suggestions would you make?
- Would you make any alterations to the structure of the experiment to yield better results? What, and why?
Data
Publish the data you have gained from your performing of the experiment here.
Analysis
Answer the following:
- Does the data seem in-line with the published data from the original author?
- Can you explain any deviations?
- How about any sources of error?
- Is the stated hypothesis adequate?
Conclusions
Answer the following:
- What conclusions can you make based on performing the experiment?
- Do you feel the experiment was adequate in obtaining a further understanding of a concept?
- Does the original author appear to have gotten some value out of performing the experiment?
- Any suggestions or observations that could improve this particular process (in general, or specifically you, or specifically for the original author).