Sidebar
Table of Contents
09/21/2015
A quick DATACOMM recap on the EEG project (sde0).
Matt has broken the program down into a series of byte checks for the initial data-check. If nothing fails or bottoms out because of a bad packet or check sum error,
[0001] chksum: 06 msglen: 0005 time_sec: d4 dtype: version 3 ERROR! Checksums do not match! [2b:fe] Valid: 2 Invalid: 1 Total: 3
the program (read.c) will print out various packet metadata.
[0027] chksum: 06 msglen: 0005 time_sec: 19 dtype: version 3
[0028] chksum: 80 msglen: 0005 time_sec: 19 dtype: Slice_End
data: 00 12 d9 93
Valid: 29 Invalid: 0 Total: 29
Finally, I had modified the source to (as requested) print the data type as a string implemented with a series of switch statements.
switch (dtype){ case 0x00: fprintf(stdout," Event"); break; case 0x02: fprintf(stdout," Slice_End"); break; case 0x03: fprintf(stdout," Version"); break; case 0x80: fprintf(stdout," Wave_Form"); break; case 0x83: fprintf(stdout," Frequincy_Bins"); break; case 0x84: fprintf(stdout," Signal"); break; case 0x8A: fprintf(stdout," Time_Stamp"); break; case 0x97 fprintf(stdout," Impedance"); break; case 0x9C: fprintf(stdout," Bad_Signal"); break; case 0x9D: fprintf(stdout," Sleep_Stage"); break; default: fprintf(stdout, "%-9.2x ", dtype); break;
The switch statements use the [unsigned char] dtype as subject matter for the switch conditions. Each switch case accounts for all of the different data type values. If not the switch defaults to printing the HEX values as the previous version did.
09/22/2015
Hello, Today I will be going over various HPC thoughts regarding the project I have decided to focus on. The lair's library!
First off, I would like to brainstorm or put pen to paper what has been my personal solutions, ideas, and understandings(via Matt's help).
Before I do so, I would like to iterate(for myself) what seems to be the general idea of implementation for this problem. Which in no way should reflect the final implementation/output of this project. Just racking my brain here.
- The library database will be a flat-file containing [book_id][book_title][book_auth][username][date][time]
- The library catalog program(done through shell scripting with whiptail) will be the primary user interface.
- Users will have no need to authenticate because they have already done so via ssh.
- The user will be able to search for books via the catalog and check out once ready.
- The database could indicate who has checked a book out along with time and date if applicable.
Various issues to keep in mind when coming up with a solution to this new lair feature.
- Preventing anomalies in the database from occurring like if two users try to checkout the same book.
- What happens if more then one user is accessing the database script?
- How will reminders be enforced?
Now, to follow up with a report regarding my current findings while trying to conjure up a solution. I have been experimenting with whiptail. I have found the menu option to be very enticing regarding traversing the catalog itself, but also being able to select a book from said menu. Experimenting with whiptail itself has been inspiring, but it's hard to want to lock into one idea as far as creating a user interface goes, because it is hard to turn back from such a decision. I think it may work very well though. I learned a neat things about whiptail's menu when figure out how to intercept a return value regarding the option that was selected in the menu. Whiptail's menu by default returns to stderr. So, for the first time ever I learned how to redirect that kind of output.
--menu "Welcome "$USER 25 50 16 0000 "UNIX SHELL PRGRAMMING"
--menu "Welcome "$USER 25 50 16 0000 "UNIX SHELL PRGRAMMING"2>output
Something just hit me! Why does output have to be a file? Could it be a variable $output.
My next step is attempting to populate that menu with the contents of the flat-file database with a loop, but I have yet to figure out the way to employ a loop that would create multiple parameters for a single call to whiptail. Parsing the information for this should be no issue.
Finally, to end on a optimistic note. Playing around with whiptail has dramatically quenched my thirst for playing with shell scripting. I almost forgot the pleasure of shell scripting, or just being in that powerful shell environment. Where our code can be tested immediately on the fly right on the command line itself. I love it. I'm dramatically shifting my focus to shell scripting, because I have been reminded of the power it holds. Using the shell for a quick solution to a problem is extremely productive. We have so many tools at our disposal it is all right at our finger tips. You may get a fraction of that kind of flexibility and power with a fancy application like an IDE glued to a GUI environment.
I just wanted to close on one more of my findings today. I found a putty alternative called KiTTY. I'm not even sure I can call it an alternative because it is essential the same thing. IT is a fork from putty version .65 which can be downloaded in a handy portable .exe version perfect for the flash drive. Which I find great, and sad at the same time. How is it that the official putty application is so sh*y in comparison to an offshoot. Yet, I presume the offshoot was that of the occurring sh*ty evolution. Then again movie sequels always had a similar reputation… KiTTY, highly recommended for windows users it does everything it is suppose to. I may be exploring other terminal emulators in the future.
09/25/2015
Hello again, time seems to be advancing in a rate at which I can't get anything done (diving into a project etc.) Bummer. However, I have gotten the chance to play around with my Kindle, and grease my wheels with some shell scripting. I don't know how this happens, but it seems like I know more than I knew yesterday. Like all that HPC/UNIX stuff is starting to make sense, and the stars are aligning. XD
The Kindle. This is more for the HPC side of things. Brushing up various administrative/reconnaissance commands. Kindle specs! The boring.
uname -a
uname -mrs
Linux 2.6.31 with a armv71 processor.
df -h
Showing a lot of disk space information. Stuff I'm honestly having trouble mentally parsing. It shows a lot of what seems to be mounted filesystems. With a Google search on the specs, I know that the kindle can hold roughly 2Gs of memory.
ps aux
Took a peek at the processes running. Most of it goes over my head. Speaking of ps! I learned that the program had evolved to accept different flavors of UNIX systems flavors of issued arguments to make everyone happy.
- GNU's long arguments preceded with double dashes. - -
- UNIX's short arguments preceded with a single dash. -
- BSD's single stand alone argument. Like aux
So, my next mission with the kindle is installing my favorite packages. Possibly the man library if it's not ungodly huge, screen and much more. Maybe update the coreutils? The issue I'm having is figuring out how to write to the kindle remotely (specifically to root's mounted file-system that is set to read-only). I do have a root home directory, and it is writable. I have also successfully tested binaries that coexist in root's home/bin/ directory. The reason I pursue this is because I could benefit from editing aliases and setting environment variables at login. I did test a shell script (number guessing game) on the kindle, and it worked!I was delighted to have done this. Now I need more tools!
The guessing game!
echo "Guess a number!" cmp=$((`date|cut -d ' ' -f 4|sed s/://g`%101)) for i in `seq 1 5` do read cho if [ "$cho" -lt "$cmp" ] then echo "too low!" elif [ "$cho" -gt "$cmp" ] then echo "too high!" if [ "$i" -eq "5" -a "$cho" -ne "$cmp" ] then echo "game over, you loose!" echo "I chose $cmp, haha." fi else echo "corect, you win!" exit 0 fi done exit 0
After doing a fair amount of C/C++ programming this year. I gotta say, I love the flexibility coupled with it's strict yet rich syntax. Very fun to write, and very fun to look for new and improved ways of accomplishing a task. With BASH scripting I'm always looking for those sick one liners that get the job done. Something that sums up what could be multiple lines of code in c. Very fun.
09/28/2015
Regarding sde0, I decided to advert my eyes back to sde0 project. I am currently working on outputting the sleep stage type. After attempting this I noticed something.
[18410] chksum: a0 msglen: 0005 time_sec: 7c dtype: Sleep_Stage sleep: 00 00 00 03
If there are only five possible sleep stages why the need for a multi byte field? Is it for possible expansion? If so, how is it calculated being that we have to read it as little-endian? Is each byte a separate field, or a total(summing the bytes) points to the correct sleep stage. I'm not sure, but it has made me wonder. Should I ignore the other bytes, and only look at the most significant byte? Decisions… Decisions… I know, I'll just ask Matt!
The way I first noticed this is because I would get an output like this after creating a switch statement in the loop where the data block gets outputted.
[18410] chksum: a0 msglen: 0005 time_sec: 7c dtype: Sleep_Stage sleep: UndefinedUndefinedUndefinedLight
216 for(i=(msglen-2); i>=0; i--) 217 { 218 if (j == 15) 219 { 220 fprintf(stdout, "\n\t "); 221 j = 0; 222 } 223 else 224 { 225 j++; 226 switch(dtype){ 227 228 case 0x9D: 229 if( *(dblock+i) == 0x00){fprintf(stdout,"Undefined");} 230 if( *(dblock+i) == 0x01){fprintf(stdout,"Conscious");} 231 if( *(dblock+i) == 0x02){fprintf(stdout,"Rem");} 232 if( *(dblock+i) == 0x03){fprintf(stdout,"Light");} 233 if( *(dblock+i) == 0x04){fprintf(stdout,"Deep");} 234 break; 235 236 default: 237 fprintf(stdout, "%.2x ", (*(dblock+i))); 238 break; 239 } 240 } 241 }
Of course I first figured I must have inserted my code into the wrong part of the loop, but I wanted to make sure before I made any edits that I fully understood the situation. Hmmm, guess those programming classes at CCC are starting to pay off. :'D
09/29/2015
Back to HPC related work! Went to work on the lair-brary.sh script I have been working on. I've decided not to stress over a complete finish product that would be actually implemented. Instead, I would try to tackle the project for better understanding. Which usually results in revisions, reiterations, and reconstruction.
Working with whiptail, I have worked my way around to of the problems I was having.
- Spanning multiple arguments for a single command.
- Returning whiptails error from STDERR to a variable.
However, they were not the solutions I was looking for, but they work. Pretty or not.
For instance, when trying to span multiple arguments for a single command. I wanted(if possible) to use a loop mid command that would iterate more arguments, but I'm starting to think this is not possible. Instead, I formatted a string that whiptail would take using quotes and spaces when appropriate. It worked, but I'm having trouble discarding certain characters without things breaking. So, now I have a moderately ugly output on the whiptail menu under the book title's name that is encased in quotes and underscores that serve as book title spaces. The first part not as annoying, but I'm not sure how my variable string is being interpreted. The second part makes absolutely no sense to me. The output of the variable has been tested many times on the command line to ensure whiptail is receiving the correct input. The problem is either the result of the shell(bad syntax on my part), or whiptail misinterpreting. My guess is the first one. :'D
When returning whiptails argument from STDERR to a variable. I found it easy enough to save to a file(even though I would have killed to get it straight to the variable instead), and then I would cat the file for it's value. After the value had been recorded I would remove the file as if nothing ever happened… But, I still wish there was a nicer way to do this. :)
As a by-product of these headaches. I have already started to rethink of things. Like, how the user can view the catalog and its data that exists on a flat-file! If a user is aloud/able to scroll through said data. Why not present the file to them as is? Save the work of inputing/checking out books, and other services for whiptail. Let the user view the read-only catalog as they wish to see what books are available, and who has checked them out.
I was reunited with sed, and regular expressions. How, I miss how much time one could spend on parsing/formatting data for their needs.
More to come!
09/30/2015
This will be a quick update due to running short on time because I have been glued to the library project, and am enjoying every bit of it.
So, I happily made a revision that was a tribute to my previous post stressing how important revisions are. I think things are starting to look like an envisioned output. At least an output I'm happy with to convey my understanding, or application of this whole library checkout thing.
I ran into a few walls here and there, but I made a lot of progress. I now have a presentable user interface with tasteful windows that work (looping back to where they should and not locking user's in which happened me today :p). The main function I'm taking my sweet time with (non-intentional) is the Get A Book feature. Where a user will enter a correct book ID, if the book has not been checked out by another user, and the ID is valid. The user is successfully advanced to a confirmation screen that reminds them to return the book by the end of the semester. The username, and time/date of the checkout is recorded in the same row as the book in the flat-file catalogue. Otherwise, If the book has been checked out you are advanced to a screen that informs you who has the book along with what time/date it was checked out.
I decided it would work smoothly if the ID book field was a four digit hexadecimal value. Therefore this would leave room for roughly a nice 65,000 books, but also be a convenient small number to memorize. The user could search the catalogue maybe greping for a books name. Then all the user needs to do is snag the four digit book ID and punch it into the Get A Book screen.
Sorry this was a frantic and poor entry, but it was a rushed one. I will follow up with more soon.
10/01/2015
Ok, I just wanted to follow up with the god awful opus entry above ^. I was a bit excited, and pressed for time.
Back to the details of the library project. The current output I have completed is a sturdy menu system. The main menu features two options [Get A Book][EXIT].
Get A Book
- Allows the user to checkout a book
- The user inputs a 4 digit hex number aka the book ID.
- If the book ID is invalid the user will be looped back to the ID entry menu, or if they escape.
- After the user inputs a valid book ID the script prompts the user with the book of the ID entered.
- The prompt is there to double check with the user that this is indeed the book they want.
- After the book is confirmed by the user. The script inserts username, time, and date the book was checked out.
- The catalog is then updated with this information. Preventing others from checking out the same book.
Exit
- An obvious escape from the main menu. Which can also be done by pressing ESC.
A list of ideas that I don't want to forget.
- Add an “invalid book ID!” line in the input box the first time the user inserts an invalid book ID.
- Add a return menu for returning books that creates another document(or the same) of books being returned.
- Create admin menus like updating returned books to ensure they were in fact returned.
- This enables me to assign certain users as “librarians” which could come along with update alerts for these users.
- Add a Add Book menu for librarians to streamlines book entry.
- For the add menu the Input field for ID would decrement from FFFF by each book entry.
1#!/bin/bash 2 menu=1 3 while [ "$menu" -eq 1 ] 4 do 5 6 if (whiptail --title "MENU" --menu "Hello, "$USER 25 25 16 "0." "GET BOOK" "1." "EXIT" 2>opt) 7 then 8 9 opt=`cat opt` 10 rm opt 11 12 if [ "$opt" == "0." ] 13 then 14 15 get=1 16 17 while [ "$get" -eq 1 ] 18 do 19 20 if (whiptail --title "Get A Book" --inputbox "Enter a book ID" 25 25 0000 2>opt) 21 then 22 opt=`cat opt` 23 rm opt 24 book=`awk "/$opt/" catalog` 25 26 if [ "`awk "/$opt/" catalog|cut -f 4`" == "NULL" ] 27 then 28 29 if (whiptail --title "Get A Book" --yesno "Are you sure you want this book?\n\n`echo "$book"|cut -f 1,2,3`" 10 100) 30 then 31 dtime=`date|cut -d ' ' -f 5`"\t"`date|cut -d ' ' -f 1,2,4,7` 32 sed "s/$book/`echo "$book"|cut -f 1,2,3`\t$USER\t$dtime/g" catalog > tester 33 34 35 whiptail --title "Get A Book" --msgbox "Be sure to return this book by the end of the semester!\n\nThank You!" 10 80 36 get=0 37 38 else 39 40 get=1 41 42 fi 43 44 fi 45 else 46 47 get=0 48 49 fi 50 51 done 52 53 elif [ "$opt" == "1." ] 54 then 55 56 menu=0 57 58 fi 59 60 else 61 62 menu=0 63 64 65 fi 66 done 67 clear 68 exit 0
Some things I learned.
I should probably get around to learning awk. For everything I have wanted to do with the catalog file when it came to modifying field/column values everything points to awk.
True false return from whiptail that counts for the ok and cancel buttons. Helps for making menus user friendly. Done with an if( ) sub shell conditional. As a pose to conditional comparison if [ ] statement. Which again reminds me of my love for bash even more. Promoting this “do it your way”, “whatever floats your boat”, or “whatever works for you” philosophy.
I should probably dedicate some time to datacomm next! :'D
10/06/2015
So, things are slowing down as we approach Columbus day break. I have been working steadily on my library project while neglecting other work… *caugh* *caugh* sde0 Sorry, but I believe I will be able to make good use of this time, and grind out some sde0, and do some HPC like activities along the way.
I was eager to play with virtual box recently. I did a reinstall of debian, and set up a host-only network so I could ssh to the Debian box using putty. I was happy because after having done last semesters HPC activities I found setting up and finding things a breeze. Not only that, but I now am starting to feel at home on a unix/linux system.
I will brainstorm some offhand HPC ideas that could possibly be tackled over the break.
- Deploy, configure, and play with squid on vbox.
- Do other various networking set ups with vms
- Continue library shell script
- Work on sde0 both gd and docuwiki's charter.
10/19/2015
Break is over now… Nothing was done… Nothing at all. It is time to re engage, and get some work done. Where I left off was with a finished demo of lairbrary. I say demo because it gets the job done, but it completely ignores the possibility of others using lairbrary while modifying the catalog file at the same time. I also worry that the output file of the whiptail command could over write another user's option file while in use. Which may not be a problem because the file is extremely temporary. I think naming the option file according to the $USER name could be a quick solution to prevent two instances of the program from deleting the option file. That is assuming these files always output to in the same location of the scripts contents as a pose to the users current working directory. Which I could be completely wrong, and sending it to their home or working directory could be another solution. I'm going to make a massive revisions of this regarding these issues. Not only that, but I would like to simplify redundant commands, standardize window sizes, fix the way text appears in menus, and better the catalog's format. Honestly, this is my favorite part of programming. When you can relive an experience with more knowledge, and understanding.
So, I know I have been promising, and promising to work on sde0. Through guilt I will dedicate more time to the project. I decided I would use the output of the C program that displays all packets specs and time, and parse that information with a handy dandy bash script! The script will output a bar graph in charter's text format to be copied and pasted in the wiki.
I'm very pumped to see Mr. Stallman too!
10/21/2015
Quick entry!
Busting my butt on the lair-brary! Matt gave me a hand with coming up with a solution for handling concurrent file writes. Where a dragon will yell at you other wise. ;P The script looks for a .lock file which means another users is modifying the library's catalog. My next obstacle after creating the ADD A BOOK features, again. Is to make the non auto indexed books be placed where they belong in the order of IDs. Which I believe will work simply using grep, counting the lines, diving by thee, and then evaluating each line to find where the book belongs in the currently indexed catalog.
My initial idea for handling concurrent file modifications was to create some sort of tokenized effort. What I called “last one out” meaning the last user to exit having modified the catalog would consolidate and update the pending updates, but this proved to complicated. I'm starting to learn that programming is about simplification short and sweet. If you over abstract your going to have a hell of a time wrapping your head around it. However, if this is an issue chances are there's a simpler answer. You just have to explain it to yourself. Which can be a clue in itself.
Finally, I thoroughly enjoyed the Richard Stallman talk. It was nice to go to an event with like minded people, and feel apart of a community.
10/26/2015
Another quick entry regarding recent finding and ambitions. Over the weekend I unintentionally worked on many hpc related things. Funny how that happens. Wound if installing my first third-party library that was offered out side of the standard application packages that are available. A pidgin plugin for facebook chat which had me scratching my head for a while until I found a repository of one of the developers. Facebook had recently moved away from using xamp, and moved to something google based. Go figure. Also, I again played a lot with virtual box (probably more than I should have), but I got the guest additions installed, virtual mounted some directories, got the usb support working which made my day.
I jumped back on the lair-braryv2 for a bit today I decided to go to fixed window sizes because I did not like the way proportioning the screen with the terminal size work, but I did try it! Finally, I have been staring at .ogg vorbis wishing I knew more about codecs, audio decoding, and audio decoding. Hopefully, I'll get around to playing with some audio libraries. Just maybe!
See ya!
11/04/2015
Wow, today was highly productive in regards to HPC. I came in with a question, and came out with an answer plus code to show! Today was a good day. So, while working on the lair-brary I was struggling finding a way where I could numerically sort the library's catalog. The issue being that each entry resides on 3 lines delimited by a newline character. The problem I ran into was trying to manipulate multiple lines of data in said way. So, I began to ask around which finally came down to the ultimatum… I have to learn how to use more advanced features with sed. I learned my first sed script today custome polished to be very own needs. :) Consider the following input!
FFFF BOOK TITLE AUTHOR NAME USERNAME DATE etc. FFFE BOOK TITLE AUTHOR NAME USERNAME DATE etc.
Supplemented with…
sed '/^[[:xdigit:]]* .*$/{ N s/\n/[-]/g N s/\n/[-]/g N s/\n/[-]/g }' input
Output looks like…
FFFF BOOK TITLE[-]AUTHOR NAME[-]USERNAME DATE etc.[-] FFFE BOOK TITLE[-]AUTHOR NAME[-]USERNAME DATE etc.
Sort!
FFFE BOOK TITLE[-]AUTHOR NAME[-]USERNAME DATE etc.[-] FFFF BOOK TITLE[-]AUTHOR NAME[-]USERNAME DATE etc.
Finally, revert back!
sed 's/\[-\]/\n/g' input
Final Output should look like this (note. techinally this example would not work due to the missing newline character on the last entry)
FFFE BOOK TITLE AUTHOR NAME USERNAME DATE etc. FFFF BOOK TITLE AUTHOR NAME USERNAME DATE etc.
More to come!
11/09/2015
So, the weekend was productive. In fact, it was the most productive weekend I have witnessed so far this semester. Tackled tic-tac-toe, and became best buddies with ncurses. I found the library to be extremely robust, but extremely accessible. Once I got the curses moving around the way I wanted the rest was history. I got sick of the lair-brary worrying about multiple user issues which led me straight to tic-tac-toe a single no more than two user experience… Maybe, I should make an undefeat-able AI?!
Things keep getting better, and better!
11/16/2015
It has been some time since my last submit. However, this time I bring good news. After mingling around on the internet, and obsessing over a potential new found hobby. Somebody dropped the line usb programming which immediately struck my fancy. What better of a project for datacomm and HPC? What really solidified this was “ioctl” a unix tool for interfacing with usb devices.
This will be where I will record all my resources and information. I hope it will be more interesting than some of my previous posts.
11/18/2015
USB/Bluetooth Notes: MAX devices 127 USB devices uses a 7bit address
“All communications on the bus are intiated by the host” There is no communication usb-device to usb-device.
“A device cannot initiate data transfering, but must wait to be asked to transfer data by the host.” The only exception being a 'suspend' that results in a remote wakeup.
USB On-The-Go, An interesting extension on the USB spec.
Cables specially designed to only connect host-to-device IE. not host-to-host, or device-to-device.
4 Wire shielded cable two of which are a twisted pair (D+, D-). The fourth wire (VBUS) provides 5V supply of power.
Additional Resources
Using libusbhid & libfcntl. Ran into libfcntl while reading somones source that reads input from a joystick. libfcntl or (file control) provides a means to accessing a file that is already open. Generally after a usb device is plugged in it is already being accessed by the host.
“A USB HID report descriptor is one of the descriptors that a USB host can request from a USB device. HID devices send data to the host using reports, and the descriptor tells the host how to interpret the data.”
/usr/share/misc/usb_hid_usages Shows ideal report descriptor formats for various usb devices.
12/02/2015
Back to USB. I've been looking in all the wrong places. The library I should have been focusing on was libusb which references all of the other libraries I have occasionally stared far too long at. Those of which were probably being referenced as a lower level layer by libusb.h.
Regarding libusb, When interfacing with usb devices you have two choices Synchronous, and Asynchronous. Synchronous offers a less involved method for interfacing with a USB device to make a transfer. Offering a “one function call” solution to make a transfer, but those calls may “sleep” inside libusb_bulk_transfer() until a transfer has completed. This could be a pain if a transfer process is taking longer than expected to complete. While Asynchronous offers a more controlled route with the ability to cancel a transfer, but is more complicated to use.
note: Asynchronous offers simultaneous I/O operations or threading. Which libusb supports with its thread-control.
USB Transfer Types
- Control Transfers
- Interrupt Transfers
- Isochronous Transfers
- Bulk Transfers
Control Transfers- Command or status operations.
Interrupt Transfers- Initiated by the device to request some action from the host.
Isochronous Transfers- Transfers which are used to carry data the delivery of which is crucial.
Bulk Transfers- Transfers which are used to carry data the delivery of which is not crucial.
“All transfers take the form as packets, which contain certain control information and error checking fields.”
The typical libusb procedure
- Get device list
- Clear device list with libusb_free_device_list)
- Claim usb interface with libusb_claim_interface
- Do the desired I/O operations
- Release the device by using libusb_release_interface
- Close the device libusb_close
- Exit session libusb_exit
