Differences

This shows you the differences between two versions of the page.

--- haas:spring2016:unix:projects:udr1 [2015/03/14 18:27] – external edit 127.0.0.1
+++ haas:spring2016:unix:projects:udr1 [2016/03/15 13:37] (current) – [Partition Layout] wedge
@@ Line 11: / Line 11: @@
 Typos and bug fixes:
-  * Please see the **README** file in the project directory (under the Public Directory)... I have offered up some optimizations that should reduce some of the common errors some are experiencing, along with a special version of data.file that, with a little extra prep-work, will allow for much faster processing. (20150314)
+  * <description> (DATESTAMP)
 =====Objective=====
-Practice and review your UNIX skills.
+Use your UNIX skills and tools at hand to enable you to solve a problem in the realm of raw data management and data recovery, this time involving hard drives and partition tables.
-=====Background=====
+=====Backstory=====
-The last project introduced us to many important concepts. It threw many people for a loop- it is still throwing some for a loop.
+It always starts the same way: you torrent a cracked version of a newly released game you've been pining for, neglecting to check for embedded malware. This time it was the pinnacle of human programming achievement- the Mudkip Simulator (move over goat simulator!) Oh, and you were sneaking internet access on the chief meme archivist's main workstation. Whoops.
-But the concepts of offsets and quantities of data at an offset is an important thing to understand. With that, udr1 continues to play in that exciting realm, offering you another chance at getting more familiar with some very powerful UNIX tools.
+Turns out this one had a few tricks up its sleeve. Of particular note: partition scrambling. Oh no! All the memes!
-For those who have had the fortune of interacting with sequential access media (tape), you may have a natural advantage in being able to more quickly grasp the abstract concepts at work here. But even if you haven't-- these concepts are everywhere, especially on the computer, where we see it in the way software and operating systems work, to how hardware is organized and communicates with the rest of the system.
+After downing two Mr. Pibbs, a Mars bar, and a pack of Necco Wafers (saving the purple ones for last), you sit down to correct your most grievous of oversights. Chiptunes blazing, gaming flash drive at the ready, and extra monitors displaying terminals covering nearly every square decimeter of desk space, you have a hard drive partition map to restore.
-I am also having you review your file permissions skills through the use of a tool called **urev**. However, it is currently unable to run, because it lacks a "gizmo", which is contained within the **data.file** you will be operating on.
+Turns out there was at least **some** redundancy... a backup of the partition table/master boot record for the drive was stored somewhere in the first part of the drive. A little digital forensics and you should be on your way...
-Extract and process **gizmo** appropriately, and then you can run **urev** and complete your file permission exercises (you have a week to do 96 exercises, but you are limited by how many you can do in one sitting, and there's a two hour time limit between sessions, so you'll also have to practice some time management).
+You need to recover the partition table, restore the disk to its accessible state, and copy pertinent files onto one of its partitions. Basic file management never seemed so enthralling.
-This is also intended for questions to be asked. I have have not reached critical mass on people asking questions for things.
+Time to dust off that hexadecimal, sharpen your dd skills, and prepare for offset-calculating awesomeness: **udr1** is ready to enhance your day!
+=====Background=====
+Your data, as we experience it today, is organized in units of files (regular files, directory files, special files). Turns out, that is further organized into things called inodes and superblocks, software organizational structures that make up the filesystem, which organizes your data.
+Below that, we have the notion of sectors (units of transactions), that are the often de-facto unit of measurement.
+So here, below the file system, in fact, what helps establish boundaries for the filesystem that we conveniently use everyday, is another organizational scheme, one that sets boundaries for where our data can reside on media. We call these partitions. And the partitions are laid out in a partition table.
 =====Obtain the file=====
-This week's project is located in the **spring2015/udr1/** directory of the UNIX Public Directory, in a file called: **data.file**
+This week's project is located in the **udr1/** subdirectory of the UNIX Public Directory.
-Make a copy of this into your home directory somewhere and set to work.
+You need to go there and run a program called **prep**, by providing it the absolute path to the directory you've made to store your files for the **udr1** project.
+For example, if you've made a **src/unix/projects/udr1/** directory, you can say:
+<cli>
+lab46:/var/public/spring2016/unix/udr1$ ./prep /home/$USER/src/unix/projects/udr1
+Processing disk.image .........................................
+MBR backup is at offset 0xXXXXXX
+Process complete. Your file is in '/home/$USER/src/unix/projects/udr1/disk.image'
+lab46:/var/public/spring2016/unix/udr1$
+</cli>
+A couple of things to note:
+  * the file you need, **disk.image** will be copied into the directory of your choosing. The **directory** must exist before you instruct **prep** to place it there (otherwise you'll get an error).
+  * You'll get an offset for the "**MBR backup**"... write this down! You'll need it to progress further. The MBR backup is located in a contiguous region spanning exactly 127 bytes (that's in decimal).
-**NOTE:** Hopefully it has been standard practice to locate project files in their own unique subdirectory, such as under **src/unix/**, where you can then add/commit/push the results to your repository (you ARE regularly putting stuff in your repository, aren't you?)
 =====Process=====
-The data you seek (2 files) is obfuscated and contained within this file.
+In order to gain access to the drive (in the form of the file called **disk.image**, which is 5MB in size), you'll need to restore the partition table/Master Boot Record (MBR).
-Plain text directions give clues on how to find both pieces of information, and it is up to you to use your skills to extract the necessary data.
+There is a backup stored on the drive... you were informed of its location when you successfully ran **prep**. Obtain and unravel it.
-Some additional information:
+Once you have it, you'll want to restore it onto the drive; it needs to occupy the first sector on the drive.
-  * The first file should be named **udr1.text** and be properly oriented.
+Once you have that in place, we can use a hex editor to study the partition table and determine the starting sectors and overall sizes of each partition.
-  * The second (big) file runs from the starting point until the very end of the file
-  * It should be named 'gizmo', and reside in your current working directory.
+You'll want to extract each partition individually from the disk image so you can better interact with it. You'll need to determine the partition type and ensure a viable filesystem is in place (think back to the puzzle boxes). Ultimately you'll want to be able to view and copy data to/from the various partitions.
-  * gizmo is binary data, and entirely reversed- you need to get its bytes back in order (last byte should be first byte, 2nd to last should be 2nd, etc.)
-    * You are to write a shell script to perform the de-reversal of the data, reading from data.file and through whatever processing is needed, produce the file called **gizmo**.
+To facilitate your efforts, please name each extracted partition files as follows:
-  * The **urev** tool has some additional constraints with respect to gizmo... running it should notify you of any details you are lacking.
+  * partition #1: **udr1.part1.img**
+  * if there's a partition #2: **udr1.part2.img**
+  * if there's a partition #3: **udr1.part3.img**
+  * if there's a partition #4: **udr1.part4.img**
+As this is a disk image of the chief meme archivist's workstation, you'll also need to do a little file organization to bring things up to spec (and ensure they never realized what happened); specifically, placing all mudkip and reptilian-themed files on the fourth partition.
+Finally, reintegrating the changed fourth partition over the old one, so that you can submit the whole restored image, along with a list of command-lines you performed (and other information gathered) to accomplish this task.
 =====Useful tools=====
 You may want to become familiar with the manual pages of the following tools (in addition to tools you've already encountered):
-  * **dd**(1)
+  * **dd**(1) -- especially the **conv=notrunc** option
   * **bc**(1)
-  * **du**(1)
+  * **mtools**(1)
-  * **bash**(1) shell scripting
-  * **od**(1)
-  * **bvi**(1)
   * **hexedit**(1)
-... along with other tools previously encountered.
+The aim is for this entire project to be performed on lab46. No external resources/tools are required. Like **udr0**, this project deals with binary data, something that is likely still throwing many of you for a conceptual loop. The more you play with this, the more it will make sense.
+Be sure to ask questions! There are bound to be many.
+=====Partition Access=====
+The partitions are defined in the partition table, located near the end of the first sector of the drive.
+Of particular note, you're interested in groups of 16 bytes starting at offset **0x1BE**.
+You may want to consult a reference of the binary layout of the MBR/partition table. An excellent source of information can be found here:
+  * http://thestarman.pcministry.com/asm/mbr/PartTables.htm
+====Partition Layout====
+A partition is defined in 16 bytes (the classic PC layout allows for four primary partitions). The byte layout is as follows:
+^  Byte  ^  Description  |
+|  0  |  Boot Indicator (0x80 indicates it is set bootable)  |
+|  1  |  Starting CHS values (you can ignore for this project)  |
+|  2  |  :::  |
+|  3  |  :::  |
+|  4  |  Partition Type ID  |
+|  5  |  Ending CHS values (you can ignore for this project)  |
+|  6  |  :::  |
+|  7  |  :::  |
+|  8  |  Starting Sector (little endian format)  |
+|  9  |  :::  |
+|  10  |  :::  |
+|  11  |  :::  |
+|  12  |  Partition Size (in Sectors)  |
+|  13  |  :::  |
+|  14  |  :::  |
+|  15  |  :::  |
+====Partition Types====
+<code>
+  Empty           24  NEC DOS         81  Minix / old Lin bf  Solaris
+  FAT12           27  Hidden NTFS Win 82  Linux swap / So c1  DRDOS/sec (FAT-
+  XENIX root      39  Plan 9          83  Linux           c4  DRDOS/sec (FAT-
+  XENIX usr       3c  PartitionMagic  84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
+  FAT16 <32M      40  Venix 80286     85  Linux extended  c7  Syrinx
+  Extended        41  PPC PReP Boot   86  NTFS volume set da  Non-FS data
+  FAT16           42  SFS             87  NTFS volume set db  CP/M / CTOS / .
+  HPFS/NTFS/exFAT 4d  QNX4.x          88  Linux plaintext de  Dell Utility
+  AIX             4e  QNX4.x 2nd part 8e  Linux LVM       df  BootIt
+  AIX bootable    4f  QNX4.x 3rd part 93  Amoeba          e1  DOS access
+ a  OS/2 Boot Manag 50  OnTrack DM      94  Amoeba BBT      e3  DOS R/O
+ b  W95 FAT32       51  OnTrack DM6 Aux 9f  BSD/OS          e4  SpeedStor
+ c  W95 FAT32 (LBA) 52  CP/M            a0  IBM Thinkpad hi eb  BeOS fs
+ e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a5  FreeBSD         ee  GPT
+ f  W95 Ext'd (LBA) 54  OnTrackDM6      a6  OpenBSD         ef  EFI (FAT-12/16/
+  OPUS            55  EZ-Drive        a7  NeXTSTEP        f0  Linux/PA-RISC b
+  Hidden FAT12    56  Golden Bow      a8  Darwin UFS      f1  SpeedStor
+  Compaq diagnost 5c  Priam Edisk     a9  NetBSD          f4  SpeedStor
+  Hidden FAT16 <3 61  SpeedStor       ab  Darwin boot     f2  DOS secondary
+  Hidden FAT16    63  GNU HURD or Sys af  HFS / HFS+      fb  VMware VMFS
+  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fc  VMware VMKCORE
+  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fd  Linux raid auto
+b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid fe  LANstep
+c  Hidden W95 FAT3 75  PC/IX           be  Solaris boot    ff  BBT
+e  Hidden W95 FAT1 80  Old Minix
+</code>
 =====Submission=====
 Successful completion will result in the following criteria being met:
-  * Resulting file with proper settings should enable you to run **urev** tool.
+  * When all is said and done, you will submit 2 files to me:
-  * You have completed all weekly exercises (96, I think) before the deadline, being mindful of the intentionally-paced nature of urev.
+    * The final, modified **disk.image** file, which:
-    * Bonus opportunity: while still performing a minimum of 3 distict **urev** sessions, how could you get around the urev-imposed time limit? (Without copying/changing urev).
+      * has the backup MBR restored to the proper place
-  * When all is said and done, you will submit 3 files:
+      * has the mudkip/reptilian files copied into the fourth partition, and that fourth partition is merged back into the disk image.
-    * **udr1.text**
+    * **steps.txt**, which contains:
-      * Append the dd line(s) as well as any other command lines needed to extract and properly re-orient the file. Also be sure to indicate what is in the file you found (content, not just type of data).
+      * a list of command-lines used to accomplish every step of this project from start to finish (excluding the successful run of **prep** and the successful **submit**).
-    * your bash script enabling the processing of data.file to produce gizmo
+      * explanations of the commands, as well as how you derived any numbers used.
-      * Be sure to include comments indicating the reasoning behind actions taken
+      * type ID (in hex as well as descriptive text) of each partition
-    * Your extracted/processed **gizmo** file
+      * which partition is marked as bootable?
+      * total size (in bytes and sectors) of the MBR/partition table region
+      * total size of each partition (as defined in the partition table)
+      * total formatted size of each filesystem
+      * amount of space used in each filesystem (when all is said and done)
+      * amount of space free in each filesystem (when all is said and done)
+      * the disk label of each partition/filesystem.
+      * what file(s) are present in the fourth partition upon copying all mudkip/reptilian-themed files?
+      * what file(s) are left out?
 ====Submit====
@@ Line 80: / Line 175: @@
 <cli>
-lab46:~/src/unix/udr1$ submit unix udr1 udr1.text getgizmo.bash gizmo
+lab46:~/src/unix/udr1$ submit unix udr1 disk.image steps.txt
 Submitting unix project "udr1":
-    -> udr1.text(OK)
+    -> disk.image(OK)
-    -> getgizmo.bash(OK)
+    -> steps.txt(OK)
-    -> gizmo(OK)
 SUCCESSFULLY SUBMITTED
 lab46:~/src/unix/udr1$
 </cli>

Lab46 Wiki

User Tools

Site Tools

Differences

Page Tools