User Tools

Site Tools


haas:system:sokraits_halfadder.offbyone.lan

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
haas:system:sokraits_halfadder.offbyone.lan [2014/08/08 19:55] – [/etc/drbd.conf AFTER OCFS2 is ready] wedgehaas:system:sokraits_halfadder.offbyone.lan [2014/10/04 14:41] (current) – [rsync to fileserver] wedge
Line 4: Line 4:
  
 ^  hostname  ^  RAM  ^  disk  ^  swap  ^  OS  ^  Kernel  ^ ^  hostname  ^  RAM  ^  disk  ^  swap  ^  OS  ^  Kernel  ^
-|  sokraits.lair.lan  |  8GB  |  32GB (/)  |  1.2GB  |  Debian 8.0 "Jessie" (AMD64)  |  3.14-2-amd64  |+|  sokraits.lair.lan  |  8GB  |  32GB (/)  |  1.2GB  |  Debian 8.0 "Jessie" (AMD64)  |  3.16-2-amd64  |
 |  :::  |  :::  |  500GB + 500GB RAID1 (/dev/md0)  |  :::  |  :::  |  :::  | |  :::  |  :::  |  500GB + 500GB RAID1 (/dev/md0)  |  :::  |  :::  |  :::  |
  
 ^  hostname  ^  RAM  ^  disk  ^  swap  ^  OS  ^  Kernel  ^ ^  hostname  ^  RAM  ^  disk  ^  swap  ^  OS  ^  Kernel  ^
-|  halfadder.lair.lan  |  8GB  |  32GB (/)  |  1.2GB  |  Debian 8.0 "Jessie" (AMD64)  |  3.14-2-amd64  |+|  halfadder.lair.lan  |  8GB  |  32GB (/)  |  1.2GB  |  Debian 8.0 "Jessie" (AMD64)  |  3.16-2-amd64  |
 |  :::  |  :::  |  500GB + 500GB RAID1 (/dev/md0)  |  :::  |  :::  |  :::  | |  :::  |  :::  |  500GB + 500GB RAID1 (/dev/md0)  |  :::  |  :::  |  :::  |
  
Line 21: Line 21:
   * Re-re-reinstalled sokraits with Debian Wheezy -> Jessie, due to failed boot drive (20140806)   * Re-re-reinstalled sokraits with Debian Wheezy -> Jessie, due to failed boot drive (20140806)
   * Re-installed halfadder with Debian Jessie (20140806)   * Re-installed halfadder with Debian Jessie (20140806)
 +  * Re-re-re-resetup sokraits as a clone of halfadder and netbooting with entire system in initrd, due to (another) failed boot drive (20141004)
 =====TODO==== =====TODO====
-  * rig up ramdisk /var and /tmp w/ periodic writes (since we have an SSD /).+  * <del>rig up ramdisk /var and /tmp w/ periodic writes (since we have an SSD /).</del> System runs in a RAMdisk.
   * find 3.5" to 5.25" drive brackets and remount sokraits data drives in case.   * find 3.5" to 5.25" drive brackets and remount sokraits data drives in case.
   * on next halfadder reboot, verify that OCFS2 /export gets mounted automatically (last time I had to run "/etc/init.d/ocfs2 restart" for it to do this).   * on next halfadder reboot, verify that OCFS2 /export gets mounted automatically (last time I had to run "/etc/init.d/ocfs2 restart" for it to do this).
Line 122: Line 123:
 =====apt configuration===== =====apt configuration=====
  
 +====use LAIR apt proxy====
 +To reduce traffic caused by package transactions, I set up a proxy on the fileserver(s), so every client will need to configure itself appropriately. Turns out this can be done most easily by creating the file **/etc/apt/apt.conf.d/00apt-cacher-ng** and putting in the following contents:
 +
 +<code>
 +Acquire::http { Proxy "http://10.80.1.3:3142"; };
 +</code>
 +
 +====no recommends====
 I wanted a small installation footprint, so I disabled the installation of recommended packages by default. I wanted a small installation footprint, so I disabled the installation of recommended packages by default.
  
Line 157: Line 166:
   smartmontools   smartmontools
   firmware-realtek   firmware-realtek
 +  qemu-system-x86-64
  
 =====GRUB Configuration===== =====GRUB Configuration=====
Line 269: Line 279:
 ## ##
 arch            = amd64 arch            = amd64
-#mirror         = http://mirror/debian/ 
 mirror          = http://ftp.us.debian.org/debian/ mirror          = http://ftp.us.debian.org/debian/
 mirror_squeeze  = http://ftp.us.debian.org/debian/ mirror_squeeze  = http://ftp.us.debian.org/debian/
-#mirror_squeeze = http://mirror/debian/ +mirror_wheezy   = http://ftp.us.debian.org/debian/ 
-mirror_wheezy   = http://mirror/debian/ +mirror_jessie   = http://ftp.us.debian.org/debian/ 
-#mirror_wheezy  = http://ftp.us.debian.org/debian/ + 
-#mirror_jessie  = http://ftp.us.debian.org/debian/ +###################################################################### 
-mirror_jessie   = http://mirror/debian/+## 
 +## Proxy Settings for repositories 
 +## 
 +apt_proxy       = http://10.80.1.3:3142/
  
 ###################################################################### ######################################################################
Line 542: Line 554:
 With the tools installed, we need to configure some files. With the tools installed, we need to configure some files.
  
-====/etc/drbd.conf BEFORE==== +====/etc/drbd.d/global_common.conf BEFORE==== 
-First up, we need to get the peers talking so we can form the volume and get OCFS2 established. Let's make the **/etc/drbd.conf** look as follows:+First up, we need to get the peers talking so we can form the volume and get OCFS2 established. Let's make the **/etc/drbd.d/global_common.conf** file look as follows:
  
 <code> <code>
Line 571: Line 583:
     protocol C;     protocol C;
 } }
 +</code>
  
 +This is only an intermediate step. Further changes are needed before we can bring it up in dual-primary mode.
 +
 +====/etc/drbd.d/xen_data.res====
 +And the resource configuration (doesn't need to change), on both peers:
 +
 +<code>
 resource xen_data resource xen_data
 { {
 +    device      /dev/drbd0;
 +    disk        /dev/md0;
 +    meta-disk   internal;
 +
     on sokraits     on sokraits
     {     {
-        device      /dev/drbd0; +        address     172.16.1.1:7788;
-        disk        /dev/md0; +
-        address     172.16.2.1:7789; +
-        meta-disk   internal;+
     }     }
  
     on halfadder     on halfadder
     {     {
-        device      /dev/drbd0; +        address     172.16.1.2:7788;
-        disk        /dev/md0; +
-        address     172.16.2.2:7789; +
-        meta-disk   internal;+
     }     }
 } }
 </code> </code>
- 
-This is only an intermediate step. Further changes are needed before we can bring it up in dual-primary mode. 
- 
 ====bootstrapping DRBD==== ====bootstrapping DRBD====
 Getting DRBD initially up-and-running has always has a bit of voodoo behind it... trying a number of commands and eventually stumbling upon something that works. I may have finally gotten the procedure down: Getting DRBD initially up-and-running has always has a bit of voodoo behind it... trying a number of commands and eventually stumbling upon something that works. I may have finally gotten the procedure down:
Line 773: Line 787:
 </code> </code>
  
-This **/etc/drbd.conf** file needs to be identical and present on BOTH DRBD peers.+This **/etc/drbd.d/global_common.conf** file needs to be identical and present on BOTH DRBD peers. 
 + 
 +Recognizing the changes does not require a reboot! The following command (run on both DRBD peers), will update the config: 
 + 
 +<cli> 
 +machine:~# drbdadm adjust xen_data 
 +</cli>
 ====Bringing OCFS2 online==== ====Bringing OCFS2 online====
 Assuming **/etc/ocfs2/cluster.conf** and **/etc/default/o2cb** are configured and identical, we can now establish OCFS2 cluster connectivity. Assuming **/etc/ocfs2/cluster.conf** and **/etc/default/o2cb** are configured and identical, we can now establish OCFS2 cluster connectivity.
Line 858: Line 878:
  
  
-====Configuring xen-tools to create Debian jessie VMs====+====Historical: Configuring xen-tools to create Debian jessie VMs==== 
 +**This is no longer needed, but may well be in the future.** 
 There are two changes needed to successfully create jessie VMs, and both are symlinks: There are two changes needed to successfully create jessie VMs, and both are symlinks:
  
Line 1086: Line 1108:
  
 Wait at least 5 minutes for data reporting to make it into graphable form. Wait at least 5 minutes for data reporting to make it into graphable form.
 +
 +=====Sync'ing to data store=====
 +Since we've been successful running the systems out of a RAMdisk, care must be taken to preserve any changes in the event of a reboot or power failure.
 +
 +====rsync to disk====
 +
 +In this light, I first had the systems rsync'ing to their local SSD (boot drive). I rigged up a custom cronjob than ran 3 times a day. It looks as follows:
 +
 +<code>
 +12 */8 *         (mkdir -p /tmp/sda1; mount /dev/sda1 /tmp/sda1; rsync -av --one-file-system / /tmp/sda1/; umount /tmp/sda1)
 +</code>
 +
 +====rsync to fileserver====
 +
 +This worked handily until sokraits lost its boot drive (again! In 2 months time!) so I decided to investigate netbooting using an NFSroot.
 +
 +In the process, I may have finally made a breakthrough in my longtime desire to put the entire system IN the initial ramdisk (so it would be running in RAM from the get-go). Turns out, according to the manual page, you merely have to put the system IN the initrd file... obviously one needs adequate memory (2x at boot- enough for the initrd, and enough to decompress it).
 +
 +My cron job changed as follows:
 +
 +<code>
 +24 */8 *         (rsync -av --one-file-system / data.lair.lan:/export/tftpboot/netboot/halfadder/disk/)
 +</code>
 +
 +I plan to rig up either some daily autogeneration of the initrd, or have a script on standby that can use to make it. This will then become the method of booting both sokraits and halfadder (and potentially freeing up a still-working SSD in the process! Which I can use in data2).
 +
 +On the fileserver, I then obtain the latest copy of the hypervisor, kernel, and generate a new all-system initrd:
 +
 +<cli>
 +data1:/export/tftpboot/netboot/halfadder# cp disk/boot/xen-4.4-amd64.gz .
 +data1:/export/tftpboot/netboot/halfadder# cp disk/boot/vmlinuz-3.16-2-amd64 linux
 +data1:/export/tftpboot/netboot/halfadder# cd disk
 +data1:/export/tftpboot/netboot/halfadder/disk# find . | cpio -c -o | gzip -9 > ../initrd.gz
 +data1:/export/tftpboot/netboot/halfadder/disk# 
 +</cli>
 +
 +====pxeboot file for sokraits/halfadder====
 +On the fileserver, in **/export/tftpboot/pxelinux.cfg/** are two files, **0A50012E** (sokraits) and **0A50012F** (halfadder)... they are named according to the machine's IP address (only in hex).
 +
 +The file(s) contain:
 +
 +<code>
 +default netboot
 +prompt 1
 +timeout 2
 +
 +label netboot
 +    kernel mboot.c32
 +    append netboot/halfadder/xen-4.4-amd64.gz --- netboot/halfadder/linux console=tty0 root=/dev/ram0 ro --- netboot/halfadder/initrd.gz
 +
 +label memtest
 +    kernel distros/memtest/memtest86+
 +</code>
  
 =====References===== =====References=====
Line 1120: Line 1195:
 ====DRBD+OCFS2==== ====DRBD+OCFS2====
   * http://www.clusterlabs.org/wiki/Dual_Primary_DRBD_%2B_OCFS2   * http://www.clusterlabs.org/wiki/Dual_Primary_DRBD_%2B_OCFS2
 +
 +====Debian from RAM====
 +  * http://reboot.pro/topic/14547-linux-load-your-root-partition-to-ram-and-boot-it/
 +  * debirf: 
 +    * http://cmrg.fifthhorseman.net/wiki/debirf
 +    * http://www.sphaero.org/blog:2012:0114_running_debian_from_ram
 +
 +====/tmp as noexec====
 +  * http://www.debian-administration.org/articles/57
 +
 +====netboot system to nfsroot====
 +  * http://www.iram.fr/~blanchet/tutorials/read-only_diskless_debian7.pdf
 +    * this led me to the initrd man page which indicated we might be able to stick the entire system in the initrd and PXE boot that. So many things become simpler at that point.
haas/system/sokraits_halfadder.offbyone.lan.1407527756.txt.gz · Last modified: 2014/08/08 19:55 by wedge