• the formular plugin is giving me errors, need to figure this out (email assignment form)
• can I install writer2latex on wildebeest herd without needing gcj??
• look into how to adequately set up subversion svn+ssh:// authentication using http:// access now
• set up symlink and cron job for userhomedirbackup.sh on nfs2; update wiki page for nfs
• update grade not-z scripts to handle/be aware of winter terms
• update system page for (new)www

Some links of interest:

• LLVM
• GLglobe
• Heavens Above
• http://wiki.debian.org/kristian_jerpetjoen
• http://www.webupd8.org/2010/11/alternative-to-200-lines-kernel-patch.html
• http://myproxylists.com/nix-brute-force
• My Plan9 Documentation page
• The Sokraits/Halfadder System Documentation page
• http://www.egghelp.org/commands/channels.htm
• http://stackoverflow.com/questions/470617/get-current-date-and-time-in-php
• http://www.rjsystems.nl/en/2100.php

With some recent developments figuring out how to do XDMCP once again on the USB-enabled pods, I've re-awakened the full wildebeest herd, and aptitude update; aptitude update'd the whole lot… updated fluxbox menus, and cleaned package cache.

Just need to configure the flakes themselves to divvy out the XDMCP host connections.

After a few days, we've finally figured out how to get decently reliable XDMCP client connections going on the Ubuntu USB-terminal flake stations.

I'll be posting more particulars as I configure everything in the coming days.

In my quest to have 4 dual-head LAIRstation machines for student use, I now have 3 of them up and operational… all running Ubuntu 10.10, autofs+LDAP, along with automatic Nvidia twinview going on boot.

I'll be setting up LAIRstation 4 in the coming days, especially as the great re-LAIR-rangement continues and dust settles.

It would appear that, by default, when web viewing an SVN repository, viewing an HTML will retrieve and display the HTML code, instead of passing it to the browser to render.

I've looked a bit into this, and there appear to be solutions… I tried one or two promising looking ones, but did not get instant gratification, so I removed the options.

A link that appears to talk about what I want (for websvn, which may ultimately be the way to go):

• http://old.nabble.com/-WebSVN--mixed-anon-auth-authentication-and-login-logoff-td28035968.html

I encountered a spiffy-looking webmail client called roundcube, which was easily installed on www.

It is now up and running (just point your browser at: http://lab46.corning-cc.edu/mail/).

I encountered one problem that has been fixed— upon successful log in, you'd immediately be logged out (you'd see your inbox, then wham! login screen).

Turns out this is a common issue:

• http://www.roundcubeforum.net/5-release-support/17-pending-issues/6172-another-immediate-log-out-after-log-problem.html
• http://www.roundcubeforum.net/3-news-announcements/12-general-discussion/6196-login-page-doesnt-take-me-anywhere.html

As it turns out, the solution was to disable session encryption for the suhosin php functionality.

I ended up having to add the following to the sites-enabled/000-default file:

Alias /mail/program/js/tiny_mce/ /usr/share/tinymce/www/
Alias /mail /var/lib/roundcube

In preparation for the new semester, I performed updates on the following machines:

• lab46
• irc
• www
• auth1, auth2, auth3
• sokraits, halfadder

I rebooted www (actually shutdown then xm create), and gave it more memory– it now has 384MB of memory.. it was hitting swap before and making some web lookups rather slow.

I need to put in extra checks to ensure grade not-z scripts do not run prior to the start of the semester… assignments, journals, etc.

attendance is probably the only thing smart enough at the moment to know better, yet doesn't offer anything that the other functionalities can immediately lock onto to ensure that we're in a valid semester and week or not.

Actually, as it turns out, the problem was with calcweek.sh, whose logic couldn't handle the new year (it would compute the absolute day from the start of the year, and do a comparison against the absolute day of the semester year)… yielding some impressive negative numbers.

I have put in a simple check for the year change, and if detected, merely defaults to a “week 0” status which will cause all the other scripts to resume cooperating.

I automated the process of disclosing Lab46 account information to new users. The following script(s) were used:

1

for entry in `cat readylist`; do
    username="`echo $entry | cut -d':' -f1`"
    first="`echo $entry | cut -d':' -f2 | cut -d'*' -f1`"
    password="`echo $entry | cut -d':' -f3`"
    echo "$first," >> $username.letter
    echo >> $username.letter
    echo "Enclosed please find your Lab46 account information. This will be" >> $username.letter
    echo "used to log onto the Lab46 system for use with your classes this" >> $username.letter
    echo "semester. If you have any questions please let me know at: " >> $username.letter
    echo >> $username.letter; echo "                    haas@corning-cc.edu" >> $username.letter
    echo >> $username.letter; echo "-------------------------------------------" >> $username.letter
    echo "    Lab46 username: $username" >> $username.letter
    echo "    Initial Password: $password" >> $username.letter
    echo "    Hostname: lab46.corning-cc.edu" >> $username.letter
    echo "-------------------------------------------" >> $username.letter
    echo >> $username.letter
    echo "-Matthew" >> $username.letter
    echo "--" >> $username.letter
    echo "  Matthew Haas" >> $username.letter
    echo "  Instructor" >> $username.letter
    echo "  Computer & Information Science" >> $username.letter
    echo "  http://lab46.corning-cc.edu/haas/home/" >> $username.letter
    echo >> $username.letter
    echo "    \"Writing should be like breathing;" >> $username.letter
    echo "      it is one of those important things we do.\" --me" >> $username.letter
done

The above script generated the letters (from my generated username file).

On Lab46 I then fired them off as e-mails with the following:

1

for letter in `/bin/ls -1A *.letter`; do
    username="`echo $letter | cut -d'.' -f1`"
    mail ${username}@corning-cc.edu wedge@lab46.corning-cc.edu -s "Lab46 Account Information" < $letter
done

There was apparently a long-standing bug in the student pages generation script, which caused some duplicates to appear.

It was one of those classic “similar substring” problems, but this time in the behavior of one of the tools used.

Original logic:

 16 # Gather some metrics
 17 for user in `/bin/ls -1 /home`; do
 18     uchk="`groups $user | egrep -o '(lair|offbyone|faculty|dslab)' | wc -l`"
 19     if [ $uchk -eq 0 ]; then
 20         fullname="`finger $user | grep -o 'Name.*$' | sed -e 's/Name: \(.*\)    $/\1/'`"
 21         jchk="`groups $user | egrep -o "$classgrp" | wc -l`"

And fixed it by adding a head -1 to the pipeline after the call to finger, as follows:

 16 # Gather some metrics
 17 for user in `/bin/ls -1 /home`; do
 18     uchk="`groups $user | egrep -o '(lair|offbyone|faculty|dslab)' | wc -l`"
 19     if [ $uchk -eq 0 ]; then
 20         fullname="`finger $user | head -1 | grep -o 'Name.*$' | sed -e 's/Name: \(.*\)    $/\1/'`"
 21         jchk="`groups $user | egrep -o "$classgrp" | wc -l`"

Problem solved.

My journalinit.sh script apparently had some bug in it (I think a remnant of the [A-Z] LOCALE problems) that allowed the fullnames to be mixed in with the usernames, creating an inotify watch list that was about 3 times larger than it should have been.

A simple fix (restoring the regex to a sane [^A-Z] value) restored proper functionality.

Also I removed the in-script semester-detection logic and replaced it with a call to semester.sh

Using my notes taken on the wiki from the spring and fall of 2010 semesters, I finally created a script to handle LDAP group membership for class-related purposes.

The script, gengrps, currently has the following contents:

1

#!/bin/bash                                                                               
#
# gengrps - script to assist with the generation of LDIF files for
#           populating class LDAP groups
#
# If run by itself, all LDIF definitions will be displayed to STDOUT.
# Ideal method of production execution:
#
#       gengrps | ldapmodify -x -W -D "cn=admin,dc=lair,dc=bits"
#
ATTNDPATH="/home/wedge/local/attendance/etc/list"
semester="`/usr/local/bin/semester.sh`"
LDAPINFO="dc=lair,dc=bits"
 
for group in `/bin/ls -1A ${ATTNDPATH}/class.${semester}.* | cut -d'.' -f3`; do
    grouplst="$grouplst $group"
done
 
for grp in $grouplst; do
    if [ "$grp" = "hpc1" ]; then
        echo "dn: cn=hpc,ou=groups,${LDAPINFO}"
    elif [ "$grp" = "hpc2" ]; then
        echo "dn: cn=hpc,ou=groups,${LDAPINFO}"
    else
        echo "dn: cn=${grp},ou=groups,${LDAPINFO}"
    fi  
 
    echo "changetype: modify"
    echo "add: memberUid"
 
    for student in `grep -v '^[A-Z]' ${ATTNDPATH}/class.${semester}.${grp}.*`; do
        echo "memberUid: ${student}"
    done
    echo
done
 
for student in `cat ${ATTNDPATH}/class.${semester}.* | grep '^[^A-Z]' | sort -u`; do
    echo "dn: cn=${student},ou=groups,${LDAPINFO}"
    echo "changetype: modify"
    echo "add: memberUid"
    echo "memberUid: wedge"
    echo
done
 
exit 0

Seems to work quite well, and when used in a pipeline with ldapmodify, operates without the need to directly create any temporary files.

The next step (at some point) will be to create a script that employs similar logic, but is used to remove group membership, such as at the end of the semester (a process I still performed manually this time around).

I took a closer look at the various scripts and facilities that need to be in place for proper grade not-z operation.

This began with getting the journal wiki pages set up.

I basically created a spring2011/ directory in the appropriate place in the directory tree, and put a start.txt and placeholder.txt there. I edited the ACLs and basically replaced the spring2010 entry with spring2011 (so existing spring2010 journals will still exist but nobody will have any permissions other than read at this point).

As the semester kicks off, I need to fire up journalinit.sh in a screen session on www, which will monitor for new journal creations. I don't want to do this now, as registration lists may still endure some changes. I should look into making it resilient enough to handle that sort of dynamic environment.

I also initialized the student data directories, which will be used by the grade not-z for collecting course metrics on the students.

I backed up the existing one (to data.fall2010) just in case I'd have a need for it, then cleared the contents of data and reinitialized it as follows (as root):

for user in `cat /home/wedge/local/attendance/etc/list/class.spring2011.* | grep -v '^[A-Z]' | sort | uniq`; do
    mkdir /home/wedge/local/data/$user
    chown -R wedge:$user /home/wedge/local/data/$user
    chmod 1770 /home/wedge/local/data/$user
done

SVN appears to be fully functional to my expectations at this point… after some further tests, both project and user repositories are accessible with appropriate permissions.

The final bit was the inclusion of my ability to check out each users' repository's submit directory (read-only), which I plan to use for various assignments.

The code to do that is as follows:

[/submit]
wedge = r

Note the lack of any repository reference… this is global for all repositories (that succumb to this ACL, which is just the user repositories).

Continuing my configuration adventures with web-based svn, I am now reading through the SVN book.

I now have functioning web-based SVN access working, with access control!

I rolled this out for the backgammon, cpu, and mgough's bigworld project. I'm going to do mostly similar for the private user repositories (I think I'm going to end up placing them under /svn/* on the URL namespace).

I needed to generate the ACL information on a per-user basis (as far as I can tell)… I wrote the following script to assist:

for entry in `/bin/ls -1A`; do
    valid="`id $entry 2> /dev/null | wc -l`"
    if [ "$valid" -gt 0 ]; then
        echo "[$entry:/]" >> /var/repos/acl/users.acl
        echo "$entry = rw" >> /var/repos/acl/users.acl
        echo >> /var/repos/acl/users.acl
        echo "[$entry:/submit]" >> /var/repos/acl/users.acl
        echo "wedge = r" >> /var/repos/acl/users.acl
        echo >> /var/repos/acl/users.acl
    fi
    echo "processed $entry ..."
done

In a similar vein, I discovered there were a lot of lingering repositories of no-longer-existing users… I decided to prune them while I was in the neighborhood. Thanks to the help of the following script:

for entry in `/bin/ls -1A`; do
    valid="`id $entry 2> /dev/null | wc -l`"
    if [ "$valid" -eq 0 ]; then
        rm -rvf $entry
    fi
done

I also updated the newuserbatch script on lab46db to reflect this new information to facilitate things on new user creation.

I attempted to set up https, which I was making progress on, but initial reactions appeared as if port 443 was not in fact open via the campus connection.

Some informative links:

• http://www.debian-administration.org/articles/31
• http://www.debian-administration.org/article/349/Setting_up_an_SSL_server_with_Apache2

Unrelated to SSL, more related to path-based authentications in SVN:

• http://svnbook.red-bean.com/nightly/en/svn.serverconfig.pathbasedauthz.html
• http://svnbook.red-bean.com/nightly/en/svn.serverconfig.httpd.html

Inching progressively closer to deploying class syllabi… updated pertinent class informations, created class attendance list files.

At current values, I have 52 total students, and 39 total unique students. It would also appear out of those 39 students, 16 are brand new to Lab46, as they do not yet have accounts.

The following script I wrote to parse through the attendance list information and generate a script-friendly input file for my lab46 user addition facilities:

for user in `cat *spring2011* | sed 's/ /*/g' | tr '\n' ':' | sed 's/:\([A-Z]\)/\n\1/g' | sed 's/:$//g' | sort | uniq`; do
    username="`echo $user | cut -d':' -f2`"
    id $username > /dev/null 2> /dev/null || echo $user | sed 's/^\(.*\):\(.*\)/\2:\1/g' | sed 's/\*/ /g'
done

With the upgrade to LDAP and the merging of repos into www, I had to update the newuserbatch script on lab46db.

I have successfully tested it with the list of new students for the semester.

In the process, I worked out a few bugs that had existed in the script for quite some time.

Slowly making progress on webdav+svn … I think yesterday I got reading successfully working, and tonight I got writing.

The problem is, I still need to get access control implemented.

I have it authenticating against pam, but I need a way to restrict access per repository to certain users or groups.

Some links that have information:

• http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=394097
• http://www.debian-administration.org/articles/208
• http://www.linuxforu.com/previews/subversion-16-security-improvements-illustrated/
• http://www.linuxjournal.com/article/7655?page=0,3
• http://www.howtoforge.com/debian_subversion_websvn
• http://www.gentoo-wiki.info/HOWTO_Apache2_with_subversion_SVN_and_DAV

I continued the migration of functionality over to the new www… copied scripts and data from /usr/local over, and made pertinent changes (all plain /var/www references changed to /var/www/lab46web).

Also installed lair-mail and lair-backup on the new www, removed oldwww from backup processing.

I noticed on some systems the existence of an /etc/cron.d/backup.orig file, which is likely a remainder of the package update process… I should put in some script logic to specifically remove this file, as the last thing we want is for the backups to happen twice.

I started preparing my spring 2011 syllabi… beginning with unix.

After two unfruitful weeks of wrestling with cherokee, on a whim of trying to get started on my semester syllabi preparations, I quickly threw it all out the window, installed apache2, and converted over the existing configuration, deploying a new squeeze-based VM for our public web needs.

www also has the functionality of repos built-in, as I hope to also make web-based subversion access a reality, so repos has also been retired. I copied over the ssh keys from repos, so existing subversion checkouts should proceed transparently, and once I get the web-based svn stuff working, it will be a new aspect of awesome.

In the meantime, both wikis have been updated to the latest dokuwiki (2010-11-07)… a quick and effective upgrade process, with seemingly no significant snags.

I'll have to re-implement some of my manual fixes (the calendar hack, the SSO-hack), but it seems as though we are good to go on the wikis. I was even successful in better partitioning the various bits of web data (the main wiki now resides in a /lab46web directory, and with the help of Apache Alias statements, appropriately redirects to other pertinent directories for content rendering).

There was a security update to 2010-11-07, which revolved around some xmlrpc functionality I wasn't using, but it was a quick replacement of two files, so that is done. Both wikis are now at the latest and greatest stable release of dokuwiki.

I encountered a couple broken links, and I thought to do a more in-depth check to see what else was lingering that might lead to any functionality problems. After a quick google, the following command-line served the purpose nicely:

machine:~# ( find / -type l | xargs file ) | grep 'broken symbolic link'

It will display each and every file that is identified by file(1) as a broken link. Amazingly, there were no further ones in my wiki directories.

Rigging up replication between the various auth servers, I ended up obsoleting my first replicating LDAP object (authrepl), and had need to remove it.

Here is the logic to do so:

auth1:~# ldapdelete -v "cn=authrepl,dc=lair,dc=bits" -D "cn=admin,dc=lair,dc=bits" -W
Enter LDAP Password: <LDAP_admin_password>
deleting entry "cn=authrepl,dc=lair,dc=bits"
auth1:~# 

Should come in handy.

I pushed through some additional tweaks, and also installed it on forklift and auth3.

I finally got around to fixing the network detection logic… pushed updates out to everyone.

With auth3 now online, I updated lair-ldap to include it in the configuration files, and pushed out updates to everyone.

I built a new VM, auth3, living on cobras, which serves as a LAIR LDAP consumer on the lair.lan domain. This will make for 3 LDAP servers in the LAIR… two on offbyone.lan (the producer and one consumer), and then this consumer on lair.lan.

This offers LDAP clients LAIR-wide more resiliency in the event of an outage.

With my continuing work on getting a VM up for bigworld, I decided to investigate setting up a centos mirror…

• http://www.howtoforge.com/creating_a_local_yum_repository_centos
• http://nerdicism.com/2009/04/how-to-setup-up-a-private-centos-mirror/
• http://mirror.r-i-t.edu/centos/5.5/isos/x86_64/

A flurry of additional updates to lair-backup took place today, finally arriving at a configuration that is both stable and resilient. The monthly backups took place this morning, and I was able to sneak in the new lair-backup logic and get a majority of the systems to use it. Next month we should be all set.

I finished my lair-mail package, and rolled it out to all the various systems. After some initial missing files, I got everything worked out, so nullmailer is now appropriately configured for each system and should forward all mail to mail.offbyone.lan.

Once again, a bug is present in my dokuwiki status rollover script! When I came to check on my status page, it recorded the previous month link as “200912”… so I know whereabouts to look for the error.

It appears there was a double decrement (I did the “right thing” at the start of the script, then also later on..)… I commented out the second decrement, all should now be fine.

Happy New Year! The new year, aside from incrementing the year, brought further stress testing to my various scripts.

One problem that was uncovered was with journalcacheclear.sh on www… turns out it does an explicit check for existing files in a given directory, and now that we're in 2011, the semester logic has kicked over (for one), and as such, the class attendance files do not yet exist.

Therefore, it was throwing errors.

I fixed it by adding in another if statement to check for the existence of the files before plowing ahead. This adequately silenced it (since it runs every 4 minutes or so).

<html><center></html>

<html></center></html>