better late than never

I have written OOP programs by imitating the style of others. Now I am reading Object-Oriented Programming by Timothy Budd at Oregon State University, and just finished chapter 3. The book is picking up the pace. It was interesting to read about the teaching tool of Responsibility Driven Design, and about using note cards to physically model and organize components.

The constraints of an index card are also a good measure of approximate complexity. A component that is expected to perform more tasks than can fit easily in this space is probably too complex...

This reminds me of many other quotes.

A Forth word should not have more than one or two arguments. This stack which people have so much trouble manipulating should never be more than three or four deep. ... And that is in my mind one of the keystones of Forth, you factor and you factor and you factor until most of your definitions are one or two lines long.

http://www.ultratechnology.com/1xforth.htm

Now, some people will claim that having 8-character indentations makes the code move too far to the right, and makes it hard to read on a 80-character terminal screen. The answer to that is that if you need more than 3 levels of indentation, you're screwed anyway, and should fix your program.

http://www.kernel.org/doc/Documentation/CodingStyle

If a function exceeds about 40 lines, think about whether it can be broken up without harming the structure of the program.

http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml#Write_Short_Functions

So far almost all of the OO code I have read has been in frameworks for web or windows applications. The components would NOT fit on note cards. I have not decided what that means yet, but it rings true to me that real world usage will add hair to a program.

Callback spaghetti sauce

http://substack.net/posts/92a7f7/Build-chainable-interfaces-in-node-js-with-chainsaw

virtually the same

My new hardware has features for virtualization. My plan was to try Fedora 14/KVM, Windows Server 2008 R2/Hyper-V, and VMware/ESXi 4.1, to see how they treated me. I also planned to try FreeBSD 8.1 as a guest and some VMWare appliance disk images from turnkeylinux.org. All this worked perfectly on KVM. Hyper-V worked, but was surprisingly fussy. Unfortunately, I could not install VMware on my hardware; it would always crash part way through the installation process.

I ran informal benchmarks using the Linux hdparm command, the FreeBSD diskinfo command, and the Unix benchmarking tools bonnie++, nbench, and unixbench. On Windows I used AIDA64 and Sandra. On all platforms I timed a 2gb file copy over http on 100Mbps Ethernet. The benchmarks were all over the place, so I will not tabulate the numbers.

Some things were faster in the guest than on the host, including buffered/cached disk read throughput and process creation.

The 2gb file copy took roughly 3.5 minutes on all platforms except for KVM/W2k8r2, which took 5 minutes.

KVM Documentation links

• Fedora Virtualization Guide


• Hints to use VMware disk images on KVM


• Using LVM snapshots to back up KVM

Use virsh to manage virtual servers (aka domains).

Fedora 14 guest on KVM


lvcreate -L 8g -n lv_f14guest vg_overt

virt-install -n f14guest -r 1024 -c /dev/dvd --os-type=linux --os-variant=fedora14 --boot=cdrom,hd --disk=/dev/vg_overt/lv_f14guest -w network:default --vnc --vnclisten=0.0.0.0 -v --virt-type=kvm

vncviewer overt


Stop and delete the smartd service.

MediaWiki appliance on KVM

Obtain appliance disk image from turnkeylinux.org/mediawiki.


lvcreate -L 4g -n lv_appliances vg_overt
mkfs -t ext4 /dev/vg_overt/lv_appliances
mkdir /mnt/appliances
mount /dev/vg_overt/lv_appliances /mnt/appliances
cd /mnt/appliances
unzip /mnt/hd/turnkey-mediawiki-2009.10-2-hardy-x86.zip

yum install xmlstarlet
alias xml=xmlstarlet

# The vmdk file name is turnkey-mediawiki-2009.10-2-hardy-x86.vmdk
# How to find vmdk file name:
xml sel -N a=http://schemas.dmtf.org/ovf/envelope/1 -T -t -m //a:File -v @a:href -n turnkey-mediawiki-2009.10-2-hardy-x86/turnkey-mediawiki-2009.10-2-hardy-x86.ovf

virt-install -n wikiguest -r 1024 --os-type=linux --os-variant=ubuntuhardy --boot=hd --import --disk=/mnt/appliances/turnkey-mediawiki-2009.10-2-hardy-x86/turnkey-mediawiki-2009.10-2-hardy-x86.vmdk,format=vmdk -w network:default --vnc --vnclisten=0.0.0.0 -v --virt-type=kvm

$ ssh user@overt
$ su -
# sesetbool sshd_forward_ports 1
# exit
$ exit
$ ssh user@overt -L 18008:192.168.122.237:80 -L 18000:192.168.122.237:12320 -L 18001:192.168.122.237:12321 -L 18002:192.168.122.237:12322


browse to http://127.0.0.1:18008/ or https://127.0.0.1:18000/

Install Vsphere/ESXi

The installer crashes.


Traceback (most recent call last):
File "ThinESX.py", line 8, in <module>
Installer.Start()
File "/usr/lib/vmware/installer/Core/Log.py", line 46, in wrapper
return func(*args, **kwargs)
File "/usr/lib/vmware/installer/ThinESXInstall.py", line 38, in Start
Install.Start(self, data)
File "/usr/lib/vmware/installer/Core/Log.py", line 46, in wrapper
return func(*args, **kwargs)
File "/usr/lib/vmware/installer/Core/Install.py", line 40, in Start
data = self.Steps[self._Dispatcher.CurrentStep](data)
File "/usr/lib/vmware/installer/Core/Log.py", line 46, in wrapper
return func(*args, **kwargs)
File "/usr/lib/vmware/installer/ThinESX/ThinESXInstallSteps.py", line 70, in TargetSelectionStep
datastores - DatastoreEnumeration(None)
File "/usr/lib/vmware/installer/Core/DatastoreEnumeration.py", line 15, in __init__
vmfs3Module.Load()
File "/lib/python2.5/vmkctl.py", line 11328, in Load
def Load(*args): return _vmkctl.ModuleImpl_Load(*args)
vmkctl.HostCtlException: Unable to load module /usr/lib/vmware/vmkmod/vmfs3: Failure
</module>


Based on the following two threads, I think that hardware support is the problem.
thread1

thread2

I removed some of the hardware but was not able to work around the crash. I could not find any gigabyte brand motherboards on the compatibility list. Some people recommend buying a used server for testing, and others recommend Supermicro/LSI.

Fedora 14 guest on Hyper-V

Fedora 14 does not come with the Hyper-V drivers. I tried the rpmfusion kmod-staging package. After loading the hv_netvsc module, I could get a DHCP lease, but the Linux kernel would crash often. The legacy network adapter performed better and was more stable.

FreeBSD 8.1 guest on Hyper-V

When I tried to install FreeBSD 8.1 x64 as a Hyper-V guest, the FreeBSD kernel crashed and Hyper-V hammered a processor core. I was not able to turn off the guest, nor get a clean shut down on the host. When I powered the host back on, the guest resumed the broken state. Granted, FreeBSD is not a supported guest, but a guest should not be able to lock up the host system.

Windows Server 2008 R2 guest on Hyper-V

When I installed Windows Server 2008 R2 as a Hyper-V guest, it would not come online. The integration services were installed and the NIC showed up in device manager, but was missing from the "ipconfig /all" command output and from the "Manage network connections" control panel. The event log showed a message about "Microsoft VMBus Network" "This device cannot start". I re-installed the integration services, rebooted, and that brought the guest online.

remote access without SSH

As an exercise, I wanted to have authenticated and encrypted shell access without SSH. Shellinabox [1] permits anyone to connect and it does not authenticate the client. Instead, I chose to use stunnel and telnet on Fedora 14 and it worked like a charm. Here are some notes.

[1]
http://code.google.com/p/shellinabox/

Install server software.


$ su -
# yum install openssl-perl stunnel telnet-server


Use stunnel to wrap telnet service.


# vi /etc/xinetd.d/telnet
:%s,\(disable.*=\) yes,\1 no,
:%s,flags.*,& NAMEINARGS,
:%s,\(server.*=\) .*,\1 /usr/bin/stunnel ,
o
server_args = stunnel /etc/stunnel/in.telnetd.conf
:wq
# vi /etc/stunnel/in.telnetd.conf
i
exec = /usr/sbin/in.telnetd
execargs = in.telnetd
CAfile = /etc/pki/CA/cacert.pem
CApath = /etc/stunnel/authorized_certs
cert = /etc/stunnel/in.telnetd.crt
key = /etc/stunnel/in.telnetd.key
verify = 3
ciphers = ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW
sslVersion = TLSv1
:wq
# service xinetd restart


Create a new SSL certificate authority to sign SSL certificates.


# man CA.pl
Use common sense when responding to CA.pl prompts.
# /etc/pki/tls/misc/CA.pl -newca
At "CA certificate filename (or enter to create)", Press Enter.


Create an SSL certificate and key for the telnet service.


# /etc/pki/tls/misc/CA.pl -newreq-nodes
# /etc/pki/tls/misc/CA.pl -signreq
# rm -f newreq.pem
# mv newcert.pem /etc/stunnel/in.telnetd.crt
# mv newkey.pem /etc/stunnel/in.telnetd.key
# chmod og-rwx /etc/stunnel/in.telnetd.key
# mkdir /etc/stunnel/authorized_certs
# restorecon -R /etc/stunnel


Create an SSL certificate and key for the client.


# /etc/pki/tls/misc/CA.pl -newreq-nodes
# /etc/pki/tls/misc/CA.pl -signreq
# rm -f newreq.pem
# cp newcert.pem /etc/stunnel/authorized_certs/telnet.pem
# c_rehash /etc/stunnel/authorized_certs


Create stunnel configuration for the client.


# mkdir .stunnel
# cp /etc/pki/CA/cacert.pem .stunnel/
# cp /etc/stunnel/in.telnetd.crt .stunnel/in.telnetd.pem
# c_rehash .stunnel
# mv newcert.pem .stunnel/telnet.crt
# mv newkey.pem .stunnel/telnet.key
$ vi .stunnel/client.conf
i
client = yes
foreground = yes
output = .stunnel/client.log
pid =

[telnet]
accept = localhost:23
connect = servername:23
CAfile = .stunnel/cacert.pem
CApath = .stunnel
cert = .stunnel/telnet.crt
key = .stunnel/telnet.key
verify = 3
ciphers = ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW
sslVersion = TLSv1
:wq
# zip -r keys.zip .stunnel
# rm -fr .stunnel
# mv keys.zip /media/FLASHDRIVE/
# umount /media/FLASHDRIVE/
# exit


Copy SSL certificate, key, and stunnel configuration to client machine.


$ cd
$ rm -fr .stunnel
$ unzip /media/FLASHDRIVE/keys.zip
$ rm -f /media/FLASHDRIVE/keys.zip
$ chmod og-rwx .stunnel/telnet.key


Run stunnel to forward telnet from client to server.


$ stunnel .stunnel/client.conf


In another terminal, telnet to the stunnel listener.


$ telnet localhost 23


dictionary

The book I am reading frequently requires me to reference a dictionary. Today dict.org went down and that gave me all the excuse I needed to write some easy scripts. The result works on the basic web browser built in to my MP3 player.

The script downloads a pre-formatted dictionary, parses the definitions, and enters them into an SQL database. Another script gives a web-based interface to search the database.

Here is the shell script that downloads and imports the definitions.


#!/bin/sh
base=ftp://ftp.dict.org/dict/pre/
fn=dict-wn-2.0-pre.tar.gz
url=$base$fn
wget -c $url
tar xf $fn
gzip -dc <wn.dict.dz |\
awk -f load-wn20.awk |\
sqlite3 wn20.sqlite


Here is the AWK script that parses the definitions.


# Input: preformatted Wordnet 2.0 dictionary from dict.org
# Output: text file formatted for the sqlite3 command

function printDef() {
if (length(word) > 0) {
gsub(/'/, "''", word)
gsub(/'/, "''", data)
gsub(/\n\n*$/, "", data)
printf("insert into wn20 values ('%s', '%s');\n", word, data);
data = ""
}
}

BEGIN {
word = ""
data = ""
print "drop table if exists wn20;"
print "create table wn20 (word text, data text);"
print "begin;"
}

/^[0-9A-Za-z]/ {
printDef()
word = toupper($1)
}

{
data = sprintf("%s%s\n", data, $0)
}

END {
printDef()
print "end;"
print "create index words on wn20 (word);"
}


Here is the shell script for the web interface.


#!/bin/sh

lookup() {
db="$DOCUMENT_ROOT/$1.sqlite"
if [ "$3" = "wild" ]
then
query="select data from $1 where word like '$2%' limit 5"
else
query="select data from $1 where word = '$2'"
fi
retval=$(sqlite3 "$db" "$query" 2>&1)
echo "$retval"
}

word=$(echo "$QUERY_STRING" |\
sed -e 's/^q=//' -e 's/[^A-Za-z0-9].*//' |\
tr a-z A-Z)
isWild=$(echo "$QUERY_STRING" | grep '\*' >/dev/null 2>&1 && echo wild)
web=$(lookup web1913 $word $isWild)
wn=$(lookup wn20 $word $isWild)
if [ -z "$web" -a -z "$wn" ]
then
data="No definition found for word '$word'."
else
data="<h2>From Webster's 1914 Dictionary:</h2>
<pre id="web">
$web
</pre>
<h2>From WordNet 2.0 Dictionary:</h2>
<pre id="wn">
$wn
</pre>
"
fi

cat <<__TOP__
Content-type: text/html

<html>
<head>
<title>dict $word</title>
</head>
<body>
<form action="$SCRIPT_URL" name="dict">
Word: <input type="text" name="q" />
<input type="submit" />
</form>
__TOP__

if [ ! -z "$word" ]
then
cat <<__MID__
<h1>$word :</h1>
$data
__MID__
fi

cat <<__END__
</body>
</html>
__END__
exit 0


The scripts are in the linked dict.zip file.

tabletop gaming

D&D comes in many flavors, and you can play old school for free with a retro clone such as Labyrinth Lord. It is a social game and many adventures are designed for 5 or 6 players. It can be played over the Internet using free tools such as Skype, Google Docs, and dicelog.com. There are many resources available, and you can adapt resources from other games. For beginners or smaller groups, Dungeon Plungin' might be a better option.

Slackware chroot on Fedora

I used Slackware to give new life to some low-end computers. I wanted to run Fedora 13 on my desktop to try SELinux, but I wanted to compile packages on my desktop instead of the weaker computers. My first approach was to install Slackware in a virtual machine, but my processor lacks VT-x and KVM ran very slowly. My next approach was to install Slackware in a chroot environment. Test builds are exactly what chroot() was made for. SELinux broke the compiler, so I made local policy to grant all permissions to Slackware. This is not so great for security, but well enough for a learning exercise. Plus, I don't have to reboot into Slackware or disable SELinux. Here's what I did.

Installed Slackware 13.1 from the first install CD, based on these notes.

# mkdir /slackware
# cd /slackware
# for f in /media/S13_1d1/slackware/a/*tgz; do tar xf $f; done
# sbin/installpkg -root /slackware /media/S13_1d1/slackware/{a,ap,d,l,n}/*.txz


Created SELinux policy to allow slackware to do everything. The following documents were helpful.

• SELinux FAQ, section 1.2, question 4.
• SELinux Project Policy Language.
• Seedit SELinux Permissions.


# cd
# mkdir slackware
# cd slackware
# touch slackware.{fc,if,te}
# cat >slackware.te <<__EOF__
policy_module(slackware, 1.0)

require {
type fs_t;
type setfiles_t;
type unconfined_t;
}

type slackware_t;

# necessary for restorecon
allow slackware_t fs_t : filesystem { associate } ;
allow setfiles_t slackware_t : file * ;
allow setfiles_t slackware_t : dir * ;
allow setfiles_t slackware_t : lnk_file * ;
allow setfiles_t slackware_t : chr_file * ;
allow setfiles_t slackware_t : blk_file * ;
allow setfiles_t slackware_t : fifo_file * ;

# necessary for programs within chroot /slackware
allow unconfined_t slackware_t : file * ;
allow unconfined_t slackware_t : dir * ;
allow unconfined_t slackware_t : lnk_file * ;
allow unconfined_t slackware_t : chr_file * ;
allow unconfined_t slackware_t : blk_file * ;
allow unconfined_t slackware_t : sock_file * ;
allow unconfined_t slackware_t : fifo_file * ;
__EOF__
# make -f /usr/share/selinux/devel/Makefile
# semodule -i slackware.pp
# semanage fcontext -a -s unconfined_u -t slackware_t '/slackware.*'
# restorecon -R /slackware


Then I tested the chroot environment. It is not shown here, but I also built several packages from http://slackbuilds.org/.

# mkdir proc sys
# mount --bind /proc /slackware/proc
# mount --bind /sys /slackware/sys
# chroot /slackware
# cd
# cat >hi.c <<__EOF__
#include <stdio.h>

int main(int argc, char *argv[]) {
puts("hi there");
return 0;
}
__EOF__
# cc -o hi hi.c
# ./hi
hi there