pbwy

better late than never

2011-03-12T12:57:00.000-08:00

I have written OOP programs by imitating the style of others. Now I am reading Object-Oriented Programming by Timothy Budd at Oregon State University, and just finished chapter 3. The book is picking up the pace. It was interesting to read about the teaching tool of Responsibility Driven Design, and about using note cards to physically model and organize components.

The constraints of an index card are also a good measure of approximate complexity. A component that is expected to perform more tasks than can fit easily in this space is probably too complex...

This reminds me of many other quotes.

A Forth word should not have more than one or two arguments. This stack which people have so much trouble manipulating should never be more than three or four deep. ... And that is in my mind one of the keystones of Forth, you factor and you factor and you factor until most of your definitions are one or two lines long.

http://www.ultratechnology.com/1xforth.htm

Now, some people will claim that having 8-character indentations makes the code move too far to the right, and makes it hard to read on a 80-character terminal screen. The answer to that is that if you need more than 3 levels of indentation, you're screwed anyway, and should fix your program.

http://www.kernel.org/doc/Documentation/CodingStyle

If a function exceeds about 40 lines, think about whether it can be broken up without harming the structure of the program.

http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml#Write_Short_Functions

So far almost all of the OO code I have read has been in frameworks for web or windows applications. The components would NOT fit on note cards. I have not decided what that means yet, but it rings true to me that real world usage will add hair to a program.

Callback spaghetti sauce

2011-01-23T05:42:00.000-08:00

http://substack.net/posts/92a7f7/Build-chainable-interfaces-in-node-js-with-chainsaw

virtually the same

2010-11-27T14:58:00.000-08:00

My new hardware has features for virtualization. My plan was to try Fedora 14/KVM, Windows Server 2008 R2/Hyper-V, and VMware/ESXi 4.1, to see how they treated me. I also planned to try FreeBSD 8.1 as a guest and some VMWare appliance disk images from turnkeylinux.org. All this worked perfectly on KVM. Hyper-V worked, but was surprisingly fussy. Unfortunately, I could not install VMware on my hardware; it would always crash part way through the installation process.

I ran informal benchmarks using the Linux hdparm command, the FreeBSD diskinfo command, and the Unix benchmarking tools bonnie++, nbench, and unixbench. On Windows I used AIDA64 and Sandra. On all platforms I timed a 2gb file copy over http on 100Mbps Ethernet. The benchmarks were all over the place, so I will not tabulate the numbers.

Some things were faster in the guest than on the host, including buffered/cached disk read throughput and process creation.

The 2gb file copy took roughly 3.5 minutes on all platforms except for KVM/W2k8r2, which took 5 minutes.

KVM Documentation links

Fedora Virtualization Guide

Hints to use VMware disk images on KVM

Using LVM snapshots to back up KVM

Use virsh to manage virtual servers (aka domains).

Fedora 14 guest on KVM


lvcreate -L 8g -n lv_f14guest vg_overt

virt-install -n f14guest -r 1024 -c /dev/dvd --os-type=linux --os-variant=fedora14 --boot=cdrom,hd --disk=/dev/vg_overt/lv_f14guest -w network:default --vnc --vnclisten=0.0.0.0 -v --virt-type=kvm

vncviewer overt

Stop and delete the smartd service.

MediaWiki appliance on KVM

Obtain appliance disk image from turnkeylinux.org/mediawiki.


lvcreate -L 4g -n lv_appliances vg_overt
mkfs -t ext4 /dev/vg_overt/lv_appliances
mkdir /mnt/appliances
mount /dev/vg_overt/lv_appliances /mnt/appliances
cd /mnt/appliances
unzip /mnt/hd/turnkey-mediawiki-2009.10-2-hardy-x86.zip

yum install xmlstarlet
alias xml=xmlstarlet

# The vmdk file name is turnkey-mediawiki-2009.10-2-hardy-x86.vmdk
# How to find vmdk file name:
xml sel -N a=http://schemas.dmtf.org/ovf/envelope/1 -T -t -m //a:File -v @a:href -n turnkey-mediawiki-2009.10-2-hardy-x86/turnkey-mediawiki-2009.10-2-hardy-x86.ovf

virt-install -n wikiguest -r 1024 --os-type=linux --os-variant=ubuntuhardy --boot=hd --import --disk=/mnt/appliances/turnkey-mediawiki-2009.10-2-hardy-x86/turnkey-mediawiki-2009.10-2-hardy-x86.vmdk,format=vmdk -w network:default --vnc --vnclisten=0.0.0.0 -v --virt-type=kvm

$ ssh user@overt
$ su -
# sesetbool sshd_forward_ports 1
# exit
$ exit
$ ssh user@overt -L 18008:192.168.122.237:80 -L 18000:192.168.122.237:12320 -L 18001:192.168.122.237:12321 -L 18002:192.168.122.237:12322

browse to http://127.0.0.1:18008/ or https://127.0.0.1:18000/

Install Vsphere/ESXi

The installer crashes.


Traceback (most recent call last):
File "ThinESX.py", line 8, in <module>
Installer.Start()
File "/usr/lib/vmware/installer/Core/Log.py", line 46, in wrapper
return func(*args, **kwargs)
File "/usr/lib/vmware/installer/ThinESXInstall.py", line 38, in Start
Install.Start(self, data)
File "/usr/lib/vmware/installer/Core/Log.py", line 46, in wrapper
return func(*args, **kwargs)
File "/usr/lib/vmware/installer/Core/Install.py", line 40, in Start
data = self.Steps[self._Dispatcher.CurrentStep](data)
File "/usr/lib/vmware/installer/Core/Log.py", line 46, in wrapper
return func(*args, **kwargs)
File "/usr/lib/vmware/installer/ThinESX/ThinESXInstallSteps.py", line 70, in TargetSelectionStep
datastores - DatastoreEnumeration(None)
File "/usr/lib/vmware/installer/Core/DatastoreEnumeration.py", line 15, in __init__
vmfs3Module.Load()
File "/lib/python2.5/vmkctl.py", line 11328, in Load
def Load(*args): return _vmkctl.ModuleImpl_Load(*args)
vmkctl.HostCtlException: Unable to load module /usr/lib/vmware/vmkmod/vmfs3: Failure
</module>

Based on the following two threads, I think that hardware support is the problem.
thread1

thread2

I removed some of the hardware but was not able to work around the crash. I could not find any gigabyte brand motherboards on the compatibility list. Some people recommend buying a used server for testing, and others recommend Supermicro/LSI.

Fedora 14 guest on Hyper-V

Fedora 14 does not come with the Hyper-V drivers. I tried the rpmfusion kmod-staging package. After loading the hv_netvsc module, I could get a DHCP lease, but the Linux kernel would crash often. The legacy network adapter performed better and was more stable.

FreeBSD 8.1 guest on Hyper-V

When I tried to install FreeBSD 8.1 x64 as a Hyper-V guest, the FreeBSD kernel crashed and Hyper-V hammered a processor core. I was not able to turn off the guest, nor get a clean shut down on the host. When I powered the host back on, the guest resumed the broken state. Granted, FreeBSD is not a supported guest, but a guest should not be able to lock up the host system.

Windows Server 2008 R2 guest on Hyper-V

When I installed Windows Server 2008 R2 as a Hyper-V guest, it would not come online. The integration services were installed and the NIC showed up in device manager, but was missing from the "ipconfig /all" command output and from the "Manage network connections" control panel. The event log showed a message about "Microsoft VMBus Network" "This device cannot start". I re-installed the integration services, rebooted, and that brought the guest online.

remote access without SSH

2010-11-25T17:06:00.000-08:00

As an exercise, I wanted to have authenticated and encrypted shell access without SSH. Shellinabox [1] permits anyone to connect and it does not authenticate the client. Instead, I chose to use stunnel and telnet on Fedora 14 and it worked like a charm. Here are some notes.

[1]
http://code.google.com/p/shellinabox/

Install server software.


$ su -
# yum install openssl-perl stunnel telnet-server

Use stunnel to wrap telnet service.


# vi /etc/xinetd.d/telnet
:%s,\(disable.*=\) yes,\1 no,
:%s,flags.*,& NAMEINARGS,
:%s,\(server.*=\) .*,\1 /usr/bin/stunnel ,
o
server_args = stunnel /etc/stunnel/in.telnetd.conf
:wq
# vi /etc/stunnel/in.telnetd.conf
i
exec = /usr/sbin/in.telnetd
execargs = in.telnetd
CAfile = /etc/pki/CA/cacert.pem
CApath = /etc/stunnel/authorized_certs
cert = /etc/stunnel/in.telnetd.crt
key = /etc/stunnel/in.telnetd.key
verify = 3
ciphers = ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW
sslVersion = TLSv1
:wq
# service xinetd restart

Create a new SSL certificate authority to sign SSL certificates.


# man CA.pl
Use common sense when responding to CA.pl prompts.
# /etc/pki/tls/misc/CA.pl -newca
At "CA certificate filename (or enter to create)", Press Enter.

Create an SSL certificate and key for the telnet service.


# /etc/pki/tls/misc/CA.pl -newreq-nodes
# /etc/pki/tls/misc/CA.pl -signreq
# rm -f newreq.pem
# mv newcert.pem /etc/stunnel/in.telnetd.crt
# mv newkey.pem /etc/stunnel/in.telnetd.key
# chmod og-rwx /etc/stunnel/in.telnetd.key
# mkdir /etc/stunnel/authorized_certs
# restorecon -R /etc/stunnel

Create an SSL certificate and key for the client.


# /etc/pki/tls/misc/CA.pl -newreq-nodes
# /etc/pki/tls/misc/CA.pl -signreq
# rm -f newreq.pem
# cp newcert.pem /etc/stunnel/authorized_certs/telnet.pem
# c_rehash /etc/stunnel/authorized_certs

Create stunnel configuration for the client.


# mkdir .stunnel
# cp /etc/pki/CA/cacert.pem .stunnel/
# cp /etc/stunnel/in.telnetd.crt .stunnel/in.telnetd.pem
# c_rehash .stunnel
# mv newcert.pem .stunnel/telnet.crt
# mv newkey.pem .stunnel/telnet.key
$ vi .stunnel/client.conf
i
client = yes
foreground = yes
output = .stunnel/client.log
pid =

[telnet]
accept = localhost:23
connect = servername:23
CAfile = .stunnel/cacert.pem
CApath = .stunnel
cert = .stunnel/telnet.crt
key = .stunnel/telnet.key
verify = 3
ciphers = ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW
sslVersion = TLSv1
:wq
# zip -r keys.zip .stunnel
# rm -fr .stunnel
# mv keys.zip /media/FLASHDRIVE/
# umount /media/FLASHDRIVE/
# exit

Copy SSL certificate, key, and stunnel configuration to client machine.


$ cd
$ rm -fr .stunnel
$ unzip /media/FLASHDRIVE/keys.zip
$ rm -f /media/FLASHDRIVE/keys.zip
$ chmod og-rwx .stunnel/telnet.key

Run stunnel to forward telnet from client to server.


$ stunnel .stunnel/client.conf

In another terminal, telnet to the stunnel listener.


$ telnet localhost 23

dictionary

2010-10-16T17:41:00.000-07:00

The book I am reading frequently requires me to reference a dictionary. Today dict.org went down and that gave me all the excuse I needed to write some easy scripts. The result works on the basic web browser built in to my MP3 player.

The script downloads a pre-formatted dictionary, parses the definitions, and enters them into an SQL database. Another script gives a web-based interface to search the database.

Here is the shell script that downloads and imports the definitions.


#!/bin/sh
base=ftp://ftp.dict.org/dict/pre/
fn=dict-wn-2.0-pre.tar.gz
url=$base$fn
wget -c $url
tar xf $fn
gzip -dc <wn.dict.dz |\
  awk -f load-wn20.awk |\
  sqlite3 wn20.sqlite

Here is the AWK script that parses the definitions.


# Input: preformatted Wordnet 2.0 dictionary from dict.org
# Output: text file formatted for the sqlite3 command

function printDef() {
  if (length(word) > 0) {
      gsub(/'/, "''", word)
      gsub(/'/, "''", data)
      gsub(/\n\n*$/, "", data)
      printf("insert into wn20 values ('%s', '%s');\n", word, data);
      data = ""
  }
}

BEGIN {
  word = ""
  data = ""
  print "drop table if exists wn20;"
  print "create table wn20 (word text, data text);"
  print "begin;"
}

/^[0-9A-Za-z]/ {
  printDef()
  word = toupper($1)
}

{
  data = sprintf("%s%s\n", data, $0)
}

END {
  printDef()
  print "end;"
  print "create index words on wn20 (word);"
}

Here is the shell script for the web interface.


#!/bin/sh

lookup() {
  db="$DOCUMENT_ROOT/$1.sqlite"
  if [ "$3" = "wild" ]
  then
      query="select data from $1 where word like '$2%' limit 5"
  else
      query="select data from $1 where word = '$2'"
  fi
  retval=$(sqlite3 "$db" "$query" 2>&1)
  echo "$retval"
}

word=$(echo "$QUERY_STRING" |\
  sed -e 's/^q=//' -e 's/[^A-Za-z0-9].*//' |\
  tr a-z A-Z)
isWild=$(echo "$QUERY_STRING" | grep '\*' >/dev/null 2>&1 && echo wild)
web=$(lookup web1913 $word $isWild)
wn=$(lookup wn20 $word $isWild)
if [ -z "$web" -a -z "$wn" ]
then
  data="No definition found for word '$word'."
else
  data="<h2>From Webster's 1914 Dictionary:</h2>
<pre id="web">
$web
</pre>
<h2>From WordNet 2.0 Dictionary:</h2>
<pre id="wn">
$wn
</pre>
"
fi

cat <<__TOP__
Content-type: text/html

<html>
<head>
<title>dict $word</title>
</head>
<body>
<form action="$SCRIPT_URL" name="dict">
Word: <input type="text" name="q" />
<input type="submit" />
</form>
__TOP__

if [ ! -z "$word" ]
then
  cat <<__MID__
<h1>$word :</h1>
$data
__MID__
fi

cat <<__END__
</body>
</html>
__END__
exit 0

The scripts are in the linked dict.zip file.

tabletop gaming

2010-09-23T05:46:00.000-07:00

D&D comes in many flavors, and you can play old school for free with a retro clone such as Labyrinth Lord. It is a social game and many adventures are designed for 5 or 6 players. It can be played over the Internet using free tools such as Skype, Google Docs, and dicelog.com. There are many resources available, and you can adapt resources from other games. For beginners or smaller groups, Dungeon Plungin' might be a better option.

Slackware chroot on Fedora

2010-06-13T17:02:00.000-07:00

I used Slackware to give new life to some low-end computers. I wanted to run Fedora 13 on my desktop to try SELinux, but I wanted to compile packages on my desktop instead of the weaker computers. My first approach was to install Slackware in a virtual machine, but my processor lacks VT-x and KVM ran very slowly. My next approach was to install Slackware in a chroot environment. Test builds are exactly what chroot() was made for. SELinux broke the compiler, so I made local policy to grant all permissions to Slackware. This is not so great for security, but well enough for a learning exercise. Plus, I don't have to reboot into Slackware or disable SELinux. Here's what I did.

Installed Slackware 13.1 from the first install CD, based on these notes.


# mkdir /slackware
# cd /slackware
# for f in /media/S13_1d1/slackware/a/*tgz; do tar xf $f; done
# sbin/installpkg -root /slackware /media/S13_1d1/slackware/{a,ap,d,l,n}/*.txz

Created SELinux policy to allow slackware to do everything. The following documents were helpful.

SELinux FAQ, section 1.2, question 4.
SELinux Project Policy Language.
Seedit SELinux Permissions.


# cd
# mkdir slackware
# cd slackware
# touch slackware.{fc,if,te}
# cat >slackware.te <<__EOF__
policy_module(slackware, 1.0)

require {
 type fs_t;
 type setfiles_t;
 type unconfined_t;
}

type slackware_t;

# necessary for restorecon
allow slackware_t fs_t : filesystem { associate } ;
allow setfiles_t slackware_t : file * ;
allow setfiles_t slackware_t : dir * ;
allow setfiles_t slackware_t : lnk_file * ;
allow setfiles_t slackware_t : chr_file * ;
allow setfiles_t slackware_t : blk_file * ;
allow setfiles_t slackware_t : fifo_file * ;

# necessary for programs within chroot /slackware
allow unconfined_t slackware_t : file * ;
allow unconfined_t slackware_t : dir * ;
allow unconfined_t slackware_t : lnk_file * ;
allow unconfined_t slackware_t : chr_file * ;
allow unconfined_t slackware_t : blk_file * ;
allow unconfined_t slackware_t : sock_file * ;
allow unconfined_t slackware_t : fifo_file * ;
__EOF__
# make -f /usr/share/selinux/devel/Makefile
# semodule -i slackware.pp
# semanage fcontext -a -s unconfined_u -t slackware_t '/slackware.*'
# restorecon -R /slackware

Then I tested the chroot environment. It is not shown here, but I also built several packages from http://slackbuilds.org/.


# mkdir proc sys
# mount --bind /proc /slackware/proc
# mount --bind /sys /slackware/sys
# chroot /slackware
# cd
# cat >hi.c <<__EOF__
#include <stdio.h>

int main(int argc, char *argv[]) {
   puts("hi there");
   return 0;
}
__EOF__
# cc -o hi hi.c
# ./hi
hi there

SELinux transition

2010-06-12T12:12:00.000-07:00

Read the RedHat SELinux User Guide this morning. So far, I think of SELinux as a generic kernel-level application firewall. It would augment and not replace the JVM sandbox. On RedHat, it seems intended for servers because there are myriads of policies written for services and very few written for users and roles. Here are some personal notes in response to the User Guide.

Chapter 3. SELinux Contexts
Section 3.1. Domain Transitions

only authorized domains, such as passwd_t, can write to files labeled with the shadow_t type. Even if other processes are running with superuser privileges, those processes can not write to files labeled with the shadow_t type, as they are not running in the passwd_t domain.

Why can root run "vi /etc/shadow" and successfully write using ":w!"? The answer comes later in chapter 4.

Chapter 4. Targeted Policy
Section 4.2. Unconfined Processes

Processes running in unconfined domains fall back to using DAC rules exclusively.

"ps -eZ | grep bash" reveals that the root shell is running in an unconfined domain. That is why it can write to /etc/shadow.

What protection does SELinux offer to an interactive root user? The answer comes later in chapter 6.

Chapter 6. Confining users
Section 6.2. Confining New Linux Users

When Linux users run in the unconfined_t domain, SELinux policy rules are applied, but policy rules exist that allow Linux users running in the unconfined_t domain almost all access. If unconfined Linux users execute an application that SELinux policy defines can transition from the unconfined_t domain to its own confined domain, unconfined Linux users are still subject to the restrictions of that confined domain. The security benefit of this is that, even though a Linux user is running unconfined, the application remains confined, and therefore, the exploitation of a flaw in the application can be limited by policy.

Going back to the example in section 3.1, the passwd command has the passwd_exec_t type, which transitions the passwd process to the passwd_t domain so that it can write to files with the shadow_t type. How can I see the policy responsible for transitioning the passwd process to the passwd_t domain? Found the answer after asking questions on IRC and reading about Domain Transitions.


# id --context
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

# ls --context /usr/bin/passwd
-rwsr-xr-x. root root system_u:object_r:passwd_exec_t:s0 /usr/bin/passwd

The following rule states that when a process of type unconfined_t executes a file of type passwd_exec_t, the process type should be changed to passwd_t if allowed by the policy (i.e. Transition from the unconfined_t domain to the passwd_t domain).


# sesearch --type --source=unconfined_t --target=passwd_exec_t
Found 1 semantic te rules:
   type_transition unconfined_t passwd_exec_t : process passwd_t;

File needs to be executable in the unconfined_t domain:


# sesearch --allow --source=unconfined_t --target=passwd_exec_t | grep -w unconfined_t
   allow unconfined_t passwd_exec_t : file { read getattr execute open } ;

The executable file needs an entry point into the passwd_t domain:


# sesearch --allow --source=passwd_t --target=passwd_exec_t
Found 1 semantic av rules:
   allow passwd_t passwd_exec_t : file { ioctl read getattr lock execute entrypoint open } ;

Process needs permission to transition into the passwd_t domain:


# sesearch --allow --source=unconfined_t --target=passwd_t | grep -w unconfined_t
   allow unconfined_t passwd_t : process transition ;

foot notes

2010-05-25T20:35:00.000-07:00

Some times while reading books I will want to make notes for later. Firefox has an add-on that can be used for this purpose.

Here is a link to install ScrapBook Plus.

Once installed, view an HTML ebook such as The Hound of the Baskervilles by Arthur Conan Doyle.

Click ScrapBook Plus. Click Capture Page. Click ScrapBook Plus. Click The Hound of the Baskervilles. Now you are viewing your local copy of the book.

To highlight, select the desired section and click Style 1.

To add a foot note, click the pencil button in the bottom tool bar. This will insert a note. You can move the note by dragging the top edge around. You can grow or shrink the note by dragging the bottom right edge.

To save your notes, click the floppy disk icon.

scrollmarks

2010-05-24T18:55:00.000-07:00

Page numbers can be a convenient way to save your place in an HTML ebook, but what if the page numbers are missing? Here is a work around to save the position in a Firefox bookmark.

Right click Save Position and click Bookmark This Link.

Right click Load Position and click Bookmark This Link.

View an HTML ebook, such as Dracula by Bram Stoker.

Scroll down to the desired section. Click Bookmarks. Click Save Position. Click Bookmarks. Click Bookmark This Page.

Close the web browser and start a new one. Click Bookmarks. Click Dracula. Click Bookmarks. Click Load Position. This will return you to the saved position.

I tend to read books in full screen mode. You can press F11 to enter full screen mode and Ctrl-Shift-O to open your bookmarks.

If you save the position in full screen mode and load the position in window mode, then do not expect it to work correctly. It would fail because the position depends on the size of the window (and font).

ebook font on Linux

2010-05-23T15:53:00.000-07:00

I read HTML ebooks in Firefox on Linux. The Firefox default serif font is legible but not stylish. I wanted to try Caslon, a font favored by Ben Franklin. Here are the steps I took.

Run xterm and type the following commands.


mkdir ~/.fonts
cd ~/.fonts
wget http://www.orbitals.com/programs/wyld.zip
unzip -LL wyld.zip *.ttf
rm wyld.zip
mkfontscale
mkfontdir
xset fp+ ~/.fonts
xset fp rehash
exit

Right click Use Caslon Font and click Bookmark This Link.

View an HTML ebook, such as The Story of My Boyhood and Youth by John Muir.

Click Bookmarks and click Use Caslon Font.