Recently I tried editing my Cisco’s ACL at home on the train. It went something like this:

• I logged in
• I started updating the ACL
• I hit a blackspot in my 3g coverage
• My command stops at “router(config)#access-”
• I get an alert saying my home internet was down

Although it is simple enough to just ask her to “flip the switch on the black box”, I still don’t like doing it. Plus, if she’s not home, I’m stuck. This accident immediately reminded me of one of a trait of the ‘reload’ command: it can be scheduled.

In the case of updating a device remotely, it is as easy as:

router# reload in 2
router# conf t
router(config)# [type in desired commands]
router(config)# exit
router# reload cancel

If the commands are entered in fine, then cancel the reload. If there is a problem, then the router will reboot and resort to the startup config.

If you want to use Graphserver to do some analysis with GTFS, you will need to convert GTFS into the database. This is how I did it.

Get an appropriate AMI from Amazon’s EC2

I used the following AMI. If you have enough memory, you don’t need to do this.

ami-7000f019

Lookup and read the GTFSDB INSTALL.txt document

Prepare system

sudo apt-get install mercurial
hg clone https://gtfsdb.googlecode.com/hg/ gtfsdb
sudo apt-get install python-setuptools
sudo easy_install psycopg2
sudo apt-get install build-essential

Download GTFS database

ubuntu@domU-12-31-39-00-5D-B8:/mnt/gtfsdb$ pwd
/mnt/gtfsdb
sudo python setup.py install
sudo wget http://cdn.kelvinism.com/google_transit.zip
sudo apt-get install python-psycopg2

EDIT 16-03-2025: I’ve since removed these files.

Prepare configuration file

#default.cfg
[options]
create = True
database = postgresql://nsw:[email protected]:5432/nsw
filename = /mnt/google_transit.zip
geospatial = True
#schema = None

Perform import

screen
python gtfsdb/scripts/load.py

I’ve had several conversations with neighbors and co-workers about the “lack” of forward thinking, or at least the lack of forward action. Of course, I keep in the back of my mind that we aren’t “experts”, and the more I learn about transport the more I learn how complex it is. Dr. Sussman’s CLIOS process (Complex, Large-
Scale, Integrated, Open Systems) appears more and more true the longer I work in and study transport. There is a plethora of excuses that can be made, but the general conclusion was that the earlier we prepare the better. I can remember working near Zhongshan 7-8 years ago and driving around on huge roads in the middle of empty fields. There weren’t even stoplights at every intersection. It was then that I had an epiphany of how smart the planning was to build the infrastructure before the masses arrived.

Sydney is estimated to increase by some 1.7 million people by 2036, and I can tell you, from a transportation (private and public) standpoint, that sort of scares me. When people ask me why transport is so difficult I justify it by with my uneducated guess that the CBD is next to the ocean, so everybody travels in from just 180 degrees instead of 360. Maybe this is why the NSW government created the “City of Cities” strategy. I realized this within the first few weeks: most people live west but work east.

Tonight (a Saturday) I was bored, and should have been studying, but wanted to create a few visualizations first.

The below maps were created using TDX data released from 131500. After converting it to GTFS I imported it into PostGIS using GTFSDB, and then could serve it via GeoServer. Finally, I could access it via WMS in QGIS. I added the stops into a map of Sydney and added some boundaries, and added the Growth Zones. The result was a map with every bus/train/ferry stop. Darker areas have stops that are closer (not necessarily more frequent service).

One of the first things I noticed is that there isn’t much physical infrastructure in these areas. There also aren’t many transit stops; I suppose this is why the South West Rail Link is going to be so important. I don’t know all of the political ramifications, but let’s hope the North West Rail Link is built as well?

I’ve been cleaning up my personal blog a bit, and I noticed that my tagging system recently broke. I’ve investigated the cause, and it appears to be because I removed some apps but the contenttypes remained. This meant that whenever I tried calling a tag with a TaggedItem that had been deleted, I was getting this error:

'NoneType' object has no attribute '_meta'

The solution is to first list all app_labels for contenttypes, and then delete any not in use.

In [61]: from django.contrib.contenttypes.models import ContentType
 
In [62]: for ct in ContentType.objects.all(): print ct.app_label
   ....:
picasaweb
lifestream
readernaut
delicious
mapfeed
comments
...

I could then delete the unused contenttypes.

ct_list = ["delicious", "flickr", "photologue", "twitter"]
 
for ct_label in ct_list:
    for ct in ContentType.objects.filter(app_label=ct_label):
        ct.delete()
    

And no more errors! For more details take a look at David’s article.

I recently had the desire to send small updates to my so called lifestream page via XMPP/GTalk. I played around with Twisted Words and several other Python XMPP clients, but I didn’t really want to keep a daemon running if unnecessary. It turns out imified took a lot of the pain out of it. The steps for me were as follows:
Create an account with imified, and create a URL, e.g. /app/api/
We then configure the urls.conf

urlpatterns = patterns('',  
    (r'^app/api/$', bot_stream),
)

We then create the necessary views. So, in views.py:

from django.shortcuts import render_to_response
from django.http import HttpResponse
from lifestream.forms import *
from datetime import datetime
from time import time
 
def bot_stream(request):
    if request.method == 'POST':
        botkey = request.POST.get('botkey')
        username = request.POST.get('user')
        msg = request.POST.get('msg')
        network = request.POST.get('network')
    
    if username == "[email protected]" or network == "debugger":
        blob_obj = Blob(id=time(), body=msg, service_name="Mobile",
        link="http://www.kelvinism.com/about-me/", published=datetime.now())
        blob_obj.save()
        resp = "OK"
    else:
        resp = "Wrong username %s" % username
    else:
        resp = "No POST data"
    return HttpResponse(resp)

To complete this little example, you can see what I used for my models.py

class Blob(models.Model):
    id = models.CharField(max_length=255, primary_key=True)
    body = models.TextField(max_length = 1024, null = True, blank = True)
    service_name = models.CharField(max_length=50, null=True, blank=True)
    link = models.URLField(max_length=255, verify_exists=False, null=True, blank=True)
    published = models.DateTimeField(null=True, blank=True)
 
def __unicode__(self):
    return self.id
 
class Meta:
    ordering = ['-published']
    verbose_name = 'Blob'
    verbose_name_plural = 'Blobs'
 
def get_absolute_url(self):
    return "/about-me/"

It maybe isn’t super elegant, but it works just fine, and maybe can provide a hint if somebody else is contemplating using a homebuilt xmpp solution, or just pawning it off on IMified.

I was having some issues with my personal mail server (Postfix) and my phone (Android). The logs depicted the below issue:

Jan  9 09:19:53 ip-11-222-23-223 postfix/smtpd[12345]: NOQUEUE: reject: RCPT from 12-13-14-15.abc.com.au[12.13.14.15]: 504 5.5.2 <localhost>: Helo command rejected: need fully-qualified hostname; from=<emailaddr kelvinism.com="kelvinism.com"> to=<emailaddr gmail.com="gmail.com"> proto=ESMTP helo=<localhost>
</localhost></emailaddr></emailaddr></localhost>

We can see here that the stock Android email client is doing a ‘helo localhost’. One part of my main.cf file specifies this:

smtpd_helo_required = yes
smtpd_helo_restrictions =
    permit_mynetworks,
    reject_non_fqdn_helo_hostname,
    reject_invalid_helo_hostname,
    permit_sasl_authenticated,
    permit

To resolve, unfortunately, just change the order to authenticated clients are permitted earlier:

smtpd_helo_required = yes
smtpd_helo_restrictions =
    permit_mynetworks,
    permit_sasl_authenticated,
    reject_non_fqdn_helo_hostname,
    reject_invalid_helo_hostname,
    permit

You also may need to do the same for smtpd_recipient_restrictions and/or smtpd_sender_restrictions (i.e. put permit_sasl_authenticated above the reject lines).

A few weeks ago I saw an opening to give a 5-10 minute lightening talk at SyPy (Sydney Python), and with two nights to prepare, decided it would be interesting to explore Splunk’s usage of Python. You can see it here

I like using Ubuntu’s UFW command, but today I needed to allow outgoing ICMP. I received results as so:

$ ping 4.2.2.2  
PING 4.2.2.2 (4.2.2.2) 56(84) bytes of data.  
ping: sendmsg: Operation not permitted  
ping: sendmsg: Operation not permitted  
ping: sendmsg: Operation not permitted  

To allow outbound icmp I edited ‘before.rules’ and added the following lines.

$ sudo vi /etc/ufw/before.rules

# allow outbound icmp
-A ufw-before-output -p icmp -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
-A ufw-before-output -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT

I recently had the need to add some HTML to the label for a form field using Django. The solution is pretty easy, except I didn’t see it written explicitly anywhere, and I missed the memo of the function I should be using.
My form first just had the HTML in the form label as so:

from django import forms
 
class AccountForm(forms.Form):
    name = forms.CharField(widget=forms.TextInput(), max_length=15, label='Your Name (<a href="//www.blogger.com/questions/whyname/" target="_blank">why</a>?')

However, when I displayed it, the form was autoescaped.

This is generally a good thing, except my form obviously didn’t display correctly. I tried autoescaping it in the template, but that didn’t work. To resolve this you’ll need to mark that individual label as safe. Thus:

from django.utils.safestring import mark_safe
from django import forms
 
class AccountForm(forms.Form):
    name = forms.CharField(widget=forms.TextInput(), max_length=15, label=mark_safe('Your Name (<a href="//www.blogger.com/questions/whyname/" target="_blank">why</a>?)'))
    

It will now display correctly:

In [1]: from myproject.forms import *
 
In [2]: form = AccountForm()
 
In [3]: form.as_ul()
Out[3]: u'
<li><label for="id_name">Your Name (<a href="//www.blogger.com/questions/whyname/" target="_blank">why</a>?):</label> <input id="id_name" maxlength="15" name="name" type="text"></li>
'

There’s maybe another easier way to do this, but this worked for me.

I’ve started to get into Google’s App Engine again, and have started developing a simple product that I had a use for. The initial first draft was a quick 200 lines in webapp, and it worked great. However, I’m starting to find certain things quite cumbersome. I’m a huge fan of Django, and but also about keeping things as simple as possible, which is why I picked webapp to begin with.
I’m now considering making a swap to Django, but there are some development issues; namely, I’m using Ubuntu 10.04, Python 2.6, and Django 1.2. This setup presents several setbacks, as GAE has the requirement of Django 1.1 and Python 2.5. There are two solutions that I found: a) use virtualenv, which I’ve detailed, or b) chroot. This document will hopefully show how to configure a chroot environment of Ubuntu 9.10 and prepare it for Django on GAE. Using a jailed environment should allow you to edit your code with your normal IDE and VCS, but use Django 1.1 and Python 2.5.
First, I installed schroot and debootstrap.

$ sudo apt-get install schroot debootstrap

Second, I edited /etc/schroot/schroot.conf and added the following section to the end.

[karmic]
description=karmic
type=directory
location=/var/chroot/karmic
priority=3
users=kelvinn #your username goes here
groups=admin
root-groups=root
run-setup-scripts=true
run-exec-scripts=true

Third, I created the directories needed for the jailed environment and installed karmic.

$ sudo mkdir -p /var/chroot/karmic
$ sudo debootstrap --arch i386 karmic /var/chroot/karmic

Forth, I logged into the jailed environment and updated packages, installed Python 2.5 / Django 1.1. Make sure to note that I don’t call ‘python’, I call ‘python2.5’.

$ sudo schroot -c karmic
(karmic)root@kelvinn-laptop:~# apt-get update
(karmic)root@kelvinn-laptop:~# apt-get install python2.5
(karmic)root@kelvinn-laptop:~# cd /usr/src
(karmic)root@kelvinn-laptop:~# apt-get install wget
(karmic)root@kelvinn-laptop:/usr/src# wget http://www.djangoproject.com/download/1.1.2/tarball/
(karmic)root@kelvinn-laptop:/usr/src# tar -xpzf Django-1.1.2.tar.gz
(karmic)root@kelvinn-laptop:/usr/src/Django-1.1.2# python2.5 setup install
(karmic)root@kelvinn-laptop:/usr/src/Django-1.1.2# exit

Lastly, I log in as my normal user, and start the app. Let’s say I have a folder called ‘~/gaeapps’ for my GAE stuff, and that’s where I put the SDK.

$ scroot -c karmic
(karmic)kelvinn@kelvinn-laptop:~/gaeapps$ ls
google_appengine  myproject
(karmic)kelvinn@kelvinn-laptop:~/gaeapps$ google_appengine/dev_appserver.py myproject