In February I gave a presentation to about 80 people at the Sydney Python group hosted by Atlassian. Firstly, Atlassian’s office was beautiful, feeling a little like Google’s Sydney office, but with beer on tap instead of cereal dispensers. Secondly, the talk before me on Cython by Aaron Defazio was exceptionally interesting, garnering lots of questions from the audience. My presentation, more of a show and tell on piping location data to Google’s Latitude through App Engine, was also meant to subtly share my views on the need for innovation in the public sector (all sectors, really).

My slides are below. I used very little text in the slides, but you can probably catch what is going on. The response from the audience was favourable, and I thank Dylan Jay for giving me the opportunity to speak.

My first trip to Kathmandu is now over, so there are some lessons learned I should scribe. Some of these are obvious, and which I abide by whenever travelling, and some I simply forgot in my (very) impromptu trip to Nepal.

• When agreeing on a price, make 100% sure the other person states the price back to you. I thought a price had been agreed to when the other person responded “ok ok, you are a lucky man”, but this does not count. As they say, reconfirm, reconfirm, reconfirm.
• When arriving at the airport, make sure you have small bills, too. I had 3x 100RS, 1x 10RS, and then a few 500RS. The price I negotiated was 440RS, and it would have been nice to have paid the exact amount.
• Kathmandu is polluted and dirty. I cannot emphasise this enough. It is dirtier than probably any other city I have been to. If I come back, I will be bringing masks. I know this sounds silly, to wear a mask, but any local on a motorbike or in a taxi wears a mask, and many just walking around. Instead of the normal cloth masks that many people use, I would probably bring a make with finer grained material - probably N95. I ultimately tried to avoid walking on main roads, but having some activated carbon absorb something would have made it a little less unpleasant.
• Bring old cloths, and throw them away after the trip. Or just bring black. If you have nice jackets or clothing they will likely come back pretty dirty.
• Bring some toilet paper. Similar to other parts of Asia, the bathrooms don’t have any.
• Bring a flashlight. The load shedding makes the city dark, and if you go out, you will want a flashlight. There aren’t any lights. I only used it a few times, but I am really glad I brought two flashlights with us.
• Bring vitamin C and lots of hand sanitizer. I did, like I always do when I travel, and I’m really glad I did. Everybody is coughing or sick, and everybody spits. It is similar to the situation in China, i.e. everyone spits. Then everybody gets sick. Bring hand sanitizer.
• If you take a bus somewhere, try to ask when you buy tickets to sit on the left side in the middle. The front is a no-go for me. I typically had seats in the rear right, but on the curvy roads I think the left middle would be safer, as oncoming buses won’t hit you. One bus on the way back had the left side decimated. If you do a search on “nepal bus crashes” in images.google.com, you will quickly see why you don’t want to be in the front row.
• My hotel rooms all typically had just one power outlet. If you bring multiple electronic devices, bring some way to  charge more than one at a time.
• Bring clothing to stay warm at night. I travelled to Nepal in winter, and all my rooms got pretty cold at night.

Enjoy!

For the last ten years I have run this website from various systems. First it was on Wordpress, then Mambo, then Joomla, and since early 2006 it has been running on custom code written using Django. I used this site as a learning tool for Django, re-wrote it after gaining more knowledge of Django, and then re-wrote it again when Google released App Engine. However, I recently realised that for the last few years I have spent more time writing little features than actually writing. I have entire trips that I never wrote because I was too busy writing code.

This week it all changed. I did the unthinkable. I moved this website to Blogger.

After evaluating some of the features of blogger, i.e. custom domains, location storing, ability to filter on labels, custom HTML/CSS, great integration with Picasa, and their mobile app, I realised I could virtually replace everything I had previously custom made.

This post gives a technical description how to migrate a site running Django, but readily applies to any blog running on custom code. I initially spent a fair bit of time trying to figure out how to convert my existing RSS feed into something Blogger could accept, but every solution required troubleshooting. I soon remembered why I love Django so much, and that it would be trivial to generate the correct XML for import.

1. Create Blogger Template
   I wanted to keep my design, so I hacked it to support Blogger. Take one of the existing templates, edit the HTML, and adjust it for your design. If you’ve worked with templates before this shouldn’t be too difficult.
2. Generate Sample XML
   The first step was to generate a sample XML file from Blogger to see what would be needed for import. Create a sample post with a unique name and a few labels, and location. In Blogger, go to Settings->Other and click Export Blog. The first 90% of the file will be for your template and other settings, but eventually you will find a section with entry elements in it. Copy this sample element out - this will become your template.
3. Format Template
   Using the sample section from the blog export, format it so the view you will create populates it correctly. A note of caution: the template needs time in ISO 8601 format, you need the id element, and the location element needs coordinates if there is a name. It won’t import later if there is a name with no coordinates. My template looks like this:

feeds/rss.html

{%  load blog_extras %}
{% for entry in entries %}
    tag:blogger.com,1999:blog-1700991654357243752.post-{% generate_id %}
        {{ entry.publish_date|date:"Y-m-d" }}T10:30:00.000123
        {{ entry.publish_date|date:"Y-m-d" }}T10:30:00.000123
        {% for tag in entry.tags %}
            {% endfor %}

        {{ entry.title }}
        {{ entry.content }}

        

        Joe Bloggs
            https://plus.google.com/12345689843655881853
            [email protected] 
{% endfor %}

This isn’t really RSS, so if you are pedantic you can name it something else. You will notice I loaded some template tags in there (“blog_extras”). This is for generating the random number, as this is needed for the ID element.. Here’s the template tag.

blog_extras.py

# 'import random' at beginning of file
def generate_id():
    id = ""
    for x in xrange(1, 7):
        id = id + str(int(random.uniform(400, 900)))
    id = id + "8"
    return {'obj': id}
register.inclusion_tag('blog/generate_id.html')(generate_id)

/blog/generate_id.html

{{ obj }}

4. Create Code To Populate View

This section should be easy if you have written your blog in Django. Simply populate the template, what I have shown as “rss.html” above

blog/views.py

def show_rss(self):
    q = Entry.all()
    q = q.filter("genre !=", "blog")
    entries = q.fetch(500)
    return render_to_response("feeds/rss.html", {
        'entries': entries,
        }, mimetype='text/plain')

I did a filter on the model to not include “blog” entries - these are my travel stories, and I exported them separately. Remember that this is all happening on App Engine, so you will need to adjust if using Django’s normal ORM.

5. Download Entries

Visit the URL you mapped to the “show_rss” function in urls.py, it should generate your list of entries. Copy and paste those entries into the exported XML from Blogger where you took out the original entry element.

6. Import Entries

Now go to Blogger and import your blog. With any luck you will have imported all your entries. You will probably need to do this a few times as you tweak the text. I had to remove some newlines from my original posts.

Optional Steps

7. Create Redirect URLS
   Links in Blogger appear to only end in .html, which is a problem for links coming from Django. Luckily, Blogger includes the ability to add redirects. Go to Settings->Other-Search Preferences. You can then edit redirects there. I generated a list of my old URLs and combined that with a list of the new URLs. Hint: you can use Yahoo Pipes to extract a list of URLS from a RSS feed. If you open any of the links in Excel and split on forward slashes, remember that it will cut off leading zeros. Set that field to TEXT during import.

I decided not to create redirects for every entry, as I didn’t really have time, and it only probably matters if somebody links directly to that page. I opened Google Analytics and looked at the Search Engine Optimisation page and sorted it by the most used inbound links. After getting down to entries that only had 1 inbound request per month I stopped creating redirects.

8. Host Stylesheets and Images Externally

Blogger won’t host host files, so you need to work around this problem. All my images are generally from Picasa, except very specific website related ones. I moved those to Amazon’s S3 and updated the links. I did the same with my CSS. You could probably store them in Google Storage, too.

9. Create Filters on Labels

If you had any previous groupings you can still link to them using label searches (in my case I actually added the “genre” as a label). The syntax is “/search/label/labelname/”, as you can see in my howtos section link.

10. Update Webmaster Tools

If your site is part of Google’s Webmaster Tools, you will want to login and take a look that things are OK. You will also probably want to update your sitemap (send Google your atom.xml feed).

TDX data has been available for a number of years on 131500.info, but many tools are GTFS specific. I also find GTFS easier to work with.

Luckily, converting from TDX to GTFS is not overly difficult, and below are some instructions. This howto is a bit old, as I am only now copying it from my “Notes” folder to put online to help others.

Note: You can now directly download GTFS from the TransportInfo website: https://tdx.131500.com.au

1. Signup for an account with EC2 (AWS), unless you have 16GB of memory available on a machine.
2. Upload TransXChange2GTFS to a place you can download from.
3. Upload the latest TDX data dump from 131500.info to a place you can download from.
4. Login to AWS and start an EC2 instance.  I picked a large instance and used 64-bit Ubuntu 10.04, us-east-1 ami-f8f40591
5. Download the Data and transxchange to /mnt

wget http://ec2-175-41-139-176.ap-southeast-1.compute.amazonaws.com/Data20110127.zip
wget http://cdn.kelvinism.com/transxchange2GoogleTransit.jar

EDIT 16-03-2025: I’ve since removed these files.

6. Install Sun JRE.

apt-get install python-software-properties
add-apt-repository "deb http://archive.canonical.com/ lucid partner"
apt-get update
apt-get install sun-java6-jre

7. Check how much memory is available

root@domU-12-31-39-10-31-B1:/mnt# free -m
             total       used       free     shared    buffers     cached
Mem:          7680        626       7053          0         11        329
-/+ buffers/cache:        285       7394
Swap:            0          0          0

8. Create a configuration file sydney.conf

url=http://131500.info
timezone=Australia/Sydney
default-route-type=2
output-directory=output
useagencyshortname=true
skipemptyservice=true
skiporhpanstops=true

9. If you’re on the train like me, start screen, and start converting. The number you pick for “-Xmx” obviously needs to fit in the amount of free memory you have.

java -Xmx104000m -jar dist\\transxchange2GoogleTransit.jar Data20120524.zip -c sydney.conf

I seldom used Facebook, and felt some kind of negative emotion after logging on each time. I don’t know if I am alone in feeling this way, but each time still felt a bit like a secret contest. A bit, not a lot, but sort of subconsciously. Honestly, though, even at an involuntary subconscious level, I don’t like this sort of negative feeling. I would rather be sharing ideas or creating something. Wish me luck.

EDIT 12-05-2025: Still off Facebook, and generally all other social media. I try to chase my dopamine through other means.

Here’s a little hack I thought I’d share. There are maybe other ways to do this, but this way took all of two minutes, which suits me. My daily routine has me taking a train along the Inner West Line (here in Sydney), and I wanted to know in advance if there was any issues with the line. 131500 provides weekely emails about upcoming trackwork, and there’s an RSS feed of delays; this is what you’d see in many of the Sydney transport-related apps. Another information source is through Twitter. For all of these sources I only really care about the one line I take every day - the quickest answer would seem to be if I could only get updates on Twitter for my line

Luckily this is all quick to do using an only service called ifttt. What I needed to do was create a task searching Twitter for “Inner West from:131500trains”, and if any new items are located it sends me a message on GTalk. This is one example of what I like about having some mostly open data - I can consume it how I want it. Let’s just say I’m looking forward to GovHack 2012

This example could be extended for other purposes, e.g. searching for the word “discount” on some suppliers Twitter stream.

Now that my Europe trip has ended, I need to leave a note of things I wish I had done better, both travel-wise, and the stuff I brought along.

I like to travel light, and find it a little comical when I see people struggling with giant 70L backpacks; I always do carry-on, no matter for a two week trip, or two months. (This means my packs have to weigh less than 7KG). Even in winter it is possible to pack light. The trick seems to be only packing what one will use regularly, with no duplicates, of appropriate size, and hopefully reusable for other purposes. Don’t pack anything you would be gutted if it broke.

The lessons below are not just to pack light, but also to reduce stress. Some really only apply to me for next time :) This is not a howto list on packing light for travel, just a few things I need to do better for next time.

Electronics
Only bring electronic devices that can be charged by USB, which leads me to my next point…

Look on eBay for a USB charger that has multiple inputs (make sure it has enough amps if you are charging certain phones). It became a hassle trying to charge four or five devices using just one USB charger. The charger, if going to Europe, should have room to accept a ground, or be able to get around the ground plug used in many places. In other words, get one like (broken old link) not like (another broken link) (basically, not like any of the block ones from eBay). Once again, try to get a 4-port one with travel adaptors included.

I’m going to consider not bringing my Kindle next time, but instead bring one paperback book. The weight might be about the same. I might not even bring a book - less stuff to break or get wet. I don’t do much reading on my travels anyway, instead writing in a journal or finding out what to do next.

Get an Eye-Fi or be able to read a USB stick with my phone. I need to figure out a way to backup photos from my camera to the cloud every night using only my phone (if the hotel has free wi-fi). Sorry if you are the guest next door…

Going Light

Don’t bring jeans, or bring only one nice pair of pants for semi-formal situation. I have to admit, I almost never wore my jeans. After only wearing them once or twice they would get wet, dirty, and/or smelly, so I would be forced to wear my Columbia hiking pants over and over! The pants are not particularly stylish, especially in European cities, but they are reasonable warm, repel water, dry quickly, and are exceptionally light. After weeks of abuse they didn’t smell, or look dirty, somehow. I’m a bit of a Columbia fan…

Try to find one pair of shoes that are ergonomic and athletic enough for walking 20KM/day, or on a trail, but also stylish enough to go into a nice restaurant and not draw too much attention. And dry quickly. I don’t really have any suggestions, since I’m still looking, but I’m certainly never going to put my Allen Edmonds in a backpack.

Bring cotton undershirts instead of normal shirts for winter travel. I never ended up wearing a t-shirt on the outside, so the shirts were basically just used to keep the rest of my clothing clean. I may look into undershirts that wick better than cotton.

Replace my money belt with something. It is just uncomfortable. I am trying to find an alternative, maybe something that goes over the shoulder instead. I only need something for my passport.

My fingerless gloves worked really well, because I could still use my mobile phone without taking off my gloves.

Staying Clean and Healthy

Bring two large dry bags for dirty clothing, a dry bag for larger electronics, a dry bag for electronic cords/cables/chargers, and a dry bag for lotions and creams. Or get indestructible bags. The zip-loc bags I brought all had holes within the first few days of travel, spewing my cables everywhere. The corners on any bottle tears straight through them. I have since bought some Loksak bags, and a year later they are still holding up.

Bring more vitamins, both Multi and C, especially if in winter. A tube from an energy capsule mix would work well to store them in, or Vitamin C fits well into a Tic-Tac container. Only bring enough pills that you need. For instance, I usually only bring two pills of Gastro-Stop, because that stuff works maybe too well. Also, vitamins were really inexpensive in parts of Europe compared to most places I have lived.

Long Johns next time I travel in winter and it looks to go below 0C. I were OK, but that extra layer would have been nice.

Buy a Mach 3 as soon as possible. I tried shaving in Prague before the opera with a normal three blade razor, and it was horrible. I had to use an entire pack just to see skin, and even then had little patches of hair that I couldn’t get. I’m never making that mistake again. Alternatively, I’m thinking of getting a portable Sanyo electric razorthat I saw my dad use when I were in New Zealand.

RELATED: I have added a similar entry on my recent trip to Nepal (12/2012).

I don’t know how many times I’ve lost tiny parts and had to spend time trying to find them. I realized after removing the back of my monitor that I’m starting to learn from those mistakes; instead of just sticking the screws in a box with other screws, and then having to dig them out, I taped them to the back of the monitor.

This sounds small, but after doing it I realised how many times I had to hunt through a box of screws trying to find the exact one that fit. Lesson learned.

If you’re like me there is a box of hard drives sitting in a dusty corner somewhere. Some are mine, some are others’, but they are all in a failed or semi-failed state. So, why have I lugged them around? I’ve been a bit paranoid about throwing them away. Some of the hard drives are encrypted, others aren’t, and the drives from friends certainly aren’t. Although the chances of somebody getting the drive from a landfill and restoring it is minimal, I never wanted to take the chance.
So, I kept lugging a bag of drives through each move.
My friend Clinton has recently returned from Europe, and he brought me a gift: a Swiss-made Victorinox, the ‘CyberTool’. After playing with it for several minutes I noticed it came with a Torx 8, 10 and 15 bit. This was a reminder that one method of mostly disabling a hard drive is to destroy the platters.
Other methods I have heard are to use a hammer and nails - which I unfortunately don’t have in our tiny apartment. Whilst finishing Dexter I started pulling apart the box of hard drives, and it surprisingly didn’t take long to disable them.
Step 1: Remove Torx screws

Hint: Don’t forget the screw covered under the paper.
Step 2: Use a flathead and pop off the cover

Step 3: Stare at the shiny platters

Step 4: Scratch platters with flathead, and bend platters if possible

Step 5: Dispose of bits and pieces

Warning! The CyberTool didn’t have a Torx small enough to open the 2.5" hard drives, but I could just use a pair of pliers to lift up the cover and jam a flathead in there. However, and here’s the warning, 2.5" platters are sometimes not made of metal. I forgot about this on the third 2.5" hard drive and covered the desk I share with YS with shards of glass.

A short while ago I wrote about visualizing transport by using 131500’s TDX data, converted to GTFS, and served by GeoServer. Because I’ve started playing around with Google’s Fusion table, I thought it would be interesting to see what all the transit stops in Sydney look like in FT. So, voila!