Transcending Frontiers

Like most programmers, I don’t like to repeat manual tasks, well, manually. We use Ubuntu Server almost exclusively on our servers and whilst package management is very simple (login, aptitude update, aptitude dist-upgrade, all done) when you have to look after many (fourteen and expanding) servers it can get repetitive.

I use Terminal (or is it called Terminal.app?) on Mac OS X as my terminal emulator, and I’ve messed around with using Applescript to open multiple tabs before, so I figured this was a problem worthy of a Sunday-morning.

What I envisaged was iterating over a list of servers and having Applescript manage a Terminal instance, opening new tabs where appropriate, and executing the upgrade command in each tab for each server. Sounds simple enough.

And it was. I used Martin Ström’s excellent article as a base for my script, and ended up with the following:

set tab_count to 0
set servers to {"astrotrain", "bumblebee", "grimlock", "ironhide", ¬
	"jazz", "laserbeak", "mirage", "prime", ¬
	"prowl", "ratchet", "ravage", "rumble", ¬
	"soundwave", "wheeljack"}
-- Update the package list, dist-upgrade and remove the downloaded
-- packages so they're not included in the backups
set dist_upgrade to ¬
	" 'sudo aptitude update &&
	  sudo aptitude dist-upgrade &&
	  sudo aptitude clean'"

-- Make our settings globally available
global tab_count, servers, dist_upgrade

-- Mainline
on main()
	tell application "Terminal"
		activate
		repeat with server in servers
			set cmd to "ssh -t " & server & dist_upgrade & " && exit"
			my open_tab(cmd)
		end repeat
	end tell
end main

on open_tab(cmd)
	tell application "Terminal" to activate
	my create_new_window_or_tab()
	tell application "Terminal" to ¬
		do script with command (cmd) in last tab of window 1
end open_tab

on create_new_window_or_tab()
	if tab_count ≤ 0 then
		tell application "Terminal" to do script ""
		set tab_count to tab_count + 1
	else
		tell application "System Events" to ¬
			tell process "Terminal" to ¬
				keystroke "t" using command down
	end if
end create_new_window_or_tab

-- Run the mainline
main()

Apologies for the highlighting - the plugin doesn’t understand Applescript.

I came from a Microsoft background, when I was there I complained about some aspects of it, but in hindsight one thing I miss a lot is a good IDE for SQL and database related development. SQL server didn’t have anything amazing, but it did the job and was stable.

I’m a big believer in visualisation and easy visualisation when working with data. It’s there to be used and if you can’t visualise it simply, if you can muck around with it and fiddle and massage it into various shapes and sizes easily then it’s more likely to be left alone. Data that’s left to rot is like leaving gold to rust. If gold rusted.

In short data is valuable, but only if it’s used.

In my quest for tools I’ve tried a few and found none that I would actually pay for, except the one I’ve been using now for a couple of months without a problem. DBVisualiser Free edition, I’d pay for that. It’s lucky too because they make a paid version with a bunch of extra features.

I’m always on the lookout for better applications to improve my performance and more importantly my enjoyment at work. So far this has been the closest application I’ve found to make MySQL a joy to use on the Mac. The free version does 99% of what I want to do, the pro version just seems to add features to make life even easier.

I’d urge people working with a database on the Mac to give DBVisualiser a try.

Last week Gregg Pollack from the Rails Envy Podcast put out the call on Twitter for reviews of their latest product, the Ruby on Rails 2.2 Envycast. I was happy to oblige and had been looking forward to checking out their unique take on screen casting for a while. Here’s what I thought.

Is it really a screen cast?

Firstly, I don’t even know if it’s correct to call it a screen cast. The viewer is not presented with a computer screen once during the 44 minute presentation. Most screen casts record the presenter’s computer desktop and have an audio commentary to accompany the video. Gregg and Jason have instead gone the “weather-man” approach and are using a green-screen as their canvas. This has some immediate benefits, but also has some downsides.

It develops the duo as the “face” of Envycasts. I first learnt of them through their audio podcast but they’ve done video before. I think this is important as it presents a teacher-student situation, and I’d expect there’s a bunch of psychology proving why face-to-face teaching succeeds. I think it works well for them, but would expect that other tech-screen casters might not be as comfortable in front of the camera.

The green-screen also allows them to manipulate the background as they see fit. For the viewer of this episode that means you’ll be treated to a world tour as they “travel” to various capital cities around the globe. It allows them to use their unique style of humour with ease too. I actually laughed out loud at a few moments, and enjoyed thembeing chased by the large ball of yarn in particular.

More importantly though, the green-screen also means that transition effects and attention-grabbing techniques (like exploding or rotating text) can direct your attention to the exact part of the code they’re discussing.

By not filming a mouse-cursor style screencast they’ve chosen to give up the opportunity to pass on subtle information outside of the scope of the specific topic that viewers might find useful. What do I mean? I’ve picked up on keyboard shortcuts or other techniques people use in their day-to-day programming by watching traditional screencasts. The best example I’ve found of this is Ryan Bates’ excellent Railscasts. I liken it to looking over the shoulder of a seasoned developer - you’ll pick up on nuances which are sometimes just as interesting.

Both Gregg and Jason speak clearly and are presenters that I enjoyed viewing. As an Australian, I didn’t find their accents overpowering in anyway.

It’s all about the content

The actual content is conveniently split into chapters (I viewed the 260MB MOV) which makes navigating between topics easy. I viewed the video on both a MacBook Pro laptop and 42″ plasma. I found the colours sometimes a little dark on the laptop but didn’t have the same issues on the plasma display. I suspect that my colourblindness was to blame :)

I would have appreciated a wide-screen format for the plasma, though I presume that the majority of viewers are watching on a computer screen.

The content is broken into the following seven sections:

• ActiveRecord
• ActiveSupport
• ActionPack
• ActionController
• Railties
• Internationalization
• Performance

The 38 minutes (85%) are spent focussing on the code-heavy components which make up the changes to Rails 2.2. The examples given are very clear and consistent across the sections and I found the afore-mentioned transition effects helpful at tracking the exact part of the code the presenter was discussing. Your eyes are drawn to the moving parts of the code. Because you’re looking at Ruby code there’s nothing that will hurt your brain, or eyes.

The last few minutes in the presentation are spent discussing the other things that have changed with Rails 2.2 in details beyond just code examples. I enjoyed Jason explaining why gems are preferred over plugins, and Gregg’s explanation of etags and how they’ve changed in Rails 2.2 was fantastic.

Wrapping up

I was skeptical that this format would would for a such a technical topic but I’m glad to say they boys pulled it off. Their lame jokes are sometimes actually funny, though I think I’ve heard enough about Rails not scaling to last me a lifetime :)

My final thought: above all else they jammed a stack of detail into an easily-digestible format that was entertaining and informative. If you’re working with Rails then you really should be checking this out.

You can buy this Envycast along with the accompanying PDF online for just US$16, which even when you consider the current global economic situation is still a steal.

Disclaimer: The Frontier Group was provided with a free copy of this Envycast for review.

This took me a few minutes today to work out and find an answer on Google so I thought I’d share it.

I had a string that I was posted from a form and while Firebug on the client said that the string was valid, on the server side PHP’s json_decode() function was returning me a NULL indicating invalid JSON. I spit out the string using good old var_dump() and I could see straight away that I would need to strip some slashes from it, but I couldn’t figure out why.

I don’t like to simply know how to fix something, I like to know why it was broken in the first place! A bit of Googling gave me the answer, it turned out to be because of the magic_quotes setting in PHP.

Instead of just wrapping everything in a stripslashes() call I decided to write a wrapper function that will take into account whether magic_quotes is on or off. It will also allow me to do whatever I want later one if I wanted to parse a particular type of data or something.

Here is the code. As you can see it’s nothing fancy, but it works and hopefully it saves people some time.

function _json_decode($string) {
	if (get_magic_quotes_gpc()) {
		$string = stripslashes($string);
	}

	return json_decode($string);
}

We have deployed a new Zimbra server to gradually replace our existing mail server. We’ve moved all of The Frontier Group’s mail over to use the new system, which gives us a good chance to run it through its paces before moving clients over.

I have been signed up to a mailing list of a Perth-based mining company, and with complete (and illegal) disregard for the Spam Act of 2003, their removal process doesn’t work. They should really use Raven Communicator.

I have rules in my mail client on my Mac that filter out this nonsense when it’s open. However, I’m also using my iPhone to retrieve mail, and the mail filtering rules you set up in Mail on your Mac don’t translate to the handheld.

All of these combine into a situation where I needed to have mail filtering applied before it hits any of my clients; iPhone, Mac Mail or the Zimbra web interface.

Thankfully this is trivial to set up in Zimbra. You login, select the Options > Mail Filters tab and create filters as necessary. The interface is very self explanatory, and any filtering rules are applied before you see your mail.

I can rest assured knowing that this mail is being marked as read and funneled into my trash.

Sometimes you’re working on a server remotely and the ssh connection gets interrupted. Usually when that happens, ssh will stall until the socket times out and you’re stuck with an unresponsive terminal. To get back to the terminal from your stalled ssh session, use this key sequence:

<enter>
<enter>
<enter>
<shift-tilde>
<period>

A handy set of cheat sheets for any web developer. The full listing can be found here over at WebAppers.

jQuery

Ruby on Rails Cheat Sheet

Prototype Cheat Sheet

Scriptaculous Cheat Sheet

Javascript Cheat Sheet

HTML Cheat Sheet

CSS Cheat Sheet

Mod_Rewrite Cheat Sheet

Regular Expressions Cheat Sheet

PHP Cheat Sheet

MySQL Cheat Sheet

In most languages there is a means of building strings and printing out messages. Many will find the printf command all too familiar. In the JavaScript space however things have been a little slow to develop.

We have our strings and to build them up we merely start adding them together but as our code gets more elaborate this gets harder and harder. The biggest problem is usually trying to figure out how to escape quotes and inverted commas as we swap back and forth between plain strings and JavaScript variables.

I’ve often thought there must be a better way and working with the Dojo toolkit there is. It can be found in the method “dojo.string.substitute”. This mirrors the idea used in other languages of passing the method a template string and an object of values to substitute in. Whilst Dojo allows for numerous ways of passing the data such as arrays and functions the one that interests me the most is a standard JavaScript object which is effectively a hash map of name and value pairs.

Here’s a simplified version of the Dojo function that only uses objects for the value map.

var tfg = {
  string:{
    substitute: function(/*String*/ template, /*Object*/ valueMap){
      return template.replace(/\$\{([^\s\:\}]+)(?:\:([^\s\:\}]+))?\}/g, function(match, key){
            return (valueMap[key])?valueMap[key]:"";
        });
    }
  }
}

The old way of doing things might look like this:

var data = {name:"bob", occupation: "builder"};
var string = "His name is "+data.name+" and is occupation is "+data.occupation+".";

Here we merely add the different parts of the string and string values together. Simple enough in a string this size but as it grows and we start working with HTML syntax things get more complicated.

Using the substitute way:

var data = {name:"bob", occupation: "builder"};
var template = "His name is ${name} and is occupation is ${occupation}.";
var str = tfg.string.substitute(template, data);

Here anything surrounded by the format “${something}” is replaced with the value found in “data.something”.

A major advantage of this approach is when we start looking at fetching JSON data sets from a remote server via Ajax. We almost always have to process the return values into something to display and if it is looping through a list of objects we can now define the template once and repeatedly create a string with our substitute method. Here’s a simple idea using JQuery for the loop.

var result = [{name:"a",company:"1"},{name:"b",company:"2"}];
var template = "<tr><td>${name}</td><td>${company}</td></tr>";
var rows = $.map(result, function(index, item){
  return tfg.string.substitute(template, item);
})
var tableBodyString = rows.join("\n");

Here we looped through the items substituting each new set of values and pushing the result into an array. At the end we merge the array into a single string with “join” and have the body of a HTML table ready to be used in the page.

This approach is fast, easier to read and much cleaner to implement when looping through data.

Lately I’ve been looking into how much time I spend in front of Mail (Apple’s email client). I feel like it’s always a struggle to get away from email for two reasons. If I close Mail I am afraid to open it again as I know it will be a chore to get to the bottom of the pile , but if I leave it open all day I find it hard to complete my day’s work without constant interruption.

Time for a personal assistant you say? 

Maybe, but there might be a better way in the interim. Like trying to find the ultimate way of organising tasks, keeping on top of email can be just as daunting (read: useless). I poked around the 43folders web site and may just have found it.

Merlin Mann is the creator of 43folders and his website about finding the time and attention to do your best creative work.

Delving in to some of the posts, I managed to get a basis of how I might go about keeping on top of emails (codename Inbox Zero).

• Better practices for staying (near) zero
• Quick tips on processing your inbox
• 5 fast email productivity tips

After having a read of this site and venturing off into Google, I finally had a plan. Create some folders in Mail (abandoning my previous structure) and action emails in the appropriate way. I’m not creating anything new and revolutionary here, but so far using these ideas is working for me (and making a massive difference to the time spent in front of email).

For anyone possibly interested in the way I am doing things, let me know in the comments. I might just find the time to follow this up.

In a previous job I dealt with the IT tasks related to a multi level marketing business. We stored everything in a database of course and due to the nature of multi level marketing most of the operations performed had to take into account the tree like structure of the people involved.

An example would be, given a party that was run there would be rewards based on future bookings made, how much the party sold and so on. The largest direct benefitor of these rewards was of course the person that ran the party. However this person had a manager, who in turn may have a manager who in turn may have a manager and so on up the tree. At each level the manager got a cut based on a number of factors such as how many people she was managing, what her position was an so on.

At the time the way I dealt with trees was via adjacency lists, mostly because that’s what was most logical to me and how it seemed everyone else did it. I thought I’d have a look around and see if anything had changed in the last couple of years and found this article on using nested sets to deal with hierarchical data.

It seems to me that this method trades insertion performance for selection performance which for most situations is fantastic! It seems to be exactly what I need given I’ll be performing far more selects than inserts and it also has given me another tool in my toolbox.

It’s long been understood that to keep someone in IT happy one of the most important things is change and new stuff. I think this constitutes new stuff for me.