BDD: Book Driven Development

(This one is also on the Pathfinder blog, but since it fits in here, I wanted the full text here...)

Jay Fields, who has been posting a very nice sequence of nuts-and-bolts Ruby and Rails guidelines, pauses to talk about creating examples. It's a topic I've wanted to write about here for a while, and this is as good a lead-in as any. Plus, I'm generally interested in how principles of software development apply or don't apply in odd cases, and software being developed specifically for example purposes certainly qualifies as an odd case.

For Professional Ruby On Rails, I knew that I wanted to run a single example application through the book. I had some grand visions of it being a "real" application or at least a real example of coding best practices. For best practices, I think it's pretty good on a method-by-method bases, but has some weaknesses as an entire app, for reasons that I think will become clear.

The first question was exactly what I wanted the sample application to do. On some level, this is a superficial question -- who really cares what the fake app in the book does? At the same time, certain application structures would make it easier or more plausible to discuss certain features. And I had a list of features I wanted to cover -- legacy databases, users and roles, navigation, and graphics. Ideally, the sample app would have some kind of consistency that would make it reasonable to have these features.

In addition, I didn't want it to be a thinly-veiled version of a site that everybody would recognize, and I wanted it to be something that I could type every day for six months without going crazy.

One of my first ideas was to resurrect an old Web 1.0 joke for my killer e-commerce plan: a web-based store for selling and delivering gasoline. But I decided that the gag was a lot less funny with gas in the $3 per gallon range, and I also decided that I didn't want the fake site to be primarily an e-commerce site.

Brainstorming ensued. I almost did a social networking site for pets, but I decided that was too silly, subsequently discovering that there are already a zillion pet social networking sites...

Eventually, I settled on Soups OnLine, a soup recipe-sharing site. It had the advantages of:

1. Having a data model that could plausibly handle RESTful nested resources (recipe and ingredients)

2. Having an actual user model.

3. Being a plausible site, while still being silly enough to keep anybody from taking the whole thing too seriously.

The first problem came up almost immediately -- site design. I felt that a site that was just dull HTML would not be credible or interesting, but at the same time, I didn't really have the time or resources to create a full site design from scratch, nor did I want the book to become about site design. My solution, which I'm not prepared to defend to the death, was to integrate a template from freewebtemplates.com. This kind of sideswipes the whole issue, providing reasonably good-looking site without having to get into the weeds debating CSS minutia.

Over time, the example mutated away from a normal application development process in a few interesting ways.

The nature of the book-writing itself was a strong anti-agile push, in much the same way that any large body of documentation is. Changes in code structure affect the tests, which is fine, but also affect some amount of text in the previous 300 pages, which there's no easy way to find. (As you can read in this post, the issue affected not just me, but also the editors on the book). At least once, a late change in Rails invalidated a helper method I used in an early chapter. Not a big deal, except I also used that helper method to demonstrate a test structure later in the book -- I wound up keeping a shell of the original method redirecting to the new Rails method.

Also, some features that I had hoped to get in, didn't quite get there -- because I ran out of time or because they didn't with the chapter structure. (If you read the book, you'll see some early structures that imply that the app will eventually convert between English and Metric, which I never quite got to.) Again, this is the kind of thing that happens in all the time in real projects, but is hard to smoothly redirect when there's all that pre-existing text describing the earlier plans.

Sometimes, the sample app has more features than a real app would, rather than less. Since the book tries to cover multiple plugins or ways to accomplish things, Soups OnLine wound up with far more plugin load than a normal app would -- a regular app wouldn't normally have RSpec and Shoulda and Dust, or RMagick and MiniMagick and ImageScience. Inevitably some conflicts happened between multiple plugins trying to cover the same space, making the final code somewhat wilder than I'd really like. (If you've read the book and want to experiment with the outstanding code, it's probably easiest to work with around chapter 9, before Globalize and the image plugins are added...)

Ultimately, of course, as Fields concludes, it's the author's fault if the example doesn't help the user understand the issues at hand. It's my job to understand the constraints of the form and still try to present as realistic an application as possible while explaining how to to build similar applications.

iPhone SDK

I'm trying to figure out exactly why I'm so psyched by the Apple iPhone SDK announcement. The basic announcement wasn't a surprise, and I don't even own an iPhone. I did, however, dig out my Cocoa programming book and start studying.

Further thoughts:

• The tools themselves seemed somewhat slicker than what was expected -- a lot of Mac developers were pleasantly surprised that Interface Builder was included (although apparently it's not in the first beta). There was a lot of speculation that the SDK release was delayed over security issues, but it also looks like polishing the tools took some time.

• The demos were exceptionally well done, a very nice range of choices, and the continual push of how easy it was to make them. The quality of the demo is clearly one of the reasons that I'm excited.

• As an aside.. say your boss came up to you and said you're going to fly to Cupertino for two weeks to build something in the super-secret iPhone SDK, which you will get to present in a full-on Steve Jobs media event. How many nanoseconds would you need to wait before accepting?

• Let's see, the iPhone 2.0 comes out the end of June. So... by August 1, it should be the top mobile gaming platform going, right? Those were some seriously cool looking games...

• The terms were more open than I expected. It's particularly nice that Apple is allowing the developers to set the price. It's particularly nice that they specifically included "free" as a price. They seemed to emphasize that Apple was going to try and get as many apps in front of developers as possible -- I'd take the inclusion of AIM as a demo as a sign that even things that AT&T might not be thrilled with will be allowed.

• The general sense of the existing indy Mac community is that the 70/30 split is reasonable given what Apple is offering. It'll be interesting to see how the price points play out. (As pointed out by Jens Alfke, the App store model enables an entirely new class of micropayment software)

• Still a number of questions: how will developers manage limited betas? Will developers be allowed to have the download be free, but have a separate license authorization? What about free but add supported (like Twitteriffic)?

Early line, here's a short list of people or apps off the top of my head I expect greatness from:

• MobileTwitteriffic, already essentially announced. The trick here will be improving on the web UI.

• Red Sweater, apparently Daniel is considering porting Black Ink, and maybe MarsEdit.

• Ambrosia Software. For a really great game -- Sketch Fighter would work...

• The Desktop Tower Defense guy. The UI would be tricky, though...

• Somebody to do the killer To-Do list or GTD app. Omni?

• A really cool doodle/draw program.

I think this is going to be a very fun ride...

Two Pathfinder Blog Posts

Two things on the Pathfinder blog.

• Agile Publishing, on publishing experiences and agile methods.

• Live Ruby: Testbed, an attempt to work through a small test and metaprogramming problem live and on the blog.

Enjoy.s

More On Test-Driven Development

My first post to one of Pathfinder's official blogs is up, it's a companion piece to the blog post here on Test-Driven Development, and you can find it here.

And Now, A Special Announcement

Sorry for the radio silence for the last week or so, but I was waiting to be able to announce this:

Today I started a new job at Pathfinder Associates, as a senior software engineer in charge of Ruby on Rails projects. Needless to say, I'm thrilled to be back in the Web game professionally, and even more thrilled to be working on Rails projects.

Everybody at Pathfinder has been super great so far, and I'm looking forward to doing exciting and fun things.

I'll also be blogging on their web site roughly once a week, either at their Technical Development blog or their Agile/Ajax blog.

If you happen to be one of the maybe five people who read this blog regularly, don't worry, I'll continue to post here with updates about the book and thoughts about programming and technical publishing that wouldn't quite fit on the Pathfinder site.

Didn't I Say I Wouldn't Compare Languages?

I posted a version of this to JJ Behrens' Blog post about Ruby, and decided it was probably worth also posting here.

I use and like both Ruby and Python, here's why...

Things I like about Ruby with respect to Python

1. I think Ruby is the only language that gets accessors right. The thing you want to do 95% of the time -- simple access -- is trivial, and the thing you want to do 5% -- something fancy in your accessor -- of the time is a pretty easy override. Plus, it's nicely encapsulated, and you never have to wonder if that thing in another class is accessed directly or via a method.


2. Blocks. I find that my kind of functional style flows pretty easily in Ruby. It's also not hard to pass named functions. When I first started with Ruby, it used to bother me that you couldn't tell from a method signature if it took a block, but I've since managed to deal.


3. Being able to add method to existing classes. Although I know this really bugs some people, sometimes adding a method to String cleans up the code significantly.


4. Expression-based syntax. The implicit return in Python always causes me at least one error per sessions. Plus I like writing a ternary operator as an actual if statement.

Things I like about Python with respect to Ruby

1. Consistency. Python enforces it. As a result, my Python code is more likely to be readable by me six months later. Ruby tends to be more concise, but some parts of Ruby are still a little too Perlish...


2. Real keyword arguments (coming to Ruby soon). The Ruby syntax magic of gathering up map pairs into a hash is sort of annoying.


3. I find Python's multiple inheritance mechanism to be easier to understand then the Ruby package/module setup. It always seems like I'm messing up the difference between 'require' and 'include'...


4. On balance, I prefer Python immutable strings to Ruby's mutable string/symbol split.

Programming Perl, Personal Edition

Due to circumstances somewhat beyond my control, I find myself working in the largest Perl project I've ever done. Now, I'm not in the least interested in a "my language is better than your language" deal because a) this strikes me as very well covered territory, and b) it seems particularly pointless as the Perl 6 team appears to be doing a nice job of taming Perl's more rococo features.

There are, of course, some things I did want to mention...

• I'd be totally lost without the very large assistance of Damian Conway's Perl Best Practices book. It's an excellent guide to the minefield of all the things Perl lets you do, pointing you to the things you actually should do (especially on a large program). It's also refreshing to see a Perl book that doesn't treat the ability to write unreadable code as some kind of rite of passage or inalienable right.


• I didn't actually intend to use all three homonyms of "write" in that last sentence, but I think I'll leave it in...


• As a result of reading Conway's book, and somewhat in self-defence against my own Perl skills, I seem to be writing the most verbose Perl in the history of the known universe. I think I may have written more compact Java.


• This is also the first time I've worked this closely with a code-tidy utility, and I think I like it. Usually, I advise against doing things like lining up equal signs because it's too much work to maintain, but with perl tidy, that's not an issue, and it is more readable.


• Perl's cultural insistence on using plain non-jargon words is usually kind of charming, and only occasionally irritating (quick, rank "carp", "cluck", and "croak" in order of severity...)


• My most consistent error to date? Forgetting to dereference the argument to grep or map, which I've been doing at the rate of about twice a day for the past two weeks. Since the reference gets promoted to a single-element array, this is a little annoying to track down. Luckily, unit tests were designed for this kind of thing...


• I wanted to quick mention a couple of formatting things that Conway recommends in his book that I've never seen in other language's style guides (maybe I should get out more...). First is the "cuddled else", which is the placement of an else line on the same line as both the close of the if block and the start of it's block: } else {. A quick google on the phrase confirms that it's mostly a Perl thing -- I've never seen a non-Perl style guide comment on this. Conway, and I see that he seems not to be alone in the Perl world, recommends dropping the else to the next line. The idea being that it's more visually clear because the else is now in line with it's close bracket just like any other block. I had always thought that the Java/C style was there to visually connect the else to the if that had just closed.


• Conway also recommends breaking up long lines before the operator, rather than right after an operator. I'm reasonably sure that's different from almost any other style guide I've ever seen, but I have to say, it works for me. The idea is that you are more likely to see the visual cue that the line is a continuation if the hanging operator is at the very left of a line rather than the right. I think it's working for me partially because the split lines look so weird to me that I look more closely to see what's going on. Whatever the reason, it does seem to be making the code more readable for me.

An Agile Musing

Of course, since I muse in an agile way, I reserve the right to change my mind based on future developments...

Software development usually takes place in a complex environment where your goal can change quickly. In general, there are two ways to deal with a complex environment. One is to try to anticipate, in advance, every possible permutation you might need to deal with, and the other is to manage your environment with the flexibility to respond to new challenges with minimum effort. Software is just a specific case where that challenge plays out.

Of course, being software engineers, we've given these approaches names. They are nicely defined by James Bach:

agile methodology: a system of methods designed to minimize the cost of change, especially in a context where important facts emerge late in a project, or where we are obliged to adapt to important uncontrolled factors.

A non-agile methodology, by comparison, is one that seeks to achieve efficiency by anticipating, controlling, or eliminating variables so as to eliminate the need for changes and associated costs of changing.

It seems worth pointing out that, at least in theory, there's no reason why you can't both make a reasonable effort to find potential changes up front, and still make your working environment as flexible as possible.

In practice, of course, this is kind of difficult to do well. There's a logistics problem -- the kinds of things you do when you're trying to control change up front generally involve creating a lot of models and documentation and the like. This is exactly the kind of thing that works against minimizing the cost of change. But it seems to me that a group determined to do some up front analysis of what will largely be an agile process could manage to work around that.

The bigger problem is an issue of mindset -- if you think that you can solve all your design problems up front, then you start to look at any change to the design or implementation with suspicion. At that point, any change is, by definition, a mistake of some kind. Which leads to suspicion of all changes and can mean all kinds of tracking and procedural overhead that pushes up the cost of change. The sort of organic, bottom-up design that you get from test-first and tight iterations doesn't fit in this model at all.

Although I should say that in my experience trying to get groups to adopt XP or agile practices over several different companies is that nearly all programmers and managers agree that automated tests are a good thing. Most programmers are at least willing to consider the idea of test-first, although getting somebody to do it consistently is tough (hey, it's tough even for somebody who's totally bought into it).

The resistance I do get tends to be around the idea that you can start programming with an incomplete design without it leading to disaster.

Waterfall-style up front design can be incredibly seductive. You're brainstorming with other smart people, you're solving all your problems before they even come up. It certainly feels like you are doing something vitally important to the success of your project. Even if you know that many of the decisions will later be revised in implementation, it still feels good to have that crisp UML diagram. You can't have bugs in a UML diagram. The issues that do get solved in design are assumed to justify the time cost of the design, because the assumption is that the time cost of later improvements will be much larger (which is true, in part, because of the amount of design work).

In contrast, starting coding with incomplete information can feel risky, especially the further you get from the actual implementation team. Any issue that gets changed later can be blamed on the relative lack of design, true or not, and the time savings from not doing as much design can be invisible. It's very hard to let go of the idea that all your design problems can be solved up front. But once you are willing to allow that some problems can only be solved in the moment, you're much better equipped to deal with the inevitability of change.

from internet import *

Three posts that caught my eye today.

Ruby School

Gregory Brown over on O'Reilly net has an article about using Ruby in Computer Science courses, at least in later algorithm classes. It's not a bad argument, but I think it'd be more convincing if the Ruby example was a little cleaner and easier to read compared to the pseudo-code.

Let's see... The last time I had to care about this issue was about eight years ago when my grad institution was going through a somewhat controversial revamp of the CS curriculum. The fight, as always, is between the theorists and the pragmatists. The theorists want to teach a lot of "pure" CS up front -- Turing machines, big "O" analysis, computational theory, that kind of thing. The pragmatists want the students to be able to get jobs.

You should know that I spent the better part of three years as part of a group working with an object-oriented class that we taught in Squeak Smalltalk. Lovely language, to be sure, but we had to spend part of every course explaining to some nervous students why we weren't teaching them C++ or Java...

At the time, the initial CS classes were moving to Java, with some relief. This is because a) nobody wanted to inflict C or C++ on unsuspecting new CS majors, and b) the previous most common language, Pascal, was woefully obsolete. Java is reasonably straightforward to teach and is actually used in real programs, both high points.

Personally, I think you can make a pretty nice case for a scripting language like Python or Ruby in the initial CS class. They are both pretty easy to get started with, the syntax is clean enough that algorithms are easy to visualize (which was Brown's original point). In Python you can do it without introducing objects (which most CS1 classes didn't do eight years ago, don't know if that's changed). In Ruby it's easy to teach meta-programming.

Cha-Ching

Paul Julius of ThoughtWorks about how CruiseControl can save you $12,535 per broken test. The money coming from the difference between the cost of fixing a bug immediately versus not catching the bug until integration testing.

I dunno. I love continuous integration, and would shout about it from the rooftops if they'd let me on the roof, and that number still sounds a bit more "look what I can do with numbers" than "look what I can do with Continuous Integration". But then I'm skeptical of nearly every numerical analysis of programming productivity.

Plus, Marshmallows

Over at Roughly Drafted, Daniel Eran goes on about the smooth, harmonious relationship between Apple and Sun. Naturally, I want to talk about one of his sidebars...

The name of Apple's Mac OS X frameworks was even named Cocoa in part to associate it with Sun's Java. The other reason was that Apple already owned the Cocoa trademark, having using it earlier for a children's programing environment.

You know, I've always wondered about that. The original Cocoa was a project that was being worked on in Apple's Advanced Technology Group the summer I interned there, plus it some buzz in Educational Technology circles for a while. Internally, it was called KidSim, but the name was changed to Cocoa when it was being prepared for release. Java was programming for grown-ups, so Cocoa was programming for kids. It seems like Apple isn't really using that connotation of the name anymore.

The project (now called Stagecast Creator) is a graphical rule-based programming language, something like a cellular automata program. The user specifies an initial arraignment of sprites on the screen, then specifies how that arrangement should change in the new time slice. Complex programs could be created with almost no typing (although, like all such programs, you still had to use drawing tools to create your own sprites -- that was still hard). Stagecast still seems to be around, although it's been ages since I tried the software. It was pretty cool, though.

Comment On This

So the other day I'm looking over some code, and I see this... (slightly paraphrased to protect the innocent -- in the original, the declaration and the getter were, of course, separated.)

/**
 * The name of the user
 */
private String m_userName;

/**
 * @return The name of the user
 */
public String getUserName() {
    return m_userName;
} 

And I thought, "I really hope some of that was generated by the editor"

And then I thought, "This is why other languages make fun of Java"

And I finally ended up with, "Most of what beginning programmers are taught about comments is useless". At least that was true when I was in school.

For the moment, I'll put aside the Java issue, to talk more generally about comments.

First though -- we're agreed the example is absurd, right? In that it repeats that the user name is, in fact, the user name five times. Which is at least four more than strictly necessary (I realize that Java more or less forces some of this duplication).

I realize that this is hardly the worst coding sin you can commit, but when I look at code like that I do think that either the programmer isn't quite sure which parts of his code are most important or that the person is rigidly following an overly formal standard. Neither of which is all that flattering.

About 95% of the time, it seems like you either get no comments, or the kind of useless comments in the example. Given the choice, I'd rather have no comments -- it's less distracting, and you can see more code at once.

However, you can do comments effectively. Here's what I think, which should not ever be confused with what I actually do.

• The best commenting is clear and accurate names for your variables, functions, and classes.



• The "No Duplication" rule applies to comments at least as much as it does to code.



• Possibly more, since no compiler is ever going to catch if your comments fall out of sync with your code.



• Therefore, under normal circumstances, comments should avoid repeating what the code does.



• However, limitations on input or output values that are not apparent from the code should be included in comments. If the user name was guaranteed to be under ten characters because of the underlying database, that would be a useful comment.



• Rationales for choosing a particular implementation are often good comments, as is the code's place within the larger program.



• If you find yourself commenting inline within a long method to explain what the next section does, odds are you'd be better of extracting that to a well-named method and skipping the comment.



• There is a cost to commenting -- it takes time to do well, and it can be distracting. It also limits the amount of code you can read at once.



• The most obvious exception to all of the above is when you are in a situation where people will read your comments without ready access to the source code. Writing an API or a framework, for example. In that case, most of the issue about duplication doesn't apply and you need to be descriptive for the benefit of users who will only see your JavaDoc or RDoc.



• However, none of that excuses writing a one line comment for getUserName().

Posting to Blogger via Ruby

TextMate has what seems to be a very nice blogging bundle for programmatically sending posts to your blogging engine of choice. Except that it doesn't work for the new Blogger API. Or at least it didn't the last time I checked. Mostly I just wanted to see if I could write my own script to send to Blogger.

This is a Ruby script based on the Python script located at http://djcraven.blogspot.com/2006/10/success-posting-to-blogger-beta-using.html -- the author of that script did the heavy lifting in terms of the GData API calls, what I did was translate it into Ruby in a somewhat more flexible structure. At least I hope so. I also hope this will look like decent, idiomatic Ruby and not like a horrific hack. And I'd like a pony. If you're asking.

Let's do this in pieces. The first piece of the puzzle is using the Google ClientLogin API to get an authentication token. The token is then passed as a parameter to later calls when we actually want to post something. Google says that the lifespan of the token is dependent on the application being used, but I don't see where they specify how long Blogger keeps them.

Here's the start of our class. We're in a module called Blogger and a class called Blog. I've got an external dependency here on BlueCloth because I'm also going to automatically translate Markdown later, but that's not something you need to do...

  module Blogger

    require 'net/https'
    require 'net/http'
    require "uri"
    require "bluecloth"

    class Blog

      attr_accessor :account, :password, :blogid

      @@auth_url = "www.google.com"
      @@auth_path = URI.parse('/accounts/ClientLogin')

      def http
        http = Net::HTTP.new(@@auth_url, 443)
        http.use_ssl = true
        http
      end

      def request_data
        ["Email=#{account}", 
         "Passwd=#{password}", 
         "service=blogger", 
         "service=TestCompany-TestApp-0.0"].join("&amp")
      end

      def auth_headers
        {'Content-Type' => 
            'application/x-www-form-urlencoded'}
      end

      def auth_token
        response, data = http.post(@@auth_path, 
            request_data, auth_headers)
        return nil unless response.code == '200'
        data.match(/Auth=(\S*)/)[1]
      end

After the setup lines, we've got three methods here. The first two define the data objects. The first one creates a Ruby HTTP object, set up for an HTTPS connection to the Google ClientLogin URL. The second builds the request data string, tying together the four pieces of data into a single string. They are both used in the auth_token method -- I'm taking advantage of one of my favorite features of Ruby, which is the lack of distinction between local variables, no-argument methods, and data field getters.

So, http, request_data, and auth_headers are just dropped into the auth_token code as if they were local, even though they are separate methods. To me, that makes the code read cleanly, and encourages moving small bits out to separate methods where they can be separately tested and documented.

The auth_token itself takes the path, the request_data, and the headers, and uses the HTTP object to make a secure post call. Request data, by the way, has account and password information that I'm assuming would be defined in an abstract subclass of this Blog class. If the response code is 200, we're good to go, and we do a little regular expression magic to extract and return the authentication token.

We use the authentication token in our actual post call, the one that contains the blog post itself. We need to build up the XML Atom document to send to the Blogger server. Step one is to get the data. Since I'm going to be doing this from TextMate, I'm going to assume we're going to start with the file name that will eventually be provided by the TextMate command. First, we'll convert it to an array of lines.

    def read_data(filename)
      text = open(filename) { |f| f.readlines }
    end

In case it's not already clear, this method and the ones that follow are all parts of the

Blog

class. We're splitting this into lines to facilitate some processing of the text. I'm assuming that the first line of my file will be the title of the post, the second line will be a comma-delimted list of category labels, and the remainder of the file will be the body, written in Markdown. Obviously, that specific format is a weird quirk of the particular blog I'm posting to, which is set up to take XML and not to preserve blank lines. In your case, do whatever you need to do. Building that data structure
is quite simple in Ruby.

    def build_data(lines)
      categories = lines[1].split(",").map {|c| c.strip}
      body = BlueCloth.new(lines[2..-1].join(" ")).to_html
      data_xml(lines[0], categories, body)
    end

The last line of build_data calls the data_xml method that builds up the XML document:

    def data_xml(title, categories, body)
      result = []
      result << "<entry xmlns='http://www.w3.org/2005/Atom'>"
      result << "  <title type='text'>#{title}</title>"
      for cat in categories
        result << "  <category scheme='http://www.blogger.com/atom/ns#' "
        result << "            term='#{cat.strip}'/>"   
      end
      result << "  <content type='xhtml'>"
      result << body.to_s
      result << "  </content>"
      result << "</entry>"
      result.join("\n")
    end

I was going to use the REXML library for this -- normally I'm a big fan of building XML programatically. However, REXML really, really didn't like it when the included content contained HTML tags. So I decided it'd be much less aggravating to build the XML from scratch. Details on the syndication format can be found in the GData and Atom online docs.

Once we can create the data, we can finally make our post:

    def post_headers
      {'Content-Type' => 'application/atom+xml', 
          'Authorization' =>"GoogleLogin auth=#{auth_token}"}
    end
    
    def uri
      "http://www.blogger.com/feeds/#{blogid}/posts/full"
    end
    
    def post(lines)
      response, content = http.post(uri, build_data(lines), post_headers)
      while response.code == '302'
        response, content = http.request(response.location, entry.headers)
      end
      return response.code == '201'
    end
    
    def post_file(filename)
      post(read_data(filename))
    end
  end
end # of module

Couple things to point out here. First, we call auth_token directly when we're building up our header in preparation for the call -- so that's when we perform the HTTPS login shown above. The while loop takes care of following along if Blogger decides to redirect the post. Finally, we return true if Blogger sends us an OK response code 201, meaning the post has been successfully added.

To use this, you need to set up your own subclass of Blog with the expected information, for example.

 class MyBlog < Blog
   def initialize
     @account = "YOUR_GMAIL_HERE"
     @password = "YOUR_PASSWORD_HERE"
     @blogid = "YOUR_BLOG_ID"
   end
 end

As written, that would be included in the Blogger module, otherwise, you'd need to qualify the name Blog. The invocation of the whole thing looks like this. This assumes that the filename with the post is in ARGV[0], placed there by TextMate.

p Blogger::MyBlog.new.post_file(ARGV[0])

One problem, though. We're making an HTTPS connection for authentication to keep our password secure, but including the password in plain text in the script file. Good point. There's a cool way around that in TextMate, and probably

GWT part 3 and 4

Sorry for not mentioning this earlier, but part three of the GWT series is now up on the IBM site at:

http://www-128.ibm.com/developerworks/opensource/library/os-ad-gwt3/

This one is about remote procedure calls, and I'm pleased with how it turned out.

I just got the proofs for the fourth and final article in this series, about deployment. I expect it to be online Tuesday, Feb 27th at:

http://www-128.ibm.com/developerworks/opensource/library/os-ad-gwt4/

Clearing The Decks

A collection of small things, half-finished posts, and pure whatnot that hopefully will lead to more posts in the future:

• Got my first Amazon affiliate statement with no less than $1.55 heading my way. Thanks to the person who clicked through...
  
  
• There's a new sidebar, for small links, via del.icio.us. The sidebar has it's own RSS feed, on the off chance somebody is interested. UPDATE: It doesn't appear to work on Safari, which is a problem... Weird, because I did the preview using Safari, and it showed up fine, but not in the actual blog. Until I figure that out, Safari users can still click on the RSS icon and see the items.
  
  
• I had a half-finished post on the iPhone that got buried by the trillions of electrons poured on this story. So, I'll distill it to this: that was one cool demo.
  
  
• I also had a half-finished post on using XML as a scripting language, as in Ant scripts. Distilled: Don't do it. It's hard to read and nigh impossible to maintain. The thing I most like about Groovy is that it lets you actually script an Ant target. Although I think the JRuby/Rake combo is going to be pretty powerful here.
  
  
• Hey, another half-finished post. This one was about the lack of list or hash literals in Java. Which bothers me just about every day that I work with Java.
  
  

Decks now cleared. Hoping to get some new content here soon.

Don't Ask Questions, It Only Encourages Him

Let me promote this from the comment section -- it's not hard to find, it's the only comment on the previous post.

What is your favorite Python IDE? Your editor choices are interesting and valid but I wondered if you have a preffered IDE for Python and wxPython work?

I may have covered this somewhere, either on this site, or in the Python 411 podcast interview. If so, I'm sorry.



When I'm on a Windows machine, my Python editor of choice is JEdit, and has been for quite a while. I should say that I only rarely use the Python debugger and Jython interpreter add-ons that are available for it. I do have a couple of custom scripts that execute the script I'm working on and so on, but it's not a very elaborate set-up. What I like about JEdit is that it has a clean interface, had familiar Windows keyboard shortcuts and menus, and had a lot of plugin and macro power. The main downside is that it's kind of a memory hog for a text editor.



When I jumped Mac-side for my personal stuff, I tried just about every free programmer's editor I could find (JEdit, like a lot of Java/Swing programs, behaves a bit oddly on a Mac). Finally, I somewhat reluctantly decided to pay for TextMate. Although Python-mode on TextMate hasn't quite gotten the love and attention that Ruby has, there's still a lot of power there. It's really easy to script and customize, and it's about the only editor I've seen that has dynamic mode. So that if you are writing, say, a Django template, TextMate knows the the HTML portion gets colored and edited under HTML scope, and inside the Django tags, you use Django/Python mode.



My main issue with the various IDE's is that I haven't seen a Python one where the gain in the IDE functionality makes up for the editor itself not being as powerful as my normal editor. Some of this is my personal workflow -- using Python and test-first programming, it's very rare for me to feel that I need a step-through debugger. I'm also not very fond of code-generating GUI builders, for reasons that are probably worth another post.



That said, one thing I do like when I'm in and IDE for whatever reason is the ability to specify a project and do useful things across project scope. (Both JEdit and TextMate have this ability, but not fully formed.) I've done some Ruby work using Eclipse with Ruby plugins, and it's nice to, say, have one-button access to running all your unit tests.



That's what works for me, but I'm always looking for new tools and finding out how other people work.

Editors I Like

Two tools I use all the time. Neither is free, and since my strong bias is to use free tools where possible, these are some really impressive editors.

IntelliJ IDEA For all my Java needs. It's got more features and is more usable than any other Java IDE out there. The only downsides are that it's not free, and there are about a half-dozen keyboard shortcuts you have to get down before you achieve anything like full Zen mastery (well, and it could be nicer about the way it arranges tabs in the editor). I converted one of my teams to IntelliJ largely by just using it in team programming session. (Sample comment: "You don't seem to be typing very much"). IntelliJ is so good at using the class and type information that it single-handedly makes it useful to have static typing.

There are a couple of killer features that I miss horribly when using another tool, like the control-n series of shortcuts to browse classes, files, or symbols in the project. Even better, in the search box, the search is camel-case aware. So if you type MBGC, it'll match the class MyBigGoofyClass, or the symbol makeBeveragesGreeenCoffee. It's brilliant, completely intuitive and makes searches through the code very quick. Ten seconds after you try it once, you're hooked.

Another cool feature -- a help menu item that lists cool features along with a count of how often you use them, so you can see what neat stuff you are missing out on.

TextMate My Mac-based editor of choice for everything other than Java. The Mac editor wars (BBEdit vs. Everybody Else) do sometimes reach a fever pitch normally associated only with critical topics like Emacs vs Vi. So I don't want this to be a bash on other Mac editors -- if I want to do that, I'll do another post. Besides, if what you are doing works for you, more power to you. What I like about TextMate is that it really was built from the bottom up to be scriptable, which gives it a tremendous amount of flexibility and power to handle different file modes, integrate shortcuts, run scripts from TextMate, just easy as could be to use that stuff, and even pretty easy to create new commands. Biggest quibbles are the single-keystroke only Undo, which I understand as a design choice even as I find it kind of frustrating, and the project mode, which is useless to me, at least partially because it will only display like five tabs across before going to the overflow menu, and that's just not enough.

Killer feature: select a chunk of text, type the left half of an enclosing pair, like a left paren or bracket. Rather than replace the selected text with the keypress, which is almost never what you want to do, TextMate will surround the selected text with the enclosing pair, which is so useful that I try and fail to use this in all my other editors now. And yes, the list of enclosing pairs is adjustable for each file mode.

wxWorld

I'm pleased to be able to link to a new article: Build cross-platform GUIs using wxWidgets available on the IBM developerWorks site. The original title was "wxWorld", and it's a quick look at wxPython, the wxWidgets toolkit, and some of the other wxWidgets language bindings. I had some fun digging through the different language tools trying to create short wx programs in each. Hope you like it.

Tips-First for Test-First

Of all the exciting ideas and revelations that came from Kent Beck's original XP book, Test-First Programming has been the one that most significantly affected the way I work on a day-to-day basis.

I love programming test-first. It's a great way to take a large, amorphous task and solve it piece by piece. It's also a nice morale boost -- "Hey, I know that my code does nine things. Let's go for ten..."

Here are a bunch of things I wish somebody had told me about test-first programming.

Unit Testing is not All Testing

So, after I started doing test-first, I walked around for about six months all, "my code is perfect because I wrote tests". My smugness came crashing down when testers found some bugs. My code was better because I wrote tests, but turns out I had made some dicey assumptions about the inputs, and so my tests passed, but were still incorrect. Test-first is not a complete test suite. You still need to do acceptance testing, you still need to do GUI testing where appropriate.

However, you can still automate a very large percentage of acceptance tests. The more you can automate the tests, the more they'll be run, and the happier you'll be.

Test-First is a structure for writing good code at least as much as it's a means for verifying code

Code that has been written in a test-first style tends to have certain qualities. Small methods, loosely coupled. Small objects, loosely coupled. Code that causes side effects (such as output) tends to be separated from code that doesn't. These are all side effects of what's easy to test -- it's easy to write small methods in a tight test-first loop. And dependencies between methods or objects make tests harder to write.

As it happens, those exact qualities -- tight cohesion and loose coupling -- are exactly what characterizes the best software architectures. My test-first experience is that I wind up with much better code architectures from test-first then I do when I try to guess the design before I start. (Which is not to say that a little bit of pre-design can't be helpful, just that it can be overdone).

Test-first is better suited for some things than others

That doesn't mean that you shouldn't try, of course. Test-first is vital in cases where you know the input and output, but not the process. It's also critical in cases where your program can be incorrect in subtle ways. It's somewhat less important for things that will visibly or loudly break. GUI's are a challenge because GUI layouts tend to change in ways that can break unit tests. GUI behaviors are more stable and easier to test. Again, though, you should try and have automated coverage of even those areas that weren't developed test-first.

Trust the process. Look ahead on tests, not implementation.

It works. The tight process is: Write a test. Run the test so that it fails. Make the simplest fix that will pass the test. Run the test so that it passes. Refactor. Keep to that tight loop. Resist the temptation to guess about what you'll need to pass the next test. What I usually do is put a list of the tests I'm going to write in comments in my test class -- that's my lookahead plan, and keeps me from forgetting something. But the design I do in my actual code comes during the refactor step, which is where I see duplication and abstraction.

The earlier you start the better off you are.

It gets increasingly hard to convert a code base to test-first the longer you wait. I've even had 20 line classes that needed significant refactoring to unit test (mostly because output was intertwined with functionality -- the code was better after the refactoring). Test-first is a good place to start, anyway -- pick something to test, and go.

Treat your tests like code and refactor them

Pretty much every test-first guide gives sort of a perfunctory nod to, "oh yeah, keep your test code clean." But I think this could stand a little more attention. For one thing, your unit tests are critically important to your ability to deliver quality code -- and they have no tests of their own. The cleaner your tests are, the better you'll be able to see issues with the tests themselves.

One thing that has worked nicely for me is extracting sets of assertions into custom assert methods. If you are continually making the same five assertions to check validity of your objects, throw them into an assert_instance method of some kind. Another common case is making the same assertion over a range of values -- move the for loop to your custom assertion and pass the range endpoints in.

There are two big advantages to doing this consistently. The first is that it's easier to see what's going on from one line of assert_person(expected_name, expected_addr) then from five lines of assert_equals. The second is that it ensures that you actually do make all the assertions every time. Hey, everybody slacks, and test-first is about making the test setups as quickly as possible. If you can trigger all umpteen tests on your class with one method call, you're more likely to do the whole set every time, rather than just picking one or two at random each time.

Don't reuse instance variables

This is another refactoring issue. It's tempting as you add new unit test cases to do something like this:

Person p = new Person("noel", "david", "rappin");
assertEquals(15, p.nameLength());
p.setLastName("flintstone");
assertEquals(19, p.nameLength());
p.setFirstName("pebbles")
assertEquals(22, p.nameLength());
p.setFirstName("betty");
assertEquals(22, p.nameLength());

The last test fails -- quick, what's it testing? Okay, now we have to trace the life of that instance variable all the way back up. It's hard to read, and prone to dangerous errors. You should never reuse an instance variable like this in a unit test -- every assertion should, where its at all feasible, be completely distinct:

assertNameLength(int expected, String first, String middle, String last) {
  Person p = new Person(first, middle, last);
  assertEquals(expected, p.nameLength())
};

testNameLength() {
  assertNameLength(15, "noel", "david", "rappin");
  assertNameLength(19, "noel", "david", "flintstone);
  assertNameLength(22, "pebbles", "david", "flintstone");
  assertNameLength(22, "betty", "david", "flintstone");
}

Now when the last test fails, you can actually see what's going on.

Avoid tautologies

The scariest issue you can have with tests is a test that passes when it should fail, allowing you to continue blithely along, ignorant of a bug you should have already caught. There will come a day, for instance, when you will forget to put any assertions in a test. There are a couple of things you can do to make tautologies less likely.

1. Follow the process. The process says each test has to fail before you add code. Adding tests that you already know will pass can easily lead to writing a test that will never fail.
   
   
2. If you have constants for text or numerical values in your code, don't reuse those in the tests -- use the literal or create a separate constant in the test.
   
   
3. Be careful with Mock Objects. Try not to test the things that you are explicitly inserting in the Mock when it's created.
   

Mock Objects Rule

Mock Objects are the missing link in helping you test all the things that are traditionally hard to unit test, like databases, GUI, web server... anything where your code is dependent on an external system or person, the Mock can get in the way and pretend to be that third-party and allow you to send and receive data in a testable way. Mock Object packages exist for a variety of languages, and using a package will save you time and effort on your tests.

Hope that helps -- go out and test something.

Why, Johnny, Why?

We interrupt Python week to bring you the following alternative programming rant. I know, Python week has sort of gone up in smoke. But one of our mottoes here is "Whenever a Hugo Award winning SF novelist writes a hyperbolic screed about BASIC in the public schools, 10 Print Hello will be there". As a motto, it's not very catchy. We're working on it.

As soon as I mentioned "Hugo Award winner", "BASIC" and "hyperbolic screed" many of you were probably able to quickly deduce that the author is David Brin, here on Salon wondering what happened to BASIC (you'll have to watch an ad to view the article):

Only, quietly and without fanfare, or even any comment or notice by software pundits, we have drifted into a situation where almost none of the millions of personal computers in America offers a line-programming language simple enough for kids to pick up fast. Not even the one that was a software lingua franca on nearly all machines, only a decade or so ago. And that is not only a problem for Ben and me; it is a problem for our nation and civilization.

Does he have your attention yet? He'll equate the loss of BASIC to an act of war later in the essay. Brin seems to be making three separate points:

• BASIC used to be available on all computers that kids touch, and that is no longer the case.

This is obviously true, but a bit less dramatic than Brin implies.

Brin implies that BASIC was available for kids for a long time, and only recently disappeared. Actually, that's close to backwards. BASIC was generally available for less than a decade, and has been fading ever since. Even though Brin says a couple of times that "20 years ago" millions of kids could have used BASIC, the fact is that by 1986 BASIC was on its way out as a standard part of home computers.

Although it was invented in the early 60's, BASIC is associated most strongly with the late '70s and early 80s generation of computers. This market would eventually be dominated by the Apple ][ line, but earlier included things like the TRS/80, and Texas Instruments. (I even remember the Bally TV based game system having a BASIC module circa 1981 or so.). In any case, neither the Mac (introduced in 1984) or the IBM PC and clones featured BASIC to that same degree. By 1985, the idea that all computers would have BASIC was much less strong, although Apple ][ and BASIC instruction lingered in schools for a few years after that Which is why Brin's son is still seeing it in math textbooks, although that says more about the textbook industry than anything else. Data point -- my middle school had an Apple ][ computer lab in 1984 or 5. In my high school, a couple of years later, the computers were already Mac & PC's without basic -- we learned Pascal.

• There is much less of an sense that kids should be taught programming (particularly in BASIC) than there may have been in the mid 80's.
  

Largely true. Another data point -- my younger relatives about ten years later were no longer taught BASIC, nor were the kids at elementary schools I studied about the same time. By now, computers had migrated into the actual classroom and were being used as reference and also for what I guess you'd have to call multimedia authoring. I do think this is a loss. But at the same time, I've always kind of suspected that the reason why elementary school kids were taught BASIC in the early 80s was because the schools were kind of floundering around for what to do with the shiny new computers. My read of the educational literature during the time I was studying educational technology was that eventually this petered out because it was not clear that teaching programming was helping students become better general learners. To be fair, that's not exactly the point Brin is arguing, but it does suggest that, perhaps, losing BASIC is not the end of civilization.

• BASIC has some magical set of properties (Brin calls it "line-programming") that makes it uniquely suitable for introducing programming concepts. Because of this, we're losing an entire generation of tinkerers. This, I don't get at all.

So, Brin goes on at some length about how the computer people he's talked to don't seem to feel that it's a problem that BASIC isn't around anymore, while he, Brin, knows better. (Anybody familiar with Brin's essay entitled "The Dogma of Otherness" should catch at least a hint of irony.) Anyway, while Brin does acknowledge that BASIC has a lot of limitations, he goes on at length about line-programming and how important it is.

I'm not completely sure what Brin means by line-programming. Google didn't give me a relevant link. I'm going to assume that it has something to do with the fact that BASIC circa the Applesoft years was coded on a line-by-line basis. Brin suggests that this was an experience that modern languages can't give:

The "scripting" languages that serve as entry-level tools for today's aspiring programmers -- like Perl and Python -- don't make this experience accessible to students in the same way. BASIC was close enough to the algorithm that you could actually follow the reasoning of the machine as it made choices and followed logical pathways. Repeating this point for emphasis: You could even do it all yourself, following along on paper, for a few iterations, verifying that the dot on the screen was moving by the sheer power of mathematics, alone. Wow! (Indeed, I would love to sit with my son and write "Pong" from scratch. The rule set -- the math -- is so simple. And he would never see the world the same, no matter how many higher-level languages he then moves on to.)

I confess, I have no idea what he's saying here, though I do like the scare-quotes around "scripting". I'm kind of trying to get my head around the idea that you can't program a mathematical algorithm in Python and follow the reasoning of the machine. I mean, there are higher level constructs, but if we're talking about loops and conditionals for 5th grade math problems... I think Python would be pretty easy to follow and would look a lot like the logical structure of the algorithm. Python even has an interactive interpreter so you could type the code in line-by-line if you wanted. That actually could be pretty cool in a learning-math setting. And you could even track it with pencil and paper.

It is true, though, that it was much more conceptually simple to do simple graphics in Applesoft BASIC than in Pascal. That's not the language's fault, and it's not because you write Python in a full text editor. It's because modern programming languages sit on an operating system that mediates access to the drawing controls, and Applesoft BASIC didn't. It wouldn't be hard to come up with a Python package that emulated the draw controls of Applesoft basic (which were on the order of "Make that pixel blue. Now make that one red").

I think what I object to is the implication that this is somehow a difficult time to be learning to program, that it's harder now to get into programming. That's totally wrong -- it's a fabulous time to be learning to program. Brin says his son is now learning C++, so I'll assume he's interested and motivated. Twenty years ago, yeah, he would have had BASIC. And that's it. Unless you wanted to pay some money. As for seeing any examples of what a real program looked like, forget it.

These days... Well, a Mac OS X box ships with what, a half-dozen or so programming languages right out of the box, with who knows how many all available for free. Want to see basic algorithm code for free? It's there. All kind of code, complex and simple, is available online. There's a whole industry of programming books, something I would have devoured as a kid. We've traded The One True Teaching Language for many different languages. An elementary school teacher explaining "20 goto 10" is now a publishing empire, plus the internet. Coloring individual dots on a screen is now building a web page, or a web application, or a sprite animation. Even an elementary school child who is motivated can do more and understand more about computers than I would have dreamed in 1985. Have we lost something? Maybe. Have we gained something? Oh yes.

Re-refactoring

Here's a little riff inspired by one of the examples in Martin Fowler's book Refactoring, which is another great programming book that deserves an appreciation post one of these days. This was actually also spawned by code that I've read, and later realized that Fowler did a similar example. Thing is, I don't think Fowler went far enough in this case.

Here's the example. (page 243 for those of you playing the home game). But, since it's Python Week here, I'll translate to Python.

if isSpecialDeal():
   total = price * 0.95
   send()
else:
   total = price * 0.98
   send()

Fowler correctly notices the duplicate call to send(), and refactors to:

if isSpecialDeal():
   total = price * 0.95
else:
   total = price * 0.98
send()

This is fine as far as it goes, but as I see it, there's a second duplication in this snippet -- the formula for calculating the total. I'd rather see something like this (using the new Python 2.5 ternary syntax:

multiplier = (0.95 if isSpecialDeal() else 0.98)
total = price * multipiler
send()

There are a couple of advantages to this last snippet. We've separated the calculation of the total from the act of gathering the data for that calculation. This makes the actual formula for the total clearer, and allows you to easily spawn the multiplier getter off to it's own method if it gets more complicated. Plus we've removed more duplication, and I think made the code structure match the logical structure of the calculation a little bit better.

This is a simple example, and you could quibble with it. The general idea of separating conditional logic from calculations is a solid way to clean up code and make it easier to maintain in the long run.

Before I leave... I'm not sold on the syntax for the Python ternary yet. I'm told that the syntax was chosen over the perhaps more consistent if cond then x else y end because it was felt that in most use cases you'd have a clear preferred choice and a clear alternate choice, and putting the preferred choice before the conditional emphasized that. I don't know if that matches how I'd use a ternary. Although I guess it's reminiscent of listcomp syntax. I need to use it in real programs to know for sure.

Some 411 of my own

Saturday, Robin and I had the pleasure of being interviewed by Ron Stephens for the excellent Python 411 podcast. I think this was the first time I've ever been interviewed for anything, and while it's always fun to talk about Python, the book, and me (not necessarily in that order), it does take some getting used to.

Anyway, I do mention this here blog during the interview, and while I don't want to talk about the actual interview in detail until I hear the edited version, it did occur to me that I might want to have some actual Python content on board in case anybody comes by to check the place out.

Python content all week, then, starting with today's Things I Love About Python:

1. Whitespace. I know that I said just a few short days ago that I wasn't going to redefend Python's whitespace blocks. That was then, this is now. Now, I'm just going to gush over them. I love using whitespace to mark blocks. It enforces what I'd be doing anyway. It encourages consistent style, with the result that other people's Python code is actually intelligible. It encourages short methods and shallow nesting, both good habits, and it lets you get about 10-25% more code on a page. Nobody is ever going to have an Obfuscated Python contest (okay, I looked it up... somebody has, but they realize it's a joke.
   
   
2. List Comprehensions. One of my favorite syntax features in any language. So concise and yet so clear... Try to describe the following any more clearly in any language, programming or not.
   
   [x.name for x in students if x.grade > 90]
   
   Okay, they do sometimes blow up if you make them too complicated.
   
   
3. First Class Functions. It's easier to pass around named function objects in Python than in just about any language not named Lisp. This is a very good thing. It enables all kinds of elegant abstractions (especially since classes and instances can all be made callable). Over time, using Python has made all my coding move to a more functional style that's easier to test, verify, and maintain.
   

Of course, not everything in Python needs to be elegant and abstracted. Last night I had a problem. I wanted to download all episodes of a popular podcast that does not have an easily accessible archive page. Rather that walk through months of postings, I decided to write a script that would take advantage of the pages naming conventions, loop to find the shows for given days, find the downloadable URL and download, then add to iTunes. Final code, just under 60 lines. Elapsed time, under 45 minutes start to first download, including downloading, installing, and using a new library (Beautiful Soup, which is a nice HTML parser). The point is not that I'm particularly good at this (the script is a little sloppy and doesn't handle error conditions well), but that Python is particularly good at this. Plus, it was fun -- no fighting with compilers and interpreters, able to find support for the libraries when I needed it.