Wednesday, March 28, 2007

Posting to Blogger via Ruby

TextMate has what seems to be a very nice blogging bundle for programmatically sending posts to your blogging engine of choice. Except that it doesn't work for the new Blogger API. Or at least it didn't the last time I checked. Mostly I just wanted to see if I could write my own script to send to Blogger.

This is a Ruby script based on the Python script located at http://djcraven.blogspot.com/2006/10/success-posting-to-blogger-beta-using.html -- the author of that script did the heavy lifting in terms of the GData API calls, what I did was translate it into Ruby in a somewhat more flexible structure. At least I hope so. I also hope this will look like decent, idiomatic Ruby and not like a horrific hack. And I'd like a pony. If you're asking.

Let's do this in pieces. The first piece of the puzzle is using the Google ClientLogin API to get an authentication token. The token is then passed as a parameter to later calls when we actually want to post something. Google says that the lifespan of the token is dependent on the application being used, but I don't see where they specify how long Blogger keeps them.

Here's the start of our class. We're in a module called Blogger and a class called Blog. I've got an external dependency here on BlueCloth because I'm also going to automatically translate Markdown later, but that's not something you need to do...


module Blogger

require 'net/https'
require 'net/http'
require "uri"
require "bluecloth"

class Blog

attr_accessor :account, :password, :blogid

@@auth_url = "www.google.com"
@@auth_path = URI.parse('/accounts/ClientLogin')

def http
http = Net::HTTP.new(@@auth_url, 443)
http.use_ssl = true
http
end

def request_data
["Email=#{account}",
"Passwd=#{password}",
"service=blogger",
"service=TestCompany-TestApp-0.0"].join("&amp")
end

def auth_headers
{'Content-Type' =>
'application/x-www-form-urlencoded'}
end

def auth_token
response, data = http.post(@@auth_path,
request_data, auth_headers)
return nil unless response.code == '200'
data.match(/Auth=(\S*)/)[1]
end


After the setup lines, we've got three methods here. The first two define the data objects. The first one creates a Ruby HTTP object, set up for an HTTPS connection to the Google ClientLogin URL. The second builds the request data string, tying together the four pieces of data into a single string. They are both used in the auth_token method -- I'm taking advantage of one of my favorite features of Ruby, which is the lack of distinction between local variables, no-argument methods, and data field getters.

So, http, request_data, and auth_headers are just dropped into the auth_token code as if they were local, even though they are separate methods. To me, that makes the code read cleanly, and encourages moving small bits out to separate methods where they can be separately tested and documented.

The auth_token itself takes the path, the request_data, and the headers, and uses the HTTP object to make a secure post call. Request data, by the way, has account and password information that I'm assuming would be defined in an abstract subclass of this Blog class. If the response code is 200, we're good to go, and we do a little regular expression magic to extract and return the authentication token.

We use the authentication token in our actual post call, the one that contains the blog post itself. We need to build up the XML Atom document to send to the Blogger server. Step one is to get the data. Since I'm going to be doing this from TextMate, I'm going to assume we're going to start with the file name that will eventually be provided by the TextMate command. First, we'll convert it to an array of lines.


def read_data(filename)
text = open(filename) { |f| f.readlines }
end


In case it's not already clear, this method and the ones that follow are all parts of the
Blog
class. We're splitting this into lines to facilitate some processing of the text. I'm assuming that the first line of my file will be the title of the post, the second line will be a comma-delimted list of category labels, and the remainder of the file will be the body, written in Markdown. Obviously, that specific format is a weird quirk of the particular blog I'm posting to, which is set up to take XML and not to preserve blank lines. In your case, do whatever you need to do. Building that data structure
is quite simple in Ruby.


def build_data(lines)
categories = lines[1].split(",").map {|c| c.strip}
body = BlueCloth.new(lines[2..-1].join(" ")).to_html
data_xml(lines[0], categories, body)
end


The last line of build_data calls the data_xml method that builds up the XML document:


def data_xml(title, categories, body)
result = []
result << "<entry xmlns='http://www.w3.org/2005/Atom'>"
result << " <title type='text'>#{title}</title>"
for cat in categories
result << " <category scheme='http://www.blogger.com/atom/ns#' "
result << " term='#{cat.strip}'/>"
end
result << " <content type='xhtml'>"
result << body.to_s
result << " </content>"
result << "</entry>"
result.join("\n")
end


I was going to use the REXML library for this -- normally I'm a big fan of building XML programatically. However, REXML really, really didn't like it when the included content contained HTML tags. So I decided it'd be much less aggravating to build the XML from scratch. Details on the syndication format can be found in the GData and Atom online docs.

Once we can create the data, we can finally make our post:


def post_headers
{'Content-Type' => 'application/atom+xml',
'Authorization' =>"GoogleLogin auth=#{auth_token}"}
end

def uri
"http://www.blogger.com/feeds/#{blogid}/posts/full"
end

def post(lines)
response, content = http.post(uri, build_data(lines), post_headers)
while response.code == '302'
response, content = http.request(response.location, entry.headers)
end
return response.code == '201'
end

def post_file(filename)
post(read_data(filename))
end
end
end # of module


Couple things to point out here. First, we call auth_token directly when we're building up our header in preparation for the call -- so that's when we perform the HTTPS login shown above. The while loop takes care of following along if Blogger decides to redirect the post. Finally, we return true if Blogger sends us an OK response code 201, meaning the post has been successfully added.

To use this, you need to set up your own subclass of Blog with the expected information, for example.


class MyBlog < Blog
def initialize
@account = "YOUR_GMAIL_HERE"
@password = "YOUR_PASSWORD_HERE"
@blogid = "YOUR_BLOG_ID"
end
end


As written, that would be included in the Blogger module, otherwise, you'd need to qualify the name Blog. The invocation of the whole thing looks like this. This assumes that the filename with the post is in ARGV[0], placed there by TextMate.


p Blogger::MyBlog.new.post_file(ARGV[0])


One problem, though. We're making an HTTPS connection for authentication to keep our password secure, but including the password in plain text in the script file. Good point. There's a cool way around that in TextMate, and probably

Wednesday, March 21, 2007

Rubies in My Coffee

Now two of the big Java IDE's are promoting Ruby language tools as a big thing. IntelliJ has a plugin in early beta, and NetBeans is also making a big deal of their new early beta support. Eclipse has had a Ruby/Rails plugin for about a year or so.

This is weird, weird, weird, that suddenly all the Java tools would feel the need to grow into somewhat ill-fitting Ruby IDE's (Eclipse has always styled itself as more of a meta-IDE, so that's a little less strange). But do we really need all these Windows Ruby IDE's to be attached to big, memory hogging, Java tools. (And in IntelliJ's case, I say that with love...)

I've played with all three now -- I've used Eclipse for Windows-side Ruby editing for a while, and for the last few days I've been going back and forth between IntelliJ and NetBeans checking them out on an existing Rails project. None of the three are going to make me ditch TextMate yet (although I'd say the big tools all have somewhat more developed project support that TextMate).

IntelliJ's plugin really strikes me as Not Ready Yet. For one thing, there's no syntax checking on .rhtml files, which is a big problem. Also, there's a cool Rails project view, that associates the .rhtml files with the appropriate controller method, but there didn't seem to be an obvious way to get to partial .rhtml files. Another minor frustration is that the Ruby syntax layout is not customizable yet, the way the Java one is. It's even hard to keep a separate setup so that Ruby files only have a two character indent. (And there's a good sociology of programmers paper, on why Ruby is canonically a two character indent, not four).

This seems like a good place to point out that I still haven't spent all that much time with any of the new tools, so it's entirely possible I just missed something. For instance, even though the IntelliJ docs say there are Live Templates for Ruby, my installation didn't seem to have them.

There's some nice stuff in the IntelliJ tool -- all the ctrl-N navigation works on Ruby classes and methods. It's very easy to run arbitrary Rake tasks from the UI (although they seem to run kind of slowly) I don't think syntax completion is there yet, but it does syntax check .rb files on the fly I'd like to see this one a few revs down the line.

The NetBeans tool is in their beta for version 6 (you have to download the tool, then grab Ruby support via the update center). I was kind of amazed to be liking this because I fled NetBeans in a huff around version 3 and haven't had any success in trying it since. The syntax coloring and layout seems to be a bit stronger. It appears to be using JRuby not just for internal parsing, but also as the default external engine for running, say, WEBrick. Which makes everything feel a little slow. There is some syntax-completion -- I found it a little hit-and-miss in terms of getting useful alternatives. The project view is nice -- it flattens out all the /app subdirectories, but I couldn't see an obvious way to run arbitrary rake tasks or the test suite. Although there is a keyboard shortcut to run an individual file, and if you try it in a view page, it attempts to open that action in a browser. My app has a slightly nonstandard route configuration, though, so it didn't quite work for me.

I also had some stability problems, mostly with trying to run the server from NetBeans, it either hung or took a really long time to load. Development on this seems to be progressing rapidly -- they are releasing new versions much more quickly than IntelliJ, so all these things could be fixed by Thursday.

Right now, NetBeans would be my pick of the three if I couldn't use TextMate. I'm as surprised as anybody by that conclusion, but it felt pretty good as I was using it. Adding easy test hooks would help. All three tools are still under development (RDT for Eclipse just released a new version, and RadRails just passed to a new owner with new development promised). So, watch this space, I guess.

Update: Looks like NetBeans has a bug in reformatting code -- it kind of messed up some of my source files. Nothing permanent, just kept indenting and not noting the end to outdent so the file kept creeping to the right.

Sunday, March 18, 2007

Wow, There Are Comments

It's true -- the way to get comments on your blog is to mention Apple... I do something like one substantive post in three months, and then two apple posts in 48 hours, and bang! Four comments within a day. I'm surprised, not least because I really wasn't sure anybody was out there.

Anyway, interesting points have been made, and I thought I'd pull them up to either agree, or whine defensively.

On the Win 95 post, Massive writes:


My observation at the time was that the 'competitve analysis' of the OS features and capabiltiies was basically bogus and totally irrelevant. I felt that Apple wasn't whistling past the graveyard so much as completely lying to itself.


You say "tomato", I say "tomahtoe". There was a lot of denial in the air, that's for sure.

Still Massive, about the impact of Win95 on end users:


You could already, with Windows 3.1, show demos rigged to look identical between Mac and the Windows apps. And people were sufficiently ignorant about computers at the time that they would actually buy Windows 3.1 and tell people that it was 'just like the Mac'.


I can't really disagree with that, but I still think the general point that Windows 95 brought the two platforms much, much closer together holds. I certainly remember that to be the theme of much of the Win95 press coverage. I also remember Apple's slogan "Windows 95 = Macintosh 89". Y'know, Apple's marketing also kind of stunk in that period...

Massive also mentions that Apple had a poor relation with the dealer channel and that didn't help. Also true. As far as I can tell, still kind of true.

On to the internet posting, a couple of people comment on some other facets of how Microsoft eventually took over after they decided to start competing in browsers. Right, but there was a time when it was not yet clear that they were going to be willing and able to do that.

One anonymous poster did bring up a long-buried memory:


Nonesense. Apple's initial 'internet strategy' was CyberDog, which was a fantastic set of tools (web browser, mail client, etc ) unfortunately chained to the technological boat anchor that was OpenDoc.


Mike, is that you? Mike Pinkerton, who went on to create Camino, was the only person I knew who used CyberDog. I remember him as being very enthusiastic about it, but it was about 10 years ago, so I could easily be wrong. It did indeed look cool, but I recall the setup as being kind of daunting, and OpenDoc was kind of a pain in the neck. Also, CyberDog came out in 1996, and by then Microsoft had already moved in.

Thanks to the commenters -- I like responding, and it's a good way to get a post, so by all means, keep writing.

Saturday, March 17, 2007

Apple Dot Net

What is it about us tech fanboys and Apple... I've always found them interesting, even when I wasn't a regular Mac user. Infuriating, sometimes. But interesting.

So here's another thing about Apple, circa 1995... That was right about the end of something like a two-year period where Apple was way ahead on internet integration and didn't really make anything out of it. By the time the internet really started to escape out of academia, the Windows world was well on it's way to catching up, and by the time it all really went mainstream, Macs would be a fading memory. (I remember as late as 1995 impressing non-techy visitors to my grad school office with demos of a web browser. But a year later, the cat was largely out of the bag.)

But in 1994/1995 it was all Macish. Just accessing the internet was much easier (although it was a separate software purchase initially). And all the best software was on Macs (well, some of it was Unix, too) Eudora was the standard email client, and it was still Mac-only, for some reason the Windows version remained ugly forever. The Mac version of Netscape (and before that Mosaic) were much prettier than the Windows versions. (In my memory, the Mac versions came out sooner, particularly for early Netscape, but I can't find a source that backs that up).

On top of which, Microsoft was in the middle of a totally clueless stage. Win 95 did not, by default, ship with a web browser, and the big Microsoft internet initiative of 1995 was something called Blackbird, which was going to be a proprietary replacement for HTML. That worked out about as well as you'd expect, evidenced by the fact that Blackbird doesn't even have a Wikipedia article. Making it less important than even the most minor anime character...

Putting that all together, in the months before the Win95 launch, Mac had about a 12-18 month lead in Internet apps and Microsoft was distracted by an attempt to privatize the nascent WWW. There's clearly some room there for Apple to have maneuvered themselves into a better position than they wound up in. Being the best internet platform around would have been an interesting counterweight to the Win 95 launch, and while I doubt they could have stopped the upcoming onslaught, they probably could have mitigated it somewhat.

Instead, Apple didn't really do anything in the space, except release an email client (Claris Emailer -- which does have a Wikipedia article, which charitably refers to it as "popular"). Microsoft, on the other hand, turned on a dime, and gave up the futile attempt to take over the internet by co-opting the authoring tools. They switched instead to the more lucrative method of taking over the internet by co-opting the client tools. Apple wandered around in a fog for a few years, until they attacked the opportunity to go after digital media. But you probably know that part...

Apple Summer of 95

As I may have mentioned here, back in 1995 I spent three months as a summer intern at Apple HQ in Cupertino.

I was buried deep in the educational technology research group so, trust me, I didn't work on anything you've heard of. It was a fun summer, though. They stuffed about 15 of us, a mix of grad students and contractors, in a room that was really long and narrow. They tried to make up for the cramped office space by supplying us with Nerf toys (the admin assistant for our group would sometimes come in and announce a Toys R' Us run, and did anybody want anything). I got about a dozen free T-shirts in three months. I also experienced my first earthquake, which went like this:

A slow rumble, like a truck going down a slowdown strip

All the non-California people: What was that?

All the California people: What was what?

None of which is why I brought this up. 1995 was near a low point for Apple, but even at that there were three or four good-sized company parties during the few months I was there. I'm talking about food, music, drinks, some entertainment, all in the inner courtyard of the Infinite Loop campus. I may be slightly exaggerating in memory, and I was assured that it was small-scale by later Valley standards, but it sure seemed elaborate to me.

In August 1995, Apple held what you'd have to describe as an anti-Windows 95 launch party. More of a morale booster, I guess. The thing I remember most is a screen projecting Windows 95, while some Apple guy pointed out it's flaws. As in, "Windows 95 will only let you have 256 characters in a path name, but if you have more than 15 directory levels, you can cause it to give you a really weird error message." Thin stuff, but enthusiastically received by a lot of people who wanted to whistle in the dark together.

Everybody there, I think, knew that Apple lost something big that day. Prior to Win95, the visual differences between Mac and PC were so clear that you could see the difference in a store just by looking at the two. Afterwards, you actually had to use them both for a while to learn the difference. This was a big change in the Mac/PC dynamic, and it took Apple about five years of flailing before they would come up with a decent response.