Checking Out Rackspace Cloud

Just spent an hour kicking around a Rackspace cloud server. While I have the technical abilities to manage an EC2 instance, the Rackspace setup/config was nice. Very simple and straightforward.

It didn’t take me long however to discover the lack of persistent storage. This of course led me to scour the internet thinking “surely there has to be a way.” I landed upon this post from Brad Something Something (what is your name Brad?) which provides and excellent comparison of Rackspace vs EC2 – I agree with everything he says.

I like Rackspace Cloud, it’s easy to admin,  great price/performance, but until they have something EBS/NAS like I’m going to stick with EC2 and Linode.

urllib2 With Multiple Network Interfaces

Normally if I have an issue which is answered in the first 2-3 results of a Google search I won’t create a post. On the other hand when I spend 2-3 hours trying to solve something which should be simple I like to take the opportunity to describe the issue & resolution in hopes that someone will find it quickly in the future.

So the task here was to find a way to specify the IP address, aka socket, aka network interface when making an http request using Python’s urllib2. Why would you want to do this you ask? Well for many web API’s the request rate is limited by whitelisting the IP address – such is the case with Twitter. In the event that you want to be able to use the same machine (with multiple network interfaces) to run jobs in parallel you need to be able to specify where the requests should be routed.

The problem is Python’s urllib2 is based on the httplib library which doesn’t let you specify which address to bind to. This person tried to get around the problem in 2005 without any luck, another guy created a patch for httplib in 2008 which hasn’t been accepted, and finally someone else created a subclass for httplib which unfortunately I couldn’t get hooked up to the urllib2 class.

The best solution I found was this “monkey patch” from Alex Martelli over on Stack Overflow.  In his example he attacks the problem using the socket library instead of the httplib. By his own admission stuff like this is not ideal, but the solution is actually very simple and elegant. I like it.

I wrapped the snippet up into a function which can be called in a Python script anytime before you invoke a urllib2 request.

def bind_alt_socket(alt_ip):

true_socket = socket.socket
def bound_socket(*a, **k):
     sock = true_socket(*a, **k)
     sock.bind((alt_ip, 0))
     return sock
socket.socket = bound_socket

Hope this can be of help to someone in the future who’s searching for the same thing I was.

All Programming is Programming

Over on Coding Horror Jeff Atwood wrote a post which claimed that “All Programming is Web Programming“. Jeff made some good points about how the web provides programmers with the ability to reach an audience of a previously unimaginable size. Definitely agree. Also mentioned was that for better or for worse JavaScript is becoming the most important language in the world of software development. I agree with this as well, though I would add that the significance of JavaScript is in user facing applications only at this point.

It was unfortunate that the post was written in such a polarizing manner and that the comment thread quickly eroded into a shouting match because there is another important point to be made here.

Something else I want to put out there to developers is that the evolution of programming should be focused less on desktop vs web vs embedded or choice of language/platform, and more on how lowering the barriers to entry for new programmers is a positive and not a negative.

To someone writing device drivers or kernel patches the idea of writing a JavaScript function to manipulate the DOM may seem “uninteresting”, but the fact is that more and more people are getting started with programming this way. All they need is a text editor and a web browser and they are on their way. This is a very good thing.

Programming is about automation and automation is about improving efficiency. The more people we can somehow involve in this process the better because in the end the web provides not only the largest number of potential users, but also the largest number of potential programmers. The exciting thing is we are just getting started.

Refactoring and Resistance

Yesterday Paul Miller wrote a blog post called “Does Linked Data need RDF?” In the comments I said something in support of Paul’s question which I want to expand upon here. My comment was not directly related to the topic (linked data), but rather the format in which it was asked. Let me trim it down to this:

What I am thinking is this…. as system architects or software engineers this is a critical question to ask not only during design, but also as our solutions age over time. When asked this way it challenges us to think about dependencies from a different perspective. Instead of “are things working? If so, add new feature“, we should equally be looking for ways to remove, replace, or consolidate components for the benefits of efficiency, simplicity and performance. Sounds simple right, then why is it so hard to do?

I don’t think we go through this exercise often enough because politically it can be a very difficult. The sheer inertia of a large project can prevent people from even thinking about the question. The thing is that writing code, choosing a framework, establishing an API…. these are all sunk costs. We often attach emotional value to the time and effort that went into doing the work, and have a hard time imagining what things would look like without them.

It’s certainly a challenge, but to build better systems we need to be able to let things go, to scrap code, and to replace components without emotions or personal bias.

Oh and for the record, I do think RDF and Linked Data are the right combination to build the Web of Data. I just hope that we can keep asking questions, challenging assumptions, and continue to have constructive debates about the future of the Web like the one which took place yesterday on Paul’s website. Great stuff.