unrouted.io Just sume stuff

Deleting an Amazon ELB properly

Recently I automated deletion of an Elastic Load Balancer and the Subnet it was in. Fairly straightforward stuff. One slight problem is that an ELB doesn't have states. When you delete one it disappears immediately. But in the background it is still there. When you try to delete the subnet it was in you get a dependency error. After a couple of minutes it does work.

The same is true if you try to delete a security group that the ELB was using.

What's going on?

If you look at the Network interfaces view in the EC2 control panel after you've deleted an ELB you will see that its network interfaces still exist and are cleaned up in the background. Internally it's in a 'deleting' state, but we just can't see that in the ELB API.

We can simulate it to some extent by polling the network interfaces API. It turns out (thanks Andrew @ AWS support) that ELB sets the description of its ENI's to 'ELB yournamehere'. So with botocore we can do something like:

description = "ELB " + your_balancers_name
for i in range(120):
    interfaces = client.describe_network_interfaces(
            {"Name": "description", "Values": [description]},
    ).get('NetworkInterfaces', [])
    if len(interfaces) == 0:
raise Exception("ELB not deleted after 2 minutes")

This is now done automatically when deleting an ELB with Touchdown. It won't consider an ELB deleted until all of its network interfaces have been cleaned up. So you shouldn't get any errors deleting subnets or security groups!

Support told me this behaviour is unlikely to change (and has been that way for at least 2 years), and they have fed back internally that the possible values for the ENI description field should be documented (they aren't right now).

Making a node virtualenv

These days npm ships with nodejs, which you can install on ubuntu with::

sudo add-apt-repository ppa:chris-lea/node.js -y
sudo apt-get update
sudo apt-get install nodejs

By default I have 2 ways of using npm::

sudo npm -g install bower

This (depending on how and where npm was installed) installs itself into /usr/local. The thing you installs goes on your PATH. But I don't really let anything near /usr unless it's in a Debian package. So I can also do::

npm install bower

This will create a node_modules directory in the current working directory. If i want to run bower I can just run::


Which is a bit yuck. Neither of these really cut it for me - I have been spoiled by virtualenv. Can I npm into my virtualenv - and have . bin/activate work for the node stuff too?

First of all I want a virtualenv::

virtualenv /home/john/myvirtualenv

npm -g tries to write into /usr/local because that is the default prefix config option. But I need to use -g so that I can get the bin/bower to appear. Helpfully you can override the prefix from the command line::

npm -g --prefix /home/john/myvirtualenv install bower

Now when i . bin/activate the JS binaries are available too.

Taking this one step further, you can install npm into a virtualenv and then set the default prefix::

npm -g --prefix /home/john/myvirtualenv install npm

cat > /home/john/myvirtualenv/lib/node_modules/npm/npmrc << EOF
prefix /home/john/myvirtualenv
global true

Now I can do this::

. /home/john/myvirtualenv/bin/activate
npm install bower lessc requirejs

And the dependencies are installed into that virtualenv - so as long as i have sourced my activate file I can just::


I've written virtualenv-js which does the 2 steps for you.

You can use it with mkvirtualenv of virtualenv wrapper by putting it in ~/.virtualenvs/ and adding this to ~/.virtualenvs/postmkvirtualenv::

# This hook is run after a new virtualenv is activated.
~/.virtualenvs/virtualenv-js $VIRTUAL_ENV

(Make sure you chmod +x virtualenv-js). (You could also just save virtualenv-js as postmkvirtualenv if you want).

Greenlets and Gio

I made a thing! It's like gevent but on top of the GLib stack instead of libev.

Most GLib async code has a sync and async variant, and the same pattern is used. So we can hook into GObjectMeta and automatically wrap them in something like this:

def some_api(*args, **kwargs):
    current = greenlet.getcurrent()

    # Call the function and pass it current.switch as the callback - this
    # is what allows the current coroutine to be resumed
    args = args + (current.switch, None)
    some_api_async(*args, **kwargs)

    # Pause the current coroutine. It will be resumed here when the
    # callback calls current.switch()
    obj, result, _ = current.parent.switch()

    # Actually return the expected value
    return some_api_finish(result)

So all use of the synchronous API's that have an asynchronous version would be replaced by a asynchronous greenlet version.

Patterns for asynchronous javascript

Isn’t writing synchronous code nice?

function do_stuff () {
  do_thing_one ();
  do_thing_two ();
  do_thing_three ();

But synchronous is bad! Bad, bad, bad. So then came async. The simple pattern is to pass a callback to the function you are calling. It’s not as bad in JS because we just can do this:

function do_stuff () {
  do_thing_one ( function (result) {
    do_thing_two ( function (result) {
      do_thing_three ( function (result) {

I’ve left out error handling. That really depends on the library.. But i imagine its messy. Now let’s see GIO style async.

function do_stuff () {
  do_thing_one_async ( function (ar) {
    var result = do_thing_one_finish (ar);
    do_thing_two_async ( function (ar) {
      var result = do_thing_two_finish (ar);
      do_thing_three_async (function (ar) {
        var result = do_thing_three_finish (ar);

I like this a lot better than how i’d do it in python. But wouldn’t it be nice if you could write async code something like this?

var do_stuff = async (function () {
  var result = yield do_thing_one ();
  yield do_thing_two ();
  yield do_thing_three ();

Or even:

var do_stuff = async (function () {
  var result = yield do_thing_one ();
  yield do_thing_two ();
    yield do_thing_three ();
  } catch (e) {
    print("Exception handled");

You can in python. You can in vala. And for JS? Well, I was going to say now you can. But while I was looking for a good Vala link, I noticed Alex already did something like this over a year ago. D’oh.

What would be really nice is if the async wrappers could be generated automatically by GI. I had a first stab at this by simply parsing the GIR xml with E4X and providing an alternative import mechanism (thanks for the suggestion jdahlin). However to get full coverage i’d have to consider interfaces and inspect every object that implements an interface as it lands in JavaScript land to ensure it is wrapped. Ew.

Playing with RDF

I’ve been playing with the master branch of tracker and i’m loving it – it looks like its finally reached the stage where I won’t just turn it straight off after a fresh install.

It now brings GNOME an RDF store with a SPARQL interface. Powerful joo-joo, but kinda scary if you haven’t seen it before. Most conversations about it lead to words like graphs, triples, ontologies… My eyes start to gloss over.. I need to learn by doing. So i’ve been playing with writing some python wrappers to hide tracker and just provide a familiar pythonic interface.

Any object type that tracker knows about will be available in python via the wonders of introspection. All properties of the class are available, with docstrings explaining the intent of the property and its type. Obviously you can get, set and delete and do simple queries. And behind the scenes are SPARQL queries in all their glory. Theres a lot still to do, but enough done that I can synchronise my address book to Tracker with Conduit (see my tracker branch).

So far it looks something like this (but its subject to very rapid change):

import tralchemy
from tralchemy.nco import Contact

# add a contact
c = Contact.create()
c.fullname = "John Carr"
c.firstname = "John"
c.nickname = "Jc2k"

# find all the people called John
for c in Contact.get(firstname="John"):
    print c.uri, c.fullname

# subscribe to any contact changes
def callback(subjects):
    print subjects
Contact.notifications.connect("SubjectsAdded", callback)

# Will probably be just:

While get() is a nice way to do simple queries, what if you wanted to do something a little more complicated. It always feels messy when you have SQL or SPARQL nested in other code. Existing SQL ORM tools are a great place to start at avoiding this, but i quite like the LINQ style generator-to-SPARQL. Something like:

q = Query(Contact.firstname for Contact in Store if Contact.nickname == 'Jc2k')


q = Query(c.firstname for c in Store if c is Contact and c.nickname == 'Jc2k')