I was asked by Jeremy to give one of the talks at the beginning of the hack day, so I chatted a bit about the work we’ve done with astrometry.net to tag Flickr photos with their celestial locations — astrotagging.

Introducing astrotags

At the Royal Observatory, Greenwich, we’ve set up a Flickr group for our photography competition, Astronomy Photographer of the Year. Photos in the group are being scanned by the astrometry.net bot, which identifies stars in a photo then solves for celestial coordinates by comparison with a reference catalogue. Astrometry.net leaves a comment on each photo, listing the coordinates of the image and the names of objects within the field of view. The comments are relatively easy to parse, and I’ve written a small utility to extract information from commented photos. However, this is the sort of problem that’s ideally suited to machine tags so we asked astrometry.net to add tags to photos in our group using the astro: prefix. The tags are:

astro:RA

Right Ascension of the centre of the photo.

astro:dec

Declination of the centre of the photo.

astro:orientation

Orientation of the photo from North, in degrees.

astro:fieldsize

Angular extent of the photo, East-West × North-South.

astro:name

Named objects in the field. One tag per name.

Some good examples of tagged photos are:

• Horsehead Nebula
• Eta Carina Nebula
• Summer Triangle and Milky Way

The simplest way to get at these machine tags and use them in your own applications is with YQL, which allows you to query the flickr search and photo info APIs using a simple, SQL-like syntax. To make life easier, I’ve written a YQL open table, flickr.photos.astro, which can be queried to return a feed of astronomical coordinates and info for a set of Flickr photos.

select * from flickr.photos.astro where astro_name = 'Horsehead Nebula'

That query, for example, returns a feed of photos tagged astro:name=’Horsehead Nebula’. For some other examples, have a look at my post on searching astro:name with YQL.

My hack

Astrometry.net have solved almost 7,000 photos on Flickr, but only around 600 of those have been tagged with astro: machine tags. For my hack, I decided to try making a YQL table based on screen-scraping Flickr for information left by the astrometry.net robot. I took the code from my astrotagging web form and put together flickr.photos.astrometry, a YQL open table that can parse astrometry information from Flickr comments. The input parameter is a Flickr photo page URL, so a query looks like

select * from flickr.photos.astrometry where url = 'http://www.flickr.com/photos/dangerous_astro/4722913544/'

This returns a feed of the coordinates and names found on that page.

To demo this, I hacked together a little javascript astronomy photo search engine. You pass in a search query in the URL:

http://eatyourgreens.org.uk/yql/scihackday.html?text=milky+way

That text is then used to run a Flickr search and grab up to 30 photos, shown as thumbnails. The search is limited to photos which have been solved by astrometry.net, and we also request the photographer’s name so that we can credit them:

select * from flickr.photos.search(30) where text = 'milky way' and extras='owner_name' and machine_tags = 'Astrometrydotnet:status=solved'

If you don’t pass in a search query, we instead return the 30 most recent photos solved by astrometry.net.

select * from flickr.photos.search(30) where extras='owner_name' and machine_tags = 'Astrometrydotnet:status=solved'

When you select a thumbnail, by clicking on it, another YQL query is run against flickr.photos.astrometry to return the coordinates and names for that photo. The coordinates are used to plot the photo on a map of the sky, while the names are listed as links so that you can go on to search for pictures of specific objects in the field of view.

It’s quite a simple little bit of code, but hopefully it shows how useful YQL can be for accessing structured data on the web, even when that data is in HTML or plain text. If anyone does use it to build anything, Astronomy Photographer of the Year has a showcase page for mashups built using the Flickr astrometry data.

If the site user wants to pick the answer ‘Ahead’ to a question, they click this button. With jQuery, I get the value of the button (2) using var answer_id = $(this).val(). Users confirm their answer by pressing a ‘Next’ button to move onto the next question:

<button id="next" value="">Next</button>

I decided to use the ‘Next’ button to store the currently selected answer: $('#next').val(answer_id). I can then look at the value of the ‘Next’ button to decide which question to ask next when they submit an answer. All fairly straightforward HTML and JavaScript, which works in any standards-compliant web browser.

However, after launching the site we started to get strange bug reports on the Solar Stormwatch forum. Buttons were displaying numbers instead of words as their labels. Questions were repeating rather than moving onto the next question for a given answer. It turns out there is a bug in IE6 and IE7, which was being picked up by our users — IE6 and IE7 do not support the value attribute on buttons.

In IE6 and IE7, calling $(this).val() returns the text of a button — ‘Ahead’ in the example given above. Similarly, setting .val() for a button sets the button text, not its value. My first reaction was to change my jQuery code to use .attr('value') to access the value attribute instead. This runs into exactly the same underlying browser bug — .attr('value') gets and sets the button text in IE6 and 7.

To solve this, and get the questions working in IE, I ended up changing the value attribute on buttons to data-value. This is invalid in HTML 4, though it will be valid in HTML 5.

<button data-value="2">Ahead</button>

This works, however, because the default behaviour of HTML parsers in all the major browsers is the same when they encounter an attribute they don’t understand. They simply add it to the DOM as a property of the current node ie. you get something like button.data-value=2. So, wherever I had used .val() in my jQuery code, I now use .attr('data-value') to get and set the value of buttons.

I’m not hugely happy with this solution. It’s invalid HTML and relies on default error-handling behaviour in HTML parsers. It does work cross-browser, and it is robust in that HTML parsers aren’t likely to change the way they handle unrecognised attributes. I wish Internet Explorer supported the standards-compliant, valid solution though.

I was stunned to hear that Jack died last Saturday night. The comments on his last blog post, accessifyforum, and the reaction on twitter and facebook are a testament to the respect with which Jack was held online. As pixeldiva says, his loss is a tragedy for his wife, his children and family and the wider web accessiblity community.

I used to swap Monty Python jokes with Jack online, so I’m going to end with a clip from The Life of Brian. I think Jack would have agreed with the sentiment.

A new urgent action, and update, from Amnesty International on the case of the jailed Azeri bloggers.

Two jailed male Azerbaijani youth activists and bloggers will have their appeal heard in the capital, Baku, on 22 December. Emin Abdullayev and Adnan Hajizade were imprisoned on charges Amnesty International believes to have been fabricated.

The two youth activists and bloggers were arrested on charges of “hooliganism” on 8 July, after reporting to police that they had been assaulted in a restaurant in Baku. Their arrest came just over a week after Adnan Hajizade posted a video critical of the Azerbaijani government on the video-sharing website YouTube. On 11 November, Adnan Hajizade was sentenced to two years, and Emin Abdullayev to two and a half years, after being convicted in an unfair trial.

On 23 November, Emin Abdullayev and Adnan Hajizade’s lawyers filed an appeal against the verdict. On 10 December the lawyers informed Amnesty International that the two youth activists and bloggers had been granted a hearing at the Baku Court of Appeal on 22 December.

Please write immediately in English, Azeri, or your own language:

• Urging the authorities to ensure that Adnan Hajizade and Emin Abdullayev receive a fair appeal hearing in line with international fair trial standards;
• Noting that Amnesty International believes them to be prisoners of conscience, detained solely for the peaceful exercise of their right to freedom of expression and association, and is calling for their immediate and unconditional release.

I have prepared updated letters to the Minister of Justice, the Ombudsperson and the Azerbaijan Embassy in London.

When the astrometry.net robot solves a photo on Flickr, it leaves a comment identifying the coordinates of the photo and listing the names of objects in the field.

Hello, this is the blind astrometry solver. Your results are:
(RA, Dec) center:(82.4668973542, 6.33857270637) degrees
(RA, Dec) center (H:M:S, D:M:S):(05:29:52.055, +6:20:18.862)
Orientation:161.45 deg E of N

Pixel scale:67.93 arcsec/pixel

Parity:Reverse (“Left-handed”)
Field size :53.14 x 39.85 degrees

Your field contains:
The star Rigel (βOri)
The star Betelgeuse (αOri)
The star Aldebaran (αTau)
The star Bellatrix (γOri)
The star Alnilam (εOri)
The star Alhena (γGem)
The star Alnitak (ζOri)
The star Saiph (κOri)
The star Mintaka (δOri)
The star Cursa (βEri)
IC 2118 / IC 2118 / Witch Head nebula
NGC 1976 / NGC 1976 / Great Nebula in Orion / M 42
NGC 1990 / NGC 1990
IC 434 / IC 434 / Horsehead nebula
IC 443 / IC 443
NGC 2264 / NGC 2264 / Christmas Tree cluster / Cone nebula

View in World Wide Telescope

—–
If you would like to have other images solved, please submit them to the astrometry group.
Posted 3 weeks ago. ( permalink | delete )

These comments are always in the same format, so it’s straightforward to parse them and extract the astrometry metadata as a list of tags. I’ve written a small form which does this, using YQL to grab the comments from a Flickr photo then parsing them using standard DOM traversal and manipulation methods.

If you have a photo which has been solved, generating tags is straightforward. Copy the address of a Flickr photo page into the tagging form and press the big blue ‘Get astrotags’ button. The script should find the comment from astrometry.net and print out the tags for celestial coordinates and names, which you can then paste into the ‘Add a tag’ form on Flickr.

The code to do this is fairly simple, and reproduced below. After initialising the page, we can take advantage of YQL’s HTML parser to fetch all of the comments for a Flickr photo page by selecting all paragraphs inside divs with a class of ‘comment-content’ at that URL.

select * from html where url='http://www.flickr.com/photos/eatyourgreens/4182924966/' and xpath='//div[@class="comment-content"]/p'

We then loop through the results of this query, looking for paragraphs which contain the text ‘blind astrometry solver’. If we have a match, we add this paragraph to the DOM so we can parse it with standard DOM methods. The code then loops through the child nodes of the comment paragraph, running regular expression matches against any text nodes it finds to extract the coordinates of the photo.

Names are slightly more tricky. For those, we grab every line of text between ‘Your field contains:’ and the line ‘—–’ above the signature, strip out whitespace, split each line on ‘/’ to get individual names and store these in an associative array, keyed on name to remove duplicates. That done, we can then just loop through the arrays of coordinates and names and print them out.

Here’s the full code:

 var url = "http://www.flickr.com/photos/skiwalker79/4174398309";
 var comment_holder;
 var position_output;
 var names_output;
 
 function init() {
   var url_input = document.getElementById('photoURL');
   var url_button = document.getElementById('updateURL');
   
   comment_holder = document.getElementById('comment');
   position_output = document.getElementById('position');
   names_output = document.getElementById('names');
   
   url_input.disabled = false;
   url_input.value = url;
   url_button.disabled = false;
   position_output.disabled = false;
   names_output.disabled = false;
   
   addEvent(url_button, 'click', function(e) {
     getFlickrPhotoComments(url_input.value);
     return false;
   });
   addEvent(url_input, 'focus', function(e) {
     url_input.select();
   });
   addEvent(position_output, 'focus', function(e) {
     position_output.select();
   });
   addEvent(names_output, 'focus', function(e) {
     names_output.select();
   });
   
   // Mark up nodes which this script updates as
   // ARIA live regions.
   comment_holder.setAttribute('aria-live', 'polite');
   position_output.setAttribute('aria-live', 'polite');
   names_output.setAttribute('aria-live', 'polite');
   
   getFlickrPhotoComments(url);    
 }

 function getFlickrPhotoComments(url) {

   // YQL query to get all comments from a Flickr photo page.
   var yql = "select * from html where url='"+url+"' and xpath='//div[@class=\"comment-content\"]/p'";
   var yql_url = 'http://query.yahooapis.com/v1/public/yql?q='+escape(yql)+'&format=xml&callback=getAstrometryComment&diagnostics=false';
   position_output.value = '';
   names_output.value = '';
   comment_holder.innerHTML = 'Looking up '+url;
   makeYQLRequest(yql_url);
 }
 
 function makeYQLRequest(yql_url) {
   var script=document.getElementById('yqlscript');
   var newscript=document.createElement('script');
   newscript.type = 'text/javascript';
   newscript.src=yql_url;
   newscript.id='yqlscript';
   document.getElementsByTagName("body")[0].removeChild(script);
   document.getElementsByTagName("body")[0].appendChild(newscript);
 }

function getAstrometryComment(data) {
  var results = data.results;
  
  var comment = 'Sorry, that photo has not been solved by <a href="http://astrometry.net">astrometry.net</a>.';
  for (var i in results) {
    var text = results[i];
    // Comments left by the solver contain the text 'blind astrometry solver'.
    if(text.match(/blind astrometry solver/gi)) {
      comment = text;
    }
  }
  parseComment(comment);
  
}

function parseComment(comment) {
  var astro = {};
  var names = {};
  var parsing_names = false;
  
  comment_holder.innerHTML = comment;
  var children = comment_holder.firstChild.childNodes;
  
  for (var i in children) {
    var child = children[i];
    var text = '';
    text = child.data;
    
    if (text) {
      if (text.match(/(RA, Dec)/g) && text.match(/degrees/g)) {
        text=text.match(/[-0-9\.]+/g);
        astro.RA = text[0];
        astro.Dec = text[1];
      } else if (text.match(/Orientation/g)) {
        text = text.match(/[-0-9\.]+/g);
        astro.orientation = text[0];
      } else if (text.match(/Pixel scale/g)) {
        text = text.match(/[0-9\.]+/g);
        astro.pixelScale = text[0];
      } else if(text.match(/Field size/g)) {
        text = text.match(/[0-9\.]+ x [0-9\.]+ (degrees|arcminutes|arcseconds)/g);
        astro.fieldsize = text[0];
      } else if(text.match(/Your field contains:/g)) {
        parsing_names = true;
      } else if (text.match(/-----/g)) {
        parsing_names = false;
      }

      if (parsing_names) {
        names = addNames(names, text);
      }
      
    }
  }

  renderPositionTags(astro);
  
  renderNameTags(names);
  
  if (astro.RA) {
    position_output.focus();
  }
}

function addNames(names, text) {
  
 text = trim(text);
 text = text.split('/');
 for (var j in text) {
   var name = text[j];
   name = trim(name);
   if (name && name !='Your field contains:'){
     names[name] = name;
   }
 }
 return names;
}

function renderPositionTags(astro) {
  
  position_output.value = '';
   for (var tag in astro) {
     position_output.value += 'astro:'+tag+'="'+astro[tag]+'" ';
   } 
}

function renderNameTags(names) {

  names_output.value = '';
   for (var name in names) {
     names_output.value += 'astro:name="'+name+'" '
   }
}

function trim(text) {
  // Trim leading and trailing whitespace from a string.
  text = text.replace(/^\s+/, '');
  text = text.replace(/\s+$/,'');
  return text;
}

function addEvent(obj, evType, fn) {
  if (obj.addEventListener) {
    obj.addEventListener(evType, fn, false);
    return true;
  } else if (obj.attachEvent) {
    var r = obj.attachEvent("on" + evType, fn);
    return r;
  } else {
    return false;
  }
}

Searching astro:name with YQL, originally uploaded by Eat your greens!.

The YQL team announced personal URLs for queries this week. I’ve used the new feature to set up a shortcut for looking up photos of astronomical objects by name. The URL is:

http://queries.yahooapis.com/v1/public/yql/eatyourgreens/astrolookup?name=M+42

You can set the name parameter in the URL to change the name of the object you are looking for. I’ve also set up a demo page to render results from this query. The URL is:

http://eatyourgreens.org.uk/testapps/yql/astronamesearch.html?name=Horsehead+nebula

Again, change the name parameter in the URL to lookup different objects. Note that it looks for an exact match with the astro:name machine tag, so looking up stars is cumbersome:

http://eatyourgreens.org.uk/testapps/yql/astronamesearch.html?name=the+star+deneb+(αcyg)

Update: it seems Flickr’s machine tag search can match just the first part of a tag, so you can search for stars by supplying the first part of the tag’s value.

http://eatyourgreens.org.uk/testapps/yql/astronamesearch.html?name=the+star+deneb

I’ve also made some changes to flickr.photos.astro in order to enable faster searching by name. Use astro_name in a query to find objects by matching on astro:name:

select * from flickr.photos.astro where astro_name = 'M 31'

or use text to run a Flickr text search across photo descriptions and titles:

select * from flickr.photos.astro where text = 'orion'

If you want to see what values have been used for astro:name on Flickr, I recommend Paul Mison’s excellent machine tag browser.

If you’re interested in the nuts and bolts of the automated astrometry robot, I recommend having a look at astrometry.net and reading Making the sky searchable: Fast geometric hashing for automated astrometry.

Tom Disch: I’m writing a book about what everyone wants most.

Moorcock: Really? Is it about elephants?

Disch: Elephants? No, it’s about becoming more intelligent.

Moorcock: Oh, what I’ve always wanted most is to be an elephant.

Azerbaijani youth activists and bloggers Emin Abdullayev (aged 30) and Adnan Hajizade (aged 26) have been sentenced to two and a half years and two years respectively in an unfair trial. Amnesty International believes the charges against them were fabricated and they have been imprisoned solely for exercising their right to freedom of expression.

On 28 June, a satirical video made by Adnan Hajizade criticizing the Azerbaijani government was posted on the video-sharing website YouTube. On 8 July, Adnan Hajizade and Emin Abdullayev (who blogs under the name Emin Milli) were approached by two men as they were dining with friends in a restaurant in the capital Baku. The men demanded to know what they were talking about. When Emin Abdullayev replied that it did not concern the two men, one of the men head-butted him and he fell to the floor. The same man then struck Adnan Hajizade, knocking him to the ground, and continued to kick and beat Adnan Hajizade and Emin Abdullayev until restaurant staff intervened.

Adnan Hajizade and Emin Abdullayev immediately went to a local police station to report the incident. However, police officers interrogated them as suspects for five hours, denying them access to a lawyer of their choosing and arrested them on charges of ‘hooliganism’. The activists were remanded in custody pending their trial. They were later also charged with inflicting minor bodily harm.

During their investigation the police and prosecutors failed to interview witnesses and to obtain video evidence from a security camera which may have recorded the incident in the restaurant. They also submitted documents to the court which wrongly stated that Adnan Hajizade was unemployed and that he had a criminal record. According to the activist’s lawyers, at the trial the court refused to consider photographs showing the injuries they sustained in the attack, as well as video evidence from mobile phones and the security camera, but provided no explanation as to why. The two men were convicted and sentenced to two and a half and two years respectively in prison. Emin Abdullayev and Adnan Hajizade intend to appeal against the decision.

There is a letter calling for the release of Emin and Adnan, which you can send to President Aliyev, the Minister of Justice, the Ombudsperson and the Azerbaijani Ambassador in London.

The Amnesty November book sale is over and we raised an amazing £3736.56, which brings our total for 2009 to a phenomenal £16,500. Many thanks to everyone who volunteered time to help set up the Church of the Ascension and also sell books on the day. Thanks also to the Ascension for allowing us to take over their church for the three days it takes to set up the November sale. Finally, thanks also to everyone who offered books to the group. We’re sorry that we haven’t been able to accept donations for the November sale. There were simply so many books leftover from June that we were kept busy trying to sell those. I will try to contact everyone who offered books and suggest some alternative places which may be collecting. We will start collecting books again in May 2010.