Category: astronomy

Last weekend, 19th & 20th June 2010, saw the first Science Hack Day at the Guardian offices in London. Jeremy Keith organised a venue, food and drink and sponsorship for around 100 people to spend two days building small science projects. Saturday morning saw a series of short talks to introduce the event,  give people some ideas of what they might make, and what tools were available to help them make it. 24 hours of hacking and building followed, with presentations and prizes for the best hacks on Sunday afternoon. Ed Gomez has a great write-up of the hack day itself, and the winning hacks. My personal favourites were the Aurorascope, which shows auroral activity by lighting up LEDs inside a ball representing the Earth, and Random Orbit, a RESTful service to track satellites in real time.

I was asked by Jeremy to give one of the talks at the beginning of the hack day, so I chatted a bit about the work we’ve done with astrometry.net to tag Flickr photos with their celestial locations — astrotagging.

Continue reading Get excited and make things with science

In December, I talked at London Web Standards about tagging astronomy photos with position and name information. I mentioned that around 400 photos have been tagged already on Flickr but this is only a tiny fraction of the 4,900 photos that have been solved by astrometry.net. It would be great if the remaining 4,500 photos could also be tagged, and it ought to be straightforward to generate tags for those photos too. Inspired by the iNaturalist Taxonomic Tagging Tool, I’ve written a little astrotagging form for Flickr photos.

When the astrometry.net robot solves a photo on Flickr, it leaves a comment identifying the coordinates of the photo and listing the names of objects in the field.

Hello, this is the blind astrometry solver. Your results are:
(RA, Dec) center:(82.4668973542, 6.33857270637) degrees
(RA, Dec) center (H:M:S, D:M:S):(05:29:52.055, +6:20:18.862)
Orientation:161.45 deg E of N

Pixel scale:67.93 arcsec/pixel

Parity:Reverse (“Left-handed”)
Field size :53.14 x 39.85 degrees

Your field contains:
The star Rigel (βOri)
The star Betelgeuse (αOri)
The star Aldebaran (αTau)
The star Bellatrix (γOri)
The star Alnilam (εOri)
The star Alhena (γGem)
The star Alnitak (ζOri)
The star Saiph (κOri)
The star Mintaka (δOri)
The star Cursa (βEri)
IC 2118 / IC 2118 / Witch Head nebula
NGC 1976 / NGC 1976 / Great Nebula in Orion / M 42
NGC 1990 / NGC 1990
IC 434 / IC 434 / Horsehead nebula
IC 443 / IC 443
NGC 2264 / NGC 2264 / Christmas Tree cluster / Cone nebula

View in World Wide Telescope

—–
If you would like to have other images solved, please submit them to the astrometry group.
Posted 3 weeks ago. ( permalink | delete )

These comments are always in the same format, so it’s straightforward to parse them and extract the astrometry metadata as a list of tags. I’ve written a small form which does this, using YQL to grab the comments from a Flickr photo then parsing them using standard DOM traversal and manipulation methods.

If you have a photo which has been solved, generating tags is straightforward. Copy the address of a Flickr photo page into the tagging form and press the big blue ‘Get astrotags’ button. The script should find the comment from astrometry.net and print out the tags for celestial coordinates and names, which you can then paste into the ‘Add a tag’ form on Flickr.

The code to do this is fairly simple, and reproduced below. After initialising the page, we can take advantage of YQL’s HTML parser to fetch all of the comments for a Flickr photo page by selecting all paragraphs inside divs with a class of ‘comment-content’ at that URL.

select * from html where url='http://www.flickr.com/photos/eatyourgreens/4182924966/' and xpath='//div[@class="comment-content"]/p'

We then loop through the results of this query, looking for paragraphs which contain the text ‘blind astrometry solver’. If we have a match, we add this paragraph to the DOM so we can parse it with standard DOM methods. The code then loops through the child nodes of the comment paragraph, running regular expression matches against any text nodes it finds to extract the coordinates of the photo.

Names are slightly more tricky. For those, we grab every line of text between ‘Your field contains:’ and the line ‘—–‘ above the signature, strip out whitespace, split each line on ‘/’ to get individual names and store these in an associative array, keyed on name to remove duplicates. That done, we can then just loop through the arrays of coordinates and names and print them out.

Here’s the full code:

 var url = "http://www.flickr.com/photos/skiwalker79/4174398309";
 var comment_holder;
 var position_output;
 var names_output;
 
 function init() {
   var url_input = document.getElementById('photoURL');
   var url_button = document.getElementById('updateURL');
   
   comment_holder = document.getElementById('comment');
   position_output = document.getElementById('position');
   names_output = document.getElementById('names');
   
   url_input.disabled = false;
   url_input.value = url;
   url_button.disabled = false;
   position_output.disabled = false;
   names_output.disabled = false;
   
   addEvent(url_button, 'click', function(e) {
     getFlickrPhotoComments(url_input.value);
     return false;
   });
   addEvent(url_input, 'focus', function(e) {
     url_input.select();
   });
   addEvent(position_output, 'focus', function(e) {
     position_output.select();
   });
   addEvent(names_output, 'focus', function(e) {
     names_output.select();
   });
   
   // Mark up nodes which this script updates as
   // ARIA live regions.
   comment_holder.setAttribute('aria-live', 'polite');
   position_output.setAttribute('aria-live', 'polite');
   names_output.setAttribute('aria-live', 'polite');
   
   getFlickrPhotoComments(url);    
 }

 function getFlickrPhotoComments(url) {

   // YQL query to get all comments from a Flickr photo page.
   var yql = "select * from html where url='"+url+"' and xpath='//div[@class=\"comment-content\"]/p'";
   var yql_url = 'http://query.yahooapis.com/v1/public/yql?q='+escape(yql)+'&format=xml&callback=getAstrometryComment&diagnostics=false';
   position_output.value = '';
   names_output.value = '';
   comment_holder.innerHTML = 'Looking up '+url;
   makeYQLRequest(yql_url);
 }
 
 function makeYQLRequest(yql_url) {
   var script=document.getElementById('yqlscript');
   var newscript=document.createElement('script');
   newscript.type = 'text/javascript';
   newscript.src=yql_url;
   newscript.id='yqlscript';
   document.getElementsByTagName("body")[0].removeChild(script);
   document.getElementsByTagName("body")[0].appendChild(newscript);
 }

function getAstrometryComment(data) {
  var results = data.results;
  
  var comment = 'Sorry, that photo has not been solved by <a href="http://astrometry.net">astrometry.net</a>.';
  for (var i in results) {
    var text = results[i];
    // Comments left by the solver contain the text 'blind astrometry solver'.
    if(text.match(/blind astrometry solver/gi)) {
      comment = text;
    }
  }
  parseComment(comment);
  
}

function parseComment(comment) {
  var astro = {};
  var names = {};
  var parsing_names = false;
  
  comment_holder.innerHTML = comment;
  var children = comment_holder.firstChild.childNodes;
  
  for (var i in children) {
    var child = children[i];
    var text = '';
    text = child.data;
    
    if (text) {
      if (text.match(/(RA, Dec)/g) && text.match(/degrees/g)) {
        text=text.match(/[-0-9\.]+/g);
        astro.RA = text[0];
        astro.Dec = text[1];
      } else if (text.match(/Orientation/g)) {
        text = text.match(/[-0-9\.]+/g);
        astro.orientation = text[0];
      } else if (text.match(/Pixel scale/g)) {
        text = text.match(/[0-9\.]+/g);
        astro.pixelScale = text[0];
      } else if(text.match(/Field size/g)) {
        text = text.match(/[0-9\.]+ x [0-9\.]+ (degrees|arcminutes|arcseconds)/g);
        astro.fieldsize = text[0];
      } else if(text.match(/Your field contains:/g)) {
        parsing_names = true;
      } else if (text.match(/-----/g)) {
        parsing_names = false;
      }

      if (parsing_names) {
        names = addNames(names, text);
      }
      
    }
  }

  renderPositionTags(astro);
  
  renderNameTags(names);
  
  if (astro.RA) {
    position_output.focus();
  }
}

function addNames(names, text) {
  
 text = trim(text);
 text = text.split('/');
 for (var j in text) {
   var name = text[j];
   name = trim(name);
   if (name && name !='Your field contains:'){
     names[name] = name;
   }
 }
 return names;
}

function renderPositionTags(astro) {
  
  position_output.value = '';
   for (var tag in astro) {
     position_output.value += 'astro:'+tag+'="'+astro[tag]+'" ';
   } 
}

function renderNameTags(names) {

  names_output.value = '';
   for (var name in names) {
     names_output.value += 'astro:name="'+name+'" '
   }
}

function trim(text) {
  // Trim leading and trailing whitespace from a string.
  text = text.replace(/^\s+/, '');
  text = text.replace(/\s+$/,'');
  return text;
}

function addEvent(obj, evType, fn) {
  if (obj.addEventListener) {
    obj.addEventListener(evType, fn, false);
    return true;
  } else if (obj.attachEvent) {
    var r = obj.attachEvent("on" + evType, fn);
    return r;
  } else {
    return false;
  }
}

Searching astro:name with YQL, originally uploaded by Eat your greens!.

The YQL team announced personal URLs for queries this week. I’ve used the new feature to set up a shortcut for looking up photos of astronomical objects by name. The URL is:

http://queries.yahooapis.com/v1/public/yql/eatyourgreens/astrolookup?name=M+42

You can set the name parameter in the URL to change the name of the object you are looking for. I’ve also set up a demo page to render results from this query. The URL is:

http://eatyourgreens.org.uk/testapps/yql/astronamesearch.html?name=Horsehead+nebula

Again, change the name parameter in the URL to lookup different objects. Note that it looks for an exact match with the astro:name machine tag, so looking up stars is cumbersome:

http://eatyourgreens.org.uk/testapps/yql/astronamesearch.html?name=the+star+deneb+(αcyg)

Update: it seems Flickr’s machine tag search can match just the first part of a tag, so you can search for stars by supplying the first part of the tag’s value.

http://eatyourgreens.org.uk/testapps/yql/astronamesearch.html?name=the+star+deneb

I’ve also made some changes to flickr.photos.astro in order to enable faster searching by name. Use astro_name in a query to find objects by matching on astro:name:

select * from flickr.photos.astro where astro_name = 'M 31'

or use text to run a Flickr text search across photo descriptions and titles:

select * from flickr.photos.astro where text = 'orion'

If you want to see what values have been used for astro:name on Flickr, I recommend Paul Mison’s excellent machine tag browser.

Here are the slides from my talk at the December LWS meetup – Introducing astrotags: astrometry, machine tags and YQL (20MB pdf). The examples and demos should all be in the YQL category on this blog.

If you’re interested in the nuts and bolts of the automated astrometry robot, I recommend having a look at astrometry.net and reading Making the sky searchable: Fast geometric hashing for automated astrometry.

I was fortunate enough to win one of the prizes at Open Hack London this weekend. I ported the javascript from my astronomy photo browser to YQL Execute, creating a new open data table which returns celestial coordinates for astrotagged flickr photos. Essentially, my hack extends the flickr API to, hopefully, enable location-based searching in the sky.

Since I only wrote my hack in about an hour, during breakfast on Sunday, I returned to it this evening and finished it off. I’ve defined an open data table at http://eatyourgreens.org.uk/yql/flickr.photos.astro.xml which returns all machine tag info in the astro: namespace for the 50 most recently tagged photos. For convenience, it also returns the photo owner, title, url and root url for thumbnail images.

There is a demo, where you can try searching based on Right Ascension and Declination (both expressed in degrees). Please try it out and leave feedback in the comments here.

Demo URL: http://eatyourgreens.org.uk/testapps/yql/locationsearch.html

Example queries

The Carina Nebula

        select * from flickr.photos.astro
        where ra > 155 and ra < 165
        and dec > -65 and dec < -55

The Orion Nebula and surroundings

        select * from flickr.photos.astro
        where ra > 70 and ra < 100
        and dec > -20 and dec < 10

Get lots of photos of Orion (may be slow)

        select * from flickr.photos.astro(0,200)
        where ra > 70 and ra < 100
        and dec > -20 and dec < 10

Find nebulae from the New General Catalogue (names beginning NGC)

      select * from flickr.photos.astro
      where name like 'NGC%'

Find nebulae from the Messier catalogue (names beginning with M )

     select id, title, url, imgroot, username, ra, dec, fov, orientation, name
      from flickr.photos.astro(40)
      where name like 'M %'

Find all photos of the Rosette nebula

      select * from flickr.photos.astro(0,200)
        where name = 'Rosette nebula'

Explicitly declare all the table columns

      select id, title, url, imgroot, username, ra, dec, fov, orientation, name
      from flickr.photos.astro

Following on from my javascript photo browser, for viewing astronomy photos in Google sky, I’ve written a feed to display Astronomy Photographer of the Year (APY) photos in Google Earth. The address is http://www.nmm.ac.uk/collections/feeds/apyKML.cfm That link should open in Google Earth. If it doesn’t, add it manually in Google Earth via ‘Add > Network Link’ (some browsers save the KML feed rather than opening it).

If you’re interested in seeing how the feed is generated, have a look at the source code. I’ll also go through the code here to try and explain how it works. I’ve written it in coldfusion, but it should be straightforward to rewrite in any other server-side language.

Continue reading Building a KML feed with YQL and coldfusion

Just a brief update on my previous post. I’ve now hacked together an astronomy photo browser which displays Flickr photos directly in Google Sky. Requires you to have the Google Earth plugin installed.

Machine tags and Google Sky, originally uploaded by eat your greens.

Astronomy photographer of the year has been open for a couple of months now, and the astrophoto Flickr group has a few hundred photos now. The amazing astrometry.net bot has been scanning the group and about 70 photos have been tagged with their celestial coordinates, using astro: machine tags.

Continue reading Mapping the sky with YQL and astrometry.net