Vicci Ho is a writer, film programmer and producer. She has written for Variety and worked for film festivals across the globe, including Toronto International Film Festival, Zurich Film Festival and directed the Hong Kong Lesbian & Gay Film Festival.  She is the President of Janitor Interactive: a production company specializing in games.

This is her personal page.

My week with the NYT API (or why the hell did I think it was a good idea???)

I have always been a huge fan of the data visualization graphics that the New York Times have been doing over the past few years, such as the 2012 election. A collection of it can be seen here: NYT's 2012 data visualization

Apologies for the genuinely awful sketch, but fist I thought it could be fun to have the text scroll through across something like a movie screen:

scrolling text with NYT

But I wanted to be more ambitious. I wanted to do something using the APIs that NYT have made available, especially the movie reviews API. I had an idea of using the API to search for a number of films (the entire archive is online), and I thought about the idea of searching through the archive and pull reviews from a specific time frame, and see how many films are made by directed by men and how many by women. The simple idea is sketched out (very poorly) below: but basically, there will be a GUI with buttons that I can press to extract that information in possibly a graph form. Though that was something I intended to experiment with AFTER I managed to pull the data.

basic interface idea

My basic game-plan is to figure out how to get the information I need, then figure out a way to code it in processing to ensure it is analyzing the data, then I would use the codes to make visual it graphs, or different circles for different sizes like the NYT...but it would be a sketch that provides a graphical representation of such data.

So I spent the weekend trying to really understand how an API works. The NYT documentation is great but for someone who kinda has no idea, it was tricky. I had to really understand the different formats I can pull out (XML, json), and to understand how to make the API query. It took me a few hours, but I can now figure out how to pull search using the API.

Then I realized the major problem: the API doesn't actually give me the information I need. I had thought about pulling data from the reviews, then building a string document that will search the director name by gender to divide them into male and female results, but the NYT search results doesn't actually give us the review. I would receive a link to the review, but I can't actually pull the full review. Even if I just try and search for all the records in a same time, the API, naturally, limits the results I can access at one time.

Here's a screenshot of me using the API to search 'war', it returns 115 results but not all are listed in the page:

NYT API search result

So the valuable lesson learned is: APIs actually doesn't mean a free for all and there are a lot of limitations as to what I can do. I am not sure I am a good enough programmer to hack it to do more than I do. But, I still want to use the API to do something with my sketch, so I just need to work out what I can do within the limitations.

The music block idea

Final project thoughts