Topix.net Weblog

July 31, 2006

Local News for the World

by skrenta at 7:10 PM

Topix.net adds 5,000 International Cities to Local News Aggregation

We've been working on adding international local news to Topix for nearly a year. The first challenge was to obtain the digital map data to drive our local Knowledge Base, which our geo-categorization engine uses to recognize local references in text. We found a suitable dataset put out by the CIA of all people. We then had to make some adjustments to our AI, you can't just scan for city names but have to disambiguate the local lat+long references, and apply some discrimination to eliminate stray mentions from the article stream.

Finally we had to bulk up our international source crawl. In addition to the international news services, which are pretty well covered in our crawl now, we also did a focused addition of world bloggers who were writing about local events in their towns. Since every blog in our crawl is editorially vetted (there is just too much spam in the blogosphere) this was a labor intensive process.

We're thrilled with the result. Currently it's english-only, and the editorial feel of the posts is probably more reliable for the business traveler than a local resident. However, at 5000 cities and growing and with coverage of 171 countries there is a lot of news. Samples:

Quito, Ecuador
...where our lead engineer is climbing a mountain (and we're looking for stories about missing tourists)

Geelong, Australia
...the home of the Australian entrepreneur behind Tinfinger

Sapporo, Japan
...where the head of product for Topix.net lived for five years

Wuxi, China
...just got some CEO-spam offering an all expenses paid trip to visit the city (the first 50 CEOs who reply get the trip!)

Tallinn, Estonia
Ross Mayfield made me think of this one

Liverpool, UK ...well, we had to pick somewhere in the UK

Check here to see the rest.