Topix.net Weblog

November 7, 2004

Linkston

by skrenta at 10:11 AM

Back in 2000, just after joining Netscape with NewHoo, I put together a simple news aggregator called Linkston.

Linkston crawled various tech sites every 20 minutes, assembling headlines into a simple html list. Linkston was a source-based crawler; anything from Techdirt, Slashdot, CNet, or the NYTimes technology section would be included. It had a small circle of 10-12 sites it could reliably crawl and include.

Linkston was a primitive scraper, and couldn't keep up very well with changes in site html and registration policies. The scrapers broke constantly. Now that we have the topix supercrawler and topic categorization AI I'd been meaning to bring back linkston.. ...finally I figured it was time to exercise some new APIs we'd added and replaced my old broken perl scrapers with a few calls into the topix.net newsbus. Presto, linkston reborn with far better breadth and relevance than before.

ObTopixFeedSalesPitch: we can help you easily add dynamic content to your site as well. Bloggers understand the relation between update frequency and traffic. If you post once a week, you get a certain level of traffic. Views jump at a post every day, but if you can do more than one post/day (Jason Calacanis requires his weblogsinc bloggers to do 50 posts/month) the traffic takes off. With content updated every 10 minutes, parnters using our feeds report significant increases in page views. Dynamic content equals more repeat visitors.