Recommended
More Related Content
What's hot
What's hot (20)
Similar to Birdpie
Similar to Birdpie (20)
Recently uploaded
Recently uploaded (20)
Birdpie
- 1. www.birdpie.com Anup Narkhede
- 2. About Me • Anup Narkhede • On Rails since 2007 • Employee at www.dynamic50.com • http://twitter.com/railsbob
- 3. Do you tweet links?
- 4. What if you want to look back through your old links?
- 5. Limitations • Search only returns last 1500 tweets • Different TinyUrls for same target links • Twitter API goes back for 3000 tweets
- 6. • Twitter web application • Aggregates all your tweeted links • Auto link tagging • Categories for link management • Shows popularity of the bookmark
- 7. • Resolves the Tiny URLs • Stores the count • 404 Errors and Deadlinks • Gets the page title
- 8. Sign Up • Twitter OAuth (Read-Only) • Private Users?
- 9. Intelligent Tags • Auto Tags • Listing by tags
- 10. Categories • Manage bookmarks by assigning a category • A Category acts like a Folder
- 11. Hashtags • Auto Assign a category by adding a hashtag RT @dynamic50: New blog post: Birdpie Launched http://bit.ly/8iyh2g #ruby Puts the bookmark under Ruby Category
- 12. Rails 3.0
- 13. Architecture • 75% Background processes Crawl Resolve Tag
- 14. Redis • Key value store • Eliminates need for counter cache • Lists, Sets, Ordered Sets • Used to manage bookmark counts
- 15. Resque • Queue Manager • Admin Sinatra Application • Inspect Failed Jobs • Add and remove workers
- 16. Process User URLs Bookmarks Worker 1 Worker 2 Worker 3 •Crawl •Resolve •Store •Tag •Update count
- 18. Redis http://code.google.com/p/redis/ http://github.com/ezmobius/redis-rb Resque http://github.com/defunkt/resque
- 19. Coming soon • Mozilla Plugin to manage your bookmarks • Retweets • Bookmark Search
Editor's Notes
- Hi All, my name is Anup. I am here to present you birdpie, an experimental project developed using Rails 3 with resque backend. This application was released in January.
- So thats me. I am working fulltime on rails since 2007. Right now I work for Dynamic50 which is a cool startup based in Croydon. You can follow me on twitter & my id is railsbob.
- You must have seen a ‘tweet this’ link on popular blogs and websites. We tweet the links, share it among our network and then forget about it.
- Won;t it be nice if I was able to look back through my links, tag them with keywords, search them, or see who else have tweeted it? There are few websites which aggregate bookmarks, but none of them are offered as personal tools.
- But there are many limitations while dealing with twitter. The search cannot go beyond last 1500 tweets We might come accoss different tiny urls pointing to same web page Also, the twitter API cannot go beyond 3000 tweets for a user.
- So we offer you Birdpie, a twitter application which aggregates all your tweeted links. It also stores contextual information like tags and allows you to assign a category.
- It also resolved the tiny urls, and keeps a track of tweeted count unique to a user. All dead and non existing links are discarded and the link is labeled with the title of the target page.
- The sign up is simple, you can log in using your twitter id. Private users can sign up too, but their bookmarks are not available for public viewing on the profile pages or search results.
- Intelligent Tags Once the url is saved, birdpie fetches the target page and passes its content to a service to extract tags. You can see an example above. You can also list all bookmarks by a particular tag.
- A category is a personal folder to manage your boomark. The dashboard allows you to assign a category for a bookmark and define new ones as you want.
- There is another way to assign a category for a bookmark, and that is using a matching hashtag. So, if I tweet a link with hashtag #ruby, it is automatically assigned under ruby cateogy.
- We developed this application on Rails 3.0 The idea was to start early and know it better This being a minimal app, we did not face any major issue. The site is up and running smoothly on production since a month.
- Birdpie is mostly a background application. There are three main tasks Crawl for new tweets, check for links Resolve target urls Tag and save
- We use Redis to manage bookmark counts and tag counts. The in-built increment method eliminates the need of a counter cache. It is pretty fast and improves the overall performance of the application. The Resque queue manager also relies on Redis for operation.
- There are few options available for background processing. We chose Resque as we wanted to break the background tasks into multiple queues with different priorities. Also, the admin app bundled with Resque is a handy tool for monitoring.
- Basically the process is spread across two queues. User queue deals with fetching content from twitter for a particular user. Bookmarks queue resolves the raw link, tags it and updates the redis index
- This is a snapshot of the Resque admin application. You can easily see if you require more workers on a particular queue and act accordingly. It also gives information about failed jobs if any.
- We selected redis and resque over delayed job as we wanted to manage the processes using multiple queues. I have covered a brief introduction here, but you can follow these links if you want to dig in more details.
- We have few more features in the pipeline, some are listed here. The mozilla plugin will provide you easy access to all your bookmarks right in the browser. Retweet within the birdpie application to save someone else’s bookmarks And finally a more extensive bookmark search.
- Thank you for listening to my talk. You can keep a watch on dynamic50 blog for new announcements about birdpie features.