Skip to content

Instantly share code, notes, and snippets.

@ginatrapani

ginatrapani/Favorites crawl log

Created November 16, 2010 23:18
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ginatrapani/702722 to your computer and use it in GitHub Desktop.
Save ginatrapani/702722 to your computer and use it in GitHub Desktop.
Download ZIP
I deleted all rows from tu_favorites, and set all the favorite-related tu_instance table fields to 0 and ran the crawler. Here's what happened.
Raw
Favorites crawl log
2010-11-16 15:24:33 | 3.27 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:Errors to tolerate: 10
2010-11-16 15:24:36 | 3.91 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:API request: https://api.twitter.com/1/account/rate_limit_status.xml
2010-11-16 15:24:36 | 3.91 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:Parsing XML data from https://api.twitter.com/1/account/rate_limit_status.xml
2010-11-16 15:24:36 | 3.91 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:45 of 350 API calls left this hour; 7 for crawler until 15:43:11
2010-11-16 15:24:36 | 3.91 MB | ginatrapani | TwitterCrawler:***in fetchInstanceUserTweets
2010-11-16 15:24:37 | 3.91 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:API request: https://api.twitter.com/1/users/show/ginatrapani.xml
2010-11-16 15:24:37 | 3.91 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:44 of 350 API calls left this hour; 6 for crawler until 15:43:11
2010-11-16 15:24:37 | 3.97 MB | ginatrapani | UserMySQLDAO:User ginatrapani updated in system.
2010-11-16 15:24:37 | 4.00 MB | ginatrapani | TwitterCrawler:Owner info set.
2010-11-16 15:24:39 | 3.99 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:API request: https://api.twitter.com/1/statuses/user_timeline/ginatrapani.xml?count=200&include_rts=true&since_id=4400967323754496
2010-11-16 15:24:39 | 3.99 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:43 of 350 API calls left this hour; 5 for crawler until 15:43:11
2010-11-16 15:24:39 | 4.38 MB | ginatrapani | TwitterCrawler:0 tweet(s) found and 0 saved
2010-11-16 15:24:39 | 4.38 MB | ginatrapani | TwitterCrawler:More than Twitter cap of 3200 already in system, moving on.
2010-11-16 15:24:39 | 4.38 MB | ginatrapani | TwitterCrawler:3201 in system; 3570 by owner
2010-11-16 15:24:39 | 4.37 MB | ginatrapani | TwitterCrawler:****working on favorites****
2010-11-16 15:24:39 | 4.37 MB | ginatrapani | TwitterCrawler:last favs count: 0
2010-11-16 15:24:39 | 4.37 MB | ginatrapani | TwitterCrawler:curr favs count: 537
2010-11-16 15:24:39 | 4.37 MB | ginatrapani | TwitterCrawler:got last_page_fetched_favorites: 0
2010-11-16 15:24:39 | 4.37 MB | ginatrapani | TwitterCrawler:owner favs: 537, instance owner favs in system: 0
2010-11-16 15:24:39 | 4.37 MB | ginatrapani | TwitterCrawler:favs missing: 537
2010-11-16 15:24:39 | 4.37 MB | ginatrapani | TwitterCrawler:new favs to add: 537
2010-11-16 15:24:39 | 4.37 MB | ginatrapani | TwitterCrawler:extra pages: 27
2010-11-16 15:24:39 | 4.37 MB | ginatrapani | TwitterCrawler:total last favs count: 0, last page fetched: 28, last fav id: 4396038618091521
2010-11-16 15:24:39 | 4.37 MB | ginatrapani | TwitterCrawler:current favs count: 537, new favs to add: 537, last page of favs: 161, mode: 0
2010-11-16 15:24:39 | 4.37 MB | ginatrapani | TwitterCrawler:in fetchInstanceFavorites: API available: 1, avail for crawler: 5
2010-11-16 15:24:40 | 4.41 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:API request: https://api.twitter.com/1/favorites/ginatrapani.xml?page=27
2010-11-16 15:24:40 | 4.41 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:42 of 350 API calls left this hour; 4 for crawler until 15:43:11
2010-11-16 15:24:40 | 4.70 MB | ginatrapani | TwitterCrawler:user id: 930061
2010-11-16 15:24:40 | 4.70 MB | ginatrapani | TwitterCrawler:working on fav: 562278652
2010-11-16 15:24:40 | 4.84 MB | ginatrapani | TwitterCrawler:working on fav: 509007822
2010-11-16 15:24:40 | 5.03 MB | ginatrapani | FavoritePostMySQLDAO:Found reply by a friend!
2010-11-16 15:24:40 | 5.03 MB | ginatrapani | TwitterCrawler:working on fav: 508714172
2010-11-16 15:24:40 | 5.04 MB | ginatrapani | TwitterCrawler:working on fav: 502000572
2010-11-16 15:24:40 | 5.04 MB | ginatrapani | TwitterCrawler:working on fav: 475311452
2010-11-16 15:24:40 | 5.05 MB | ginatrapani | TwitterCrawler:working on fav: 472278402
2010-11-16 15:24:40 | 5.06 MB | ginatrapani | TwitterCrawler:working on fav: 468900002
2010-11-16 15:24:40 | 5.07 MB | ginatrapani | TwitterCrawler:working on fav: 456902652
2010-11-16 15:24:40 | 5.07 MB | ginatrapani | TwitterCrawler:working on fav: 452057862
2010-11-16 15:24:40 | 5.08 MB | ginatrapani | TwitterCrawler:working on fav: 450223352
2010-11-16 15:24:40 | 5.09 MB | ginatrapani | TwitterCrawler:working on fav: 419364772
2010-11-16 15:24:40 | 5.09 MB | ginatrapani | TwitterCrawler:working on fav: 329153442
2010-11-16 15:24:40 | 5.10 MB | ginatrapani | TwitterCrawler:working on fav: 321110172
2010-11-16 15:24:40 | 5.11 MB | ginatrapani | TwitterCrawler:working on fav: 202675042
2010-11-16 15:24:40 | 5.11 MB | ginatrapani | TwitterCrawler:working on fav: 62063322
2010-11-16 15:24:40 | 5.12 MB | ginatrapani | TwitterCrawler:working on fav: 5029923
2010-11-16 15:24:40 | 4.88 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:API request: https://api.twitter.com/1/favorites/ginatrapani.xml?page=26
2010-11-16 15:24:40 | 4.88 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:41 of 350 API calls left this hour; 3 for crawler until 15:43:11
2010-11-16 15:24:40 | 5.11 MB | ginatrapani | TwitterCrawler:user id: 930061
2010-11-16 15:24:40 | 5.11 MB | ginatrapani | TwitterCrawler:working on fav: 772534247
2010-11-16 15:24:40 | 5.12 MB | ginatrapani | TwitterCrawler:working on fav: 769704709
2010-11-16 15:24:40 | 5.12 MB | ginatrapani | TwitterCrawler:working on fav: 769687692
2010-11-16 15:24:40 | 5.13 MB | ginatrapani | TwitterCrawler:working on fav: 769670931
2010-11-16 15:24:40 | 5.14 MB | ginatrapani | TwitterCrawler:working on fav: 767143925
2010-11-16 15:24:40 | 5.14 MB | ginatrapani | TwitterCrawler:working on fav: 767128994
2010-11-16 15:24:40 | 5.15 MB | ginatrapani | TwitterCrawler:working on fav: 765472751
2010-11-16 15:24:40 | 5.16 MB | ginatrapani | TwitterCrawler:working on fav: 732900612
2010-11-16 15:24:40 | 5.16 MB | ginatrapani | TwitterCrawler:working on fav: 716141652
2010-11-16 15:24:40 | 5.17 MB | ginatrapani | TwitterCrawler:working on fav: 712836472
2010-11-16 15:24:40 | 5.18 MB | ginatrapani | TwitterCrawler:working on fav: 708396642
2010-11-16 15:24:40 | 5.18 MB | ginatrapani | TwitterCrawler:working on fav: 704944102
2010-11-16 15:24:40 | 5.20 MB | ginatrapani | FavoritePostMySQLDAO:Found reply by a friend!
2010-11-16 15:24:40 | 5.19 MB | ginatrapani | TwitterCrawler:working on fav: 700565682
2010-11-16 15:24:40 | 5.20 MB | ginatrapani | TwitterCrawler:working on fav: 691215592
2010-11-16 15:24:40 | 5.21 MB | ginatrapani | TwitterCrawler:working on fav: 677271892
2010-11-16 15:24:40 | 5.21 MB | ginatrapani | TwitterCrawler:working on fav: 637827842
2010-11-16 15:24:40 | 5.22 MB | ginatrapani | TwitterCrawler:working on fav: 624384312
2010-11-16 15:24:40 | 5.23 MB | ginatrapani | TwitterCrawler:working on fav: 586175542
2010-11-16 15:24:40 | 5.23 MB | ginatrapani | TwitterCrawler:working on fav: 576626232
2010-11-16 15:24:40 | 5.24 MB | ginatrapani | TwitterCrawler:working on fav: 573024942
2010-11-16 15:24:42 | 4.93 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:API request: https://api.twitter.com/1/favorites/ginatrapani.xml?page=25
2010-11-16 15:24:42 | 4.93 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:40 of 350 API calls left this hour; 2 for crawler until 15:43:11
2010-11-16 15:24:42 | 5.16 MB | ginatrapani | TwitterCrawler:user id: 930061
2010-11-16 15:24:42 | 5.16 MB | ginatrapani | TwitterCrawler:working on fav: 796099688
2010-11-16 15:24:42 | 5.17 MB | ginatrapani | TwitterCrawler:working on fav: 794890211
2010-11-16 15:24:42 | 5.17 MB | ginatrapani | TwitterCrawler:working on fav: 793800186
2010-11-16 15:24:42 | 5.18 MB | ginatrapani | TwitterCrawler:working on fav: 791644658
2010-11-16 15:24:42 | 5.19 MB | ginatrapani | TwitterCrawler:working on fav: 790863906
2010-11-16 15:24:42 | 5.19 MB | ginatrapani | TwitterCrawler:working on fav: 790675801
2010-11-16 15:24:42 | 5.20 MB | ginatrapani | TwitterCrawler:working on fav: 784647266
2010-11-16 15:24:42 | 5.21 MB | ginatrapani | TwitterCrawler:working on fav: 784581707
2010-11-16 15:24:42 | 5.22 MB | ginatrapani | TwitterCrawler:working on fav: 783035592
2010-11-16 15:24:42 | 5.22 MB | ginatrapani | TwitterCrawler:working on fav: 782883198
2010-11-16 15:24:42 | 5.23 MB | ginatrapani | TwitterCrawler:working on fav: 782486648
2010-11-16 15:24:42 | 5.24 MB | ginatrapani | TwitterCrawler:working on fav: 781222571
2010-11-16 15:24:42 | 5.24 MB | ginatrapani | TwitterCrawler:working on fav: 781108063
2010-11-16 15:24:42 | 5.25 MB | ginatrapani | TwitterCrawler:working on fav: 780931842
2010-11-16 15:24:42 | 5.26 MB | ginatrapani | TwitterCrawler:working on fav: 778883072
2010-11-16 15:24:42 | 5.26 MB | ginatrapani | TwitterCrawler:working on fav: 778878851
2010-11-16 15:24:42 | 5.28 MB | ginatrapani | FavoritePostMySQLDAO:Found reply by a friend!
2010-11-16 15:24:42 | 5.27 MB | ginatrapani | TwitterCrawler:working on fav: 777644754
2010-11-16 15:24:42 | 5.28 MB | ginatrapani | TwitterCrawler:working on fav: 777532894
2010-11-16 15:24:42 | 5.28 MB | ginatrapani | TwitterCrawler:working on fav: 774641446
2010-11-16 15:24:42 | 5.29 MB | ginatrapani | TwitterCrawler:working on fav: 772912108
2010-11-16 15:24:44 | 4.98 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:API request: https://api.twitter.com/1/favorites/ginatrapani.xml?page=24
2010-11-16 15:24:44 | 4.98 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:39 of 350 API calls left this hour; 1 for crawler until 15:43:11
2010-11-16 15:24:44 | 5.21 MB | ginatrapani | TwitterCrawler:user id: 930061
2010-11-16 15:24:44 | 5.21 MB | ginatrapani | TwitterCrawler:working on fav: 808696666
2010-11-16 15:24:44 | 5.22 MB | ginatrapani | TwitterCrawler:working on fav: 807265330
2010-11-16 15:24:44 | 5.22 MB | ginatrapani | TwitterCrawler:working on fav: 806896288
2010-11-16 15:24:44 | 5.23 MB | ginatrapani | TwitterCrawler:working on fav: 806842781
2010-11-16 15:24:44 | 5.24 MB | ginatrapani | TwitterCrawler:working on fav: 806755375
2010-11-16 15:24:44 | 5.24 MB | ginatrapani | TwitterCrawler:working on fav: 806316774
2010-11-16 15:24:44 | 5.25 MB | ginatrapani | TwitterCrawler:working on fav: 806190537
2010-11-16 15:24:44 | 5.26 MB | ginatrapani | TwitterCrawler:working on fav: 805852158
2010-11-16 15:24:44 | 5.26 MB | ginatrapani | TwitterCrawler:working on fav: 805730717
2010-11-16 15:24:44 | 5.27 MB | ginatrapani | TwitterCrawler:working on fav: 805220828
2010-11-16 15:24:44 | 5.28 MB | ginatrapani | TwitterCrawler:working on fav: 804947188
2010-11-16 15:24:44 | 5.29 MB | ginatrapani | TwitterCrawler:working on fav: 804878071
2010-11-16 15:24:44 | 5.29 MB | ginatrapani | TwitterCrawler:working on fav: 803595517
2010-11-16 15:24:44 | 5.30 MB | ginatrapani | TwitterCrawler:working on fav: 801438216
2010-11-16 15:24:44 | 5.31 MB | ginatrapani | TwitterCrawler:working on fav: 801291223
2010-11-16 15:24:44 | 5.32 MB | ginatrapani | TwitterCrawler:working on fav: 801268707
2010-11-16 15:24:44 | 5.32 MB | ginatrapani | TwitterCrawler:working on fav: 801266125
2010-11-16 15:24:44 | 5.33 MB | ginatrapani | TwitterCrawler:working on fav: 799699098
2010-11-16 15:24:44 | 5.34 MB | ginatrapani | TwitterCrawler:working on fav: 799101177
2010-11-16 15:24:44 | 5.34 MB | ginatrapani | TwitterCrawler:working on fav: 796899529
2010-11-16 15:24:46 | 5.03 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:API request: https://api.twitter.com/1/favorites/ginatrapani.xml?page=23
2010-11-16 15:24:46 | 5.03 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:38 of 350 API calls left this hour; 0 for crawler until 15:43:11
2010-11-16 15:24:46 | 5.26 MB | ginatrapani | TwitterCrawler:user id: 930061
2010-11-16 15:24:46 | 5.26 MB | ginatrapani | TwitterCrawler:working on fav: 824200811
2010-11-16 15:24:46 | 5.27 MB | ginatrapani | TwitterCrawler:working on fav: 822937906
2010-11-16 15:24:46 | 5.27 MB | ginatrapani | TwitterCrawler:working on fav: 822672148
2010-11-16 15:24:46 | 5.28 MB | ginatrapani | TwitterCrawler:working on fav: 822632869
2010-11-16 15:24:46 | 5.29 MB | ginatrapani | TwitterCrawler:working on fav: 822082216
2010-11-16 15:24:46 | 5.30 MB | ginatrapani | TwitterCrawler:working on fav: 821258496
2010-11-16 15:24:46 | 5.30 MB | ginatrapani | TwitterCrawler:working on fav: 818571369
2010-11-16 15:24:46 | 5.31 MB | ginatrapani | TwitterCrawler:working on fav: 817911551
2010-11-16 15:24:46 | 5.32 MB | ginatrapani | TwitterCrawler:working on fav: 817780288
2010-11-16 15:24:46 | 5.32 MB | ginatrapani | TwitterCrawler:working on fav: 817142820
2010-11-16 15:24:46 | 5.33 MB | ginatrapani | TwitterCrawler:working on fav: 816059399
2010-11-16 15:24:46 | 5.34 MB | ginatrapani | TwitterCrawler:working on fav: 815987691
2010-11-16 15:24:46 | 5.34 MB | ginatrapani | TwitterCrawler:working on fav: 815925822
2010-11-16 15:24:46 | 5.35 MB | ginatrapani | TwitterCrawler:working on fav: 812647009
2010-11-16 15:24:46 | 5.36 MB | ginatrapani | TwitterCrawler:working on fav: 812068023
2010-11-16 15:24:46 | 5.36 MB | ginatrapani | TwitterCrawler:working on fav: 811469762
2010-11-16 15:24:46 | 5.37 MB | ginatrapani | TwitterCrawler:working on fav: 811279687
2010-11-16 15:24:46 | 5.38 MB | ginatrapani | TwitterCrawler:working on fav: 810379055
2010-11-16 15:24:46 | 5.38 MB | ginatrapani | TwitterCrawler:working on fav: 809705976
2010-11-16 15:24:46 | 5.39 MB | ginatrapani | TwitterCrawler:working on fav: 809426751
2010-11-16 15:24:46 | 5.03 MB | ginatrapani | TwitterCrawler:new_favs_to_add: 537, fcount: 96
2010-11-16 15:24:46 | 5.03 MB | ginatrapani | TwitterCrawler:new 'last fav id': 4396038618091521
2010-11-16 15:24:46 | 5.03 MB | ginatrapani | TwitterCrawler:*********in cleanUpMissedFavsUnFavs**********
2010-11-16 15:24:46 | 5.03 MB | ginatrapani | TwitterCrawler:****owner user id: 930061
2010-11-16 15:24:46 | 5.03 MB | ginatrapani | TwitterCrawler:start page: 2, with 1 cleanup pages
2010-11-16 15:24:46 | 5.05 MB | ginatrapani | InstanceMySQLDAO:Updated ginatrapani's system status.
2010-11-16 15:24:46 | 5.04 MB | ginatrapani | TwitterCrawler:Friend archive loaded
2010-11-16 15:24:46 | 5.04 MB | ginatrapani | TwitterCrawler:Follower archive is not loaded; fetch should begin.
2010-11-16 15:24:46 | 5.41 MB | ginatrapani | TwitterCrawler:734 stray replied-to tweets to load for user ID 930061 on twitter
2010-11-16 15:24:46 | 5.04 MB | ginatrapani | TwitterCrawler:0 unloaded follower details to load.
2010-11-16 15:24:47 | 5.06 MB | ginatrapani | InstanceMySQLDAO:Updated ginatrapani's system status.
2010-11-16 15:24:47 | 5.05 MB | ginatrapani | TwitterCrawler:****working on cleanUpFollows****
2010-11-16 15:24:47 | 5.06 MB | ginatrapani | InstanceMySQLDAO:Updated ginatrapani's system status.
<!--second run-->
2010-11-16 15:25:29 | 3.27 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:Errors to tolerate: 10
2010-11-16 15:25:30 | 3.91 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:API request: https://api.twitter.com/1/account/rate_limit_status.xml
2010-11-16 15:25:30 | 3.91 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:Parsing XML data from https://api.twitter.com/1/account/rate_limit_status.xml
2010-11-16 15:25:30 | 3.91 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:38 of 350 API calls left this hour; 2 for crawler until 15:43:11
2010-11-16 15:25:30 | 3.91 MB | ginatrapani | TwitterCrawler:***in fetchInstanceUserTweets
2010-11-16 15:25:32 | 3.91 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:API request: https://api.twitter.com/1/users/show/ginatrapani.xml
2010-11-16 15:25:32 | 3.91 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:37 of 350 API calls left this hour; 1 for crawler until 15:43:11
2010-11-16 15:25:32 | 3.97 MB | ginatrapani | UserMySQLDAO:User ginatrapani updated in system.
2010-11-16 15:25:32 | 4.00 MB | ginatrapani | TwitterCrawler:Owner info set.
2010-11-16 15:25:33 | 3.99 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:API request: https://api.twitter.com/1/statuses/user_timeline/ginatrapani.xml?count=200&include_rts=true&since_id=4400967323754496
2010-11-16 15:25:33 | 3.99 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:36 of 350 API calls left this hour; 0 for crawler until 15:43:11
2010-11-16 15:25:33 | 4.38 MB | ginatrapani | TwitterCrawler:0 tweet(s) found and 0 saved
2010-11-16 15:25:33 | 4.38 MB | ginatrapani | TwitterCrawler:More than Twitter cap of 3200 already in system, moving on.
2010-11-16 15:25:33 | 4.38 MB | ginatrapani | TwitterCrawler:3201 in system; 3570 by owner
2010-11-16 15:25:33 | 4.37 MB | ginatrapani | TwitterCrawler:****working on favorites****
2010-11-16 15:25:33 | 4.37 MB | ginatrapani | TwitterCrawler:last favs count: 537
2010-11-16 15:25:33 | 4.37 MB | ginatrapani | TwitterCrawler:curr favs count: 537
2010-11-16 15:25:33 | 4.37 MB | ginatrapani | TwitterCrawler:got last_page_fetched_favorites: 23
2010-11-16 15:25:33 | 4.37 MB | ginatrapani | TwitterCrawler:owner favs: 537, instance owner favs in system: 96
2010-11-16 15:25:33 | 4.37 MB | ginatrapani | TwitterCrawler:favs missing: 441
2010-11-16 15:25:33 | 4.37 MB | ginatrapani | TwitterCrawler:new favs to add: 0
2010-11-16 15:25:33 | 4.37 MB | ginatrapani | TwitterCrawler:extra pages: 0
2010-11-16 15:25:33 | 4.37 MB | ginatrapani | TwitterCrawler:total last favs count: 537, last page fetched: 23, last fav id: 4396038618091521
2010-11-16 15:25:33 | 4.37 MB | ginatrapani | TwitterCrawler:current favs count: 537, new favs to add: 0, last page of favs: 161, mode: 0
2010-11-16 15:25:33 | 4.37 MB | ginatrapani | TwitterCrawler:in fetchInstanceFavorites: API available: 1, avail for crawler: 0
2010-11-16 15:25:33 | 4.37 MB | ginatrapani | TwitterCrawler:new_favs_to_add: 0, fcount: 0
2010-11-16 15:25:33 | 4.37 MB | ginatrapani | TwitterCrawler:new 'last fav id': 4396038618091521
2010-11-16 15:25:33 | 4.37 MB | ginatrapani | TwitterCrawler:*********in cleanUpMissedFavsUnFavs**********
2010-11-16 15:25:33 | 4.37 MB | ginatrapani | TwitterCrawler:****owner user id: 930061
2010-11-16 15:25:33 | 4.48 MB | ginatrapani | TwitterCrawler:start page: 2, with 1 cleanup pages
2010-11-16 15:25:33 | 4.50 MB | ginatrapani | InstanceMySQLDAO:Updated ginatrapani's system status.
2010-11-16 15:25:33 | 4.65 MB | ginatrapani | TwitterCrawler:Friend archive loaded
2010-11-16 15:25:33 | 4.65 MB | ginatrapani | TwitterCrawler:Follower archive is not loaded; fetch should begin.
2010-11-16 15:25:33 | 5.01 MB | ginatrapani | TwitterCrawler:734 stray replied-to tweets to load for user ID 930061 on twitter
2010-11-16 15:25:33 | 4.65 MB | ginatrapani | TwitterCrawler:0 unloaded follower details to load.
2010-11-16 15:25:33 | 4.67 MB | ginatrapani | InstanceMySQLDAO:Updated ginatrapani's system status.
2010-11-16 15:25:33 | 4.66 MB | ginatrapani | TwitterCrawler:****working on cleanUpFollows****
2010-11-16 15:25:34 | 4.67 MB | ginatrapani | InstanceMySQLDAO:Updated ginatrapani's system status.
<!--third run-->
2010-11-16 15:26:09 | 3.27 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:Errors to tolerate: 10
2010-11-16 15:26:10 | 3.91 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:API request: https://api.twitter.com/1/account/rate_limit_status.xml
2010-11-16 15:26:10 | 3.91 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:Parsing XML data from https://api.twitter.com/1/account/rate_limit_status.xml
2010-11-16 15:26:10 | 3.91 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:36 of 350 API calls left this hour; 2 for crawler until 15:43:11
2010-11-16 15:26:10 | 3.91 MB | ginatrapani | TwitterCrawler:***in fetchInstanceUserTweets
2010-11-16 15:26:11 | 3.91 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:API request: https://api.twitter.com/1/users/show/ginatrapani.xml
2010-11-16 15:26:11 | 3.91 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:35 of 350 API calls left this hour; 1 for crawler until 15:43:11
2010-11-16 15:26:11 | 3.97 MB | ginatrapani | UserMySQLDAO:User ginatrapani updated in system.
2010-11-16 15:26:11 | 4.00 MB | ginatrapani | TwitterCrawler:Owner info set.
2010-11-16 15:26:12 | 3.99 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:API request: https://api.twitter.com/1/statuses/user_timeline/ginatrapani.xml?count=200&include_rts=true&since_id=4400967323754496
2010-11-16 15:26:12 | 3.99 MB | ginatrapani | CrawlerTwitterAPIAccessorOAuth:34 of 350 API calls left this hour; 0 for crawler until 15:43:11
2010-11-16 15:26:12 | 4.38 MB | ginatrapani | TwitterCrawler:0 tweet(s) found and 0 saved
2010-11-16 15:26:12 | 4.38 MB | ginatrapani | TwitterCrawler:More than Twitter cap of 3200 already in system, moving on.
2010-11-16 15:26:12 | 4.38 MB | ginatrapani | TwitterCrawler:3201 in system; 3570 by owner
2010-11-16 15:26:12 | 4.37 MB | ginatrapani | TwitterCrawler:****working on favorites****
2010-11-16 15:26:12 | 4.37 MB | ginatrapani | TwitterCrawler:last favs count: 537
2010-11-16 15:26:12 | 4.37 MB | ginatrapani | TwitterCrawler:curr favs count: 537
2010-11-16 15:26:12 | 4.37 MB | ginatrapani | TwitterCrawler:got last_page_fetched_favorites: 23
2010-11-16 15:26:12 | 4.37 MB | ginatrapani | TwitterCrawler:owner favs: 537, instance owner favs in system: 96
2010-11-16 15:26:12 | 4.37 MB | ginatrapani | TwitterCrawler:favs missing: 441
2010-11-16 15:26:12 | 4.37 MB | ginatrapani | TwitterCrawler:new favs to add: 0
2010-11-16 15:26:12 | 4.37 MB | ginatrapani | TwitterCrawler:extra pages: 0
2010-11-16 15:26:12 | 4.37 MB | ginatrapani | TwitterCrawler:total last favs count: 537, last page fetched: 23, last fav id: 4396038618091521
2010-11-16 15:26:12 | 4.37 MB | ginatrapani | TwitterCrawler:current favs count: 537, new favs to add: 0, last page of favs: 161, mode: 0
2010-11-16 15:26:12 | 4.37 MB | ginatrapani | TwitterCrawler:in fetchInstanceFavorites: API available: 1, avail for crawler: 0
2010-11-16 15:26:12 | 4.37 MB | ginatrapani | TwitterCrawler:new_favs_to_add: 0, fcount: 0
2010-11-16 15:26:12 | 4.37 MB | ginatrapani | TwitterCrawler:new 'last fav id': 4396038618091521
2010-11-16 15:26:12 | 4.37 MB | ginatrapani | TwitterCrawler:*********in cleanUpMissedFavsUnFavs**********
2010-11-16 15:26:12 | 4.37 MB | ginatrapani | TwitterCrawler:****owner user id: 930061
2010-11-16 15:26:12 | 4.48 MB | ginatrapani | TwitterCrawler:start page: 2, with 1 cleanup pages
2010-11-16 15:26:12 | 4.50 MB | ginatrapani | InstanceMySQLDAO:Updated ginatrapani's system status.
2010-11-16 15:26:12 | 4.65 MB | ginatrapani | TwitterCrawler:Friend archive loaded
2010-11-16 15:26:12 | 4.65 MB | ginatrapani | TwitterCrawler:Follower archive is not loaded; fetch should begin.
2010-11-16 15:26:12 | 5.01 MB | ginatrapani | TwitterCrawler:734 stray replied-to tweets to load for user ID 930061 on twitter
2010-11-16 15:26:12 | 4.65 MB | ginatrapani | TwitterCrawler:0 unloaded follower details to load.
2010-11-16 15:26:12 | 4.67 MB | ginatrapani | InstanceMySQLDAO:Updated ginatrapani's system status.
2010-11-16 15:26:12 | 4.66 MB | ginatrapani | TwitterCrawler:****working on cleanUpFollows****
2010-11-16 15:26:13 | 4.67 MB | ginatrapani | InstanceMySQLDAO:Updated ginatrapani's system status.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment