Skip to content

Instantly share code, notes, and snippets.

@charles-pinkston

charles-pinkston/gist:ff7e3edc5da7b92bc0027fb331069819

Created August 23, 2019 21:49
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save charles-pinkston/ff7e3edc5da7b92bc0027fb331069819 to your computer and use it in GitHub Desktop.
Save charles-pinkston/ff7e3edc5da7b92bc0027fb331069819 to your computer and use it in GitHub Desktop.
Download ZIP
Scrubbed Sample fess-crawler.log
Raw
gistfile1.txt
2019-08-23 01:52:11,536 [Crawler-20190822000000-1-2] INFO Crawling URL: https://mywebsite.com/what-if-more-than-one-user-is-involved
2019-08-23 01:52:11,545 [Crawler-20190822000000-1-2] INFO Redirect to URL: https://mywebsite.com/what-if-more-than-one-user-is-involved/
2019-08-23 01:52:21,547 [Crawler-20190822000000-1-2] INFO Crawling URL: https://mywebsite.com/what-if-the-user-does-not-respond-within-30-days
2019-08-23 01:52:21,554 [Crawler-20190822000000-1-2] INFO Redirect to URL: https://mywebsite.com/what-if-the-user-does-not-respond-within-30-days/
2019-08-23 01:52:23,426 [Crawler-20190822000000-1-1] INFO Crawling URL: https://mywebsite.com/how-does-documentation-address-issues
2019-08-23 01:52:23,434 [Crawler-20190822000000-1-1] INFO Redirect to URL: https://mywebsite.com/how-does-documentation-address-issues/
2019-08-23 01:52:23,958 [Crawler-20190822000000-1-3] INFO Crawling URL: https://mywebsite.com/how-does-a-change-occur
2019-08-23 01:52:23,966 [Crawler-20190822000000-1-3] INFO Redirect to URL: https://mywebsite.com/how-does-a-change-occur/
2019-08-23 01:52:29,689 [Crawler-20190822000000-1-5] INFO Crawling URL: https://mywebsite.com/who-do-i-consult-with-for-help
2019-08-23 01:52:29,708 [Crawler-20190822000000-1-5] INFO Redirect to URL: https://mywebsite.com/who-do-i-consult-with-for-help/
2019-08-23 01:52:30,821 [Crawler-20190822000000-1-4] INFO Crawling URL: https://mywebsite.com/what-is-consultation
2019-08-23 01:52:30,830 [Crawler-20190822000000-1-4] INFO Redirect to URL: https://mywebsite.com/what-is-consultation/
2019-08-23 01:52:33,436 [Crawler-20190822000000-1-1] INFO Crawling URL: https://mywebsite.com/when-does-an-issue-have-no-result
2019-08-23 01:52:33,444 [Crawler-20190822000000-1-1] INFO Redirect to URL: https://mywebsite.com/when-does-an-issue-have-no-result/
2019-08-23 01:52:33,967 [Crawler-20190822000000-1-3] INFO Crawling URL: https://mywebsite.com/what-are-the-general-responsibilities
2019-08-23 01:52:33,975 [Crawler-20190822000000-1-3] INFO Redirect to URL: https://mywebsite.com/what-are-the-general-responsibilities/
2019-08-23 01:54:14,835 [Crawler-20190822000000-1-1] INFO Crawling URL: https://mywebsite.com/where-are-special-areas-located
2019-08-23 01:54:14,844 [Crawler-20190822000000-1-1] INFO Redirect to URL: https://mywebsite.com/where-are-special-areas-located/
2019-08-23 01:54:22,522 [Crawler-20190822000000-1-4] INFO Crawling URL: https://mywebsite.com/what-if-my-issue-impacts-special-areas
2019-08-23 01:54:22,531 [Crawler-20190822000000-1-4] INFO Redirect to URL: https://mywebsite.com/what-if-my-issue-impacts-special-areas/
2019-08-23 01:54:22,844 [Crawler-20190822000000-1-2] INFO Crawling URL: https://mywebsite.com/are-there-any-exceptions-to-requirements
2019-08-23 01:54:22,852 [Crawler-20190822000000-1-2] INFO Redirect to URL: https://mywebsite.com/are-there-any-exceptions-to-requirements/
2019-08-23 01:54:31,303 [Crawler-20190822000000-1-5] INFO Crawling URL: https://mywebsite.com/how-do-i-coordinate-with-users
2019-08-23 01:54:31,312 [Crawler-20190822000000-1-5] INFO Redirect to URL: https://mywebsite.com/how-do-i-coordinate-with-users/
2019-08-23 01:54:32,854 [Crawler-20190822000000-1-2] INFO Crawling URL: https://mywebsite.com/how-can-a-user-comment
2019-08-23 01:54:32,862 [Crawler-20190822000000-1-2] INFO Redirect to URL: https://mywebsite.com/how-can-a-user-comment/
2019-08-23 04:25:34,055 [Crawler-20190822000000-1-3] INFO Crawling URL: https://mywebsite.com/what-if-more-than-one-user-is-involved
2019-08-23 04:25:34,064 [Crawler-20190822000000-1-3] INFO Redirect to URL: https://mywebsite.com/what-if-more-than-one-user-is-involved/
2019-08-23 04:25:35,284 [Crawler-20190822000000-1-4] INFO Crawling URL: https://mywebsite.com/what-are-the-general-responsibilities
2019-08-23 04:25:35,292 [Crawler-20190822000000-1-4] INFO Redirect to URL: https://mywebsite.com/what-are-the-general-responsibilities/
2019-08-23 04:25:46,714 [Crawler-20190822000000-1-5] INFO Crawling URL: https://mywebsite.com/what-if-the-user-does-not-respond-within-30-days
2019-08-23 04:25:46,722 [Crawler-20190822000000-1-5] INFO Redirect to URL: https://mywebsite.com/what-if-the-user-does-not-respond-within-30-days/
2019-08-23 04:25:47,130 [Crawler-20190822000000-1-2] INFO Crawling URL: https://mywebsite.com/how-does-documentation-address-issues
2019-08-23 04:25:47,146 [Crawler-20190822000000-1-2] INFO Redirect to URL: https://mywebsite.com/how-does-documentation-address-issues/
2019-08-23 04:25:54,347 [Crawler-20190822000000-1-1] INFO Crawling URL: https://mywebsite.com/how-does-a-change-occur
2019-08-23 04:25:54,357 [Crawler-20190822000000-1-1] INFO Redirect to URL: https://mywebsite.com/how-does-a-change-occur/
2019-08-23 04:25:54,377 [Crawler-20190822000000-1-3] INFO Crawling URL: https://mywebsite.com/who-do-i-consult-with-for-help
2019-08-23 04:25:54,385 [Crawler-20190822000000-1-3] INFO Redirect to URL: https://mywebsite.com/who-do-i-consult-with-for-help/
2019-08-23 04:25:56,724 [Crawler-20190822000000-1-5] INFO Crawling URL: https://mywebsite.com/what-is-consultation
2019-08-23 04:25:56,731 [Crawler-20190822000000-1-5] INFO Redirect to URL: https://mywebsite.com/what-is-consultation/
2019-08-23 04:26:04,359 [Crawler-20190822000000-1-1] INFO Crawling URL: https://mywebsite.com/when-does-an-issue-have-no-result
2019-08-23 04:26:04,370 [Crawler-20190822000000-1-1] INFO Redirect to URL: https://mywebsite.com/when-does-an-issue-have-no-result/
2019-08-23 04:26:35,535 [Crawler-20190822000000-1-4] INFO Crawling URL: https://mywebsite.com/where-are-special-areas-located
2019-08-23 04:26:35,543 [Crawler-20190822000000-1-4] INFO Redirect to URL: https://mywebsite.com/where-are-special-areas-located/
2019-08-23 04:26:37,014 [Crawler-20190822000000-1-5] INFO Crawling URL: https://mywebsite.com/what-if-my-issue-impacts-special-areas
2019-08-23 04:26:37,022 [Crawler-20190822000000-1-5] INFO Redirect to URL: https://mywebsite.com/what-if-my-issue-impacts-special-areas/
2019-08-23 04:26:37,497 [Crawler-20190822000000-1-2] INFO Crawling URL: https://mywebsite.com/are-there-any-exceptions-to-requirements
2019-08-23 04:26:37,504 [Crawler-20190822000000-1-2] INFO Redirect to URL: https://mywebsite.com/are-there-any-exceptions-to-requirements/
2019-08-23 04:26:44,409 [Crawler-20190822000000-1-1] INFO Crawling URL: https://mywebsite.com/how-do-i-coordinate-with-users
2019-08-23 04:26:44,424 [Crawler-20190822000000-1-1] INFO Redirect to URL: https://mywebsite.com/how-do-i-coordinate-with-users/
2019-08-23 04:26:44,950 [Crawler-20190822000000-1-3] INFO Crawling URL: https://mywebsite.com/how-can-a-user-comment
2019-08-23 04:26:44,959 [Crawler-20190822000000-1-3] INFO Redirect to URL: https://mywebsite.com/how-can-a-user-comment/
2019-08-23 09:38:34,216 [Crawler-20190822000000-1-4] INFO Crawling URL: https://mywebsite.com/what-is-consultation
2019-08-23 09:38:34,224 [Crawler-20190822000000-1-4] INFO Redirect to URL: https://mywebsite.com/what-is-consultation/
2019-08-23 09:38:34,246 [Crawler-20190822000000-1-5] INFO Crawling URL: https://mywebsite.com/what-if-more-than-one-user-is-involved
2019-08-23 09:38:34,254 [Crawler-20190822000000-1-5] INFO Redirect to URL: https://mywebsite.com/what-if-more-than-one-user-is-involved/
2019-08-23 09:38:42,162 [Crawler-20190822000000-1-1] INFO Crawling URL: https://mywebsite.com/what-are-the-general-responsibilities
2019-08-23 09:38:42,172 [Crawler-20190822000000-1-1] INFO Redirect to URL: https://mywebsite.com/what-are-the-general-responsibilities/
2019-08-23 09:38:43,442 [Crawler-20190822000000-1-2] INFO Crawling URL: https://mywebsite.com/what-if-the-user-does-not-respond-within-30-days
2019-08-23 09:38:43,451 [Crawler-20190822000000-1-2] INFO Redirect to URL: https://mywebsite.com/what-if-the-user-does-not-respond-within-30-days/
2019-08-23 09:38:44,226 [Crawler-20190822000000-1-4] INFO Crawling URL: https://mywebsite.com/how-does-documentation-address-issues
2019-08-23 09:38:44,234 [Crawler-20190822000000-1-4] INFO Redirect to URL: https://mywebsite.com/how-does-documentation-address-issues/
2019-08-23 09:38:44,255 [Crawler-20190822000000-1-5] INFO Crawling URL: https://mywebsite.com/how-does-a-change-occur
2019-08-23 09:38:44,263 [Crawler-20190822000000-1-5] INFO Redirect to URL: https://mywebsite.com/how-does-a-change-occur/
2019-08-23 09:38:44,334 [Crawler-20190822000000-1-3] INFO Crawling URL: https://mywebsite.com/who-do-i-consult-with-for-help
2019-08-23 09:38:44,341 [Crawler-20190822000000-1-3] INFO Redirect to URL: https://mywebsite.com/who-do-i-consult-with-for-help/
2019-08-23 09:38:54,236 [Crawler-20190822000000-1-4] INFO Crawling URL: https://mywebsite.com/when-does-an-issue-have-no-result
2019-08-23 09:38:54,243 [Crawler-20190822000000-1-4] INFO Redirect to URL: https://mywebsite.com/when-does-an-issue-have-no-result/
2019-08-23 09:39:03,464 [Crawler-20190822000000-1-2] INFO Crawling URL: https://mywebsite.com/where-are-special-areas-located
2019-08-23 09:39:03,473 [Crawler-20190822000000-1-2] INFO Redirect to URL: https://mywebsite.com/where-are-special-areas-located/
2019-08-23 09:39:04,354 [Crawler-20190822000000-1-3] INFO Crawling URL: https://mywebsite.com/what-if-my-issue-impacts-special-areas
2019-08-23 09:39:04,362 [Crawler-20190822000000-1-3] INFO Redirect to URL: https://mywebsite.com/what-if-my-issue-impacts-special-areas/
2019-08-23 09:39:04,425 [Crawler-20190822000000-1-5] INFO Crawling URL: https://mywebsite.com/are-there-any-exceptions-to-requirements
2019-08-23 09:39:04,433 [Crawler-20190822000000-1-5] INFO Redirect to URL: https://mywebsite.com/are-there-any-exceptions-to-requirements/
2019-08-23 09:39:14,256 [Crawler-20190822000000-1-4] INFO Crawling URL: https://mywebsite.com/how-do-i-coordinate-with-users
2019-08-23 09:39:14,264 [Crawler-20190822000000-1-4] INFO Redirect to URL: https://mywebsite.com/how-do-i-coordinate-with-users/
2019-08-23 09:39:14,435 [Crawler-20190822000000-1-5] INFO Crawling URL: https://mywebsite.com/how-can-a-user-comment
2019-08-23 09:39:14,442 [Crawler-20190822000000-1-5] INFO Redirect to URL: https://mywebsite.com/how-can-a-user-comment/
2019-08-23 15:38:39,382 [Crawler-20190822000000-1-2] INFO Crawling URL: https://mywebsite.com/what-if-the-user-does-not-respond-within-30-days
2019-08-23 15:38:39,391 [Crawler-20190822000000-1-2] INFO Redirect to URL: https://mywebsite.com/what-if-the-user-does-not-respond-within-30-days/
2019-08-23 15:38:40,527 [Crawler-20190822000000-1-1] INFO Crawling URL: https://mywebsite.com/how-does-a-change-occur
2019-08-23 15:38:40,536 [Crawler-20190822000000-1-1] INFO Redirect to URL: https://mywebsite.com/how-does-a-change-occur/
2019-08-23 15:38:42,551 [Crawler-20190822000000-1-4] INFO Crawling URL: https://mywebsite.com/who-do-i-consult-with-for-help
2019-08-23 15:38:42,560 [Crawler-20190822000000-1-4] INFO Redirect to URL: https://mywebsite.com/who-do-i-consult-with-for-help/
2019-08-23 15:38:42,582 [Crawler-20190822000000-1-3] INFO Crawling URL: https://mywebsite.com/what-is-consultation
2019-08-23 15:38:42,590 [Crawler-20190822000000-1-3] INFO Redirect to URL: https://mywebsite.com/what-is-consultation/
2019-08-23 15:38:49,493 [Crawler-20190822000000-1-5] INFO Crawling URL: https://mywebsite.com/when-does-an-issue-have-no-result
2019-08-23 15:38:49,501 [Crawler-20190822000000-1-5] INFO Redirect to URL: https://mywebsite.com/when-does-an-issue-have-no-result/
2019-08-23 15:38:50,538 [Crawler-20190822000000-1-1] INFO Crawling URL: https://mywebsite.com/what-if-more-than-one-user-is-involved
2019-08-23 15:38:50,546 [Crawler-20190822000000-1-1] INFO Redirect to URL: https://mywebsite.com/what-if-more-than-one-user-is-involved/
2019-08-23 15:38:52,562 [Crawler-20190822000000-1-4] INFO Crawling URL: https://mywebsite.com/what-are-the-general-responsibilities
2019-08-23 15:38:52,570 [Crawler-20190822000000-1-4] INFO Redirect to URL: https://mywebsite.com/what-are-the-general-responsibilities/
2019-08-23 15:38:52,592 [Crawler-20190822000000-1-3] INFO Crawling URL: https://mywebsite.com/how-does-documentation-address-issues
2019-08-23 15:38:52,599 [Crawler-20190822000000-1-3] INFO Redirect to URL: https://mywebsite.com/how-does-documentation-address-issues/
2019-08-23 15:39:20,761 [Crawler-20190822000000-1-1] INFO Crawling URL: https://mywebsite.com/where-are-special-areas-located
2019-08-23 15:39:20,768 [Crawler-20190822000000-1-1] INFO Redirect to URL: https://mywebsite.com/where-are-special-areas-located/
2019-08-23 15:39:24,082 [Crawler-20190822000000-1-3] INFO Crawling URL: https://mywebsite.com/what-if-my-issue-impacts-special-areas
2019-08-23 15:39:24,088 [Crawler-20190822000000-1-4] INFO Crawling URL: https://mywebsite.com/are-there-any-exceptions-to-requirements
2019-08-23 15:39:24,091 [Crawler-20190822000000-1-3] INFO Redirect to URL: https://mywebsite.com/what-if-my-issue-impacts-special-areas/
2019-08-23 15:39:24,095 [Crawler-20190822000000-1-4] INFO Redirect to URL: https://mywebsite.com/are-there-any-exceptions-to-requirements/
2019-08-23 15:39:30,065 [Crawler-20190822000000-1-2] INFO Crawling URL: https://mywebsite.com/how-do-i-coordinate-with-users
2019-08-23 15:39:30,076 [Crawler-20190822000000-1-2] INFO Redirect to URL: https://mywebsite.com/how-do-i-coordinate-with-users/
2019-08-23 15:39:30,238 [Crawler-20190822000000-1-5] INFO Crawling URL: https://mywebsite.com/how-can-a-user-comment
2019-08-23 15:39:30,247 [Crawler-20190822000000-1-5] INFO Redirect to URL: https://mywebsite.com/how-can-a-user-comment/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment