Created
March 27, 2015 21:41
-
-
Save corey-cole/94279e100b0e7561c696 to your computer and use it in GitHub Desktop.
Logstash and SAS
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Logstash startup completed | |
| 2015-03-27T21:40:12.772Z localhost.localdomain NOTE: There were 376935 observations read from the data set RMTGT.X_EXTND_PRICE_GRID_TX951. | |
| 2015-03-27T21:40:12.779Z localhost.localdomain NOTE: There were 1320 observations read from the data set WORK.WBE0CKXK. | |
| 2015-03-27T21:40:12.781Z localhost.localdomain NOTE: DATA statement used (Total process time): | |
| real time 0.16 seconds | |
| cpu time 0.17 seconds |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| NOTE: There were 376935 observations read from the data set RMTGT.X_EXTND_PRICE_GRID_TX951. | |
| NOTE: There were 284 observations read from the data set RMTGT.X_ANALYTIC_RUN_TX951. | |
| ♀12 The SAS System 13:54 Friday, March 27, 2015 | |
| NOTE: There were 1320 observations read from the data set WORK.WBE0CKXK. | |
| NOTE: DATA statement used (Total process time): | |
| real time 0.16 seconds | |
| cpu time 0.17 seconds | |
| 458 | |
| 459 options missing='.'; | |
| 460 | |
| 461 %rcSet(&syserr); | |
| 462 %rcSet(&sysrc); | |
| 463 %rcSet(&sqlrc); | |
| 464 | |
| 465 | |
| 466 | |
| 467 /** Step end File Writer v2 **/ | |
| 468 | |
| 469 %let etls_endTime = %sysfunc(datetime(),datetime.); | |
| 470 | |
| NOTE: SAS Institute Inc., SAS Campus Drive, Cary, NC USA 27513-2414 | |
| NOTE: The SAS System used: | |
| real time 0.44 seconds | |
| cpu time 0.41 seconds |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| input { | |
| file { | |
| codec => multiline { | |
| pattern => "^\s" | |
| what => "previous" | |
| } | |
| path => "/tmp/output.log" | |
| } | |
| } | |
| filter { | |
| # Lines starting with a form feed (0x0c) are the SAS output header | |
| # The negation regex below takes care of _most_ of these, but occasionally the | |
| # form feed character rolls up into a multiline match | |
| # Oniguruma documentation: http://www.geocities.jp/kosako3/oniguruma/doc/RE.txt | |
| if ([message] =~ /^\f/) { | |
| drop{} | |
| } | |
| # Just noise | |
| if ([message] =~ /SAS Campus Drive/) { | |
| drop{} | |
| } | |
| if ([message] !~ /^(ERROR|WARNING|NOTE):/) { | |
| drop{} | |
| } | |
| mutate { | |
| strip => "message" | |
| } | |
| grok { | |
| match => [ "message", "^ERROR:%{SPACE}%{GREEDYDATA:log_message}" ] | |
| add_field => [ "loglevel", "error" ] | |
| } | |
| grok { | |
| match => [ "message", "^WARNING:%{SPACE}%{GREEDYDATA:log_message}" ] | |
| add_field => [ "loglevel", "warning" ] | |
| } | |
| grok { | |
| match => [ "message", "^NOTE:%{SPACE}%{GREEDYDATA:log_message}" ] | |
| add_field => [ "loglevel", "info" ] | |
| } | |
| } | |
| output { | |
| stdout { } | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Are there any other examples of working with SAS logs? Did you find any ready-made examples from SAS? I'm going your way from scratch now. I would be grateful for any information.