Skip to content

Instantly share code, notes, and snippets.

@Sjeanpierre
Created December 6, 2018 06:26
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save Sjeanpierre/122d5acf8e6a297e0c8bbf85575a2716 to your computer and use it in GitHub Desktop.
Save Sjeanpierre/122d5acf8e6a297e0c8bbf85575a2716 to your computer and use it in GitHub Desktop.
Cloudflare logs Athena DDL - Create Athena schema for Cloudflare logs
CREATE EXTERNAL TABLE `cf_logs`(
`cachecachestatus` string COMMENT 'from deserializer',
`cacheresponsebytes` int COMMENT 'from deserializer',
`cacheresponsestatus` int COMMENT 'from deserializer',
`cachetieredfill` boolean COMMENT 'from deserializer',
`clientasn` int COMMENT 'from deserializer',
`clientcountry` string COMMENT 'from deserializer',
`clientdevicetype` string COMMENT 'from deserializer',
`clientip` string COMMENT 'from deserializer',
`clientipclass` string COMMENT 'from deserializer',
`clientrequestbytes` int COMMENT 'from deserializer',
`clientrequesthost` string COMMENT 'from deserializer',
`clientrequestmethod` string COMMENT 'from deserializer',
`clientrequestpath` string COMMENT 'from deserializer',
`clientrequestprotocol` string COMMENT 'from deserializer',
`clientrequestreferer` string COMMENT 'from deserializer',
`clientrequesturi` string COMMENT 'from deserializer',
`clientrequestuseragent` string COMMENT 'from deserializer',
`clientsslcipher` string COMMENT 'from deserializer',
`clientsslprotocol` string COMMENT 'from deserializer',
`clientsrcport` int COMMENT 'from deserializer',
`edgecoloid` int COMMENT 'from deserializer',
`edgeendtimestamp` string COMMENT 'from deserializer',
`edgepathingop` string COMMENT 'from deserializer',
`edgepathingsrc` string COMMENT 'from deserializer',
`edgepathingstatus` string COMMENT 'from deserializer',
`edgerequesthost` string COMMENT 'from deserializer',
`edgeresponsebytes` int COMMENT 'from deserializer',
`edgeresponsecompressionratiof` float COMMENT 'from deserializer',
`edgeresponsecontenttype` string COMMENT 'from deserializer',
`edgeresponsestatus` int COMMENT 'from deserializer',
`edgeserverip` string COMMENT 'from deserializer',
`edgestarttimestamp` string COMMENT 'from deserializer',
`originip` string COMMENT 'from deserializer',
`originresponsebytes` int COMMENT 'from deserializer',
`originresponsehttpexpires` string COMMENT 'from deserializer',
`originresponsehttplastmodified` string COMMENT 'from deserializer',
`originresponsestatus` int COMMENT 'from deserializer',
`originresponsetime` bigint COMMENT 'from deserializer',
`originsslprotocol` string COMMENT 'from deserializer',
`rayid` string COMMENT 'from deserializer',
`securitylevel` string COMMENT 'from deserializer',
`wafaction` string COMMENT 'from deserializer',
`wafflags` string COMMENT 'from deserializer',
`wafmatchedvar` string COMMENT 'from deserializer',
`wafprofile` string COMMENT 'from deserializer',
`wafruleid` string COMMENT 'from deserializer',
`wafrulemessage` string COMMENT 'from deserializer',
`zoneid` int COMMENT 'from deserializer')
ROW FORMAT SERDE
'org.openx.data.jsonserde.JsonSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat'
LOCATION
's3://****-cloudflare-logpush/****/logs'
TBLPROPERTIES (
'has_encrypted_data'='false',
'transient_lastDdlTime'='1544076993')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment