Skip to content

Instantly share code, notes, and snippets.

@andrewkroh

andrewkroh/field_count.csv Secret

Created March 17, 2022 18:06
Show Gist options
  • Save andrewkroh/885e28b1cdafbeacf0fca20b062a6de2 to your computer and use it in GitHub Desktop.
Save andrewkroh/885e28b1cdafbeacf0fca20b062a6de2 to your computer and use it in GitHub Desktop.
Download ZIP
Count of Fields in each Fleet Integration Data Stream (commit a987263b6)
Raw
field_count.csv
Integration Data Stream Field Count
1password item_usages 39
1password signin_attempts 41
activemq audit 90
activemq broker 68
activemq log 62
activemq queue 72
activemq topic 71
akamai siem 111
apache access 90
apache error 79
apache status 72
atlassian_bitbucket audit 81
atlassian_confluence audit 82
atlassian_jira audit 81
auditd log 186
auth0 logs 105
aws billing 72
aws cloudfront_logs 93
aws cloudtrail 117
aws cloudwatch_logs 41
aws cloudwatch_metrics 53
aws dynamodb 79
aws ebs 64
aws ec2_logs 41
aws ec2_metrics 97
aws elb_logs 100
aws elb_metrics 109
aws firewall_logs 119
aws firewall_metrics 60
aws lambda 68
aws natgateway 67
aws rds 133
aws route53_public_logs 71
aws route53_resolver_logs 75
aws s3_daily_storage 56
aws s3_request 70
aws s3_storage_lens 81
aws s3access 101
aws sns 69
aws sqs 63
aws transitgateway 60
aws usage 58
aws vpcflow 95
aws vpn 57
aws waf 73
aws_logs generic 47
azure activitylogs 121
azure auditlogs 133
azure eventhub 104
azure platformlogs 112
azure signinlogs 162
azure springcloudlogs 113
azure_application_insights app_insights 58
azure_application_insights app_state 75
azure_billing billing 66
azure_metrics compute_vm 56
azure_metrics compute_vm_scaleset 56
azure_metrics container_instance 56
azure_metrics container_registry 56
azure_metrics container_service 56
azure_metrics database_account 56
azure_metrics monitor 56
azure_metrics storage_account 56
barracuda spamfirewall 801
barracuda waf 801
bluecoat director 801
carbonblack_edr log 227
cassandra log 20
cassandra metrics 143
cef log 258
checkpoint firewall 563
cisco asa 182
cisco ftd 197
cisco ios 109
cisco meraki 830
cisco nexus 830
cisco_asa log 191
cisco_duo admin 59
cisco_duo auth 105
cisco_duo offline_enrollment 47
cisco_duo summary 44
cisco_duo telephony 45
cisco_ftd log 198
cisco_ios log 111
cisco_meraki log 830
cisco_nexus log 830
cisco_secure_endpoint event 141
cisco_umbrella log 147
cloudflare audit 71
cloudflare logpull 181
cockroachdb status 48
containerd blkio 16
containerd cpu 17
containerd memory 27
crowdstrike falcon 193
crowdstrike fdr 405
cyberark corepas 801
cyberarkpas audit 140
cylance protect 801
docker container 33
docker cpu 40
docker diskio 44
docker event 33
docker healthcheck 32
docker image 32
docker info 31
docker memory 37
docker network 35
elastic_agent apm_server_logs 40
elastic_agent apm_server_metrics 115
elastic_agent auditbeat_logs 40
elastic_agent auditbeat_metrics 115
elastic_agent elastic_agent_logs 40
elastic_agent elastic_agent_metrics 181
elastic_agent endpoint_security_metrics 48
elastic_agent endpoint_sercurity_logs 40
elastic_agent filebeat_logs 40
elastic_agent filebeat_metrics 115
elastic_agent fleet_server_logs 40
elastic_agent fleet_server_metrics 115
elastic_agent heartbeat_logs 40
elastic_agent heartbeat_metrics 115
elastic_agent metricbeat_logs 40
elastic_agent metricbeat_metrics 115
elastic_agent osquerybeat_logs 40
elastic_agent osquerybeat_metrics 115
elastic_agent packetbeat_logs 39
elastic_agent packetbeat_metrics 115
elasticsearch audit 36
elasticsearch ccr 78
elasticsearch cluster_stats 63
elasticsearch deprecation 13
elasticsearch enrich 31
elasticsearch gc 32
elasticsearch index 95
elasticsearch index_recovery 51
elasticsearch index_summary 53
elasticsearch ml_job 25
elasticsearch node 26
elasticsearch node_stats 145
elasticsearch pending_tasks 23
elasticsearch server 18
elasticsearch shard 32
elasticsearch slowlog 25
etcd leader 19
etcd metrics 28
etcd self 24
etcd store 27
f5 bigipafm 804
f5 bigipapm 806
fireeye nx 144
fortinet clientendpoint 830
fortinet firewall 573
fortinet fortimail 830
fortinet fortimanager 833
gcp audit 120
gcp dns 75
gcp firewall 109
gcp vpcflow 104
gcp_pubsub generic 42
github audit 45
google_workspace admin 190
google_workspace drive 122
google_workspace groups 104
google_workspace login 101
google_workspace saml 99
google_workspace user_accounts 93
haproxy info 96
haproxy log 112
haproxy stat 126
hashicorp_vault audit 83
hashicorp_vault log 16
hashicorp_vault metrics 289
hid_bravura_monitor log 180
hid_bravura_monitor winlog 185
http_endpoint generic 12
httpjson generic 10
iis access 88
iis application_pool 52
iis error 68
iis webserver 88
iis website 54
imperva securesphere 801
infoblox nios 801
iptables log 116
journald log 64
juniper junos 830
juniper netscreen 830
juniper srx 807
juniper_junos log 830
juniper_netscreen log 830
juniper_srx log 807
kafka broker 66
kafka consumergroup 57
kafka log 52
kafka partition 59
keycloak log 96
kibana audit 14
kibana log 23
kibana stats 57
kibana status 16
kubernetes apiserver 81
kubernetes audit_logs 73
kubernetes container 86
kubernetes container_logs 71
kubernetes controllermanager 84
kubernetes event 71
kubernetes node 74
kubernetes pod 79
kubernetes proxy 64
kubernetes scheduler 86
kubernetes state_container 73
kubernetes state_cronjob 61
kubernetes state_daemonset 57
kubernetes state_deployment 57
kubernetes state_job 67
kubernetes state_node 64
kubernetes state_persistentvolume 56
kubernetes state_persistentvolumeclaim 58
kubernetes state_pod 67
kubernetes state_replicaset 57
kubernetes state_resourcequota 57
kubernetes state_service 62
kubernetes state_statefulset 58
kubernetes state_storageclass 57
kubernetes system 61
kubernetes volume 60
linux conntrack 47
linux entropy 41
linux iostat 53
linux ksm 46
linux memory 55
linux network_summary 44
linux pageinfo 73
linux raid 50
linux service 75
linux socket 57
linux users 52
log log 33
logstash log 11
logstash node 47
logstash node_stats 69
logstash slowlog 14
mattermost audit 96
microsoft defender_atp 104
microsoft dhcp 830
microsoft_defender_endpoint log 104
microsoft_dhcp log 40
microsoft_sqlserver audit 112
mimecast audit_events 73
mimecast dlp_logs 54
mimecast siem_logs 114
mimecast threat_intel_malware_customer 62
mimecast threat_intel_malware_grid 62
mimecast ttp_ap_logs 66
mimecast ttp_ip_logs 67
mimecast ttp_url_logs 71
modsecurity auditlog 82
mongodb collstats 60
mongodb dbstats 54
mongodb log 47
mongodb metrics 158
mongodb replstatus 72
mongodb status 204
mysql error 51
mysql galera_status 79
mysql performance 16
mysql slowlog 88
mysql status 119
mysql_enterprise audit 91
nats connection 20
nats connections 12
nats log 31
nats route 20
nats routes 12
nats stats 27
nats subscriptions 19
netflow log 1747
netscout sightline 801
netskope alerts 470
netskope events 365
network_traffic amqp 122
network_traffic cassandra 154
network_traffic dhcpv4 116
network_traffic dns 120
network_traffic flow 83
network_traffic http 105
network_traffic icmp 90
network_traffic memcached 125
network_traffic mongodb 94
network_traffic mysql 90
network_traffic nfs 100
network_traffic pgsql 88
network_traffic redis 85
network_traffic sip 153
network_traffic thrift 87
network_traffic tls 179
nginx access 78
nginx error 47
nginx stubstatus 49
nginx_ingress_controller access 80
nginx_ingress_controller error 46
o365 audit 214
okta system 160
oracle database_audit 79
osquery result 233
osquery_manager result 1560
panw panos 228
panw_cortex_xdr alerts 189
pfsense log 235
postgresql activity 57
postgresql bgwriter 52
postgresql database 59
postgresql log 78
postgresql statement 62
prometheus collector 44
prometheus query 40
prometheus remote_write 44
proofpoint emailsecurity 801
pulse_connect_secure log 90
qnap_nas log 51
rabbitmq connection 57
rabbitmq exchange 49
rabbitmq log 41
rabbitmq node 76
rabbitmq queue 57
radware defensepro 801
redis info 151
redis key 43
redis keyspace 42
redis log 43
redis slowlog 41
santa log 68
snort log 113
snyk audit 37
snyk vulnerabilities 71
sonicwall firewall 805
sophos utm 801
sophos xg 363
squid log 816
stan channels 17
stan log 31
stan stats 18
stan subscriptions 18
suricata eve 373
symantec_endpoint log 222
synthetics browser 278
synthetics browser_network 271
synthetics browser_screenshot 259
synthetics http 260
synthetics icmp 251
synthetics tcp 251
system application 173
system auth 98
system core 64
system cpu 76
system diskio 74
system filesystem 45
system fsstat 52
system load 54
system memory 78
system network 74
system process 191
system process_summary 65
system security 342
system socket_summary 74
system syslog 64
system system 181
system uptime 36
tcp generic 12
tenable_sc asset 72
tenable_sc plugin 91
tenable_sc vulnerability 113
ti_abusech malware 65
ti_abusech malwarebazaar 83
ti_abusech url 72
ti_anomali limo 77
ti_anomali threatstream 93
ti_cybersixgill threat 79
ti_misp threat 136
ti_otx threat 75
ti_recordedfuture threat 78
ti_threatq threat 81
tomcat log 812
traefik access 62
traefik health 14
udp generic 12
vsphere datastore 33
vsphere host 35
vsphere log 46
vsphere virtualmachine 40
windows forwarded 453
windows perfmon 38
windows powershell 235
windows powershell_operational 235
windows service 45
windows sysmon_operational 295
winlog winlog 255
zeek capture_loss 54
zeek connection 101
zeek dce_rpc 88
zeek dhcp 83
zeek dnp3 87
zeek dns 121
zeek dpd 83
zeek files 86
zeek ftp 107
zeek http 125
zeek intel 92
zeek irc 96
zeek kerberos 120
zeek modbus 86
zeek mysql 88
zeek notice 108
zeek ntlm 93
zeek ntp 105
zeek ocsp 61
zeek pe 67
zeek radius 93
zeek rdp 99
zeek rfb 91
zeek signature 100
zeek sip 105
zeek smb_cmd 101
zeek smb_files 101
zeek smb_mapping 86
zeek smtp 105
zeek snmp 90
zeek socks 92
zeek ssh 94
zeek ssl 149
zeek stats 73
zeek syslog 84
zeek traceroute 76
zeek tunnel 81
zeek weird 83
zeek x509 106
zerofox alerts 97
zookeeper connection 44
zookeeper mntr 57
zookeeper server 52
zoom webhook 231
zscaler zia 811
zscaler_zia alerts 48
zscaler_zia dns 77
zscaler_zia firewall 75
zscaler_zia tunnel 77
zscaler_zia web 97
zscaler_zpa app_connector_status 81
zscaler_zpa audit 70
zscaler_zpa browser_access 93
zscaler_zpa user_activity 96
zscaler_zpa user_status 68
@andrewkroh
Copy link
Author

Generated with:

go install github.com/andrewkroh/go-examples/fields-yml@main

echo "Integration,Data Stream,Field Count" > field_count.csv

for pkg in $(ls); do for ds in $(ls $pkg/data_stream); do echo -n "$pkg, $ds,"; fields-yml ${pkg}/data_stream/${ds}/fields/* | wc -l; done ; done | sed 's/ //g' >> field_count.csv

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment