Skip to content

Instantly share code, notes, and snippets.

View ianrumford's full-sized avatar

Ian Rumford ianrumford

9 followers · 0 following
View GitHub Profile
@ianrumford
ianrumford / print_file.clj
Created September 28, 2012 12:59
Cascalog print auditd log file
(ns aud_cas.print_file
(:use cascalog.api))
(defn print-file
"Use cascalog to print a file"
[file-path]
(let [file-tap (lfs-textline file-path)]
(?<- (stdout) [?line] (file-tap :> ?line))))
(defn -main
@ianrumford
ianrumford / project.clj
Created September 28, 2012 15:50
Cascalog project file
(defproject aud-cas "0.1.0-SNAPSHOT"
:description "Using cascalog, cascading and clojure for auditd log ETL"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "1.4.0"]
[cascalog "1.10.0"]
;;[cascalog "1.9.0"]
[org.clojure/data.json "0.1.2"]
[clj-time "0.4.3"]
@ianrumford
ianrumford / blog_audit1.log
Created September 29, 2012 13:29
Cascalog auditd log file
node=cdh4flumevm1 type=DAEMON_START msg=audit(1342114506.467:9723): auditd start, ver=1.7.18 format=raw kernel=3.2.0-26-generic auid=4294967295 pid=1054 subj=unconfined res=success
node=cdh4flumevm1 type=CONFIG_CHANGE msg=audit(1342114506.571:24): audit_backlog_limit=8192 old=64 auid=4294967295 ses=4294967295 res=1
node=cdh4flumevm1 type=CONFIG_CHANGE msg=audit(1342114506.571:25): audit_failure=2 old=1 auid=4294967295 ses=4294967295 res=1
node=cdh4flumevm1 type=CONFIG_CHANGE msg=audit(1342114506.579:105): audit_enabled=1 old=1 auid=4294967295 ses=4294967295 res=1
node=cdh4flumevm1 type=LOGIN msg=audit(1342114506.751:106): login pid=1104 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=1
node=cdh4flumevm1 type=LOGIN msg=audit(1342114517.503:107): login pid=1447 uid=0 old auid=4294967295 new auid=1000 old ses=4294967295 new ses=2
node=cdh4flumevm1 type=SYSCALL msg=audit(1342114517.511:108): arch=c000003e syscall=87 success=no exit=-2 a0=e273d0 a1=0 a2=e22620 a3=7ffffcd967e0 items=1 ppid=1447 p
@ianrumford
ianrumford / *scratch*.el
Created September 29, 2012 13:35
Cascalog print auditd log fields output
DAEMON_START cdh4flumevm1 audit(1342114506.467:9723)
CONFIG_CHANGE cdh4flumevm1 audit(1342114506.571:24)
CONFIG_CHANGE cdh4flumevm1 audit(1342114506.571:25)
CONFIG_CHANGE cdh4flumevm1 audit(1342114506.579:105)
LOGIN cdh4flumevm1 audit(1342114506.751:106)
LOGIN cdh4flumevm1 audit(1342114517.503:107)
SYSCALL cdh4flumevm1 audit(1342114517.511:108)
CWD cdh4flumevm1 audit(1342114517.511:108)
PATH cdh4flumevm1 audit(1342114517.511:108)
SYSCALL cdh4flumevm1 audit(1342114517.547:109)
@ianrumford
ianrumford / print_fields.clj
Created September 29, 2012 16:17
Cascalog print auditd log fields
(ns aud_cas.print_fields
(:use cascalog.api)
(:require [clojure.string :as str])
)
(defn parse_input_record
"Parse the text of the input record into fields in a map"
[input_record]
(let [prefix_string (get (str/split input_record #"\: ") 0)
prefix_pairs (str/split prefix_string #" ")
@ianrumford
ianrumford / save_fields.clj
Created September 29, 2012 16:36
Cascalog save auditd log fields
(ns aud_cas.save_fields
(:use cascalog.api)
(:require (cascalog [workflow :as w])
[clojure.string :as str])
(:import [java.util UUID ]
[org.apache.hadoop.hbase.util Bytes]
[com.twitter.maple.hbase HBaseTap HBaseScheme]))
(defmapop dmo-uuid [& any] [(.toString (UUID/randomUUID))]) ;; custom operation to generate UUIDs
@ianrumford
ianrumford / filter_fields.clj
Created September 29, 2012 18:01
Cascalog filter auditd log fields
(ns aud_cas.filter_fields
(:use cascalog.api)
(:require (cascalog [workflow :as w])
[clojure.string :as str])
(:import [java.util UUID ]
[org.apache.hadoop.hbase.util Bytes]
[com.twitter.maple.hbase HBaseTap HBaseScheme]))
(defmapop dmo-uuid [& any] [(.toString (UUID/randomUUID))]) ;; custom operation to generate UUIDs
@ianrumford
ianrumford / project.clj
Created October 10, 2012 13:20
VMFest VBox 4.2 examples project.clj
(defproject vmfest_examples "0.1.0-SNAPSHOT"
:description "VMFest and VirtualBox 4.2 examples"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "1.4.0"]
[org.clojure/tools.logging "0.2.3"]
[local/vboxjws "4.2.0"]
[local/vmfest "0.2.5-vbox4.2.0-SNAPSHOT"]
[ch.qos.logback/logback-classic "1.0.0"]
@ianrumford
ianrumford / blog_vmfest_image0.clj
Created October 12, 2012 12:50
VMFest VBox 4.2 examples blog_vmfest_image0.clj
(ns vmfest_examples.blog_vmfest_image0
(:require [vmfest.manager :as vman]))
;; First connect to the VBox web service.
(def my-server (vman/server "http://localhost:18083" "ian" "<password goes here>"))
;; Name of the machine
(def my-server-name "blog-vmfest-image0")
@ianrumford
ianrumford / blog_vmfest_model0.clj
Created October 12, 2012 13:31
VMFest VBox 4.2 examples blog_vmfest_model0.clj
(ns vmfest_examples.blog_vmfest_model0
(:require [vmfest.manager :as vman]))
;; Models must be kept in ~/.vmfest/models
;; First connect to the VBox web service.
(def my-server (vman/server "http://localhost:18083" "ian" "<password goes here>"))
;; Name of the machine