Skip to content

Instantly share code, notes, and snippets.

-- invoke with two arguments, the input file , and the output file. -input /bps/gen -output /bps/analytics
-- FYI...
-- If you run into errors, you can see them in
-- ./target/failsafe-reports/TEST-org.bigtop.bigpetstore.integration.BigPetStorePigIT.xml
-- First , we load data in from a file, as tuples.
-- in pig, relations like tables in a relational database
-- so each relation is just a bunch of tuples.
-- in this case csvdata will be a relation,
package org.apache.hadoop.fs.glusterfs;
import java.net.URI;
import java.io.File;
import java.io.RandomAccessFile;
import java.io.IOException;
import java.nio.channels.FileLock;
@mattf
mattf / gettrustedglusterfspathinfo.c
Created February 11, 2013 13:49
Tool to get the trusted.glusterfs.pathinfo extended attribute from a file
/*
* Allow retrieval of ONLY the trusted.glusterfs.pathinfo extended
* attribute on a file.
*/
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
@mattf
mattf / wallaby_accounting_group_porcelain.txt
Created November 14, 2012 20:12
wallaby accounting group porcelain
This is porcelain for managing an AccountingGroups feature.
The AccountingGroups feature should be independently added to nodes
potentially running a condor_negotiator.
A group: Name (str), Static (int), Dynamic (float), Surplus (tri)
A name: may have multiple dots (.), e.g. A, A.B, B, C, C.D,
representing a Parent.Child relationship
Static and Dynamic are mutually exclusive
Surplus is Yes/No/Default
@mattf
mattf / gist:3911872
Created October 18, 2012 13:40
7.6 -> 7.8 schedd stats
WINDOWED_STAT_WIDTH STATISTICS_WINDOW_SECONDS // quantized to schedd_stats_window_quantum = 200
WindowedStatWidth RecentStatsLifetime
JobsSubmitted RecentJobsSubmitted
JobsSubmittedCumulative JobsSubmitted
JobsStarted RecentJobsStarted
JobsStartedCumulative JobsStarted
@mattf
mattf / setuid-eagain.c
Created July 26, 2012 21:35
Demonstrate poor setuid error checking
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>
int
main(int argc, char *argv[])
{
if (argc < 2) return 1;
@mattf
mattf / mef_plugin.cpp
Created July 26, 2012 20:57
Condor File Transfer plugin (basically here-data://)
#include <string.h>
#include <iostream>
#include <fstream>
/**
* This is an example file transfer plugin. It takes a uri of
* mef://blob/filename & a file path and writes blob the file.
*
* There are two valid invocations:
* 0) argv = {_, "-classad"}
@mattf
mattf / condor_ec2_q.sh
Created December 28, 2011 18:38
EC2 details from condor_q
#!/bin/sh
# NOTE:
# . Requires condor_q >= 7.5.2, old classads do not
# have %
# . When running, jobs show RUN_TIME of their current
# run, not accumulated, which would require adding
# in RemoteWallClockTime
# . See condor_utils/condor_q.cpp:encode_status for
# JobStatus map
@mattf
mattf / condor_ec2_link.sh
Created December 28, 2011 18:37
Import EC2 instance into Schedd queue
@mattf
mattf / memcached.job
Created December 28, 2011 18:34
memcached managed from Condor
cmd = memcached.sh
args = -m $$(Memory)
log = memcached.log
kill_sig = SIGTERM
# Want chirp functionality
+WantIOProxy = TRUE