Skip to content

Instantly share code, notes, and snippets.

View abemusic's full-sized avatar

Abe Music abemusic

View GitHub Profile
@abemusic
abemusic / 01-without-sentry-properties
Created August 20, 2014 15:54
Various stuff related to my attempts to get Hive working with Sentry
hive> create database foo;
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: org.apache.hadoop.security.AccessControlException Permission denied: user=stackdio, access=WRITE, inode="/user/hive/warehouse":hive:hive:drwxrwx--x
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkFsPermission(FSPermissionChecker.java:271)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:257)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:238)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:179)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5584)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5566)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:5540)
at org.apache.had
salt-call --versions-report
Salt: 2014.1.4
Python: 2.6.6 (r266:84292, Jul 10 2013, 22:48:45)
Jinja2: 2.2.1
M2Crypto: 0.20.2
msgpack-python: 0.1.13
msgpack-pure: Not Installed
pycrypto: 2.0.1
PyYAML: 3.10
PyZMQ: 2.2.0.1
foo-dn-100:
- {error: 'The function "state.sls" is running as PID 16463 and was started at 2014,
May 15 19:14:03.733974 with jid 20140515191403733974'}
foo-dn-143:
- {error: 'The function "state.sls" is running as PID 16910 and was started at 2014,
May 15 19:14:03.733974 with jid 20140515191403733974'}
foo-dn-17:
- {error: 'The function "state.sls" is running as PID 17276 and was started at 2014,
May 15 19:14:03.733974 with jid 20140515191403733974'}
foo-dn-185:
[2014-05-15 11:56:34,333: CRITICAL/MainProcess] Task stacks.orchestrate[65bdf379-0086-40d0-8e20-112d05730fbc] INTERNAL ERROR: DatabaseError(1153, "Got a packet bigger than 'max_allowed_packet' bytes")
Traceback (most recent call last):
File "/home/stackdio/.virtualenvs/stackdio/lib/python2.6/site-packages/celery/task/trace.py", line 247, in trace_task
R = I.handle_error_state(task, eager=eager)
File "/home/stackdio/.virtualenvs/stackdio/lib/python2.6/site-packages/celery/task/trace.py", line 108, in handle_error_state
}[self.state](task, store_errors=store_errors)
File "/home/stackdio/.virtualenvs/stackdio/lib/python2.6/site-packages/celery/task/trace.py", line 138, in handle_failure
task.backend.mark_as_failure(req.id, exc, einfo.traceback)
File "/home/stackdio/.virtualenvs/stackdio/lib/python2.6/site-packages/celery/backends/base.py", line 119, in mark_as_failure
traceback=traceback)
'20140514162935419088':
Arguments:
- cdh4.hbase.regionserver
- test.user1
Function: state.sls
Returned:
- foo-dn-19
- foo-dn-13
- foo-dn-17
- jid
@abemusic
abemusic / overstate.sls
Created May 13, 2014 15:39
Overstate file used by salt-run state.overstate
group_0:
match: G@stack_id:20 and G@roles:qatp.base and G@roles:qatp.ipa and G@roles:java.oracle.jdk7
sls:
- qatp.base
- qatp.ipa
- java.oracle.jdk7
group_1:
match: G@stack_id:20 and G@roles:cdh4.hadoop.namenode
sls:
- cdh4.hadoop.namenode

Keybase proof

I hereby claim:

  • I am abemusic on github.
  • I am abemusic (https://keybase.io/abemusic) on keybase.
  • I have a public key whose fingerprint is 5247 0540 50DF 7B87 2D16 938E 7795 D1DA A198 1F80

To claim this, I am signing this object:

@abemusic
abemusic / gist:10903470
Created April 16, 2014 16:31
salt-master auth log
2014-04-16 16:31:31,196 [salt.master ][INFO ] Clear payload received with command _auth
2014-04-16 16:31:31,246 [salt.master ][INFO ] Authentication request from scalerun-382-dn-83
2014-04-16 16:31:31,249 [salt.master ][INFO ] Authentication accepted from scalerun-382-dn-83
2014-04-16 16:31:31,261 [salt.master ][INFO ] Clear payload received with command _auth
2014-04-16 16:31:31,329 [salt.master ][INFO ] Authentication request from scalerun-382-dn-106
2014-04-16 16:31:31,329 [salt.master ][INFO ] Authentication accepted from scalerun-382-dn-106
2014-04-16 16:31:31,423 [salt.master ][INFO ] Clear payload received with command _auth
2014-04-16 16:31:31,424 [salt.master ][INFO ] Authentication request from scalerun-382-dn-25
2014-04-16 16:31:31,424 [salt.master ][INFO ] Authentication accepted from scalerun-382-dn-25
@abemusic
abemusic / gist:10758293
Created April 15, 2014 18:47
Blueprint schema validation
BLUEPRINT_SCHEMA = {
'$schema': 'http://json-schema.org/draft-04/schema#',
'title': 'Blueprint',
'description': '',
'type': 'object',
'required': ['title'],
'properties': {
'title': {
# load the default settings
from django.conf import settings
settings.configure()
from django.db.utils import load_backend
from dj_database_url import parse
def try_dsn(dsn):
try: