Skip to content

Instantly share code, notes, and snippets.

View ibspoof's full-sized avatar

Brad Vernon ibspoof

52 followers · 45 following
View GitHub Profile
@PatrickCallaghan
PatrickCallaghan / SSTables per partition
Last active February 23, 2017 16:54
for f in *Data.db;
do ~/dse/resources/cassandra/tools/bin/sstabledump -e $f >> out.log;
done;
sed $'s/,/\\\n/g' out.log | sort | uniq -c | grep -v '^ *1 ' | sort -nk1 > duplicate_keys.out
OR
for f in *Data.db;
@bradfordcp
bradfordcp / README.md
Last active July 28, 2023 10:10
Setting up Apache Spark to use Apache Shiro for authentication of Spark Master dashboard.

Securing Apache Spark with Apache Shiro

  1. Download shiro-core-1.2.5.jar Apache Shiro Downloads
  2. Download shiro-web-1.2.5.jar Apache Shiro Downloads
  3. Note the location of the JAR files and shiro.ini. I placed it in the root of my Spark download
  4. Update the spark-env.sh file with the Shiro JARs and add an entry for the path where the shiro.ini resides
  5. Start the Spark master sbin/start-master.sh
  6. Navigate to the Spark master dashboard
  7. Authenticate with credentials in shiro.ini

Note this was developed / tested with Apache Spark 1.4.1, but should work with newer versions as well.

@codekoala
codekoala / vault_intermediate_ca_setup.sh
Last active April 21, 2021 03:52 — forked from bhouse/vault_intermediate_ca_setup.sh
Setting Up Hashicorp Vault with an intermediate CA based on https://gist.github.com/jefferai/092d2cd728ff66089f17
#!/bin/bash -e
# Setup a Root CA in vault
# Generate and sign an Intermediate cert
#
# Requires:
# * A running vault server already initialzed and unsealed
# * Environment variable VAULT_TOKEN is set
# * vault cli (https://www.vaultproject.io)
# * httpie (https://github.com/jkbrzt/httpie)
@jeromatron
jeromatron / user-defined-compaction.md
Last active April 12, 2018 13:12
Using jmxsh to compact specfic sstables

Sometimes you'll want to compact specific sstables and you won't want to major compact. You may not have the ability to run jconsole and want to use a command-line tool. There's a ticket (CASSANDRA-10660) to be able to do this with nodetool, but in the meantime, you can use something like jmxsh (or my mirror for when google pulls the plug on google code).

The jmx mbean to do user defined compaction is in the org.apache.cassandra.db package in the CompactionExecutor and the method is called forceUserDefinedCompaction. You'll need to get a list of sstables that you want to compact.

Start jmxsh with java -jar jmxsh-R5.jar -h localhost -p 7199:

jeremy$ java -jar jmxsh-R5.jar -h localhost -p 7199
jmxsh v1.0, Tue Jan 22 10:23:12 CST 2008
@yogidevendra
yogidevendra / WeightedBiasRandomElementGenerator.java
Last active February 22, 2022 17:41
Program to generate pick elements from array random following given weighted distribution
package com.blogspot.yogidevendra.codesnippets;
import java.util.Arrays;
import java.util.Random;
/**
* To generate Random numbers based on weighted bias.
*/
public class WeightedBiasRandomElementGenerator<T>
@mburrows02
mburrows02 / CustomSimulation.scala
Last active April 18, 2023 23:23
Gatling simulation with dynamic scenarios and injection profiles
package test
import io.gatling.core.Predef._
import io.gatling.core.structure.PopulatedScenarioBuilder
import io.gatling.core.controller.inject.InjectionStep
import io.gatling.http.Predef._
import io.gatling.jdbc.Predef._
import scala.concurrent.duration._
import scala.collection.mutable.ArraySeq
import org.json.JSONArray;
import org.json.JSONObject;