Skip to content

Instantly share code, notes, and snippets.

View doubleirish's full-sized avatar

Conor Redmond doubleirish

14 followers · 2 following
  • Seattle Washington, United States
View GitHub Profile
@doubleirish
doubleirish / aggregateScrapesFromWebCrawl.sh
Created February 1, 2017 17:57
bash script to automatically Log in to target page , extracts all relative urls in target page, crawl through each child page , scrape a html table and aggregate it
export OS_USERNAME="yourUser"
export OS_PASSWORD="yourPass"
# logs into protected page (your formdata params may be different), finds relative urls, converts them to full urls and stores them in a local file
wget -O- --post-data "os_username=${OS_USERNAME}&os_password=${OS_PASSWORD}" https://www.example.com/display/IT/Release+Management \
| grep href \
| sed -n 's/.*href="\([^"]*\).*/https\:\/\/www\.example\.com\1/p' \
| sort | uniq > urls.txt
# iterates through file of urls and aggrregates screen scrapes of selected xpath elements in each url into a single summary files
echo "<html><head><title>summary</title></head><body>" >summary.html;
@doubleirish
doubleirish / deploy_springboot_to_cloud_foundry.md
Last active August 5, 2016 23:11
deploy a spring boot app to a cloud foundry host implemented by pivotal web services

To download CF CLI as binary

you can download the CF CLI from https://github.com/cloudfoundry/cli#downloads or use the curl command below

cd ~/bin
curl -L "https://cli.run.pivotal.io/stable?release=linux64-binary&amp;source=github" | tar -zx
@doubleirish
doubleirish / kafka_wireshark.md
Created August 4, 2016 16:41
Using Wireshark to sniff kafka message packets on ubuntu

#Using wireshark to sniff kafka messages

Install the latest WireShark application

 sudo add-apt-repository ppa:wireshark-dev/stable
 sudo apt-get update
 sudo apt-get install wireshark
@doubleirish
doubleirish / gist:713001237292fff96079
Created May 19, 2015 23:50
No Hits on my L2 Cache ?
@RunWith(SpringJUnit4ClassRunner.class)
@TransactionConfiguration(transactionManager = "txManager", defaultRollback = true)
...
public class BookDaoTest extends AbstractTransactionalJUnit4SpringContextTests {
@Test public void testBookEntityCaching() {
bookDao.findById(3);
bookDao.findById(3);
Statistics stats = bookDao.getHibernateStatistics();
String regionName = "edu.uw.data.model.Book";
SecondLevelCacheStatistics level2BookEntityStats =