Skip to content

Instantly share code, notes, and snippets.

View ioscott's full-sized avatar

ioscott

  • import.io
  • London
View GitHub Profile
1. Create Application run Configuration on MainGrizzly with Environment variable CLUSTER_MODE=bees
2. Ensure private sqs queue created - set the queue in to default.conf
3. Start the application
4. Hit the url like `curl -v -XPOST "http://localhost:5000/bees/run/extractor/0b1edc0f-9562-45c4-b9f1-b797dcbb4841"`
Setup tunnel to redis instance then access via local. lstaging is local ssh configuration to staging environment.
`ssh lstaging -L 9595:bees-staging.xfnopy.ng.0001.use1.cache.amazonaws.com:6379`
`redis-cli -p 9595`
@ioscott
ioscott / Change.md
Last active April 8, 2016 11:59
RuntimeConfiguraiton - add ExtractorId

Run the following DDL against the DB:

alter table RuntimeConfiguration add column field_extractorId BINARY(16);

Staging: Copy the migration-scripts/schema-migrations/ONE-683/update-es.sh to the gateway box. scp ONE-683/{update-es.sh,00000000-0000-0000-0000-000000000000.json,RuntimeConfiguration.json} gateway:

Run the script like follows

@ioscott
ioscott / LightningCrawler.md
Last active March 30, 2016 12:17
Lightning Batch Processor examples.

The following are created using Ian's ID so that the Extractor he trained can be used.

If not logged in, run the following from the command line.

curl -c da-user-cookies.txt -XPOST -d "username=your-name&password=your-password" "https://api.staging-owl.com/auth/login"

The Extractor is only able to keep track of one run at a time. If multiple people are following these instructions then they will interfere with each other, it is not a bug. The UI will prevent this however curl doesn't care.