Skip to content

Instantly share code, notes, and snippets.

View Alex-Ikanow's full-sized avatar

Alex Alex-Ikanow

10 followers · 0 following
View GitHub Profile
@Alex-Ikanow
Alex-Ikanow / gist:3040974
Created July 3, 2012 16:52
Example of complex scripting in Infinit.e
{
"description": "test",
"extractType": "File",
"file": {
"XmlIgnoreValues": [
"DefiningCharacteristicList",
"TargetedCharacteristicList",
"WeaponTypeList",
"PerpetratorList",
"VictimList",
@Alex-Ikanow
Alex-Ikanow / gist:4040414
Created November 8, 2012 17:58
Query resulting in MongoDB hang
QUERY:
{
"$or" : [ { "searchCycle_secs" : { "$exists" : false}} , { "$where" : "(null == this.harvest) || (null == this.harvest.harvested) || (null == this.searchCycle_secs) || ((this.harvest.harvested.getTime() + 1000*this.searchCycle_secs) <= 1352397384946)"}] ,
"isApproved" : true , "harvestBadSource" : { "$ne" : true} , "extractType" : { "$in" : [ "Feed" , "feed"]}
}
FIELDS:
@Alex-Ikanow
Alex-Ikanow / gist:4979336
Created February 18, 2013 18:08
Collection stats
{
"sharded" : true,
"ns" : "doc_metadata.metadata",
"count" : 3003302,
"numExtents" : 42,
"size" : 27260679308,
"storageSize" : 28334976896,
"totalIndexSize" : 9223868864,
"indexSizes" : {
"_id_" : 92773072,
@Alex-Ikanow
Alex-Ikanow / gist:4979375
Created February 18, 2013 18:14
Example document containing "philip morris/company" entity
{
"_id" : ObjectId("4f996119e4b053cd3218c2fd"),
"associations" : [
{
"entity1" : "clay wanta",
"entity1_index" : "clay wanta/person",
"verb" : "current",
"verb_category" : "career",
"entity2" : "police detective",
"entity2_index" : "police detective/position",
@Alex-Ikanow
Alex-Ikanow / gist:5008868
Last active December 14, 2015 01:49
Chunk migration errors after adding second shard
--------------------------------
SHARD1 LOGS:
Soon after adding the second shard:
Mon Feb 18 16:10:31 [conn68] moveChunk updating self version to: 2|1||000000000000000000000000 through { index: "000007bdc6f2a4f19f36149b08cac55" } -> { index: "01d8d8f3fcac4efa684888aa46d9008f" } for collection 'feature.association'
---
Mon Feb 18 16:11:18 [conn29881] assertion 13388 [feature.association] shard version not ok in Client::Context: version mismatch detected for feature.association, stored major version 2 does not match received 1 ( ns : feature.association, received : 1|84||000000000000000000000000, wanted : 2|0||000000000000000000000000, send ) ( ns : feature.association, received : 1|84||000000000000000000000000, wanted : 2|0||000000000000000000000000, send ) ns:feature.$cmd query:{ findandmodify: "association", query: { index: "14be3fa342209dacb92cfa69073829e0", communityId: ObjectId('4c927585d591d31d7b37097a')}, fields: { doccount: 1, entity1: 1, entity2: 1, verb: 1, db_sync_time: 1, db_sync_doccount: 1 },
@Alex-Ikanow
Alex-Ikanow / gist:5445445
Last active December 16, 2015 13:59
Kundera / MongoDB issues (Kundera 2.2.1)
/////////////////////////////////////////////
//
// 1] persistence.xml (in classpath - when I move it a get a different set of errors)
<persistence xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd" version="2.0">
<persistence-unit name="mongoPU">
<provider>com.impetus.kundera.KunderaPersistence</provider>
<properties>
<property name="kundera.client.lookup.class" value="com.impetus.client.mongodb.MongoDBClientFactory"/>
<property name="kundera.nodes" value="localhost" />
@Alex-Ikanow
Alex-Ikanow / gist:5445543
Last active December 16, 2015 13:59
Map kunderaMongoConfig = new HashMap();
kunderaMongoConfig.put("kundera.nodes", "testnode.ikanow.com");
kunderaMongoConfig.put("kundera.port", "27017");
EntityManagerFactory factory = Persistence.createEntityManagerFactory("mongoPU", kunderaMongoConfig);
// (Also tried creating a KunderaPersistence, and also instantiating EntityManagerFactoryImpl directly)
@Alex-Ikanow
Alex-Ikanow / Kundera_dynamic_class example.java
Last active December 16, 2015 15:28
Sample code showing how to persist a dynamically created (actually modified) class to MongoDB using Kundera (updated to demonstrate JPQL queries, which requires slight hack for classes not discovered by persistence.xml)
// 1] persistence.xml, this can be anywhere in the classpath
//<persistence xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd" version="2.0">
//<persistence-unit name="mongoPU">
// <provider>com.impetus.kundera.KunderaPersistence</provider>
// <class>com.ikanow.infinit.e.harvest.test.NoSqlHarvestTest$TemplateClass</class>
// <properties>
// <property name="kundera.client.lookup.class" value="com.impetus.client.mongodb.MongoDBClientFactory"/>
// <property name="kundera.port" value="27017" />
// <property name="kundera.dialect" value="mongodb" />
@Alex-Ikanow
Alex-Ikanow / Kundera_true_dynamic_class_handling.java
Created April 25, 2013 19:11
A more complicated version that supports multiple dynamic classes (you can only create one class per declared template because once they're loaded they can't be modified)
package com.ikanow.infinit.e.harvest.test;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.EntityManager;
@Alex-Ikanow
Alex-Ikanow / gist:5577084
Created May 14, 2013 15:56
Modified version of brace-fold.js, handles arrays
CodeMirror.braceRangeFinder = function(cm, start) {
var line = start.line, lineText = cm.getLine(line);
var at = lineText.length, startChar, tokenType;
for (; at > 0;) {
var found = lineText.lastIndexOf("{", at);
var startToken = '{';
var endToken = '}';
if (found < start.ch) {
var found = lineText.lastIndexOf("[", at);
if (found < start.ch) break;