Skip to content

Instantly share code, notes, and snippets.

View vthacker's full-sized avatar

Varun Thacker vthacker

  • Slack
  • San Francisco
View GitHub Profile
@vthacker
vthacker / StructuredConcurrencyTest
Last active October 8, 2023 01:59
Fan out queries and aggregate results. Some queries can take too long and some might throw exception.
class StructuredConcurrencyTest {
public static void main(String[] args) {
Test test = new Test();
test.query();
}
}
class Test {
public void query() {
try (CollectingScope<String> scope = new CollectingScope<>()) {
public static void main(String args[]) {
Jep338FullMaskVectorOperations test = new Jep338FullMaskVectorOperations();
float[] a = new float[]{1.0f, 2.0f, 2.0f, 1.0f, 1.0f, 1.0f, 1.0f};
float[] b = new float[]{3.0f, 4.0f, 8.0f, 9.9f, 1.0f, 1.0f, 1.0f};
// this returns NaN
// That's because "int bound = species.loopBound(v1.length);" returns 0 when length <=7 on my macbook pro
// which means the for loop is never executed.
// I think in this example we'd need to change the for loop to be v1.length?
System.out.println(test.cosineSimilarity(a,b));
}
// Method that overrides the gRPC
@Override
public void search(
SearchRequest request,
StreamObserver<SearchResult> responseObserver) {
doSearch(request)
.whenCompleteAsync(
(result, t) -> {
{
"@timestamp": {
"$resolver": "timestamp",
"pattern": {
"format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
"timeZone": "UTC"
}
},
"level": {
"$resolver": "level",
@vthacker
vthacker / gist:5874727
Created June 27, 2013 07:59
Example Lucene Indexing and Searching
/** Indexing part **/
Map<String,Analyzer> analyzerPerField = new HashMap<String,Analyzer>();
analyzerPerField.put("state", new StandardAnalyzer(Version.LUCENE_50));
analyzerPerField.put("region", new StandardAnalyzer(Version.LUCENE_50));
PerFieldAnalyzerWrapper aWrapper = new PerFieldAnalyzerWrapper(new StandardAnalyzer(Version.LUCENE_50), analyzerPerField); //This is the part where after basic setup enhancements should be made
IndexWriter iw;
Directory dir = new MMapDirectory(new File("/tmp/lucene"));
IndexWriterConfig iwConfig = new IndexWriterConfig(Version.LUCENE_50, aWrapper);
[junit4] 2> 1954119 WARN (zkConnectionManagerCallback-3743-thread-1-EventThread) [ ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
[junit4] 2> 1954119 INFO (zkConnectionManagerCallback-3743-thread-1-EventThread) [ ] o.a.s.c.c.ConnectionManager zkClient Connected:false
[junit4] 2> 1954120 INFO (zkConnectionManagerCallback-3743-thread-1-EventThread) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1954120 INFO (zkConnectionManagerCallback-3743-thread-1-EventThread) [ ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
[junit4] 2> 1954120 INFO (zkConnectionManagerCallback-3743-thread-1-EventThread) [ ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
[junit4] 2> 1954124 INFO (zkConnectionManagerCallback-3743-thread-1-EventThread) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader dummy.host.com:8984_solr
[junit4] 2> 1954125 INFO
2017-12-14 17:42:40.079 INFO (updateExecutor-2-thread-88620-processing-n:host05.domain:8985_solr) [c:collection_name s:shard4_3 r:core_node86 x:collection_name_shard4_3_replica2] o.a.s.u.DefaultSolrCoreState Running recovery
2017-12-14 17:42:40.083 INFO (recoveryExecutor-3-thread-11-processing-n:host05.domain:8985_solr x:collection_name_shard4_3_replica2 s:shard4_3 c:collection_name r:core_node86) [c:collection_name s:shard4_3 r:core_node86 x:collection_name_shard4_3_replica2] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=false
2017-12-14 17:44:20.269 INFO (recoveryExecutor-3-thread-11-processing-n:host05.domain:8985_solr x:collection_name_shard4_3_replica2 s:shard4_3 c:collection_name r:core_node86) [c:collection_name s:shard4_3 r:core_node86 x:collection_name_shard4_3_replica2] o.a.s.u.PeerSync PeerSync: core=collection_name_shard4_3_replica2 url=https://host05.domain:8985/solr Received 199819 versions from https://host04.domain:8983/solr/collection_name_shard4_3_replica1/
INFO - 2017-11-14 23:17:27.309; [ ] org.apache.solr.handler.admin.CollectionsHandler; Invoked Collection Action :splitshard with params action=splitshard&shard=shard1_1&collection=test_split and sendToOCPQueue=true
DEBUG - 2017-11-14 23:17:27.336; [ ] org.apache.solr.cloud.OverseerCollectionMessageHandler; OverseerCollectionMessageHandler.processMessage : splitshard , {
"collection":"test_split",
"shard":"shard1_1",
"operation":"splitshard"}
INFO - 2017-11-14 23:17:27.336; [ ] org.apache.solr.cloud.SplitShardCmd; Split shard invoked
DEBUG - 2017-11-14 23:17:27.336; [ ] org.apache.solr.common.cloud.ZkStateReader; Refreshing lazily-loaded state for collection test_split
INFO - 2017-11-14 23:17:27.359; [ ] org.apache.solr.cloud.SplitShardCmd; Creating slice shard1_1_0 of collection test_split on 192.168.10.101:8984_solr
DEBUG - 2017-11-14 23:17:27.382; [ ] org.apache.solr.cloud.OverseerCollectionMessageHandler; Waiting for slice shard1_1_0 of collection test_split to be available
DEBUG - 201
@vthacker
vthacker / gist:6535332
Created September 12, 2013 10:08
Perculator
package com.varun.perculator;
import java.util.ArrayList;
import java.util.List;
import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.core.SimpleAnalyzer;
import org.apache.lucene.index.Term;
import org.apache.lucene.index.memory.MemoryIndex;
import org.apache.lucene.queryparser.classic.ParseException;
@vthacker
vthacker / gist:9270623
Created February 28, 2014 12:59
JSON Block Join Example
{
"add": [
{
"id": "1",
"title": "Solr adds block join support",
"content_type": "parentDocument",
"_childDocuments_": [
{
"id": "2",
"comments": "SolrCloud supports it too!"