Last active
September 17, 2018 01:58
-
-
Save joel-bernstein/025789728494e360b71192a0fc97968c to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
plist(parallel(workerCollection, | |
numWorkers="1", | |
sort="year asc, month asc, day asc", | |
plist(facet(week1, zkHost="zk1", q="*:*", buckets="year, month, day", bucketSorts="year asc, month asc, day asc", count(*)), | |
facet(week2, zkHost="zk1", q="*:*", buckets="year, month, day", bucketSorts="year asc, month asc, day asc", count(*)), | |
facet(week3, zkHost="zk1", q="*:*", buckets="year, month, day", bucketSorts="year asc, month asc, day asc", count(*)))), | |
parallel(workerCollection, | |
numWorkers="1", | |
sort="year asc, month asc, day asc", | |
plist(facet(week4, zkHost="zk2", q="*:*", buckets="year, month, day", bucketSorts="year asc, month asc, day asc", count(*)), | |
facet(week5, zkHost="zk2", q="*:*", buckets="year, month, day", bucketSorts="year asc, month asc, day asc", count(*)), | |
facet(week6, zkHost="zk2", q="*:*", buckets="year, month, day", bucketSorts="year asc, month asc, day asc", count(*)))), | |
parallel(workerCollection, | |
numWorkers="1", | |
sort="year asc, month asc, day asc", | |
plist(facet(week7, zkHost="zk3", q="*:*", buckets="year, month, day", bucketSorts="year asc, month asc, day asc", count(*)), | |
facet(week8, zkHost="zk3", q="*:*", buckets="year, month, day", bucketSorts="year asc, month asc, day asc", count(*)), | |
facet(week9, zkHost="zk3", q="*:*", buckets="year, month, day", bucketSorts="year asc, month asc, day asc", count(*))))) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The data is organized in chronological Solr Cloud collections by week. New collections are added every week. Each collection has two shards and two replicas and 200+ million records. Aggregating across 10 years of data would involve:
104,000,000,000 documents
520 collections
2080 Solr Instances