Skip to content

Instantly share code, notes, and snippets.

@chatman

chatman/Logs

Created June 16, 2020 14:38
Show Gist options
  • Save chatman/c25a5ca373b56a9dc8552de3fe8a0617 to your computer and use it in GitHub Desktop.
Save chatman/c25a5ca373b56a9dc8552de3fe8a0617 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Logs
ishan@instance-4:~/solr-bench$ unzip -l logs-2020-16-06_14.35.26.zip
Archive: logs-2020-16-06_14.35.26.zip
Length Date Time Name
--------- ---------- ----- ----
133120 2020-06-16 14:35 solrlogs-solr-1.tar
40960 2020-06-16 14:35 solrlogs-solr-2.tar
--------- -------
174080 2 files
Raw
output.log
ishan@instance-4:~/solr-bench$ ./start.sh config.json
Downloading gs://solr-wikipedia-benchmarks/solr-custom.tgz
Copying gs://solr-wikipedia-benchmarks/solr-custom.tgz...
\ [1 files][178.1 MiB/178.1 MiB]
Operation completed over 1 objects/178.1 MiB.
--2020-06-16 14:30:05-- http://enos.itcollege.ee/~jpoial/allalaadimised/jdk8/jdk-8u241-linux-x64.tar.gz
Resolving enos.itcollege.ee (enos.itcollege.ee)... 193.40.231.204, 2001:bb8:4006:fff:193:40:231:204
Connecting to enos.itcollege.ee (enos.itcollege.ee)|193.40.231.204|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 194545143 (186M) [application/x-gzip]
Saving to: ‘jdk-8u241-linux-x64.tar.gz’
jdk-8u241-linux-x64.tar.gz 100%[=========================================================================================================================================>] 185.53M 922KB/s in 3m 17s
2020-06-16 14:33:22 (964 KB/s) - ‘jdk-8u241-linux-x64.tar.gz’ saved [194545143/194545143]
--2020-06-16 14:33:22-- https://archive.apache.org/dist/zookeeper/zookeeper-3.5.6/apache-zookeeper-3.5.6-bin.tar.gz
Resolving archive.apache.org (archive.apache.org)... 138.201.131.134, 2a01:4f8:172:2ec5::2
Connecting to archive.apache.org (archive.apache.org)|138.201.131.134|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9230052 (8.8M) [application/x-gzip]
Saving to: ‘apache-zookeeper-3.5.6-bin.tar.gz’
apache-zookeeper-3.5.6-bin.tar.gz 100%[=========================================================================================================================================>] 8.80M 7.41MB/s in 1.2s
2020-06-16 14:33:24 (7.41 MB/s) - ‘apache-zookeeper-3.5.6-bin.tar.gz’ saved [9230052/9230052]
Downloading gs://solr-wikipedia-benchmarks/queryFile
Copying gs://solr-wikipedia-benchmarks/queryFile...
/ [1 files][ 1.0 KiB/ 1.0 KiB]
Operation completed over 1 objects/1.0 KiB.
Downloading gs://solr-wikipedia-benchmarks/small-enwiki.tsv.gz
Copying gs://solr-wikipedia-benchmarks/small-enwiki.tsv.gz...
- [1 files][ 35.7 MiB/ 35.7 MiB]
Operation completed over 1 objects/35.7 MiB.
Using Terraform provisioner
rm: cannot remove 'terraform/id_rsa*': No such file or directory
Generating public/private rsa key pair.
Your identification has been saved in terraform/id_rsa.
Your public key has been saved in terraform/id_rsa.pub.
The key fingerprint is:
SHA256:f7lliEC4GrFkDNPw1j2sMVG2LipWaF25FqByRiy2DE0 ishan@instance-4
The key's randomart image is:
+---[RSA 2048]----+
| +Eo. ..o |
|ooo*.o B . |
|=.= B O * |
| * * + X . |
| o = * S |
| . . = . o . o |
| o o o + o |
| . . . + |
| . |
+----[SHA256]-----+
Initializing modules...
- solr_node in compute
- zookeeper in compute
Initializing the backend...
Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "google" (hashicorp/google) 3.20.0...
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
module.zookeeper.google_compute_instance.instance[0]: Creating...
module.zookeeper.google_compute_instance.instance[0]: Still creating... [10s elapsed]
module.zookeeper.google_compute_instance.instance[0]: Creation complete after 18s [id=projects/bold-gadget-222718/zones/us-central1-a/instances/zookeeper-1]
module.solr_node.google_compute_instance.instance[0]: Creating...
module.solr_node.google_compute_instance.instance[1]: Creating...
module.solr_node.google_compute_instance.instance[0]: Still creating... [10s elapsed]
module.solr_node.google_compute_instance.instance[1]: Still creating... [10s elapsed]
module.solr_node.google_compute_instance.instance[1]: Creation complete after 17s [id=projects/bold-gadget-222718/zones/us-central1-a/instances/solr-2]
module.solr_node.google_compute_instance.instance[0]: Creation complete after 17s [id=projects/bold-gadget-222718/zones/us-central1-a/instances/solr-1]
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
Outputs:
solr_node_details = [
{
"allow_stopping_for_update" = true
"attached_disk" = []
"boot_disk" = [
{
"auto_delete" = true
"device_name" = "solr-device"
"disk_encryption_key_raw" = ""
"disk_encryption_key_sha256" = ""
"initialize_params" = [
{
"image" = "https://www.googleapis.com/compute/v1/projects/centos-cloud/global/images/centos-8-v20191210"
"labels" = {}
"size" = 10
"type" = "pd-ssd"
},
]
"kms_key_self_link" = ""
"mode" = "READ_WRITE"
"source" = "https://www.googleapis.com/compute/v1/projects/bold-gadget-222718/zones/us-central1-a/disks/solr-1"
},
]
"can_ip_forward" = false
"cpu_platform" = "Intel Haswell"
"current_status" = "RUNNING"
"deletion_protection" = false
"description" = "Solr node - benchmarking"
"desired_status" = "RUNNING"
"enable_display" = false
"guest_accelerator" = []
"hostname" = ""
"id" = "projects/bold-gadget-222718/zones/us-central1-a/instances/solr-1"
"instance_id" = "4540944520485103266"
"label_fingerprint" = "D_kp_8lVJkE="
"labels" = {
"automation" = "true"
"product" = "solr"
"purpose" = "benchmarking"
}
"machine_type" = "n1-standard-4"
"metadata" = {
"ssh-keys" = "solruser:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDD02AK+PYnh7L02TWqN5qzJAV0Z/xdjtiH1yANPfJgu8Tj6PgSuXRziL51gaWLkZVQesTuvDc7uFf26bwPOgoKUmZErB2EAfN1AeSwKk+U5mbWfnqdKHi8D3SN7pXK8PBaF4+PN0l9n37vVLRPCKXE1RVl4Pzcs3JPrNZsSG91BRYkez04ZlyjQ9e0pL9G9CYasKOKF0+KktXOCM4brRIJB+dRc+UJBJl6ntQYcPJsKzykgZ+YBm+Ku4DQgqA7pfvel8AekzpefPMK4rLi5hBIBC1ojbnP0ZOmFRAp/dCSr3DcWJfbtEq6Gj08Gpk7LGDTlkKlmcmF/pnC3N4N1gHl ishan@instance-4\n"
}
"metadata_fingerprint" = "FZpOHJbiKWk="
"metadata_startup_script" = ""
"min_cpu_platform" = ""
"name" = "solr-1"
"network_interface" = [
{
"access_config" = [
{
"nat_ip" = "34.72.82.125"
"network_tier" = "PREMIUM"
"public_ptr_domain_name" = ""
},
]
"alias_ip_range" = []
"name" = "nic0"
"network" = "https://www.googleapis.com/compute/v1/projects/bold-gadget-222718/global/networks/default"
"network_ip" = "10.128.0.53"
"subnetwork" = "https://www.googleapis.com/compute/v1/projects/bold-gadget-222718/regions/us-central1/subnetworks/default"
"subnetwork_project" = "bold-gadget-222718"
},
]
"project" = "bold-gadget-222718"
"scheduling" = [
{
"automatic_restart" = true
"node_affinities" = []
"on_host_maintenance" = "MIGRATE"
"preemptible" = false
},
]
"scratch_disk" = []
"self_link" = "https://www.googleapis.com/compute/v1/projects/bold-gadget-222718/zones/us-central1-a/instances/solr-1"
"service_account" = []
"shielded_instance_config" = []
"tags" = [
"benchmarking",
"solr",
"temp",
]
"tags_fingerprint" = "hJw6ln0JFuM="
"zone" = "us-central1-a"
},
{
"allow_stopping_for_update" = true
"attached_disk" = []
"boot_disk" = [
{
"auto_delete" = true
"device_name" = "solr-device"
"disk_encryption_key_raw" = ""
"disk_encryption_key_sha256" = ""
"initialize_params" = [
{
"image" = "https://www.googleapis.com/compute/v1/projects/centos-cloud/global/images/centos-8-v20191210"
"labels" = {}
"size" = 10
"type" = "pd-ssd"
},
]
"kms_key_self_link" = ""
"mode" = "READ_WRITE"
"source" = "https://www.googleapis.com/compute/v1/projects/bold-gadget-222718/zones/us-central1-a/disks/solr-2"
},
]
"can_ip_forward" = false
"cpu_platform" = "Intel Haswell"
"current_status" = "RUNNING"
"deletion_protection" = false
"description" = "Solr node - benchmarking"
"desired_status" = "RUNNING"
"enable_display" = false
"guest_accelerator" = []
"hostname" = ""
"id" = "projects/bold-gadget-222718/zones/us-central1-a/instances/solr-2"
"instance_id" = "6606075207126181538"
"label_fingerprint" = "D_kp_8lVJkE="
"labels" = {
"automation" = "true"
"product" = "solr"
"purpose" = "benchmarking"
}
"machine_type" = "n1-standard-4"
"metadata" = {
"ssh-keys" = "solruser:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDD02AK+PYnh7L02TWqN5qzJAV0Z/xdjtiH1yANPfJgu8Tj6PgSuXRziL51gaWLkZVQesTuvDc7uFf26bwPOgoKUmZErB2EAfN1AeSwKk+U5mbWfnqdKHi8D3SN7pXK8PBaF4+PN0l9n37vVLRPCKXE1RVl4Pzcs3JPrNZsSG91BRYkez04ZlyjQ9e0pL9G9CYasKOKF0+KktXOCM4brRIJB+dRc+UJBJl6ntQYcPJsKzykgZ+YBm+Ku4DQgqA7pfvel8AekzpefPMK4rLi5hBIBC1ojbnP0ZOmFRAp/dCSr3DcWJfbtEq6Gj08Gpk7LGDTlkKlmcmF/pnC3N4N1gHl ishan@instance-4\n"
}
"metadata_fingerprint" = "FZpOHJbiKWk="
"metadata_startup_script" = ""
"min_cpu_platform" = ""
"name" = "solr-2"
"network_interface" = [
{
"access_config" = [
{
"nat_ip" = "34.72.76.15"
"network_tier" = "PREMIUM"
"public_ptr_domain_name" = ""
},
]
"alias_ip_range" = []
"name" = "nic0"
"network" = "https://www.googleapis.com/compute/v1/projects/bold-gadget-222718/global/networks/default"
"network_ip" = "10.128.15.235"
"subnetwork" = "https://www.googleapis.com/compute/v1/projects/bold-gadget-222718/regions/us-central1/subnetworks/default"
"subnetwork_project" = "bold-gadget-222718"
},
]
"project" = "bold-gadget-222718"
"scheduling" = [
{
"automatic_restart" = true
"node_affinities" = []
"on_host_maintenance" = "MIGRATE"
"preemptible" = false
},
]
"scratch_disk" = []
"self_link" = "https://www.googleapis.com/compute/v1/projects/bold-gadget-222718/zones/us-central1-a/instances/solr-2"
"service_account" = []
"shielded_instance_config" = []
"tags" = [
"benchmarking",
"solr",
"temp",
]
"tags_fingerprint" = "hJw6ln0JFuM="
"zone" = "us-central1-a"
},
]
zookeeper_details = [
{
"allow_stopping_for_update" = true
"attached_disk" = []
"boot_disk" = [
{
"auto_delete" = true
"device_name" = "zookeeper-device"
"disk_encryption_key_raw" = ""
"disk_encryption_key_sha256" = ""
"initialize_params" = [
{
"image" = "https://www.googleapis.com/compute/v1/projects/centos-cloud/global/images/centos-8-v20191210"
"labels" = {}
"size" = 10
"type" = "pd-ssd"
},
]
"kms_key_self_link" = ""
"mode" = "READ_WRITE"
"source" = "https://www.googleapis.com/compute/v1/projects/bold-gadget-222718/zones/us-central1-a/disks/zookeeper-1"
},
]
"can_ip_forward" = false
"cpu_platform" = "Intel Haswell"
"current_status" = "RUNNING"
"deletion_protection" = false
"description" = "Zookeeper node - benchmarking"
"desired_status" = "RUNNING"
"enable_display" = false
"guest_accelerator" = []
"hostname" = ""
"id" = "projects/bold-gadget-222718/zones/us-central1-a/instances/zookeeper-1"
"instance_id" = "6056394525101090516"
"label_fingerprint" = "D_kp_8lVJkE="
"labels" = {
"automation" = "true"
"product" = "solr"
"purpose" = "benchmarking"
}
"machine_type" = "n1-standard-2"
"metadata" = {
"ssh-keys" = "solruser:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDD02AK+PYnh7L02TWqN5qzJAV0Z/xdjtiH1yANPfJgu8Tj6PgSuXRziL51gaWLkZVQesTuvDc7uFf26bwPOgoKUmZErB2EAfN1AeSwKk+U5mbWfnqdKHi8D3SN7pXK8PBaF4+PN0l9n37vVLRPCKXE1RVl4Pzcs3JPrNZsSG91BRYkez04ZlyjQ9e0pL9G9CYasKOKF0+KktXOCM4brRIJB+dRc+UJBJl6ntQYcPJsKzykgZ+YBm+Ku4DQgqA7pfvel8AekzpefPMK4rLi5hBIBC1ojbnP0ZOmFRAp/dCSr3DcWJfbtEq6Gj08Gpk7LGDTlkKlmcmF/pnC3N4N1gHl ishan@instance-4\n"
}
"metadata_fingerprint" = "FZpOHJbiKWk="
"metadata_startup_script" = ""
"min_cpu_platform" = ""
"name" = "zookeeper-1"
"network_interface" = [
{
"access_config" = [
{
"nat_ip" = "35.238.191.59"
"network_tier" = "PREMIUM"
"public_ptr_domain_name" = ""
},
]
"alias_ip_range" = []
"name" = "nic0"
"network" = "https://www.googleapis.com/compute/v1/projects/bold-gadget-222718/global/networks/default"
"network_ip" = "10.128.0.48"
"subnetwork" = "https://www.googleapis.com/compute/v1/projects/bold-gadget-222718/regions/us-central1/subnetworks/default"
"subnetwork_project" = "bold-gadget-222718"
},
]
"project" = "bold-gadget-222718"
"scheduling" = [
{
"automatic_restart" = true
"node_affinities" = []
"on_host_maintenance" = "MIGRATE"
"preemptible" = false
},
]
"scratch_disk" = []
"self_link" = "https://www.googleapis.com/compute/v1/projects/bold-gadget-222718/zones/us-central1-a/instances/zookeeper-1"
"service_account" = []
"shielded_instance_config" = []
"tags" = [
"benchmarking",
"solr",
"temp",
]
"tags_fingerprint" = "hJw6ln0JFuM="
"zone" = "us-central1-a"
},
]
wait-for-it.sh: waiting for zookeeper-1:22 without a timeout
wait-for-it.sh: zookeeper-1:22 is available after 0 seconds
Starting Zookeeper on zookeeper-1
Warning: Permanently added 'zookeeper-1,10.128.0.48' (ECDSA) to the list of known hosts.
14:34:07 up 0 min, 0 users, load average: 1.58, 0.40, 0.13
apache-zookeeper-3.5.6-bin.tar.gz 100% 9014KB 102.5MB/s 00:00
jdk-8u241-linux-x64.tar.gz 100% 186MB 100.4MB/s 00:01
ZooKeeper JMX enabled by default
Using config: /home/solruser/apache-zookeeper-3.5.6-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
Starting Solr on solr-1
Starting Solr from solr-custom.tgz in solr-custom.tgz...
wait-for-it.sh: waiting for solr-1:22 without a timeout
wait-for-it.sh: solr-1:22 is available after 0 seconds
wait-for-it.sh: waiting for zookeeper-1:2181 without a timeout
wait-for-it.sh: zookeeper-1:2181 is available after 0 seconds
Warning: Permanently added 'solr-1,10.128.0.53' (ECDSA) to the list of known hosts.
14:34:21 up 0 min, 0 users, load average: 1.34, 0.32, 0.11
solr-custom.tgz 100% 178MB 98.7MB/s 00:01
lsof 100% 171KB 37.8MB/s 00:00
jdk-8u241-linux-x64.tar.gz 100% 186MB 102.4MB/s 00:01
*** [WARN] *** Your open file limit is currently 1024.
It should be set to 65000 to avoid operational disruption.
If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to false in your profile or solr.in.sh
*** [WARN] *** Your Max Processes Limit is currently 59363.
It should be set to 65000 to avoid operational disruption.
If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to false in your profile or solr.in.sh
Using Solr root directory: /home/solruser/solr-8.3.0
Using Java: /home/solruser/jdk1.8.0_241/bin/java
java version "1.8.0_241"
Java(TM) SE Runtime Environment (build 1.8.0_241-b07)
Java HotSpot(TM) 64-Bit Server VM (build 25.241-b07, mixed mode)
Starting Solr using the following settings:
JAVA = /home/solruser/jdk1.8.0_241/bin/java
SOLR_SERVER_DIR = /home/solruser/solr-8.3.0/server
SOLR_HOME = /home/solruser/solr-8.3.0/server/solr
SOLR_HOST =
SOLR_PORT = 8983
STOP_PORT = 7983
JAVA_MEM_OPTS = -Xms1g -Xmx1g
GC_TUNE = -XX:+UseG1GC -XX:+PerfDisableSharedMem -XX:+ParallelRefProcEnabled -XX:MaxGCPauseMillis=250 -XX:+UseLargePages -XX:+AlwaysPreTouch
GC_LOG_OPTS = -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/home/solruser/solr-8.3.0/server/logs/solr_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=9 -XX:GCLogFileSize=20M
SOLR_TIMEZONE = UTC
CLOUD_MODE_OPTS = -DzkClientTimeout=15000 -DzkHost=zookeeper-1:2181
SOLR_OPTS = -Dsolr.host=solr-1 -Xss256k -Dsolr.jetty.https.port=8983
Waiting up to 180 seconds to see Solr running on port 8983 [\]
Started Solr server on port 8983 (pid=6460). Happy searching!
Starting Solr on solr-2
Starting Solr from solr-custom.tgz in solr-custom.tgz...
wait-for-it.sh: waiting for solr-2:22 without a timeout
wait-for-it.sh: solr-2:22 is available after 0 seconds
wait-for-it.sh: waiting for zookeeper-1:2181 without a timeout
wait-for-it.sh: zookeeper-1:2181 is available after 0 seconds
Warning: Permanently added 'solr-2,10.128.15.235' (ECDSA) to the list of known hosts.
14:34:44 up 0 min, 0 users, load average: 0.73, 0.24, 0.08
solr-custom.tgz 100% 178MB 98.6MB/s 00:01
lsof 100% 171KB 41.2MB/s 00:00
jdk-8u241-linux-x64.tar.gz 100% 186MB 95.6MB/s 00:01
*** [WARN] *** Your open file limit is currently 1024.
It should be set to 65000 to avoid operational disruption.
If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to false in your profile or solr.in.sh
*** [WARN] *** Your Max Processes Limit is currently 59363.
It should be set to 65000 to avoid operational disruption.
If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to false in your profile or solr.in.sh
Using Solr root directory: /home/solruser/solr-8.3.0
Using Java: /home/solruser/jdk1.8.0_241/bin/java
java version "1.8.0_241"
Java(TM) SE Runtime Environment (build 1.8.0_241-b07)
Java HotSpot(TM) 64-Bit Server VM (build 25.241-b07, mixed mode)
Starting Solr using the following settings:
JAVA = /home/solruser/jdk1.8.0_241/bin/java
SOLR_SERVER_DIR = /home/solruser/solr-8.3.0/server
SOLR_HOME = /home/solruser/solr-8.3.0/server/solr
SOLR_HOST =
SOLR_PORT = 8983
STOP_PORT = 7983
JAVA_MEM_OPTS = -Xms1g -Xmx1g
GC_TUNE = -XX:+UseG1GC -XX:+PerfDisableSharedMem -XX:+ParallelRefProcEnabled -XX:MaxGCPauseMillis=250 -XX:+UseLargePages -XX:+AlwaysPreTouch
GC_LOG_OPTS = -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/home/solruser/solr-8.3.0/server/logs/solr_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=9 -XX:GCLogFileSize=20M
SOLR_TIMEZONE = UTC
CLOUD_MODE_OPTS = -DzkClientTimeout=15000 -DzkHost=zookeeper-1:2181
SOLR_OPTS = -Dsolr.host=solr-2 -Xss256k -Dsolr.jetty.https.port=8983
Waiting up to 180 seconds to see Solr running on port 8983 [\]
Started Solr server on port 8983 (pid=6469). Happy searching!
Running suite from working directory: /home/ishan/solr-bench
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
ERROR StatusLogger No Log4j 2 configuration file found. Using default configuration (logging only errors to the console), or user programmatically provided configurations. Set system property 'log4j2.debug' to show Log4j 2 internal initialization logging. See https://logging.apache.org/log4j/2.x/manual/configuration.html for instructions on how to configure Log4j 2
646 [main] INFO org.apache.solr.benchmarks.solrcloud.SolrCloud - Provisioning method: terraform-gcp
Solr nodes: [solr-1, solr-2]
ZK node: zookeeper-1
675 [main] INFO org.apache.solr.benchmarks.BenchmarksMain - SolrCloud initialized...
677 [main] INFO org.apache.solr.benchmarks.BenchmarksMain - Starting indexing benchmarks...
678 [main] INFO org.apache.solr.benchmarks.BenchmarksMain - Creating collection: small-wikipedia
1069 [main] WARN org.apache.solr.benchmarks.BenchmarksMain - Error trying to delete collection: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://solr-1:8983/solr: Could not find collection : small-wikipedia
5098 [main] INFO org.apache.solr.benchmarks.solrcloud.SolrCloud -
5100 [main] INFO org.apache.solr.benchmarks.solrcloud.SolrCloud - Collection created: {
"responseHeader":{
"status":0,
"QTime":4021},
"success":{"10.128.0.53:8983_solr":{
"responseHeader":{
"status":0,
"QTime":3064},
"core":"small-wikipedia_shard1_replica_n1"}},
"warning":"Using _default configset. Data driven schema functionality is enabled by default, which is NOT RECOMMENDED for production use. To turn it off: curl http://{host:port}/solr/small-wikipedia/config -d '{\"set-user-property\": {\"update.autoCreateFields\":\"false\"}}'"}
[doctitle, docdate, body]
21709 [main] INFO org.apache.solr.benchmarks.BenchmarksMain - Starting querying benchmarks...
Total queries: 42
exiting,time over at 1592318126423 time elapsed : 423total tasks : 21, benchmarked queries: 16
Took time: 425
Thread: 1, Median latency: 3.3673064999999998, 95th latency: 3.770454
Final results: {indexing-benchmarks={CLOUD_INDEXING={cloud_2x2=[{threads=2, total-time=16.562568591}]}}, query-benchmarks={Terms queries=[{threads=1, 50th=3.3673064999999998, 90th=3.6406306, 95th=3.770454, mean=3.3663038125, total-queries=16}]}, solr-metrics={solr-2:8983={jvm/solr.jvm/memory.heap.used=[9.175068E7, 9.5944984E7, 9.699356E7, 9.699356E7, 9.699356E7, 9.699356E7, 9.699356E7, 9.699356E7, 9.699356E7, 9.699356E7, 9.699356E7], jvm/solr.jvm/os.systemCpuLoad=[0.05673758865248227, 0.008567931456548349, 0.002484472049689441, 0.0037220843672456576, 0.0024630541871921183, 0.004962779156327543, 0.0024752475247524753, 0.0037267080745341614, 0.0024271844660194173, 0.0012453300124533001, 0.0024875621890547263]}, solr-1:8983={jvm/solr.jvm/memory.heap.used=[1.22159384E8, 1.46276632E8, 6.29540752E8, 1.49029176E8, 6.01489768E8, 1.70300064E8, 5.20666224E8, 2.8995012E8, 2.92084392E8, 3.5753892E8, 8.27301256E8], jvm/solr.jvm/os.systemCpuLoad=[0.007959782153330541, 0.08352668213457076, 0.701363073110285, 0.8308457711442786, 0.9950433705080545, 0.6263871763255241, 0.8525402726146221, 0.8447204968944099, 0.8034610630407911, 0.8768292682926829, 0.5143212951432129]}}, configuration={configuration=org.apache.solr.benchmarks.beans.Configuration@1b535c0d}}
Pulling logs
solrlogs-solr-1.tar 100% 130KB 72.8MB/s 00:00
adding: solrlogs-solr-1.tar (deflated 86%)
solrlogs-solr-2.tar 100% 40KB 37.6MB/s 00:00
updating: solrlogs-solr-1.tar (deflated 86%)
adding: solrlogs-solr-2.tar (deflated 85%)
Removing the hostname entry from ~/.ssh/known_hosts, so that another run can be possible afterwards
# Host solr-1 found: line 5
/home/ishan/.ssh/known_hosts updated.
Original contents retained as /home/ishan/.ssh/known_hosts.old
# Host solr-2 found: line 6
/home/ishan/.ssh/known_hosts updated.
Original contents retained as /home/ishan/.ssh/known_hosts.old
Host "zookeeper-1" not found in /home/ishan/.ssh/known_hosts
Copying file://results-2020-16-06_14.35.26.json [Content-Type=application/json]...
/ [1 files][ 2.8 KiB/ 2.8 KiB]
Operation completed over 1 objects/2.8 KiB.
Copying file://logs-2020-16-06_14.35.26.zip [Content-Type=application/zip]...
/ [1 files][ 24.1 KiB/ 24.1 KiB]
Operation completed over 1 objects/24.1 KiB.
module.zookeeper.google_compute_instance.instance[0]: Refreshing state... [id=projects/bold-gadget-222718/zones/us-central1-a/instances/zookeeper-1]
module.solr_node.google_compute_instance.instance[1]: Refreshing state... [id=projects/bold-gadget-222718/zones/us-central1-a/instances/solr-2]
module.solr_node.google_compute_instance.instance[0]: Refreshing state... [id=projects/bold-gadget-222718/zones/us-central1-a/instances/solr-1]
module.solr_node.google_compute_instance.instance[1]: Destroying... [id=projects/bold-gadget-222718/zones/us-central1-a/instances/solr-2]
module.solr_node.google_compute_instance.instance[0]: Destroying... [id=projects/bold-gadget-222718/zones/us-central1-a/instances/solr-1]
module.solr_node.google_compute_instance.instance[1]: Still destroying... [id=projects/bold-gadget-222718/zones/us-central1-a/instances/solr-2, 10s elapsed]
module.solr_node.google_compute_instance.instance[0]: Still destroying... [id=projects/bold-gadget-222718/zones/us-central1-a/instances/solr-1, 10s elapsed]
module.solr_node.google_compute_instance.instance[1]: Still destroying... [id=projects/bold-gadget-222718/zones/us-central1-a/instances/solr-2, 20s elapsed]
module.solr_node.google_compute_instance.instance[0]: Still destroying... [id=projects/bold-gadget-222718/zones/us-central1-a/instances/solr-1, 20s elapsed]
module.solr_node.google_compute_instance.instance[0]: Destruction complete after 21s
module.solr_node.google_compute_instance.instance[1]: Destruction complete after 21s
module.zookeeper.google_compute_instance.instance[0]: Destroying... [id=projects/bold-gadget-222718/zones/us-central1-a/instances/zookeeper-1]
module.zookeeper.google_compute_instance.instance[0]: Still destroying... [id=projects/bold-gadget-222718/zones/us-central1-a/instances/zookeeper-1, 10s elapsed]
module.zookeeper.google_compute_instance.instance[0]: Still destroying... [id=projects/bold-gadget-222718/zones/us-central1-a/instances/zookeeper-1, 20s elapsed]
module.zookeeper.google_compute_instance.instance[0]: Destruction complete after 21s
Destroy complete! Resources: 3 destroyed.
Raw
results.json
ishan@instance-4:~/solr-bench$ cat results-2020-16-06_14.35.26.json
{"indexing-benchmarks":{"CLOUD_INDEXING":{"cloud_2x2":[{"threads":2,"total-time":"16.562568591"}]}},"query-benchmarks":{"Terms queries":[{"threads":1,"50th":3.3673064999999998,"90th":3.6406306,"95th":3.770454,"mean":3.3663038125,"total-queries":16}]},"solr-metrics":{"solr-2:8983":{"jvm/solr.jvm/memory.heap.used":[9.175068E7,9.5944984E7,9.699356E7,9.699356E7,9.699356E7,9.699356E7,9.699356E7,9.699356E7,9.699356E7,9.699356E7,9.699356E7],"jvm/solr.jvm/os.systemCpuLoad":[0.05673758865248227,0.008567931456548349,0.002484472049689441,0.0037220843672456576,0.0024630541871921183,0.004962779156327543,0.0024752475247524753,0.0037267080745341614,0.0024271844660194173,0.0012453300124533001,0.0024875621890547263]},"solr-1:8983":{"jvm/solr.jvm/memory.heap.used":[1.22159384E8,1.46276632E8,6.29540752E8,1.49029176E8,6.01489768E8,1.70300064E8,5.20666224E8,2.8995012E8,2.92084392E8,3.5753892E8,8.27301256E8],"jvm/solr.jvm/os.systemCpuLoad":[0.007959782153330541,0.08352668213457076,0.701363073110285,0.8308457711442786,0.9950433705080545,0.6263871763255241,0.8525402726146221,0.8447204968944099,0.8034610630407911,0.8768292682926829,0.5143212951432129]}},"configuration":{"configuration":{"index-benchmarks":[{"name":"CLOUD_INDEXING","description":"Wikipedia dataset on SolrCloud","replication-type":"cloud","dataset-file":"small-enwiki.tsv.gz","file-format":"tsv","setups":[{"setup-name":"cloud_2x2","collection":"small-wikipedia","configset":null,"replication-factor":1,"shards":1,"min-threads":2,"max-threads":2,"thread-step":2}],"duration-secs":null,"run-count":null,"max-docs":2147483647,"rpm":null,"batch-size":1000,"id-field":"id"}],"query-benchmarks":[{"rpm":3000,"name":"Terms queries","description":null,"collection":"small-wikipedia","query-file":"queryFile","json-query":false,"uri":"/select","shuffle":true,"client-type":null,"min-threads":1,"max-threads":1,"duration-secs":null,"total-count":21,"warm-count":5,"template-values":null}],"cluster":{"num-solr-nodes":1,"zk-instance-config":null,"solr-instance-config":null,"ssh-flags":null,"jdk-url":"http://enos.itcollege.ee/~jpoial/allalaadimised/jdk8/jdk-8u241-linux-x64.tar.gz","jdk-tarball":"jdk-8u241-linux-x64.tar.gz","jdk-directory":"jdk1.8.0_241","startup-params":"-m 1g","terraform-gcp-config":{"project_id":"bold-gadget-222718","zookeeper_machine_type":"n1-standard-2","zookeeper_disk_type":"pd-ssd","solr_node_count":2,"solr_machine_type":"n1-standard-4","solr_disk_type":"pd-ssd","tags":["benchmarking","solr","temp"],"public_key_path":"id_rsa.pub","user":"solruser"},"provisioning-method":"terraform-gcp"},"repository":null,"metrics":["jvm/solr.jvm/memory.heap.used","jvm/solr.jvm/os.systemCpuLoad"],"solr-package":"gs://solr-wikipedia-benchmarks/solr-custom.tgz","pre-download":["gs://solr-wikipedia-benchmarks/queryFile","gs://solr-wikipedia-benchmarks/small-enwiki.tsv.gz"],"results-upload-location":"gs://solr-wikipedia-benchmarks/"}}}ishan@instance-4:~/solr-bench$
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment