rushirajnenuji

{
	"report_header": {
		"report_name": "Dataset Master Report",
		"report_id": "DSR1",
		"release": "RD1",
		"report-filters": [
			  {
				"Name": "Begin-Date",
				"Value": "2018-02-01"
			  },

rushirajnenuji

{
    "status": "ok",
    "message-type": "work",
    "message-version": "1.0.0",
    "message": {
        "indexed": {
            "date-parts": [
                [
                    2018,
                    5,

rushirajnenuji

{
    "LinkPublicationDate": "2017-05-31T03:01:02Z",
    "LinkProvider": [
        {
            "Name": "crossref"
        }
    ],
    "RelationshipType": {
        "Name": "References"
    },

rushirajnenuji

Setup and Operation of Elasticsearch Event Index

Log events travel a long path to get into the elasticsearch index:

1. MN or CN event
2. Log aggregation process running on a CN collects event
3. Log aggregation processes event, augmenting and pushing into solr
4. Python script copies events from solr index to log files on disk
5. Filebeat watches log files, sends entries to logstash

rushirajnenuji

Setup and Operation of Elasticsearch Event Index

Architecture of the Metrics Service

[url=https://ibb.co/nO8tLz][img]https://preview.ibb.co/mqXL0z/mdc_log_processing_architecture.png[/img][/url]

Log events travel a long path to get into the elasticsearch index:

rushirajnenuji

{
  "took": 58,
  "timed_out": false,
  "_shards": {
    "total": 2,
    "successful": 2,
    "skipped": 0,
    "failed": 0
  },
  "hits": {

rushirajnenuji

"""
Metrics Analyzer module

Implemented as a falcon web application, https://falcon.readthedocs.io/en/stable/


"""
import json
import falcon
from urllib.parse import urlparse

rushirajnenuji

{
  "report-header": {
    "report-name": "Dataset Master Report",
    "report-id": "dsr",
    "release": "rd1",
    "reporting-period": {
      "begin-date": "2013-11-01",
      "end-date": "2013-12-01"
    },
    "created": "2018-11-02",

rushirajnenuji

{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 2,
    "successful": 2,
    "skipped": 0,
    "failed": 0
  },
  "hits": {

rushirajnenuji

List of datasets with increasing counts as the version progresses

Start
1. 
- (V4) https://handy-owl.nceas.ucsb.edu/metacatui/view/doi:10.5063/F1MW2FC2
- (V3) https://handy-owl.nceas.ucsb.edu/metacatui/view/urn:uuid:e27ba7ef-2758-4a07-9bc5-419dac6f61d7
- (V2) https://handy-owl.nceas.ucsb.edu/metacatui/view/doi:10.5063/F1C53J27
- (V1) https://handy-owl.nceas.ucsb.edu/metacatui/view/urn:uuid:c93cac3c-789e-4c44-9052-5c95e416d5e1