Skip to content

Instantly share code, notes, and snippets.

Brian Tingle tingletech

Block or report user

Report or block tingletech

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View Z.txt
scala> var r = spark.sql("select substring(first(repository_name)[0],0,50) as repository_name, first(campus_name)[0] as campus, count(title[0]) as total, count(distinct(title[0])) as uniq, count(distinct(title[0]))/count(title[0]) as uniq_ratio from calisphere group by repository_url[0] order by uniq_ratio desc, uniq desc")
r: org.apache.spark.sql.DataFrame = [repository_name: string, campus: string ... 3 more fields]
scala> r.show(300,false)
+--------------------------------------------------+----------------+------+------+-------------------+
|repository_name |campus |total |uniq |uniq_ratio |
+--------------------------------------------------+----------------+------+------+-------------------+
|Television Academy Foundation |null |904 |904 |1.0 |
|California State University, Stanislaus. Library |null |260 |260 |1.0 |
|Architecture and Design Collection, Art, Design an|UC S
View fields.py
# solr schema fields that have a `_ss` varient for facets
UCLDC_SCHEMA_FACETS = [
"title",
"alternative_title",
"contributor",
"coverage",
"creator",
"date",
"extent",
"format",
View calisphere_metadata_spark.py
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import argparse
import os
import sys
import json
from urllib.parse import urlparse
import boto3
from pyspark.sql import SparkSession
from icecream import ic
View sort.py
# -*- coding: UTF-8 -*-
import unicodedata
import re
RE_ALPHANUMSPACE = re.compile(r'[^0-9A-Za-z\s]*') # \W include "_" as does A-z
def normalize_sort_field(sort_field,
default_missing='~title unknown',
missing_equivalents=['title unknown']):
View nxid.txt
usage: nxid [-h] [--mint] [--create] [--update] [--no-noop-report]
[--show-erc] [--ezid-username EZID_USERNAME]
[--ezid-password EZID_PASSWORD] [--shoulder SHOULDER]
[--owner OWNER] [--status STATUS] [--publisher PUBLISHER]
[--location LOCATION] [--loglevel LOGLEVEL] [--rcfile RCFILE]
path
nxid finds top level objects in Nuxeo and syncs them up with EZID
positional arguments:
View ucldc_schema.xsd
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:nxs="http://www.nuxeo.org/ecm/project/schemas/tingle-california-digita/ucldc_schema" xmlns:nxsv="http://www.nuxeo.org/ecm/schemas/core/validation/" xmlns:ref="http://www.nuxeo.org/ecm/schemas/core/external-references/" targetNamespace="http://www.nuxeo.org/ecm/project/schemas/tingle-california-digita/ucldc_schema">
<!-- helper XSD definitions for list types -->
<xs:complexType name="content">
<xs:sequence>
<xs:element name="encoding" type="xs:string"/>
<xs:element name="mime-type" type="xs:string"/>
<xs:element name="data" type="xs:base64Binary"/>
<xs:element name="name" type="xs:string"/>
View exception.py
class MyException(Exception):
pass
class MyException2(Exception):
pass
def f1():
View keybase.md

Keybase proof

I hereby claim:

  • I am tingletech on github.
  • I am tingle (https://keybase.io/tingle) on keybase.
  • I have a public key ASBAVyvLjP7prJs_6mYWqtCGbCs12jxzEJwrCXIM43ZhgAo

To claim this, I am signing this object:

View 1-subject.json
{
"responseHeader":{
"status":0,
"QTime":3,
"params":{
"q":"collection_data:*/79/*",
"fl":"subject,url_item",
"rows":"500"}},
"response":{"numFound":427,"start":0,"docs":[
{
@tingletech
tingletech / CHANGES.txt
Created Jan 25, 2017
XTF changes pulled into dsc.cdlib.org
View CHANGES.txt
changeset: 2203:b35bb4856e74
parent: 2174:2a619674504a
user: Martin Haye <martin.haye@ucop.edu>
date: Thu Apr 17 14:50:18 2014 -0700
summary: Adding .gitignore, derived from old .hgignore.
changeset: 2204:ae21e14ae0d0
parent: 2174:2a619674504a
user: BRIAN TINGLE <brian.tingle.cdlib.org@gmail.com>
date: Fri Apr 11 19:11:42 2014 -0700
You can’t perform that action at this time.