lawlesst

September 25, 2015

Steps to run ldf-server on Heroku with an HDT backend.

• Read the Node troubleshooting guide from Heroku for background.
• Run the commands in the Start with a Blank Slate section of the guide.
• Add package.json to your git repository if it isn't already.
• For my project, I had to explicitly add hdt - 1.2.2 as a dependendency.

package.json

lawlesst

petl

Python ETL

• Documentation: https://petl.readthedocs.org/en/latest/index.html
• Source: https://github.com/alimanfoo/petl
• Mailing list: http://groups.google.com/group/python-etl

Many in the VIVO community use Python for data manipulation and transformation tasks (ETL). petl is a framework for reading data from various sources, transforming it, and passing it along to another source.

lawlesst

Sample files for pulling data into a VIVO instance from Linked Data Fragments, Wikidata, and Wikipedia.

lawlesst

"""

Used chardet library to find encoding - it reported:
{'confidence': 0.7158930206047512, 'encoding': 'ISO-8859-2'}

Use line writing to print ntriples as they are read to avoid
memory issues with storing all statements in an in memory 
RDFLib Graph.

"""

lawlesst

@prefix :        <http://localhost/jena_example/#> .
@prefix rdf:     <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs:    <http://www.w3.org/2000/01/rdf-schema#> .
@prefix tdb:     <http://jena.hpl.hp.com/2008/tdb#> .
@prefix ja:      <http://jena.hpl.hp.com/2005/11/Assembler#> .
@prefix text:    <http://jena.apache.org/text#> .
@prefix fuseki:  <http://jena.apache.org/fuseki#> .

## Example of a TDB dataset and text index
## Initialize TDB

lawlesst

##University Linking

Using OrgRef to link universities represented in VIVOs to DBPedia and to each other.

• Exploring how VIVO instances could share data like this and reuse URIs.
• Crawled eight VIVO instances and pulled back data about Universities.
• Matched each site to OrgRef using simple string normalization and equality.
• Matched OrgRef to DBPedia using the Wikipedia Page IDs included by OrgRef.
• Matched VIVOs to DBPedia
• Loaded data into a Linked Data Fragments server: http://ldf-vivo.herokuapp.com/orgref

lawlesst

"""

Example code for connecting to Stardog (http://stardog.com/) with
Python's RDFLib (http://github.com/rdflib).

See longer description: http://lawlesst.github.io/notebook/rdflib-stardog.html

"""

lawlesst

#!/bin/sh -e

set -x # Print commands and their arguments as they are executed.

JETTY_HOME=/dock/jetty7.6.14
VITRO_HOME=/tmp/vitrohome
STAGE_DIR=/tmp/vbuild

ant clean distribute -Dskiptests=true
#ant distribute -Dskiptests=true

lawlesst

from shibboleth.middleware import ShibbolethRemoteUserMiddleware, ShibbolethValidationError

class MyAppAuthenticationMiddleware(ShibbolethRemoteUserMiddleware):
    """
    Handle login via Shibboleth and permissions for viewing and
    editing profiles and resources.
    """
    header = "Shibboleth-localId"

lawlesst

# Set up a reader and a writer
# First we need to require the reader/writer we want

require 'traject'
require 'traject/marc4j_reader'
require 'traject/debug_writer'

settings do
    provide "reader_class_name", "Traject::Marc4JReader"
    provide "marc_source.encoding", "UTF-8"