bbpennel

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:task="http://www.springframework.org/schema/task"
    xmlns:p="http://www.springframework.org/schema/p"
    xmlns:util="http://www.springframework.org/schema/util"
    xsi:schemaLocation="
    http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
    http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd

bbpennel

// snippet from FedoraTransactionsIT

    @Test
    public void testConcurrentPutsAndCommits() throws Exception {
        final String baseId = getRandomUniqueId();
        final HttpPut putBase = new HttpPut(serverAddress + "/" + baseId);
        try (final CloseableHttpResponse response = execute(putBase)) {
            assertEquals(CREATED.getStatusCode(), getStatus(response));
        } catch (IOException e) {
            e.printStackTrace();

bbpennel

Example structure

• my_book - AG
  • pages - Direct container, defining pcdm:hasFile
    • page1.png - internal binary with user supplied triples
    • page2.png - internal binary, no user triples
  • originals - container, no user triples
    • original_scan.pdf - external binary, no user triples

Persisted structure

File names are path components from fedora identifiers relative to the identifier of the AG.

bbpennel

#!/usr/bin/env bash

USAGE="$(basename "$0") [-h] [-d] [-e] target_dir 

Replaces all non-ASCII characters with another character in filenames
for all files and directories contained within the target directory.

where:
    -h, --help               Show this help text
    -d, --dry-run            Display the expected changes, but do not perform them. If

bbpennel

Example Persistence Layout for FSDB

In this example, there is:

• an object, my_book, which is an AG
  • It has been assigned a fedora id info:fedora/91f6d9c2-dc27-4ab5-b626-df0a45e4519e
• a head version contains a folder containing two binaries
• one previous version, which contained only one binary
• an active transaction that adds another folder and binary, and deletes a binary

File system representation

bbpennel

Example structure

• my_book - AG
  • pages - folder with no user supplied triples
    • page1.png - internal binary with user supplied triples
    • page2.png - internal binary, no user triples
  • original_scan.pdf - external binary, no user triples

Sidecar fcrepo dir

.fcrepo/ # sidecar files containing fcrepo server managed triples

bbpennel

Fedora structure:
obj1 -> LDPC
    └── file.txt -> LDPNR -> reference to memento 20190130120000
    └── subdir -> LDPC
        └── file1.txt -> LDPNR
        └── file2.txt -> LDPNR
            ...
        └── file1000000.txt -> LDPNR
        
[storage root]

bbpennel

[object root]
    ├── 0=ocfl_object_1.0
    ├── inventory.json ( “id”: “obj1” )
    ├── inventory.json.sha512
    └── v1
        ├── inventory.json
        ├── inventory.json.sha512
        └── content
            └── file.txt
            └── subdir

bbpennel

Sibling timemaps

External Uri => Node Path

• /5bc56ed8-aa1c-44c7-a7fa-43d02e54e353 (binary)
  • => /5bc56ed8-aa1c-44c7-a7fa-43d02e54e353/jcr:content
• /5bc56ed8-aa1c-44c7-a7fa-43d02e54e353/fcr:metadata (description)
  • => /5bc56ed8-aa1c-44c7-a7fa-43d02e54e353
• /5bc56ed8-aa1c-44c7-a7fa-43d02e54e353/fcr:versions (binary timemap)
  • => /5bc56ed8-aa1c-44c7-a7fa-43d02e54e353/fedora:binaryTimemap
• /5bc56ed8-aa1c-44c7-a7fa-43d02e54e353/fcr:metadata/fcr:versions (description timemap)

bbpennel

Data: https://docs.google.com/spreadsheets/d/1el-h-UPaLsXMxtOjnx-B26AvYqX57a8iHWTMo0GinzM/edit?usp=sharing

Source: https://github.com/bbpennel/rdf-serialization-metrics

Jena RDF Format Performance Comparison

When serializing a model, the impact of the RDF format is generally minor, with all 3 formats serializing 10000 properties/literals in < 1 second. At 1,000,000 literals, JSON-LD starts to under perform by a factor of around 5, but still only took 4.5 seconds.