bbpennel

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:task="http://www.springframework.org/schema/task"
    xmlns:p="http://www.springframework.org/schema/p"
    xmlns:util="http://www.springframework.org/schema/util"
    xsi:schemaLocation="
    http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
    http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd

bbpennel

// snippet from FedoraTransactionsIT

    @Test
    public void testConcurrentPutsAndCommits() throws Exception {
        final String baseId = getRandomUniqueId();
        final HttpPut putBase = new HttpPut(serverAddress + "/" + baseId);
        try (final CloseableHttpResponse response = execute(putBase)) {
            assertEquals(CREATED.getStatusCode(), getStatus(response));
        } catch (IOException e) {
            e.printStackTrace();

bbpennel

Example structure

• my_book - AG
  • pages - Direct container, defining pcdm:hasFile
    • page1.png - internal binary with user supplied triples
    • page2.png - internal binary, no user triples
  • originals - container, no user triples
    • original_scan.pdf - external binary, no user triples

Persisted structure

File names are path components from fedora identifiers relative to the identifier of the AG.

bbpennel

Example Persistence Layout for FSDB

In this example, there is:

• an object, my_book, which is an AG
  • It has been assigned a fedora id info:fedora/91f6d9c2-dc27-4ab5-b626-df0a45e4519e
• a head version contains a folder containing two binaries
• one previous version, which contained only one binary
• an active transaction that adds another folder and binary, and deletes a binary

File system representation

bbpennel

Example structure

• my_book - AG
  • pages - folder with no user supplied triples
    • page1.png - internal binary with user supplied triples
    • page2.png - internal binary, no user triples
  • original_scan.pdf - external binary, no user triples

Sidecar fcrepo dir

.fcrepo/ # sidecar files containing fcrepo server managed triples

bbpennel

#!/usr/bin/env bash

USAGE="$(basename "$0") [-h] [-d] [-e] target_dir 

Replaces all non-ASCII characters with another character in filenames
for all files and directories contained within the target directory.

where:
    -h, --help               Show this help text
    -d, --dry-run            Display the expected changes, but do not perform them. If

bbpennel

Fedora structure:
obj1 -> LDPC
    └── file.txt -> LDPNR -> reference to memento 20190130120000
    └── subdir -> LDPC
        └── file1.txt -> LDPNR
        └── file2.txt -> LDPNR
            ...
        └── file1000000.txt -> LDPNR
        
[storage root]

bbpennel

[object root]
    ├── 0=ocfl_object_1.0
    ├── inventory.json ( “id”: “obj1” )
    ├── inventory.json.sha512
    └── v1
        ├── inventory.json
        ├── inventory.json.sha512
        └── content
            └── file.txt
            └── subdir

bbpennel

Sibling timemaps

External Uri => Node Path

• /5bc56ed8-aa1c-44c7-a7fa-43d02e54e353 (binary)
  • => /5bc56ed8-aa1c-44c7-a7fa-43d02e54e353/jcr:content
• /5bc56ed8-aa1c-44c7-a7fa-43d02e54e353/fcr:metadata (description)
  • => /5bc56ed8-aa1c-44c7-a7fa-43d02e54e353
• /5bc56ed8-aa1c-44c7-a7fa-43d02e54e353/fcr:versions (binary timemap)
  • => /5bc56ed8-aa1c-44c7-a7fa-43d02e54e353/fedora:binaryTimemap
• /5bc56ed8-aa1c-44c7-a7fa-43d02e54e353/fcr:metadata/fcr:versions (description timemap)

bbpennel

Data: https://docs.google.com/spreadsheets/d/1el-h-UPaLsXMxtOjnx-B26AvYqX57a8iHWTMo0GinzM/edit?usp=sharing

Source: https://github.com/bbpennel/rdf-serialization-metrics

Jena RDF Format Performance Comparison

When serializing a model, the impact of the RDF format is generally minor, with all 3 formats serializing 10000 properties/literals in < 1 second. At 1,000,000 literals, JSON-LD starts to under perform by a factor of around 5, but still only took 4.5 seconds.