Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Fix Corrupt Jackrabbit Oak Version Histories

The latest version of this article now exists here instead: https://github.com/cqsupport/fix-instructions/blob/master/version-issues/

Old steps below:

Issue

When deleting versions from Apache Jackrabbit Oak you observe errors in the logs similar to the one below:

java.lang.NullPointerException: null
        at org.apache.jackrabbit.oak.plugins.version.ReadWriteVersionManager.removeVersion(ReadWriteVersionManager.java:210)

For the full stack see error.txt below.

Solution

There are a number of scripts available for validating and fixing Apache Oak version histories. The steps for running these are below.

A. Run the repair scripts

  1. Install the groovy script console for apache felix: http://felix.apache.org/documentation/subprojects/apache-felix-script-console-plugin.html
  2. Download these two jars and install them to the /system/console/bundles UI https://repo1.maven.org/maven2/org/codehaus/groovy/groovy-all/2.4.6/groovy-all-2.4.6.jar https://repo1.maven.org/maven2/org/apache/felix/org.apache.felix.webconsole.plugins.scriptconsole/1.0.2/org.apache.felix.webconsole.plugins.scriptconsole-1.0.2.jar
  3. Go to http://host/system/console/configMgr/org.apache.sling.jcr.base.internal.LoginAdminWhitelist
  4. Add org.apache.felix.webconsole.plugins.scriptconsole to "Whitelist regexp" and save
  5. Go to http://host/system/console/sc
  6. Select "Groovy" as the language
  7. Copy / paste the contents of script FixCorruptVersions-OAK-5193.groovy to the console and run it - output goes to the error.log by default
  8. Copy / paste the contents of script FixOrphanedVersions-GRANITE-25586.groovy to the console and run it - output goes to the error.log by default

B. Fix the cost calculation and optimize the /oak:index/versionStoreIndex

If you run version purge and see this error then you need to fix the cost calculation of the /oak:index/versionStoreIndex.

  1. Go to /crx/de/index.jsp and log in as admin
  2. Browse to /oak:index/versionStoreIndex and set these two properties:
    refresh (Boolean) = true
    entryCount (Long) = 2000
    evaluatePathRestrictions (Boolean) = true
    reindex (Boolean) = true
    
  3. Save - now monitor the logs for org.apache.jackrabbit.oak.plugins.index to see the progress of reindexing
  4. Now you should be able to run version purge without it failing.

C. (If it still fails) Remove mixins on nodes which are missing their version histories

If version purge still fails, it might be due to versionable nodes that have missing version histories. To fix this, perform the steps below:

  1. Download and install this package to AEM: https://documentcloud.adobe.com/link/track?uri=urn%3Aaaid%3Ascds%3AUS%3A07e61ad8-7fad-4682-b7e0-6ef75781ce9f
  2. Go to http://host:port/system/console/slinglog and add a log for org.apache.jsp.apps.tools.components.checkversions
  3. Go to http://host:port/apps/tools/components/checkversions/run.html and click "Start"
  4. In the log file from step 2 it will report if there are any version corruptions.
  5. Nodes that are missing version histories will output an error like this:
    11.06.2019 13:08:13.006 *ERROR* [Thread-1136] org.apache.jsp.apps.tools.components.checkversions.POST_jsp$VersionCheckThread VERSION ERROR: Node /content/dam/geometrixx-outdoors/activities/hiking/PDP_2_c05.jpg contains a jcr:versionHistory property that points to non-existing node with uuid e98b4045-b145-47d1-8832-3df194ef6e4a
    
  6. Copy / paste the contents of script remove-version-props-from-node.groovy
  7. For each node missing a version history, add a line like this one:
    removeVersionProps("/content/dam/path/to/item/missing/versionhistory/example.pdf")
    
  8. Run the script and it will remove mix:versionable and related properties with version history references from the nodes.

D. Corrupt /oak:index/reference index

If version purge succeeds but throws errors like the one below then it is likely that the /oak:index/reference index has some inconsistencies.

To fix this we need to reindex it: Follow the steps in this article to reindex it with minimal impact on system performance.

16.07.2019 03:19:57.446 *ERROR* [sling-threadpool-576514bd-c94a-41c0-ae83-fe58543ae1b3-(apache-sling-job-thread-pool)-80-Maintenance Queue(com/adobe/granite/maintenance/job/VersionPurgeTask)] com.day.cq.wcm.core.impl.VersionManagerImpl Unable to purge version 1.1 for /content/dam/geometrixx-outdoors/activities/hiking/PDP_2_c05.jpg : OakIntegrity0001: Unable to delete referenced node
javax.jcr.ReferentialIntegrityException: OakIntegrity0001: Unable to delete referenced node
at org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:235)
at org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:212)
at org.apache.jackrabbit.oak.jcr.version.ReadWriteVersionManager.removeVersion(ReadWriteVersionManager.java:243)
at org.apache.jackrabbit.oak.jcr.delegate.VersionManagerDelegate.removeVersion(VersionManagerDelegate.java:226)
at org.apache.jackrabbit.oak.jcr.delegate.VersionHistoryDelegate.removeVersion(VersionHistoryDelegate.java:209)
at org.apache.jackrabbit.oak.jcr.version.VersionHistoryImpl$11.performVoid(VersionHistoryImpl.java:240)
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performVoid(SessionDelegate.java:274)
at org.apache.jackrabbit.oak.jcr.version.VersionHistoryImpl.removeVersion(VersionHistoryImpl.java:236)
at com.day.cq.wcm.core.impl.VersionManagerImpl.purgeVersions(VersionManagerImpl.java:504)
at com.day.cq.wcm.core.impl.VersionPurgeTask.process(VersionPurgeTask.java:121)
at org.apache.sling.event.impl.jobs.queues.JobQueueImpl.startJob(JobQueueImpl.java:293)
at org.apache.sling.event.impl.jobs.queues.JobQueueImpl.access$100(JobQueueImpl.java:60)
at org.apache.sling.event.impl.jobs.queues.JobQueueImpl$1.run(JobQueueImpl.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.jackrabbit.oak.api.CommitFailedException: OakIntegrity0001: Unable to delete referenced node
at org.apache.jackrabbit.oak.plugins.index.reference.ReferenceEditor.checkReferentialIntegrity(ReferenceEditor.java:340)
at org.apache.jackrabbit.oak.plugins.index.reference.ReferenceEditor.leave(ReferenceEditor.java:187)
at org.apache.jackrabbit.oak.plugins.index.IndexUpdate.leave(IndexUpdate.java:329)
at org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
at org.apache.jackrabbit.oak.spi.commit.CompositeEditor.leave(CompositeEditor.java:74)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:56)
at org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:55)
at org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(CompositeHook.java:61)
at org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(CompositeHook.java:61)
at org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.prepare(SegmentNodeStore.java:604)
at org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.optimisticMerge(SegmentNodeStore.java:634)
at org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.execute(SegmentNodeStore.java:690)
at org.apache.jackrabbit.oak.segment.SegmentNodeStore.merge(SegmentNodeStore.java:334)
at org.apache.jackrabbit.oak.core.MutableRoot.commit(MutableRoot.java:249)
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.commit(SessionDelegate.java:347)
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.commit(SessionDelegate.java:372)
at org.apache.jackrabbit.oak.jcr.version.ReadWriteVersionManager.removeVersion(ReadWriteVersionManager.java:239)
... 13 common frames omitted
*ERROR* [sling-threadpool-c3fa8155-1e6e-4e71-8b2b-99077022d228-(apache-sling-job-thread-pool)-1-Maintenance Queue(com/adobe/granite/maintenance/job/VersionPurgeTask)] org.apa
che.sling.event.impl.jobs.queues.JobQueueImpl.Maintenance Queue [] Unhandled error occured in job processor null while processing job Sling Job [topic=com/adobe/granite/maintenance/job/VersionPurgeT
ask, id=2019/6/11/2/0/aa44baef-fecd-4539-a57c-dfdce6a47680_0, properties=slingevent:application=aa44baef-fecd-4539-a57c-dfdce6a47680,jcr:created=java.util.GregorianCalendar(Tue Jun 11 02:00:00 EDT 2
019),slingevent:created=java.util.GregorianCalendar(Tue Jun 11 02:00:00 EDT 2019),event.job.queuename=Maintenance Queue,event.job.queued.time=java.util.GregorianCalendar(Tue Jun 11 02:00:00 EDT 2019
),jcr:createdBy=sling-event,sling:resourceType=slingevent:Job,event.job.application=aa44baef-fecd-4539-a57c-dfdce6a47680,event.job.retries=0,event.job.started.time=java.util.GregorianCalendar(Tue Ju
n 11 02:00:00 EDT 2019),stoppable=false,jcr:primaryType=slingevent:Job,window=granite_daily,event.job.retrycount=0]
java.lang.NullPointerException: null
at org.apache.jackrabbit.oak.plugins.version.ReadWriteVersionManager.removeVersion(ReadWriteVersionManager.java:210)
at org.apache.jackrabbit.oak.plugins.version.VersionStorageEditor.childNodeDeleted(VersionStorageEditor.java:107)
at org.apache.jackrabbit.oak.spi.commit.CompositeEditor.childNodeDeleted(CompositeEditor.java:135)
at org.apache.jackrabbit.oak.spi.commit.VisibleEditor.childNodeDeleted(VisibleEditor.java:114)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeDeleted(EditorDiff.java:166)
at org.apache.jackrabbit.oak.segment.MapRecord$3.childNodeDeleted(MapRecord.java:446)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:479)
at org.apache.jackrabbit.oak.segment.MapRecord.compareBranch(MapRecord.java:568)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:467)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:433)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:608)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:555)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:555)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
at org.apache.jackrabbit.oak.segment.MapRecord$3.childNodeChanged(MapRecord.java:441)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:489)
at org.apache.jackrabbit.oak.segment.MapRecord.compareBranch(MapRecord.java:568)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:467)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:433)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:608)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
at org.apache.jackrabbit.oak.segment.MapRecord$3.childNodeChanged(MapRecord.java:441)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:489)
at org.apache.jackrabbit.oak.segment.MapRecord.compareBranch(MapRecord.java:568)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:467)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:433)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:608)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
at org.apache.jackrabbit.oak.segment.MapRecord$3.childNodeChanged(MapRecord.java:441)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:489)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:433)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:608)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
at org.apache.jackrabbit.oak.segment.MapRecord$3.childNodeChanged(MapRecord.java:441)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:489)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:433)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:608)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:52)
at org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:54)
at org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(CompositeHook.java:61)
at org.apache.jackrabbit.oak.plugins.version.VersionHook.processCommit(VersionHook.java:83)
at org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(CompositeHook.java:61)
at org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(CompositeHook.java:61)
at org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.prepare(SegmentNodeStore.java:603)
at org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.optimisticMerge(SegmentNodeStore.java:634)
at org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.execute(SegmentNodeStore.java:690)
at org.apache.jackrabbit.oak.segment.SegmentNodeStore.merge(SegmentNodeStore.java:334)
at org.apache.jackrabbit.oak.core.MutableRoot.commit(MutableRoot.java:249)
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.commit(SessionDelegate.java:347)
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.commit(SessionDelegate.java:372)
at org.apache.jackrabbit.oak.jcr.version.ReadWriteVersionManager.removeVersion(ReadWriteVersionManager.java:239)
at org.apache.jackrabbit.oak.jcr.delegate.VersionManagerDelegate.removeVersion(VersionManagerDelegate.java:226)
at org.apache.jackrabbit.oak.jcr.delegate.VersionHistoryDelegate.removeVersion(VersionHistoryDelegate.java:209)
at org.apache.jackrabbit.oak.jcr.version.VersionHistoryImpl$11.performVoid(VersionHistoryImpl.java:240)
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performVoid(SessionDelegate.java:274)
at org.apache.jackrabbit.oak.jcr.version.VersionHistoryImpl.removeVersion(VersionHistoryImpl.java:236)
at com.day.cq.wcm.core.impl.VersionManagerImpl.purgeVersions(VersionManagerImpl.java:482)
at com.day.cq.wcm.core.impl.VersionPurgeTask.process(VersionPurgeTask.java:121)
at org.apache.sling.event.impl.jobs.queues.JobQueueImpl.startJob(JobQueueImpl.java:291)
at org.apache.sling.event.impl.jobs.queues.JobQueueImpl.access$100(JobQueueImpl.java:58)
at org.apache.sling.event.impl.jobs.queues.JobQueueImpl$1.run(JobQueueImpl.java:227)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
import com.google.common.collect.BiMap
import com.google.common.collect.HashBiMap
import com.google.common.collect.Maps
import com.google.common.collect.Sets
import groovy.transform.EqualsAndHashCode
import org.apache.commons.lang3.StringUtils
import org.apache.jackrabbit.oak.api.CommitFailedException
import org.apache.jackrabbit.oak.api.PropertyState
import org.apache.jackrabbit.oak.api.Type
import org.apache.jackrabbit.oak.plugins.nodetype.TypePredicate
import org.apache.jackrabbit.oak.spi.commit.CommitInfo
import org.apache.jackrabbit.oak.spi.commit.DefaultEditor
import org.apache.jackrabbit.oak.spi.commit.Editor
import org.apache.jackrabbit.oak.spi.commit.EditorProvider
import org.apache.jackrabbit.oak.spi.commit.EmptyHook
import org.apache.jackrabbit.oak.spi.state.ChildNodeEntry
import org.apache.jackrabbit.oak.spi.state.NodeBuilder
import org.apache.jackrabbit.oak.spi.state.NodeState
import org.apache.jackrabbit.oak.spi.state.NodeStore
import org.slf4j.Logger
import org.slf4j.LoggerFactory
import java.util.ArrayList
import java.util.HashSet
import java.util.List
import java.util.Set
import static org.apache.jackrabbit.JcrConstants.JCR_BASEVERSION
import static org.apache.jackrabbit.JcrConstants.JCR_FROZENMIXINTYPES
import static org.apache.jackrabbit.JcrConstants.JCR_FROZENNODE
import static org.apache.jackrabbit.JcrConstants.JCR_ISCHECKEDOUT
import static org.apache.jackrabbit.JcrConstants.JCR_MIXINTYPES
import static org.apache.jackrabbit.JcrConstants.JCR_PREDECESSORS
import static org.apache.jackrabbit.JcrConstants.JCR_ROOTVERSION
import static org.apache.jackrabbit.JcrConstants.JCR_SUCCESSORS
import static org.apache.jackrabbit.JcrConstants.JCR_SYSTEM
import static org.apache.jackrabbit.JcrConstants.JCR_UUID
import static org.apache.jackrabbit.JcrConstants.JCR_VERSIONHISTORY
import static org.apache.jackrabbit.JcrConstants.JCR_VERSIONSTORAGE
import static org.apache.jackrabbit.JcrConstants.MIX_VERSIONABLE
import static org.apache.jackrabbit.JcrConstants.NT_FROZENNODE
import static org.apache.jackrabbit.JcrConstants.NT_VERSION
import static org.apache.jackrabbit.JcrConstants.NT_VERSIONHISTORY
import static org.apache.jackrabbit.oak.api.Type.NAMES
import static org.apache.jackrabbit.oak.api.Type.REFERENCE
import static org.apache.jackrabbit.oak.api.Type.REFERENCES
import static org.apache.jackrabbit.oak.plugins.memory.MultiGenericPropertyState.nameProperty
import org.slf4j.Logger
import org.slf4j.LoggerFactory
/**
* This script removes invalid successor and predecessor references which may be
* result of the OAK-5193 bug (https://issues.apache.org/jira/browse/OAK-5193).
* It should be run with the oak-run:
*
* <pre>
* java -jar oak-run-*.jar console repository/segmentstore ":load OAK-5193-fix.groovy" | tee script.log
* </pre>
*/
class Oak5193Fixer {
final Logger log = LoggerFactory.getLogger(getClass())
private final NodeStore ns
private final NodeBuilder rootBuilder
private final TypePredicate isVersionHistory
private final TypePredicate isVersion
private Oak5193Fixer(NodeBuilder rootBuilder, NodeStore ns) {
this.rootBuilder = rootBuilder
this.ns = ns
this.isVersionHistory = new TypePredicate(rootBuilder.getNodeState(), NT_VERSIONHISTORY)
this.isVersion = new TypePredicate(rootBuilder.getNodeState(), NT_VERSION)
}
private void traverse() {
traverse(rootBuilder.getChildNode(JCR_SYSTEM).getChildNode(JCR_VERSIONSTORAGE))
ns.merge(rootBuilder, EmptyHook.INSTANCE, CommitInfo.EMPTY)
doLog("Merged fixes")
}
private void traverse(NodeBuilder builder) {
for (String childName : builder.getChildNodeNames()) {
NodeBuilder child = builder.getChildNode(childName)
if (isVersionHistory.apply(child.getNodeState())) {
fixVersionHistory(child)
} else {
traverse(child)
}
}
}
private boolean fixVersionHistory(NodeBuilder versionHistory) {
log.debug("Validating " + versionHistory)
BiMap<String, String> uuidName = HashBiMap.create()
for (String childName : versionHistory.getChildNodeNames()) {
NodeBuilder child = versionHistory.getChildNode(childName)
if (isVersion.apply(child.getNodeState())) {
String uuid = child.getProperty(JCR_UUID).getValue(Type.STRING)
uuidName.put(uuid, childName)
}
}
for (String versionName : uuidName.values()) {
NodeBuilder version = versionHistory.getChildNode(versionName)
Set<String> predecessors = Sets.newLinkedHashSet((Iterable<String>) version.getProperty(JCR_PREDECESSORS).getValue(Type.REFERENCES))
Set<String> invalidPredecessors = Sets.difference(predecessors, uuidName.keySet())
for (String invalidPredecessorUuid : invalidPredecessors) {
String validPredecessor = fixInvalidPredecessor(versionHistory, uuidName, invalidPredecessorUuid, versionName, version)
doLog(versionHistory.toString() + " (!) fixed predecessor " + versionName + " <- " + validPredecessor)
}
for (String invalidSuccessorUuid : intersectReferences(version, JCR_SUCCESSORS, uuidName.keySet())) {
doLog(versionHistory.toString() + " (!) removed invalid successors " + versionName + " -> " + invalidSuccessorUuid)
}
}
for (String versionName : uuidName.values()) {
NodeBuilder version = versionHistory.getChildNode(versionName)
for (String predecessor : setMissingSuccessors(versionHistory, uuidName, version)) {
doLog(versionHistory.toString() + " (!) created missing successor " + predecessor + " -> " + versionName)
}
for (String successor : removeRedundantSuccessors(versionHistory, uuidName, version)) {
doLog(versionHistory.toString() + " (!) removed redundant successor " + versionName + " -> " + successor)
}
}
}
private String fixInvalidPredecessor(NodeBuilder versionHistory, BiMap<String, String> uuidName, String invalidPredecessorUuid, String versionName, NodeBuilder version) {
String validPredecessorName = closestVersion(versionName, uuidName.values())
Set<String> predecessors = Sets.newLinkedHashSet(version.getProperty(JCR_PREDECESSORS).getValue(Type.REFERENCES))
predecessors.remove(invalidPredecessorUuid)
if (validPredecessorName != null) {
predecessors.add(uuidName.inverse().get(validPredecessorName))
}
version.setProperty(JCR_PREDECESSORS, predecessors, Type.REFERENCES)
return validPredecessorName
}
private Set<String> intersectReferences(NodeBuilder builder, String propertyName, Set<String> references) {
Set<String> currentValue = Sets.newLinkedHashSet((Iterable<String>) builder.getProperty(propertyName).getValue(Type.REFERENCES))
Set<String> intersection = Sets.intersection(currentValue, references)
if (currentValue != intersection) {
builder.setProperty(propertyName, intersection, Type.REFERENCES)
return Sets.difference(currentValue, references)
} else {
return Collections.emptySet()
}
}
private static List<String> setMissingSuccessors(NodeBuilder versionHistory, BiMap<String, String> uuidName, NodeBuilder version) {
String uuid = version.getProperty(JCR_UUID).getValue(Type.STRING)
Set<String> predecessors = Sets.newLinkedHashSet((Iterable<String>) version.getProperty(JCR_PREDECESSORS).getValue(Type.REFERENCES))
List<String> fixedPredecessors = new ArrayList<>()
for (String pUuid : predecessors) {
String pName = uuidName.get(pUuid)
NodeBuilder predecessor = versionHistory.getChildNode(pName)
Set<String> successors = Sets.newLinkedHashSet((Iterable<String>) predecessor.getProperty(JCR_SUCCESSORS).getValue(Type.REFERENCES))
if (!successors.contains(uuid)) {
successors.add(uuid)
predecessor.setProperty(JCR_SUCCESSORS, successors, Type.REFERENCES)
fixedPredecessors.add(pName)
}
}
return fixedPredecessors
}
private static List<String> removeRedundantSuccessors(NodeBuilder versionHistory, BiMap<String, String> uuidName, NodeBuilder version) {
String uuid = version.getProperty(JCR_UUID).getValue(Type.STRING)
Set<String> successors = Sets.newLinkedHashSet((Iterable<String>) version.getProperty(JCR_SUCCESSORS).getValue(Type.REFERENCES))
List<String> removedSuccessors = new ArrayList<>()
Iterator<String> successorsIt = successors.iterator()
while (successorsIt.hasNext()) {
String sUuid = successorsIt.next()
String sName = uuidName.get(sUuid)
NodeBuilder successor = versionHistory.getChildNode(sName)
Set<String> predecessors = Sets.newLinkedHashSet((Iterable<String>) successor.getProperty(JCR_PREDECESSORS).getValue(Type.REFERENCES))
if (!predecessors.contains(uuid)) {
removedSuccessors.add(sName)
successorsIt.remove()
}
}
if (!removedSuccessors.isEmpty()) {
version.setProperty(JCR_SUCCESSORS, successors, Type.REFERENCES)
}
return removedSuccessors
}
static String closestVersion(String version, Set<String> versions) {
String v = version
while (true) {
v = decreaseVersion(v)
if (v == JCR_ROOTVERSION || versions.contains(v)) {
return v
}
}
}
static String decreaseVersion(String version) {
if (version == JCR_ROOTVERSION) {
return null
} else if (version == "1.0") {
return JCR_ROOTVERSION
}
String[] split = StringUtils.split(version, '.')
String[] result
int lastSegment = Integer.valueOf(split[split.length - 1])
if (lastSegment == 0) {
result = new String[split.length - 1]
for (int i = 0; i < result.length; i++) {
result[i] = split[i]
}
} else {
result = split
result[split.length - 1] = String.valueOf(lastSegment - 1)
}
return StringUtils.join(result, '.')
}
private void doLog(String message) {
log.info(message);
}
}
def test() {
def decSeq = { String version ->
List<String> seq = new ArrayList<>()
String v = version
while (v != null) {
seq.add(v)
v = Oak5193Fixer.decreaseVersion(v)
}
return seq
}
assert decSeq("1.2.1.6") == ["1.2.1.6", "1.2.1.5", "1.2.1.4", "1.2.1.3", "1.2.1.2", "1.2.1.1", "1.2.1.0", "1.2.1", "1.2.0", "1.2", "1.1", "1.0", "jcr:rootVersion"]
assert decSeq("1.0") == ["1.0", "jcr:rootVersion"]
assert decSeq("jcr:rootVersion") == ["jcr:rootVersion"]
assert Oak5193Fixer.closest("1.2.1.6", ["1.3", "1.2.1.6", "1.2.1", "1.2", "jcr:rootVersion"]) == "1.2.1"
assert Oak5193Fixer.closest("1.2.1.6", ["1.3", "1.2.1.5", "1.2.1", "1.2", "jcr:rootVersion"]) == "1.2.1.5"
assert Oak5193Fixer.closest("1.3", ["1.3", "1.1", "jcr:rootVersion"]) == "1.1"
assert Oak5193Fixer.closest("1.3", ["1.3", "jcr:rootVersion"]) == "jcr:rootVersion"
assert Oak5193Fixer.closest("1.0", ["1.0", "jcr:rootVersion"]) == "jcr:rootVersion"
assert Oak5193Fixer.closest("jcr:rootVersion", ["jcr:rootVersion"]) == null
}
def runFixer(session) {
NodeStore ns = session.getRootNode().sessionDelegate.root.store
def rootBuilder = ns.root.builder()
new Oak5193Fixer(rootBuilder, ns).traverse()
null
}
def fixVersions() {
def repo = osgi.getService(org.apache.sling.jcr.api.SlingRepository)
def session = repo.loginAdministrative(null)
try {
runFixer(session)
} finally {
session.logout()
}
}
fixVersions()
import org.apache.jackrabbit.oak.plugins.nodetype.TypePredicate
import org.apache.jackrabbit.oak.spi.commit.CommitInfo
import org.apache.jackrabbit.oak.spi.commit.EmptyHook
import org.apache.jackrabbit.oak.spi.state.NodeBuilder
import org.apache.jackrabbit.oak.spi.state.NodeStore
import static org.apache.jackrabbit.JcrConstants.JCR_SYSTEM
import static org.apache.jackrabbit.JcrConstants.JCR_VERSIONSTORAGE
import static org.apache.jackrabbit.JcrConstants.NT_VERSIONHISTORY
import org.slf4j.Logger
import org.slf4j.LoggerFactory
/**
* This script removes version histories with no crx.default property (referencing
* the versionable).
*/
class NPR21684Fixer {
final Logger log = LoggerFactory.getLogger(getClass())
private final NodeStore ns
private final NodeBuilder rootBuilder
private final TypePredicate isVersionHistory
private NPR21684Fixer(NodeBuilder rootBuilder, NodeStore ns) {
this.ns = ns
this.rootBuilder = rootBuilder
this.isVersionHistory = new TypePredicate(rootBuilder.getNodeState(), NT_VERSIONHISTORY)
}
private void traverse() {
doLog("Removing orphaned version histories")
traverse(rootBuilder.getChildNode(JCR_SYSTEM).getChildNode(JCR_VERSIONSTORAGE))
ns.merge(rootBuilder, EmptyHook.INSTANCE, CommitInfo.EMPTY)
doLog("Removed orphaned version histories")
}
private void traverse(NodeBuilder builder) {
for (String childName : builder.getChildNodeNames()) {
NodeBuilder child = builder.getChildNode(childName)
if (isVersionHistory.apply(child.getNodeState())) {
fixVersionHistory(child)
} else {
traverse(child)
}
}
}
private boolean fixVersionHistory(NodeBuilder versionHistory) {
if (!versionHistory.hasProperty("crx.default")) {
doLog("Removing " + versionHistory)
versionHistory.remove()
}
}
private void doLog(String message) {
log.info(message);
}
}
def runFixer(session) {
NodeStore ns = session.getRootNode().sessionDelegate.root.store
def rootBuilder = ns.root.builder()
new NPR21684Fixer(rootBuilder, ns).traverse()
null
}
def fixVersions() {
def repo = osgi.getService(org.apache.sling.jcr.api.SlingRepository)
def session = repo.loginAdministrative(null)
try {
runFixer(session)
} finally {
session.logout()
}
}
fixVersions()
17.05.2019 06:03:33.842 *WARN* [pool-24-thread-1] org.apache.jackrabbit.oak.query.FilterIterators The query read or traversed more than 100000 nodes.
java.lang.UnsupportedOperationException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70)
at org.apache.jackrabbit.oak.plugins.index.Cursors$TraversingCursor.fetchNext(Cursors.java:335)
at org.apache.jackrabbit.oak.plugins.index.Cursors$TraversingCursor.next(Cursors.java:314)
at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:436)
at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:824)
at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:851)
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:141)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:155)
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:141)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:155)
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:141)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:155)
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:141)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:155)
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:141)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:155)
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$4.fetch(QueryResultImpl.java:181)
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$4.<init>(QueryResultImpl.java:176)
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl.getNodes(QueryResultImpl.java:169)
at com.day.cq.wcm.core.impl.VersionManagerImpl$3.fetchBatch(VersionManagerImpl.java:747)
at com.day.cq.wcm.core.impl.VersionManagerImpl$3.hasNext(VersionManagerImpl.java:833)
at com.day.cq.wcm.core.impl.VersionManagerImpl.mergeVersions(VersionManagerImpl.java:1032)
at com.day.cq.wcm.core.impl.VersionManagerImpl.access$600(VersionManagerImpl.java:92)
at com.day.cq.wcm.core.impl.VersionManagerImpl$2.iterator(VersionManagerImpl.java:675)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at com.day.cq.wcm.core.impl.VersionManagerImpl.purgeVersions(VersionManagerImpl.java:504)
at com.day.cq.wcm.core.impl.VersionPurgeTask.process(VersionPurgeTask.java:171)
at com.adobe.granite.maintenance.impl.MaintenanceJobsManagerImpl.execute(MaintenanceJobsManagerImpl.java:195)
at com.adobe.granite.maintenance.impl.MaintenanceJobsManagerImpl.startJobInternal(MaintenanceJobsManagerImpl.java:175)
at com.adobe.granite.maintenance.impl.MaintenanceJobsManagerImpl.lambda$startJob$0(MaintenanceJobsManagerImpl.java:129)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
17.05.2019 06:03:33.843 *INFO* [pool-24-thread-1] com.adobe.granite.repository Service [39452, [org.apache.jackrabbit.oak.api.jmx.SessionMBean]] ServiceEvent UNREGISTERING
17.05.2019 06:03:33.850 *INFO* [pool-24-thread-1] com.adobe.granite.maintenance.impl.MaintenanceJobsManagerImpl Name='com.day.cq.wcm.core.impl.VersionPurgeTask', Status='ERROR', Result='Unexpected exception: 'java.lang.UnsupportedOperationException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.', Details='{"created":1558072189067,"started":1558072189067,"finished":1558073013844,"duration":824777}'
java.lang.UnsupportedOperationException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70)
at org.apache.jackrabbit.oak.plugins.index.Cursors$TraversingCursor.fetchNext(Cursors.java:335)
at org.apache.jackrabbit.oak.plugins.index.Cursors$TraversingCursor.next(Cursors.java:314)
at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:436)
at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:824)
at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:851)
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:141)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:155)
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:141)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:155)
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:141)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:155)
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:141)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:155)
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:141)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:155)
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$4.fetch(QueryResultImpl.java:181)
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$4.<init>(QueryResultImpl.java:176)
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl.getNodes(QueryResultImpl.java:169)
at com.day.cq.wcm.core.impl.VersionManagerImpl$3.fetchBatch(VersionManagerImpl.java:747)
at com.day.cq.wcm.core.impl.VersionManagerImpl$3.hasNext(VersionManagerImpl.java:833)
at com.day.cq.wcm.core.impl.VersionManagerImpl.mergeVersions(VersionManagerImpl.java:1032)
at com.day.cq.wcm.core.impl.VersionManagerImpl.access$600(VersionManagerImpl.java:92)
at com.day.cq.wcm.core.impl.VersionManagerImpl$2.iterator(VersionManagerImpl.java:675)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at com.day.cq.wcm.core.impl.VersionManagerImpl.purgeVersions(VersionManagerImpl.java:504)
at com.day.cq.wcm.core.impl.VersionPurgeTask.process(VersionPurgeTask.java:171)
at com.adobe.granite.maintenance.impl.MaintenanceJobsManagerImpl.execute(MaintenanceJobsManagerImpl.java:195)
at com.adobe.granite.maintenance.impl.MaintenanceJobsManagerImpl.startJobInternal(MaintenanceJobsManagerImpl.java:175)
at com.adobe.granite.maintenance.impl.MaintenanceJobsManagerImpl.lambda$startJob$0(MaintenanceJobsManagerImpl.java:129)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
import org.apache.jackrabbit.oak.spi.commit.CommitInfo
import org.apache.jackrabbit.oak.spi.commit.EmptyHook
import org.apache.jackrabbit.oak.spi.state.NodeStore
import org.apache.jackrabbit.oak.commons.PathUtils
import com.google.common.collect.Lists
import java.util.List
import org.slf4j.Logger
import org.slf4j.LoggerFactory
public class VersionPropRemover {
final Logger log = LoggerFactory.getLogger(getClass())
def out;
public VersionPropRemover(def out) {
this.out = out
}
public removeVersionProps(def session, def argpath, def isDryRun) {
if (!PathUtils.isValid(argpath)) {
doLog("Not a valid path: " + argpath);
return;
}
String path;
if (PathUtils.isAbsolute(argpath)) {
path = argpath;
}
List<String> elements = Lists.newArrayList();
PathUtils.elements(path).each{String element ->
if (PathUtils.denotesParent(element)) {
if (!elements.isEmpty()) {
elements.remove(elements.size() - 1);
}
} else if (!PathUtils.denotesCurrent(element)) {
elements.add(element);
}
}
NodeStore nstore = session.getRootNode().sessionDelegate.root.store
def rs = nstore.root
def ns = rs
def rnb = rs.builder()
//def nb = rnb;
elements.each {
if(it.size() > 0) {
ns = ns.getChildNode(it)
}
}
removeVersionPropsRecurse(session, nstore, argpath, ns, isDryRun)
session.refresh(true);
}
private removeVersionPropsRecurse(def session, def nodeStore, def curPath, def ns, def isDryRun) {
def entryIter = ns.getChildNodeEntries()
def rnb = nodeStore.root.builder()
def nb = getNodeBuilderForPath(rnb, curPath);
removeVersionPropertiesFromNode(nodeStore, ns, nb, rnb, curPath, isDryRun)
/*//doLog(curPath);
entryIter.each {
def cnb = getNodeBuilderForPath(rnb, curPath + "/" + it.getName());
removeVersionPropertiesFromNode(nodeStore, it, cnb, rnb, curPath + "/" + it.getName(), isDryRun)
removeVersionPropsRecurse(session, nodeStore, curPath + "/" + it.getName(), it, isDryRun)
}*/
}
private removeVersionPropertiesFromNode(def nodeStore, def ns, def nb, def rnb, def curPath, def isDryRun) {
def props = ns.getProperties()
props.each { prop ->
if("jcr:mixinTypes".equals(prop.getName())) {
def newMixins = null;
def firstVal = true;
def hasMixVersionable = false;
def mixins = prop.getValue(prop.getType())
mixins.each{ mixin ->
if(!"mix:versionable".equals(mixin)) {
if(firstVal) {
newMixins = Lists.newArrayList(mixin)
firstVal = false
} else {
newMixins.add(mixin)
}
} else {
hasMixVersionable = true;
}
}
if(hasMixVersionable) {
doLog(((isDryRun)?"(Dry run) ":"") + "Removing mix:versionable from " + prop + " from " + curPath)
if(newMixins == null) {
if(!isDryRun) nb.setProperty(prop.getName(), Lists.newArrayList(), org.apache.jackrabbit.oak.api.Type.REFERENCES)
} else {
if(!isDryRun) nb.setProperty(prop.getName(), newMixins, org.apache.jackrabbit.oak.api.Type.REFERENCES)
}
}
}
if("jcr:baseVersion".equals(prop.getName())) {
doLog(((isDryRun)?"(Dry run) ":"") + "Removing " + prop + " from " + curPath)
if(!isDryRun) nb.removeProperty("jcr:baseVersion")
}
if("jcr:predecessors".equals(prop.getName())) {
doLog(((isDryRun)?"(Dry run) ":"") + "Removing " + prop + " from " + curPath)
if(!isDryRun) nb.removeProperty("jcr:predecessors")
}
if("jcr:versionHistory".equals(prop.getName())) {
doLog(((isDryRun)?"(Dry run) ":"") + "Removing " + prop + " from " + curPath)
if(!isDryRun) nb.removeProperty("jcr:versionHistory")
}
if(!isDryRun) nodeStore.merge(rnb, EmptyHook.INSTANCE, CommitInfo.EMPTY);
}
}
private getNodeBuilderForPath(def rootNodeBuilder, def argpath) {
def nb = rootNodeBuilder
String path;
if (PathUtils.isAbsolute(argpath)) {
path = argpath;
} else {
path = PathUtils.concat(session.getWorkingPath(), argpath);
}
List<String> elements = Lists.newArrayList();
PathUtils.elements(path).each{String element ->
if (PathUtils.denotesParent(element)) {
if (!elements.isEmpty()) {
elements.remove(elements.size() - 1);
}
} else if (!PathUtils.denotesCurrent(element)) {
elements.add(element);
}
}
elements.each {
if(it.size() > 0) {
nb = nb.getChildNode(it)
}
}
return nb
}
private void doLog(def message) {
out.println(message)
//log.info(message);
}
}
def removeVersionProps(session, argpath) {
NodeStore ns = session.getRootNode().sessionDelegate.root.store
def rootBuilder = ns.root.builder()
// remove versions of test.pdf
new VersionPropRemover(out).removeVersionProps(session, argpath, false)
null
}
def removeVersionProps(def argpath) {
def repo = osgi.getService(org.apache.sling.jcr.api.SlingRepository)
def session = repo.loginAdministrative(null)
try {
removeVersionProps(session, argpath)
} finally {
session.logout()
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment