Skip to content

Instantly share code, notes, and snippets.

@pwood
Created February 6, 2023 14:59
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save pwood/be117e1f96d034f90f6c706c6cc315ea to your computer and use it in GitHub Desktop.
Save pwood/be117e1f96d034f90f6c706c6cc315ea to your computer and use it in GitHub Desktop.
fdbbackup encrypted failure
[root@ip-10-1-244-15 ec2-user]# fdbbackup start -d blobstore://XXX@s3.eu-west-1.amazonaws.com:443/main-dev-2023020601?bucket=XXX -s 604800 -z --encryption-key-file /mnt/fdb/backup-encryption-key.dat --log --logdir .
Internal Error @ /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbclient/ClientKnobCollection.h 45:
addr2line -e fdbbackup.debug -p -C -f -i 0xe4b63d 0xd7a935 0xe08f14 0xe0975b 0xdf338a 0xe34df7 0xe2bf3e 0xdf36e4 0x62582e 0x60b394 0x6487b1 0x5fb070 0x5620ed 0x54c2ca 0x7f63d716113a
ERROR: Could not create backup container: An internal error occurred
ERROR: An error was encountered during submission
Fatal Error: Backup error
-r--r----- 1 foundationdb foundationdb 16 Feb 3 11:46 /mnt/fdb/backup-encryption-key.dat
<?xml version="1.0"?>
<Trace>
<Event Severity="10" Time="1675694244.860449" DateTime="2023-02-06T14:37:24Z" Type="Net2Starting" ID="0000000000000000" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.861176" DateTime="2023-02-06T14:37:24Z" Type="ProgramStart" ID="0000000000000000" SourceVersion="dd2e68b9b3175667914673539f06b8e1071c07d6" Version="7.1.23" PackageName="7.1" ActualTime="1675694244" CommandLine="fdbbackup start -d blobstore://XXX@s3.eu-west-1.amazonaws.com:443/main-dev-2023020601?bucket=XXX -s 604800 -z --encryption-key-file /mnt/fdb/backup-encryption-key.dat --log --logdir ." MemoryLimit="8589934592" ThreadID="15164710854328060805" TrackLatestType="Original" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="ClientStart" ID="0000000000000000" SourceVersion="dd2e68b9b3175667914673539f06b8e1071c07d6" Version="7.1.23" PackageName="7.1" ActualTime="1675694244" ApiVersion="-1" ClientLibrary="fdbbackup" ImageOffset="(nil)" Primary="1" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" TrackLatestType="Original" />
<Event Severity="10" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="MachineLoadDetail" ID="0000000000000000" User="621457" Nice="7306" System="368244" Idle="52461200" IOWait="9849" IRQ="0" SoftIRQ="53198" Steal="571045" Guest="0" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="Net2TLSConfig" ID="0000000000000000" CAPath="/mnt/fdb/tls/trust.pem" CertificatePath="/mnt/fdb/tls/cert.pem" KeyPath="/mnt/fdb/tls/key.pem" HasPassword="0" VerifyPeers="Check.Valid=1" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="NotifyAddressHealthy" ID="0000000000000000" SuppressedEventCount="0" Address="10.1.244.28:4500" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="DatabaseContextCreated" ID="8fc5af4a36e54811" Backtrace="addr2line -e fdbbackup.debug -p -C -f -i 0xee35bc 0xaa0411 0xaa486d 0x561f29 0x565b65 0x54c291 0x7f63d716113a" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="ConnectToDatabase" ID="8fc5af4a36e54811" Version="7.1.23" ClusterFile="file:///mnt/fdb/4500/fdb.cluster" ConnectionString="seed:q1GWPhfJe4AT2nUfbmeKtMzFANguaEyG@10.1.244.28:4500,10.1.244.104:4500,10.1.244.212:4500,10.1.245.121:4500,10.1.245.128:4500,10.1.245.157:4500,10.1.246.58:4500,10.1.246.89:4500,10.1.246.132:4500" ClientLibrary="fdbbackup" Primary="1" Internal="1" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" TrackLatestType="Original" />
<Event Severity="10" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="AFCUnderlyingOpenBegin" ID="0000000000000000" Filename="/mnt/fdb/backup-encryption-key.dat" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="AsyncFileKAIOOpen" ID="0000000000000000" Filename="/mnt/fdb/backup-encryption-key.dat" Flags="196610" Mode="256" Fd="8" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="40" ErrorKind="BugDetected" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="InternalError" ID="0000000000000000" Error="internal_error" ErrorDescription="An internal error occurred" ErrorCode="4100" File="/home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbclient/ClientKnobCollection.h" Line="45" ThreadID="15164710854328060805" Backtrace="addr2line -e fdbbackup.debug -p -C -f -i 0xee35bc 0xee21d0 0xee25ce 0xe4b736 0xd7a935 0xe08f14 0xe0975b 0xdf338a 0xe34df7 0xe2bf3e 0xdf36e4 0x62582e 0x60b394 0x6487b1 0x5fb070 0x5620ed 0x54c2ca 0x7f63d716113a" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="40" ErrorKind="Unset" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="SystemError" ID="0000000000000000" Error="internal_error" ErrorDescription="An internal error occurred" ErrorCode="4100" ThreadID="15164710854328060805" Backtrace="addr2line -e fdbbackup.debug -p -C -f -i 0xee35bc 0xee21d0 0xee25ce 0xe4b52c 0xe4b748 0xd7a935 0xe08f14 0xe0975b 0xdf338a 0xe34df7 0xe2bf3e 0xdf36e4 0x62582e 0x60b394 0x6487b1 0x5fb070 0x5620ed 0x54c2ca 0x7f63d716113a" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="30" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="FailedToOpenEncryptionKeyFile" ID="0000000000000000" Error="internal_error" ErrorDescription="An internal error occurred" ErrorCode="4100" FileName="/mnt/fdb/backup-encryption-key.dat" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="FBA_SubmitBackup" ID="0000000000000000" TagName="default" StopWhenDone="0" UsePartitionedLog="0" OutContainer="blobstore://XXX@s3.eu-west-1.amazonaws.com:443/main-dev-2023020601?bucket=XXX" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="CodeCoverage" ID="0000000000000000" File="/home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbclient/ReadYourWrites.actor.cpp" Line="1529" Condition="true" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="StartingRunLoopProfilingThread" ID="0000000000000000" Interval="0.125" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="Net2Running" ID="0000000000000000" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.860528" DateTime="2023-02-06T14:37:24Z" Type="ProcessTimeOffset" ID="0000000000000000" ProcessTime="1675694244.869501" SystemTime="1675694244.869501" OffsetFromSystemTime="-0.000000" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.869539" DateTime="2023-02-06T14:37:24Z" Type="ConnectingTo" ID="0000000000000000" SuppressedEventCount="0" PeerAddr="10.1.246.89:4500" PeerReferences="3" FailureStatus="OK" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.869539" DateTime="2023-02-06T14:37:24Z" Type="ConnectionExchangingConnectPacket" ID="8d188ed82d673aee" SuppressedEventCount="0" PeerAddr="10.1.246.89:4500" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.871209" DateTime="2023-02-06T14:37:24Z" Type="ConnectionEstablished" ID="8d188ed82d673aee" SuppressedEventCount="0" Peer="10.1.246.89:4500" ConnectionId="1" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.871209" DateTime="2023-02-06T14:37:24Z" Type="ConnectedOutgoing" ID="0000000000000000" SuppressedEventCount="0" PeerAddr="10.1.246.89:4500" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.878584" DateTime="2023-02-06T14:37:24Z" Type="LBDistant" ID="0000000000000000" SuppressedEventCount="0" Distance="2" BackOff="0" TriedAllOptions="0" Alternatives="ba229796be2d6bc8,bd24c7d0e769f094,446f0ca2ad6acfc2" Token="4163ed4df849070f" Total="3" Best="3" Attempts="0" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.925371" DateTime="2023-02-06T14:37:24Z" Type="S3BlobStoreEndpointNewConnection" ID="0000000000000000" SuppressedEventCount="0" RemoteEndpoint="52.92.19.40:443:tls(fromHostname)" ExpiresIn="120" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.925525" DateTime="2023-02-06T14:37:24Z" Type="AsyncFileOpened" ID="0000000000000000" SuppressedEventCount="0" Filename="/mnt/fdb/backup-credentials.json" Fd="17" Flags="2228225" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.925740" DateTime="2023-02-06T14:37:24Z" Type="AsyncFileClosed" ID="0000000000000000" SuppressedEventCount="0" Fd="17" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.960965" DateTime="2023-02-06T14:37:24Z" Type="S3BlobStoreEndpointReusingConnected" ID="0000000000000000" SuppressedEventCount="0" RemoteEndpoint="52.92.19.40:443:tls(fromHostname)" ExpiresIn="119.965" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="40" ErrorKind="Unset" Time="1675694244.997581" DateTime="2023-02-06T14:37:24Z" Type="StopAfterError" ID="0000000000000000" Error="backup_error" ErrorDescription="Backup error" ErrorCode="2300" ThreadID="15164710854328060805" Backtrace="addr2line -e fdbbackup.debug -p -C -f -i 0xee35bc 0xee21d0 0xee25ce 0x56e721 0x56e962 0x599a29 0x575a25 0x5761f5 0x599a29 0x58c8da 0x58dd04 0x599a29 0x725e31 0x7261a7 0x599a29 0x5637e1 0x599a29 0x648076 0x653585 0x6537d7 0x653cb8 0xc76538 0xc88cd8 0xc88f77 0xc892d8 0xc7c1e0 0xddac58 0xddbad8 0xddc8e0 0xddcb63 0xddf808 0xdd99b8 0xdd9bd5 0x59e448 0x60a5b0 0xdd9170 0xdd971d 0x5fde30 0xe7d42e 0x97e1e2 0x54ac7b 0x7f63d716113a" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.997581" DateTime="2023-02-06T14:37:24Z" Type="MachineLoadDetail" ID="0000000000000000" User="621459" Nice="7306" System="368244" Idle="52461223" IOWait="9849" IRQ="0" SoftIRQ="53198" Steal="571046" Guest="0" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.997581" DateTime="2023-02-06T14:37:24Z" Type="ProcessMetrics" ID="0000000000000000" Elapsed="0.133836" CPUSeconds="0.013204" MainThreadCPUSeconds="0.011541" UptimeSeconds="0.137053" Memory="499064832" ResidentMemory="19980288" UnusedAllocatedMemory="391936" MbpsSent="0.212798" MbpsReceived="0.407842" DiskTotalBytes="0" DiskFreeBytes="0" DiskQueueDepth="0" DiskIdleSeconds="0" DiskReads="0" DiskReadSeconds="0" DiskWrites="0" DiskWriteSeconds="0" DiskReadsCount="0" DiskWritesCount="0" DiskWriteSectors="0" DiskReadSectors="0" FileWrites="0" FileReads="2" CacheReadBytes="0" CacheFinds="0" CacheWritesBlocked="0" CacheReadsBlocked="0" CachePageReadsMerged="0" CacheWrites="0" CacheReads="0" CacheHits="0" CacheMisses="0" CacheEvictions="0" DCID="[not set]" ZoneID="[not set]" MachineID="[not set]" AIOSubmitCount="0" AIOCollectCount="0" AIOSubmitLag="0" AIODiskStall="0" CurrentConnections="7" ConnectionsEstablished="52.3028" ConnectionsClosed="0" ConnectionErrors="0" TLSPolicyFailures="0" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" TrackLatestType="Original" />
<Event Severity="10" Time="1675694244.997581" DateTime="2023-02-06T14:37:24Z" Type="MemoryMetrics" ID="0000000000000000" TotalMemory16="0" ApproximateUnusedMemory16="0" ActiveThreads16="0" TotalMemory32="131072" ApproximateUnusedMemory32="0" ActiveThreads32="1" TotalMemory64="655360" ApproximateUnusedMemory64="391936" ActiveThreads64="2" TotalMemory96="131040" ApproximateUnusedMemory96="0" ActiveThreads96="1" TotalMemory128="131072" ApproximateUnusedMemory128="0" ActiveThreads128="1" TotalMemory256="262144" ApproximateUnusedMemory256="0" ActiveThreads256="2" TotalMemory512="0" ApproximateUnusedMemory512="0" ActiveThreads512="0" TotalMemory1024="0" ApproximateUnusedMemory1024="0" ActiveThreads1024="0" TotalMemory2048="0" ApproximateUnusedMemory2048="0" ActiveThreads2048="0" TotalMemory4096="0" ApproximateUnusedMemory4096="0" ActiveThreads4096="0" TotalMemory8192="0" ApproximateUnusedMemory8192="0" ActiveThreads8192="0" TotalMemory16384="0" ApproximateUnusedMemory16384="0" ActiveThreads16384="0" HugeArenaMemory="0" DCID="[not set]" ZoneID="[not set]" MachineID="[not set]" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.997581" DateTime="2023-02-06T14:37:24Z" Type="FastAllocMemoryUsage" ID="0000000000000000" TotalMemory="1310688" UnusedMemory="391936" Utilization="70.096926%" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" />
<Event Severity="10" Time="1675694244.997581" DateTime="2023-02-06T14:37:24Z" Type="NetworkMetrics" ID="0000000000000000" Elapsed="0.133836" CantSleep="0" WontSleep="2" Yields="2" YieldCalls="13" YieldCallsTrue="1" RunLoopProfilingSignals="0" YieldBigStack="0" RunLoopIterations="40" TimersExecuted="30" TasksExecuted="73" ASIOEventsProcessed="85" ReadCalls="31" WriteCalls="12" ReadProbes="19" WriteProbes="0" PacketsRead="10" PacketsGenerated="11" WouldBlock="19" LaunchTime="1.28746e-05" ReactTime="0.000568151" DCID="[not set]" ZoneID="[not set]" MachineID="[not set]" SlowTask1M="1" SlowTask4M="1" PriorityBusy0="0.121771" PriorityStarvedBelow1="0.00627923" PriorityMaxStarvedBelow1="0.00289512" PriorityStarvedBelow3500="0.00627923" PriorityMaxStarvedBelow3500="0.00289512" PriorityStarvedBelow7000="0.00608993" PriorityMaxStarvedBelow7000="0.00289512" PriorityStarvedBelow7500="0.00554943" PriorityMaxStarvedBelow7500="0.00286031" PriorityStarvedBelow8500="0.00498581" PriorityMaxStarvedBelow8500="0.00286031" PriorityStarvedBelow8900="0.00225782" PriorityMaxStarvedBelow8900="0.000430584" PriorityStarvedBelow10500="0.000358105" PriorityMaxStarvedBelow10500="0.000430584" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" TrackLatestType="Original" />
<Event Severity="10" Time="1675694244.997581" DateTime="2023-02-06T14:37:24Z" Type="MachineMetrics" ID="0000000000000000" Elapsed="0.133836" MbpsSent="1.2252" MbpsReceived="1.10828" OutSegs="77" RetransSegs="1" CPUSeconds="0.0154426" TotalMemory="8228167680" CommittedMemory="3200102400" AvailableMemory="5028065280" DCID="[not set]" ZoneID="[not set]" MachineID="[not set]" ThreadID="15164710854328060805" Machine="10.1.244.15:13931" LogGroup="default" ClientDescription="primary-7.1.23-15164710854328060805" TrackLatestType="Original" />
</Trace>
[root@ip-10-1-244-15 ec2-user]# addr2line -e fdbbackup.debug.x86_64 -p -C -f -i 0xee35bc 0xee21d0 0xee25ce 0xe4b736 0xd7a935 0xe08f14 0xe0975b 0xdf338a 0xe34df7 0xe2bf3e 0xdf36e4 0x62582e 0x60b394 0x6487b1 0x5fb070 0x5620ed 0x54c2ca 0x7f63d716113a
std::string::_M_rep() const at /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/basic_string.h:3322
(inlined by) std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::string const&) at /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/basic_string.tcc:613
(inlined by) std::basic_string<char, std::char_traits<char>, std::allocator<char> > std::operator+<char, std::char_traits<char>, std::allocator<char> >(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, char const*) at /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/basic_string.h:5967
(inlined by) BaseTraceEvent::backtrace(std::string const&) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/flow/Trace.cpp:1216
std::string::_M_rep() const at /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/basic_string.h:3322
(inlined by) std::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string() at /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/basic_string.h:3640
(inlined by) BaseTraceEvent::log() at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/flow/Trace.cpp:1235
BaseTraceEvent::~BaseTraceEvent() at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/flow/Trace.cpp:1275
internal_error_impl(char const*, int) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/flow/Error.cpp:61
ClientKnobCollection::getServerKnobs() const at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbclient/ClientKnobCollection.h:45
AsyncFileKAIO::AsyncFileKAIO(int, int, std::string const&) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbrpc/AsyncFileKAIO.actor.h:652 (discriminator 1)
Reference<AsyncFileKAIO>::Reference(AsyncFileKAIO*) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/flow/FastRef.h:101
(inlined by) AsyncFileKAIO::open(std::string, int, int, void*) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbrpc/AsyncFileKAIO.actor.h:153
Net2FileSystem::open(std::string const&, long, long) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbrpc/Net2FileSystem.cpp:70 (discriminator 3)
Future<Reference<IAsyncFile> >::Future(Future<Reference<IAsyncFile> > const&) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/flow/flow.h:807
(inlined by) StrictFuture<Reference<IAsyncFile> >::StrictFuture(Future<Reference<IAsyncFile> > const&) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/flow/flow.h:893
(inlined by) AsyncFileCached::Open_implActorState<AsyncFileCached::Open_implActor>::a_body1(int) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbrpc/AsyncFileCached.actor.h:341
AsyncFileCached::Open_implActor::Open_implActor(std::string const&, int const&, int const&, Reference<EvictablePageCache> const&) at /home/foundationdb_ci/foundationdb_build_output/dbdbdbdbdbdbdbdbdbdbdbdbdbdbdbdb/fdbrpc/AsyncFileCached.actor.g.h:1248
(inlined by) AsyncFileCached::open_impl(std::string const&, int const&, int const&, Reference<EvictablePageCache> const&) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbrpc/AsyncFileCached.actor.h:331
(inlined by) AsyncFileCached::open_impl(std::string, int, int) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbrpc/AsyncFileCached.actor.cpp:81
std::string::_M_rep() const at /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/basic_string.h:3322
(inlined by) std::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string() at /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/basic_string.h:3640
(inlined by) AsyncFileCached::open(std::string, int, int) at /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/basic_string.h:3639
(inlined by) Net2FileSystem::open(std::string const&, long, long) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbrpc/Net2FileSystem.cpp:59
Future<Reference<IAsyncFile> >::Future(Future<Reference<IAsyncFile> > const&) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/flow/flow.h:807
(inlined by) StrictFuture<Reference<IAsyncFile> >::StrictFuture(Future<Reference<IAsyncFile> > const&) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/flow/flow.h:893
(inlined by) BackupContainerFileSystemImpl::ReadEncryptionKeyActorState<BackupContainerFileSystemImpl::ReadEncryptionKeyActor>::a_body1(int) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbclient/BackupContainerFileSystem.actor.cpp:1148
BackupContainerFileSystemImpl::ReadEncryptionKeyActor::ReadEncryptionKeyActor(std::string const&) at /home/foundationdb_ci/foundationdb_build_output/dbdbdbdbdbdbdbdbdbdbdbdbdbdbdbdb/fdbclient/BackupContainerFileSystem.actor.g.cpp:6054
(inlined by) BackupContainerFileSystemImpl::readEncryptionKey(std::string const&) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbclient/BackupContainerFileSystem.actor.cpp:1144
(inlined by) BackupContainerFileSystem::setEncryptionKey(Optional<std::string> const&) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbclient/BackupContainerFileSystem.actor.cpp:1485
std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > >::begin() const at /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/stl_map.h:364
(inlined by) BackupContainerS3BlobStore::BackupContainerS3BlobStore(Reference<S3BlobStoreEndpoint>, std::string const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&, Optional<std::string> const&) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbclient/BackupContainerS3BlobStore.actor.cpp:152
Reference<S3BlobStoreEndpoint>::~Reference() at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/flow/FastRef.h:124
(inlined by) Reference<BackupContainerS3BlobStore> makeReference<BackupContainerS3BlobStore, Reference<S3BlobStoreEndpoint>&, std::string&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > >&, Optional<std::string> const&>(Reference<S3BlobStoreEndpoint>&, std::string&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > >&, Optional<std::string> const&) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/flow/FastRef.h:185
(inlined by) IBackupContainer::openContainer(std::string const&, Optional<std::string> const&, Optional<std::string> const&) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbclient/BackupContainer.actor.cpp:284
Reference<IBackupContainer>::operator=(Reference<IBackupContainer>&&) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/flow/FastRef.h:140
(inlined by) openBackupContainer(char const*, std::string const&, Optional<std::string> const&, Optional<std::string> const&) at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbbackup/backup.actor.cpp:2311
Reference<IBackupContainer>::~Reference() at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/flow/FastRef.h:124
(inlined by) main at /home/foundationdb_ci/src/oOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOoOo/foundationdb/fdbbackup/backup.actor.cpp:4014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment