Skip to content

Instantly share code, notes, and snippets.

@r-rmcgibbo

r-rmcgibbo/backblaze-b2

Created March 19, 2021 11:47
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save r-rmcgibbo/8cdcef696f9ac2cedea2b1de034564b1 to your computer and use it in GitHub Desktop.
Save r-rmcgibbo/8cdcef696f9ac2cedea2b1de034564b1 to your computer and use it in GitHub Desktop.
Download ZIP
system: x86_64-linux | build_time: 11 seconds | https://github.com/NixOS/nixpkgs/pull/116831
Raw
backblaze-b2
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing setuptools-build-hook
Using setuptoolsBuildPhase
Using setuptoolsShellHook
Sourcing pip-install-hook
Using pipInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
Sourcing setuptools-check-hook
Using setuptoolsCheckPhase
Sourcing pytest-check-hook
Using pytestCheckPhase
Removing setuptoolsCheckPhase
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/nsvj2j4akavyjn5hhx0qdmfjck9w14nc-source
source root is source
setting SOURCE_DATE_EPOCH to timestamp 315619200 of file source/test/unit/test_console_tool.py
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
no configure script, doing nothing
@nix { "action": "setPhase", "phase": "buildPhase" }
building
Executing setuptoolsBuildPhase
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/b2
copying b2/__init__.py -> build/lib/b2
copying b2/json_encoder.py -> build/lib/b2
copying b2/version.py -> build/lib/b2
copying b2/__main__.py -> build/lib/b2
copying b2/arg_parser.py -> build/lib/b2
copying b2/console_tool.py -> build/lib/b2
creating build/lib/test
creating build/lib/test/integration
copying test/integration/__init__.py -> build/lib/test/integration
copying test/integration/test_b2_command_line.py -> build/lib/test/integration
creating build/lib/test/unit
copying test/unit/test_arg_parser.py -> build/lib/test/unit
copying test/unit/test_base.py -> build/lib/test/unit
copying test/unit/__init__.py -> build/lib/test/unit
copying test/unit/test_console_tool.py -> build/lib/test/unit
creating build/lib/test/static
copying test/static/__init__.py -> build/lib/test/static
copying test/static/test_licenses.py -> build/lib/test/static
creating build/lib/test/unit/fixtures
copying test/unit/fixtures/__init__.py -> build/lib/test/unit/fixtures
creating build/lib/test/unit/fixtures/test_source_mod
copying test/unit/fixtures/test_source_mod/z.py -> build/lib/test/unit/fixtures/test_source_mod
copying test/unit/fixtures/test_source_mod/__init__.py -> build/lib/test/unit/fixtures/test_source_mod
creating build/lib/test/unit/fixtures/test_target_mod
copying test/unit/fixtures/test_target_mod/z.py -> build/lib/test/unit/fixtures/test_target_mod
copying test/unit/fixtures/test_target_mod/__init__.py -> build/lib/test/unit/fixtures/test_target_mod
creating build/lib/test/unit/fixtures/test_source_mod/c
copying test/unit/fixtures/test_source_mod/c/d.py -> build/lib/test/unit/fixtures/test_source_mod/c
copying test/unit/fixtures/test_source_mod/c/__init__.py -> build/lib/test/unit/fixtures/test_source_mod/c
creating build/lib/test/unit/fixtures/test_target_mod/c
copying test/unit/fixtures/test_target_mod/c/__init__.py -> build/lib/test/unit/fixtures/test_target_mod/c
creating build/lib/test/unit/fixtures/test_target_mod/a
copying test/unit/fixtures/test_target_mod/a/b.py -> build/lib/test/unit/fixtures/test_target_mod/a
copying test/unit/fixtures/test_target_mod/a/__init__.py -> build/lib/test/unit/fixtures/test_target_mod/a
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/b2
copying build/lib/b2/__init__.py -> build/bdist.linux-x86_64/wheel/b2
copying build/lib/b2/json_encoder.py -> build/bdist.linux-x86_64/wheel/b2
copying build/lib/b2/version.py -> build/bdist.linux-x86_64/wheel/b2
copying build/lib/b2/__main__.py -> build/bdist.linux-x86_64/wheel/b2
copying build/lib/b2/arg_parser.py -> build/bdist.linux-x86_64/wheel/b2
copying build/lib/b2/console_tool.py -> build/bdist.linux-x86_64/wheel/b2
creating build/bdist.linux-x86_64/wheel/test
creating build/bdist.linux-x86_64/wheel/test/integration
copying build/lib/test/integration/__init__.py -> build/bdist.linux-x86_64/wheel/test/integration
copying build/lib/test/integration/test_b2_command_line.py -> build/bdist.linux-x86_64/wheel/test/integration
creating build/bdist.linux-x86_64/wheel/test/unit
copying build/lib/test/unit/test_arg_parser.py -> build/bdist.linux-x86_64/wheel/test/unit
creating build/bdist.linux-x86_64/wheel/test/unit/fixtures
creating build/bdist.linux-x86_64/wheel/test/unit/fixtures/test_source_mod
copying build/lib/test/unit/fixtures/test_source_mod/z.py -> build/bdist.linux-x86_64/wheel/test/unit/fixtures/test_source_mod
copying build/lib/test/unit/fixtures/test_source_mod/__init__.py -> build/bdist.linux-x86_64/wheel/test/unit/fixtures/test_source_mod
creating build/bdist.linux-x86_64/wheel/test/unit/fixtures/test_source_mod/c
copying build/lib/test/unit/fixtures/test_source_mod/c/d.py -> build/bdist.linux-x86_64/wheel/test/unit/fixtures/test_source_mod/c
copying build/lib/test/unit/fixtures/test_source_mod/c/__init__.py -> build/bdist.linux-x86_64/wheel/test/unit/fixtures/test_source_mod/c
copying build/lib/test/unit/fixtures/__init__.py -> build/bdist.linux-x86_64/wheel/test/unit/fixtures
creating build/bdist.linux-x86_64/wheel/test/unit/fixtures/test_target_mod
copying build/lib/test/unit/fixtures/test_target_mod/z.py -> build/bdist.linux-x86_64/wheel/test/unit/fixtures/test_target_mod
copying build/lib/test/unit/fixtures/test_target_mod/__init__.py -> build/bdist.linux-x86_64/wheel/test/unit/fixtures/test_target_mod
creating build/bdist.linux-x86_64/wheel/test/unit/fixtures/test_target_mod/c
copying build/lib/test/unit/fixtures/test_target_mod/c/__init__.py -> build/bdist.linux-x86_64/wheel/test/unit/fixtures/test_target_mod/c
creating build/bdist.linux-x86_64/wheel/test/unit/fixtures/test_target_mod/a
copying build/lib/test/unit/fixtures/test_target_mod/a/b.py -> build/bdist.linux-x86_64/wheel/test/unit/fixtures/test_target_mod/a
copying build/lib/test/unit/fixtures/test_target_mod/a/__init__.py -> build/bdist.linux-x86_64/wheel/test/unit/fixtures/test_target_mod/a
copying build/lib/test/unit/test_base.py -> build/bdist.linux-x86_64/wheel/test/unit
copying build/lib/test/unit/__init__.py -> build/bdist.linux-x86_64/wheel/test/unit
copying build/lib/test/unit/test_console_tool.py -> build/bdist.linux-x86_64/wheel/test/unit
creating build/bdist.linux-x86_64/wheel/test/static
copying build/lib/test/static/__init__.py -> build/bdist.linux-x86_64/wheel/test/static
copying build/lib/test/static/test_licenses.py -> build/bdist.linux-x86_64/wheel/test/static
running install_egg_info
running egg_info
creating b2.egg-info
writing b2.egg-info/PKG-INFO
writing dependency_links to b2.egg-info/dependency_links.txt
writing entry points to b2.egg-info/entry_points.txt
writing requirements to b2.egg-info/requires.txt
writing top-level names to b2.egg-info/top_level.txt
writing manifest file 'b2.egg-info/SOURCES.txt'
reading manifest file 'b2.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'b2.egg-info/SOURCES.txt'
Copying b2.egg-info to build/bdist.linux-x86_64/wheel/b2-2.1.0-py3.8.egg-info
running install_scripts
adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
creating build/bdist.linux-x86_64/wheel/b2-2.1.0.dist-info/WHEEL
creating 'dist/b2-2.1.0-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'b2/__init__.py'
adding 'b2/__main__.py'
adding 'b2/arg_parser.py'
adding 'b2/console_tool.py'
adding 'b2/json_encoder.py'
adding 'b2/version.py'
adding 'test/integration/__init__.py'
adding 'test/integration/test_b2_command_line.py'
adding 'test/static/__init__.py'
adding 'test/static/test_licenses.py'
adding 'test/unit/__init__.py'
adding 'test/unit/test_arg_parser.py'
adding 'test/unit/test_base.py'
adding 'test/unit/test_console_tool.py'
adding 'test/unit/fixtures/__init__.py'
adding 'test/unit/fixtures/test_source_mod/__init__.py'
adding 'test/unit/fixtures/test_source_mod/z.py'
adding 'test/unit/fixtures/test_source_mod/c/__init__.py'
adding 'test/unit/fixtures/test_source_mod/c/d.py'
adding 'test/unit/fixtures/test_target_mod/__init__.py'
adding 'test/unit/fixtures/test_target_mod/z.py'
adding 'test/unit/fixtures/test_target_mod/a/__init__.py'
adding 'test/unit/fixtures/test_target_mod/a/b.py'
adding 'test/unit/fixtures/test_target_mod/c/__init__.py'
adding 'b2-2.1.0.dist-info/LICENSE'
adding 'b2-2.1.0.dist-info/METADATA'
adding 'b2-2.1.0.dist-info/WHEEL'
adding 'b2-2.1.0.dist-info/entry_points.txt'
adding 'b2-2.1.0.dist-info/top_level.txt'
adding 'b2-2.1.0.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Finished executing setuptoolsBuildPhase
@nix { "action": "setPhase", "phase": "installPhase" }
installing
Executing pipInstallPhase
/build/source/dist /build/source
Processing ./b2-2.1.0-py3-none-any.whl
Requirement already satisfied: phx-class-registry==3.0.5 in /nix/store/5ckm0gnqpz61avh4irkyfqxfn3sswqjf-python3.8-phx-class-registry-3.0.5/lib/python3.8/site-packages (from b2==2.1.0) (3.0.5)
Requirement already satisfied: b2sdk<1.3.0,>=1.2.0 in /nix/store/g0wvxyaxpba3d2iy948g88g91bf7xnyq-python3.8-b2sdk-1.2.0/lib/python3.8/site-packages (from b2==2.1.0) (1.2.0)
Requirement already satisfied: arrow>=0.8.0 in /nix/store/60zlp1qka1rm5ji43h6rk4jc783pnkxa-python3.8-arrow-1.0.3/lib/python3.8/site-packages (from b2==2.1.0) (1.0.3)
Requirement already satisfied: python-dateutil>=2.7.0 in /nix/store/gw25zshrvnbazhlwkc4rflhqk252r24x-python3.8-python-dateutil-2.8.1/lib/python3.8/site-packages (from arrow>=0.8.0->b2==2.1.0) (2.8.1)
Requirement already satisfied: requests>=2.9.1 in /nix/store/l8f4hijm41zy4adi78cw6fkyir3f3rn3-python3.8-requests-2.25.1/lib/python3.8/site-packages (from b2sdk<1.3.0,>=1.2.0->b2==2.1.0) (2.25.1)
Requirement already satisfied: logfury>=0.1.2 in /nix/store/jlylha5finv4nmz519xhd4fgylhqid7w-python3.8-logfury-0.1.2/lib/python3.8/site-packages (from b2sdk<1.3.0,>=1.2.0->b2==2.1.0) (0.1.2)
Requirement already satisfied: tqdm>=4.5.0 in /nix/store/cml6hm0dvrc8w61i4ln0a837zs41048r-python3.8-tqdm-4.58.0/lib/python3.8/site-packages (from b2sdk<1.3.0,>=1.2.0->b2==2.1.0) (4.58.0)
Requirement already satisfied: six>=1.10 in /nix/store/q62ngh3im96ljqjml5j6iaqzbq6daryy-python3.8-six-1.15.0/lib/python3.8/site-packages (from logfury>=0.1.2->b2sdk<1.3.0,>=1.2.0->b2==2.1.0) (1.15.0)
Requirement already satisfied: funcsigs in /nix/store/ff80c5rsa3hzwd6z7dygsyw5glv09xmn-python3.8-funcsigs-1.0.2/lib/python3.8/site-packages (from logfury>=0.1.2->b2sdk<1.3.0,>=1.2.0->b2==2.1.0) (1.0.2)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /nix/store/c3nv9ii9a8hjf2s463q9l74s6z8za6lm-python3.8-urllib3-1.26.3/lib/python3.8/site-packages (from requests>=2.9.1->b2sdk<1.3.0,>=1.2.0->b2==2.1.0) (1.26.3)
Requirement already satisfied: chardet<5,>=3.0.2 in /nix/store/ywgq28kqk3jr1ax70cb4js8ziyccqv3q-python3.8-chardet-3.0.4/lib/python3.8/site-packages (from requests>=2.9.1->b2sdk<1.3.0,>=1.2.0->b2==2.1.0) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /nix/store/254vcd1660z23qh7r3vcdc5wlzb4n6dp-python3.8-certifi-2020.12.5/lib/python3.8/site-packages (from requests>=2.9.1->b2sdk<1.3.0,>=1.2.0->b2==2.1.0) (2020.12.5)
Requirement already satisfied: idna<3,>=2.5 in /nix/store/9hnaj1ja3vvlqzsqw15lbkd04zran5mh-python3.8-idna-2.10/lib/python3.8/site-packages (from requests>=2.9.1->b2sdk<1.3.0,>=1.2.0->b2==2.1.0) (2.10)
Installing collected packages: b2
Successfully installed b2-2.1.0
/build/source
Finished executing pipInstallPhase
@nix { "action": "setPhase", "phase": "fixupPhase" }
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/08gh4zksjxqpwwmybaw25gnzkazp17hl-backblaze-b2-2.1.0
strip is /nix/store/cp1sa3xxvl71cypiinw2c62i5s33chlr-binutils-2.35.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/08gh4zksjxqpwwmybaw25gnzkazp17hl-backblaze-b2-2.1.0/lib /nix/store/08gh4zksjxqpwwmybaw25gnzkazp17hl-backblaze-b2-2.1.0/bin
patching script interpreter paths in /nix/store/08gh4zksjxqpwwmybaw25gnzkazp17hl-backblaze-b2-2.1.0
checking for references to /build/ in /nix/store/08gh4zksjxqpwwmybaw25gnzkazp17hl-backblaze-b2-2.1.0...
Rewriting #!/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/bin/python3.8 to #!/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8
wrapping `/nix/store/08gh4zksjxqpwwmybaw25gnzkazp17hl-backblaze-b2-2.1.0/bin/backblaze-b2'...
Executing pythonRemoveTestsDir
Finished executing pythonRemoveTestsDir
@nix { "action": "setPhase", "phase": "installCheckPhase" }
running install tests
no Makefile or custom installCheckPhase, doing nothing
@nix { "action": "setPhase", "phase": "pythonCatchConflictsPhase" }
pythonCatchConflictsPhase
@nix { "action": "setPhase", "phase": "pythonRemoveBinBytecodePhase" }
pythonRemoveBinBytecodePhase
@nix { "action": "setPhase", "phase": "pythonImportsCheckPhase" }
pythonImportsCheckPhase
Executing pythonImportsCheckPhase
@nix { "action": "setPhase", "phase": "pytestCheckPhase" }
pytestCheckPhase
Executing pytestCheckPhase
============================= test session starts ==============================
platform linux -- Python 3.8.8, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /build/source
collecting ...
collecting 5 items
collected 52 items / 2 deselected / 50 selected
test/unit/test_arg_parser.py .F. [ 6%]
test/unit/test_console_tool.py ......................................... [ 88%]
...FF. [100%]
=================================== FAILURES ===================================
__________ TestCustomArgTypes.test_parse_millis_from_float_timestamp ___________
self = <test.unit.test_arg_parser.TestCustomArgTypes testMethod=test_parse_millis_from_float_timestamp>
def test_parse_millis_from_float_timestamp(self):
> self.assertEqual(1367900664000, parse_millis_from_float_timestamp('1367900664'))
test/unit/test_arg_parser.py:25:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '1367900664'
def parse_millis_from_float_timestamp(s):
"""
Parse timestamp, e.g. 1367900664 or 1367900664.152
"""
> return int(arrow.get(float(s)).format('XSSS'))
E ValueError: invalid literal for int() with base 10: '1367900664.0000'
b2/arg_parser.py:83: ValueError
__________ TestConsoleTool.test_sync_exclude_if_modified_after_exact ___________
self = ArgumentParser(prog='b2 sync', usage=None, description='\n Copies multiple files from source to destination. Optiona...r uploading)\n', formatter_class=<class 'b2.arg_parser.RawTextHelpFormatter'>, conflict_handler='error', add_help=True)
action = _StoreAction(option_strings=['--excludeIfModifiedAfter'], dest='excludeIfModifiedAfter', nargs=None, const=None, defau...one, type=<function parse_millis_from_float_timestamp at 0x7ffff68b8820>, choices=None, help=None, metavar='TIMESTAMP')
arg_string = '1367600664.152'
def _get_value(self, action, arg_string):
type_func = self._registry_get('type', action.type, action.type)
if not callable(type_func):
msg = _('%r is not callable')
raise ArgumentError(action, msg % type_func)
# convert the value to the appropriate type
try:
> result = type_func(arg_string)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:2422:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '1367600664.152'
def parse_millis_from_float_timestamp(s):
"""
Parse timestamp, e.g. 1367900664 or 1367900664.152
"""
> return int(arrow.get(float(s)).format('XSSS'))
E ValueError: invalid literal for int() with base 10: '1367600664.152152'
b2/arg_parser.py:83: ValueError
During handling of the above exception, another exception occurred:
self = ArgumentParser(prog='b2 sync', usage=None, description='\n Copies multiple files from source to destination. Optiona...r uploading)\n', formatter_class=<class 'b2.arg_parser.RawTextHelpFormatter'>, conflict_handler='error', add_help=True)
args = ['--threads', '1', '--noProgress', '--excludeIfModifiedAfter', '1367600664.152', '/build/tmpaq6grdce', ...]
namespace = Namespace(allowEmptySource=False, compareThreshold=None, compareVersions='modTime', debugLogs=False, delete=False, des...Days=None, logConfig=None, noProgress=True, replaceNewer=False, skipNewer=False, source=None, threads=1, verbose=False)
def parse_known_args(self, args=None, namespace=None):
if args is None:
# args default to the system args
args = _sys.argv[1:]
else:
# make sure that args are mutable
args = list(args)
# default Namespace built from parser defaults
if namespace is None:
namespace = Namespace()
# add any action defaults that aren't present
for action in self._actions:
if action.dest is not SUPPRESS:
if not hasattr(namespace, action.dest):
if action.default is not SUPPRESS:
setattr(namespace, action.dest, action.default)
# add any parser defaults that aren't present
for dest in self._defaults:
if not hasattr(namespace, dest):
setattr(namespace, dest, self._defaults[dest])
# parse the arguments and exit if there are any errors
try:
> namespace, args = self._parse_known_args(args, namespace)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1800:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = ArgumentParser(prog='b2 sync', usage=None, description='\n Copies multiple files from source to destination. Optiona...r uploading)\n', formatter_class=<class 'b2.arg_parser.RawTextHelpFormatter'>, conflict_handler='error', add_help=True)
arg_strings = ['--threads', '1', '--noProgress', '--excludeIfModifiedAfter', '1367600664.152', '/build/tmpaq6grdce', ...]
namespace = Namespace(allowEmptySource=False, compareThreshold=None, compareVersions='modTime', debugLogs=False, delete=False, des...Days=None, logConfig=None, noProgress=True, replaceNewer=False, skipNewer=False, source=None, threads=1, verbose=False)
def _parse_known_args(self, arg_strings, namespace):
# replace arg strings that are file references
if self.fromfile_prefix_chars is not None:
arg_strings = self._read_args_from_files(arg_strings)
# map all mutually exclusive arguments to the other arguments
# they can't occur with
action_conflicts = {}
for mutex_group in self._mutually_exclusive_groups:
group_actions = mutex_group._group_actions
for i, mutex_action in enumerate(mutex_group._group_actions):
conflicts = action_conflicts.setdefault(mutex_action, [])
conflicts.extend(group_actions[:i])
conflicts.extend(group_actions[i + 1:])
# find all option indices, and determine the arg_string_pattern
# which has an 'O' if there is an option at an index,
# an 'A' if there is an argument, or a '-' if there is a '--'
option_string_indices = {}
arg_string_pattern_parts = []
arg_strings_iter = iter(arg_strings)
for i, arg_string in enumerate(arg_strings_iter):
# all args after -- are non-options
if arg_string == '--':
arg_string_pattern_parts.append('-')
for arg_string in arg_strings_iter:
arg_string_pattern_parts.append('A')
# otherwise, add the arg to the arg strings
# and note the index if it was an option
else:
option_tuple = self._parse_optional(arg_string)
if option_tuple is None:
pattern = 'A'
else:
option_string_indices[i] = option_tuple
pattern = 'O'
arg_string_pattern_parts.append(pattern)
# join the pieces together to form the pattern
arg_strings_pattern = ''.join(arg_string_pattern_parts)
# converts arg strings to the appropriate and then takes the action
seen_actions = set()
seen_non_default_actions = set()
def take_action(action, argument_strings, option_string=None):
seen_actions.add(action)
argument_values = self._get_values(action, argument_strings)
# error if this argument is not allowed with other previously
# seen arguments, assuming that actions that use the default
# value don't really count as "present"
if argument_values is not action.default:
seen_non_default_actions.add(action)
for conflict_action in action_conflicts.get(action, []):
if conflict_action in seen_non_default_actions:
msg = _('not allowed with argument %s')
action_name = _get_action_name(conflict_action)
raise ArgumentError(action, msg % action_name)
# take the action if we didn't receive a SUPPRESS value
# (e.g. from a default)
if argument_values is not SUPPRESS:
action(self, namespace, argument_values, option_string)
# function to convert arg_strings into an optional action
def consume_optional(start_index):
# get the optional identified at this index
option_tuple = option_string_indices[start_index]
action, option_string, explicit_arg = option_tuple
# identify additional optionals in the same arg string
# (e.g. -xyz is the same as -x -y -z if no args are required)
match_argument = self._match_argument
action_tuples = []
while True:
# if we found no optional action, skip it
if action is None:
extras.append(arg_strings[start_index])
return start_index + 1
# if there is an explicit argument, try to match the
# optional's string arguments to only this
if explicit_arg is not None:
arg_count = match_argument(action, 'A')
# if the action is a single-dash option and takes no
# arguments, try to parse more single-dash options out
# of the tail of the option string
chars = self.prefix_chars
if arg_count == 0 and option_string[1] not in chars:
action_tuples.append((action, [], option_string))
char = option_string[0]
option_string = char + explicit_arg[0]
new_explicit_arg = explicit_arg[1:] or None
optionals_map = self._option_string_actions
if option_string in optionals_map:
action = optionals_map[option_string]
explicit_arg = new_explicit_arg
else:
msg = _('ignored explicit argument %r')
raise ArgumentError(action, msg % explicit_arg)
# if the action expect exactly one argument, we've
# successfully matched the option; exit the loop
elif arg_count == 1:
stop = start_index + 1
args = [explicit_arg]
action_tuples.append((action, args, option_string))
break
# error if a double-dash option did not use the
# explicit argument
else:
msg = _('ignored explicit argument %r')
raise ArgumentError(action, msg % explicit_arg)
# if there is no explicit argument, try to match the
# optional's string arguments with the following strings
# if successful, exit the loop
else:
start = start_index + 1
selected_patterns = arg_strings_pattern[start:]
arg_count = match_argument(action, selected_patterns)
stop = start + arg_count
args = arg_strings[start:stop]
action_tuples.append((action, args, option_string))
break
# add the Optional to the list and return the index at which
# the Optional's string args stopped
assert action_tuples
for action, args, option_string in action_tuples:
take_action(action, args, option_string)
return stop
# the list of Positionals left to be parsed; this is modified
# by consume_positionals()
positionals = self._get_positional_actions()
# function to convert arg_strings into positional actions
def consume_positionals(start_index):
# match as many Positionals as possible
match_partial = self._match_arguments_partial
selected_pattern = arg_strings_pattern[start_index:]
arg_counts = match_partial(positionals, selected_pattern)
# slice off the appropriate arg strings for each Positional
# and add the Positional and its args to the list
for action, arg_count in zip(positionals, arg_counts):
args = arg_strings[start_index: start_index + arg_count]
start_index += arg_count
take_action(action, args)
# slice off the Positionals that we just parsed and return the
# index at which the Positionals' string args stopped
positionals[:] = positionals[len(arg_counts):]
return start_index
# consume Positionals and Optionals alternately, until we have
# passed the last option string
extras = []
start_index = 0
if option_string_indices:
max_option_string_index = max(option_string_indices)
else:
max_option_string_index = -1
while start_index <= max_option_string_index:
# consume any Positionals preceding the next option
next_option_string_index = min([
index
for index in option_string_indices
if index >= start_index])
if start_index != next_option_string_index:
positionals_end_index = consume_positionals(start_index)
# only try to parse the next optional if we didn't consume
# the option string during the positionals parsing
if positionals_end_index > start_index:
start_index = positionals_end_index
continue
else:
start_index = positionals_end_index
# if we consumed all the positionals we could and we're not
# at the index of an option string, there were extra arguments
if start_index not in option_string_indices:
strings = arg_strings[start_index:next_option_string_index]
extras.extend(strings)
start_index = next_option_string_index
# consume the next optional and any arguments for it
> start_index = consume_optional(start_index)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:2006:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
start_index = 3
def consume_optional(start_index):
# get the optional identified at this index
option_tuple = option_string_indices[start_index]
action, option_string, explicit_arg = option_tuple
# identify additional optionals in the same arg string
# (e.g. -xyz is the same as -x -y -z if no args are required)
match_argument = self._match_argument
action_tuples = []
while True:
# if we found no optional action, skip it
if action is None:
extras.append(arg_strings[start_index])
return start_index + 1
# if there is an explicit argument, try to match the
# optional's string arguments to only this
if explicit_arg is not None:
arg_count = match_argument(action, 'A')
# if the action is a single-dash option and takes no
# arguments, try to parse more single-dash options out
# of the tail of the option string
chars = self.prefix_chars
if arg_count == 0 and option_string[1] not in chars:
action_tuples.append((action, [], option_string))
char = option_string[0]
option_string = char + explicit_arg[0]
new_explicit_arg = explicit_arg[1:] or None
optionals_map = self._option_string_actions
if option_string in optionals_map:
action = optionals_map[option_string]
explicit_arg = new_explicit_arg
else:
msg = _('ignored explicit argument %r')
raise ArgumentError(action, msg % explicit_arg)
# if the action expect exactly one argument, we've
# successfully matched the option; exit the loop
elif arg_count == 1:
stop = start_index + 1
args = [explicit_arg]
action_tuples.append((action, args, option_string))
break
# error if a double-dash option did not use the
# explicit argument
else:
msg = _('ignored explicit argument %r')
raise ArgumentError(action, msg % explicit_arg)
# if there is no explicit argument, try to match the
# optional's string arguments with the following strings
# if successful, exit the loop
else:
start = start_index + 1
selected_patterns = arg_strings_pattern[start:]
arg_count = match_argument(action, selected_patterns)
stop = start + arg_count
args = arg_strings[start:stop]
action_tuples.append((action, args, option_string))
break
# add the Optional to the list and return the index at which
# the Optional's string args stopped
assert action_tuples
for action, args, option_string in action_tuples:
> take_action(action, args, option_string)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1946:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
action = _StoreAction(option_strings=['--excludeIfModifiedAfter'], dest='excludeIfModifiedAfter', nargs=None, const=None, defau...one, type=<function parse_millis_from_float_timestamp at 0x7ffff68b8820>, choices=None, help=None, metavar='TIMESTAMP')
argument_strings = ['1367600664.152']
option_string = '--excludeIfModifiedAfter'
def take_action(action, argument_strings, option_string=None):
seen_actions.add(action)
> argument_values = self._get_values(action, argument_strings)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1858:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = ArgumentParser(prog='b2 sync', usage=None, description='\n Copies multiple files from source to destination. Optiona...r uploading)\n', formatter_class=<class 'b2.arg_parser.RawTextHelpFormatter'>, conflict_handler='error', add_help=True)
action = _StoreAction(option_strings=['--excludeIfModifiedAfter'], dest='excludeIfModifiedAfter', nargs=None, const=None, defau...one, type=<function parse_millis_from_float_timestamp at 0x7ffff68b8820>, choices=None, help=None, metavar='TIMESTAMP')
arg_strings = ['1367600664.152']
def _get_values(self, action, arg_strings):
# for everything but PARSER, REMAINDER args, strip out first '--'
if action.nargs not in [PARSER, REMAINDER]:
try:
arg_strings.remove('--')
except ValueError:
pass
# optional argument produces a default when not present
if not arg_strings and action.nargs == OPTIONAL:
if action.option_strings:
value = action.const
else:
value = action.default
if isinstance(value, str):
value = self._get_value(action, value)
self._check_value(action, value)
# when nargs='*' on a positional, if there were no command-line
# args, use the default if it is anything other than None
elif (not arg_strings and action.nargs == ZERO_OR_MORE and
not action.option_strings):
if action.default is not None:
value = action.default
else:
value = arg_strings
self._check_value(action, value)
# single argument or optional argument produces a single value
elif len(arg_strings) == 1 and action.nargs in [None, OPTIONAL]:
arg_string, = arg_strings
> value = self._get_value(action, arg_string)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:2389:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = ArgumentParser(prog='b2 sync', usage=None, description='\n Copies multiple files from source to destination. Optiona...r uploading)\n', formatter_class=<class 'b2.arg_parser.RawTextHelpFormatter'>, conflict_handler='error', add_help=True)
action = _StoreAction(option_strings=['--excludeIfModifiedAfter'], dest='excludeIfModifiedAfter', nargs=None, const=None, defau...one, type=<function parse_millis_from_float_timestamp at 0x7ffff68b8820>, choices=None, help=None, metavar='TIMESTAMP')
arg_string = '1367600664.152'
def _get_value(self, action, arg_string):
type_func = self._registry_get('type', action.type, action.type)
if not callable(type_func):
msg = _('%r is not callable')
raise ArgumentError(action, msg % type_func)
# convert the value to the appropriate type
try:
result = type_func(arg_string)
# ArgumentTypeErrors indicate errors
except ArgumentTypeError:
name = getattr(action.type, '__name__', repr(action.type))
msg = str(_sys.exc_info()[1])
raise ArgumentError(action, msg)
# TypeErrors or ValueErrors also indicate errors
except (TypeError, ValueError):
name = getattr(action.type, '__name__', repr(action.type))
args = {'type': name, 'value': arg_string}
msg = _('invalid %(type)s value: %(value)r')
> raise ArgumentError(action, msg % args)
E argparse.ArgumentError: argument --excludeIfModifiedAfter: invalid parse_millis_from_float_timestamp value: '1367600664.152'
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:2435: ArgumentError
During handling of the above exception, another exception occurred:
self = <test.unit.test_console_tool.TestConsoleTool testMethod=test_sync_exclude_if_modified_after_exact>
def test_sync_exclude_if_modified_after_exact(self):
self._authorize_account()
self._create_my_bucket()
with TempDir() as temp_dir:
for file, mtime in (('test.txt', 1367900664.152), ('test2.txt', 1367600664.152)):
self._make_local_file(temp_dir, file)
path = os.path.join(temp_dir, file)
os.utime(path, (mtime, mtime))
expected_stdout = '''
upload test2.txt
'''
command = [
'sync', '--threads', '1', '--noProgress', '--excludeIfModifiedAfter',
'1367600664.152', temp_dir, 'b2://my-bucket'
]
> self._run_command(command, expected_stdout, '', 0)
test/unit/test_console_tool.py:1330:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test/unit/test_console_tool.py:1559: in _run_command
actual_status = console_tool.run_command(['b2'] + argv)
b2/console_tool.py:1513: in run_command
args = b2_command.get_parser().parse_args(argv[1:])
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1768: in parse_args
args, argv = self.parse_known_args(args, namespace)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1800: in parse_known_args
namespace, args = self._parse_known_args(args, namespace)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1988: in _parse_known_args
positionals_end_index = consume_positionals(start_index)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1965: in consume_positionals
take_action(action, args)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1874: in take_action
action(self, namespace, argument_values, option_string)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1159: in __call__
subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1807: in parse_known_args
self.error(str(err))
b2/arg_parser.py:65: in error
self.exit(2, '\n%(prog)s: error: %(message)s\n' % args)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = ArgumentParser(prog='b2 sync', usage=None, description='\n Copies multiple files from source to destination. Optiona...r uploading)\n', formatter_class=<class 'b2.arg_parser.RawTextHelpFormatter'>, conflict_handler='error', add_help=True)
status = 2
message = "\nb2 sync: error: argument --excludeIfModifiedAfter: invalid parse_millis_from_float_timestamp value: '1367600664.152'\n"
def exit(self, status=0, message=None):
if message:
self._print_message(message, _sys.stderr)
> _sys.exit(status)
E SystemExit: 2
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:2508: SystemExit
----------------------------- Captured stdout call -----------------------------
b2 sync [-h] [--noProgress] [--dryRun] [--allowEmptySource]
[--excludeAllSymlinks] [--threads THREADS]
[--compareVersions {none,modTime,size}] [--compareThreshold MILLIS]
[--excludeRegex REGEX] [--includeRegex REGEX]
[--excludeDirRegex REGEX] [--excludeIfModifiedAfter TIMESTAMP]
[--skipNewer | --replaceNewer] [--delete | --keepDays DAYS]
source destination
Copies multiple files from source to destination. Optionally
deletes or hides destination files that the source does not have.
The synchronizer can copy files:
- From a B2 bucket to a local destination.
- From a local source to a B2 bucket.
- From one B2 bucket to another.
- Between different folders in the same B2 bucket.
Use "b2://<bucketName>/<prefix>" for B2 paths, e.g. "b2://my-bucket-name/a/path/prefix/".
Progress is displayed on the console unless '--noProgress' is
specified. A list of actions taken is always printed.
Specify '--dryRun' to simulate the actions that would be taken.
To allow sync to run when the source directory is empty, potentially
deleting all files in a bucket, specify '--allowEmptySource'.
The default is to fail when the specified source directory doesn't exist
or is empty. (This check only applies to version 1.0 and later.)
Users with high-performance networks, or file sets with very small
files, will benefit from multi-threaded uploads. The default number
of threads is 10. Experiment with the --threads parameter if the
default is not working well.
Users with low-performance networks may benefit from reducing the
number of threads. Using just one thread will minimize the impact
on other users of the network.
Note that using multiple threads will usually be detrimental to
the other users on your network.
You can specify --excludeRegex to selectively ignore files that
match the given pattern. Ignored files will not copy during
the sync operation. The pattern is a regular expression
that is tested against the full path of each file.
You can specify --includeRegex to selectively override ignoring
files that match the given --excludeRegex pattern by an
--includeRegex pattern. Similarly to --excludeRegex, the pattern
is a regular expression that is tested against the full path
of each file.
Note that --includeRegex cannot be used without --excludeRegex.
You can specify --excludeAllSymlinks to skip symlinks when
syncing from a local source.
When a directory is excluded by using --excludeDirRegex, all of
the files within it are excluded, even if they match an --includeRegex
pattern. This means that there is no need to look inside excluded
directories, and you can exclude directories containing files for which
you don't have read permission and avoid getting errors.
The --excludeDirRegex is a regular expression that is tested against
the full path of each directory. The path being matched does not have
a trailing '/', so don't include on in your regular expression.
Multiple regex rules can be applied by supplying them as pipe
delimited instructions. Note that the regex for this command
is Python regex. Reference: https://docs.python.org/2/library/re.html.
Regular expressions are considered a match if they match a substring
starting at the first character. ".*e" will match "hello". This is
not ideal, but we will maintain this behavior for compatibility.
If you want to match the entire path, put a "$" at the end of the
regex, such as ".*llo$".
You can specify --excludeIfModifiedAfter to selectively ignore file versions
(including hide markers) which were synced after given time (for local source)
or ignore only specific file versions (for b2 source).
Ignored files or file versions will not be taken for consideration during sync.
The time should be given as a seconds timestamp (e.g. "1367900664")
If you need milliseconds precision, put it after the comma (e.g. "1367900664.152")
Files are considered to be the same if they have the same name
and modification time. This behaviour can be changed using the
--compareVersions option. Possible values are:
'none': Comparison using the file name only
'modTime': Comparison using the modification time (default)
'size': Comparison using the file size
A future enhancement may add the ability to compare the SHA1 checksum
of the files.
Fuzzy comparison of files based on modTime or size can be enabled by
specifying the --compareThreshold option. This will treat modTimes
(in milliseconds) or sizes (in bytes) as the same if they are within
the comparison threshold. Files that match, within the threshold, will
not be synced. Specifying --verbose and --dryRun can be useful to
determine comparison value differences.
When a destination file is present that is not in the source, the
default is to leave it there. Specifying --delete means to delete
destination files that are not in the source.
When the destination is B2, you have the option of leaving older
versions in place. Specifying --keepDays will delete any older
versions more than the given number of days old, based on the
modification time of the file. This option is not available when
the destination is a local folder.
Files at the source that have a newer modification time are always
copied to the destination. If the destination file is newer, the
default is to report an error and stop. But with --skipNewer set,
those files will just be skipped. With --replaceNewer set, the
old file from the source will replace the newer one in the destination.
To make the destination exactly match the source, use:
b2 sync --delete --replaceNewer ... ...
WARNING: Using '--delete' deletes files! We recommend not using it.
If you use --keepDays instead, you will have some time to recover your
files if you discover they are missing on the source end.
To make the destination match the source, but retain previous versions
for 30 days:
b2 sync --keepDays 30 --replaceNewer ... b2://...
Example of sync being used with excludeRegex. This will ignore .DS_Store files
and .Spotlight-V100 folders
b2 sync -excludeRegex '(.*\.DS_Store)|(.*\.Spotlight-V100)' ... b2://...
Requires capabilities: listFiles, readFiles (for downloading), writeFiles (for uploading)
positional arguments:
source
destination
optional arguments:
-h, --help show this help message and exit
--noProgress
--dryRun
--allowEmptySource
--excludeAllSymlinks
--threads THREADS
--compareVersions {none,modTime,size}
--compareThreshold MILLIS
--excludeRegex REGEX
--includeRegex REGEX
--excludeDirRegex REGEX
--excludeIfModifiedAfter TIMESTAMP
--skipNewer
--replaceNewer
--delete
--keepDays DAYS
----------------------------- Captured stderr call -----------------------------
b2 sync: error: argument --excludeIfModifiedAfter: invalid parse_millis_from_float_timestamp value: '1367600664.152'
_________ TestConsoleTool.test_sync_exclude_if_modified_after_in_range _________
self = ArgumentParser(prog='b2 sync', usage=None, description='\n Copies multiple files from source to destination. Optiona...r uploading)\n', formatter_class=<class 'b2.arg_parser.RawTextHelpFormatter'>, conflict_handler='error', add_help=True)
action = _StoreAction(option_strings=['--excludeIfModifiedAfter'], dest='excludeIfModifiedAfter', nargs=None, const=None, defau...one, type=<function parse_millis_from_float_timestamp at 0x7ffff68b8820>, choices=None, help=None, metavar='TIMESTAMP')
arg_string = '1367700664.152'
def _get_value(self, action, arg_string):
type_func = self._registry_get('type', action.type, action.type)
if not callable(type_func):
msg = _('%r is not callable')
raise ArgumentError(action, msg % type_func)
# convert the value to the appropriate type
try:
> result = type_func(arg_string)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:2422:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '1367700664.152'
def parse_millis_from_float_timestamp(s):
"""
Parse timestamp, e.g. 1367900664 or 1367900664.152
"""
> return int(arrow.get(float(s)).format('XSSS'))
E ValueError: invalid literal for int() with base 10: '1367700664.152152'
b2/arg_parser.py:83: ValueError
During handling of the above exception, another exception occurred:
self = ArgumentParser(prog='b2 sync', usage=None, description='\n Copies multiple files from source to destination. Optiona...r uploading)\n', formatter_class=<class 'b2.arg_parser.RawTextHelpFormatter'>, conflict_handler='error', add_help=True)
args = ['--threads', '1', '--noProgress', '--excludeIfModifiedAfter', '1367700664.152', '/build/tmpfba8sec3', ...]
namespace = Namespace(allowEmptySource=False, compareThreshold=None, compareVersions='modTime', debugLogs=False, delete=False, des...Days=None, logConfig=None, noProgress=True, replaceNewer=False, skipNewer=False, source=None, threads=1, verbose=False)
def parse_known_args(self, args=None, namespace=None):
if args is None:
# args default to the system args
args = _sys.argv[1:]
else:
# make sure that args are mutable
args = list(args)
# default Namespace built from parser defaults
if namespace is None:
namespace = Namespace()
# add any action defaults that aren't present
for action in self._actions:
if action.dest is not SUPPRESS:
if not hasattr(namespace, action.dest):
if action.default is not SUPPRESS:
setattr(namespace, action.dest, action.default)
# add any parser defaults that aren't present
for dest in self._defaults:
if not hasattr(namespace, dest):
setattr(namespace, dest, self._defaults[dest])
# parse the arguments and exit if there are any errors
try:
> namespace, args = self._parse_known_args(args, namespace)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1800:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = ArgumentParser(prog='b2 sync', usage=None, description='\n Copies multiple files from source to destination. Optiona...r uploading)\n', formatter_class=<class 'b2.arg_parser.RawTextHelpFormatter'>, conflict_handler='error', add_help=True)
arg_strings = ['--threads', '1', '--noProgress', '--excludeIfModifiedAfter', '1367700664.152', '/build/tmpfba8sec3', ...]
namespace = Namespace(allowEmptySource=False, compareThreshold=None, compareVersions='modTime', debugLogs=False, delete=False, des...Days=None, logConfig=None, noProgress=True, replaceNewer=False, skipNewer=False, source=None, threads=1, verbose=False)
def _parse_known_args(self, arg_strings, namespace):
# replace arg strings that are file references
if self.fromfile_prefix_chars is not None:
arg_strings = self._read_args_from_files(arg_strings)
# map all mutually exclusive arguments to the other arguments
# they can't occur with
action_conflicts = {}
for mutex_group in self._mutually_exclusive_groups:
group_actions = mutex_group._group_actions
for i, mutex_action in enumerate(mutex_group._group_actions):
conflicts = action_conflicts.setdefault(mutex_action, [])
conflicts.extend(group_actions[:i])
conflicts.extend(group_actions[i + 1:])
# find all option indices, and determine the arg_string_pattern
# which has an 'O' if there is an option at an index,
# an 'A' if there is an argument, or a '-' if there is a '--'
option_string_indices = {}
arg_string_pattern_parts = []
arg_strings_iter = iter(arg_strings)
for i, arg_string in enumerate(arg_strings_iter):
# all args after -- are non-options
if arg_string == '--':
arg_string_pattern_parts.append('-')
for arg_string in arg_strings_iter:
arg_string_pattern_parts.append('A')
# otherwise, add the arg to the arg strings
# and note the index if it was an option
else:
option_tuple = self._parse_optional(arg_string)
if option_tuple is None:
pattern = 'A'
else:
option_string_indices[i] = option_tuple
pattern = 'O'
arg_string_pattern_parts.append(pattern)
# join the pieces together to form the pattern
arg_strings_pattern = ''.join(arg_string_pattern_parts)
# converts arg strings to the appropriate and then takes the action
seen_actions = set()
seen_non_default_actions = set()
def take_action(action, argument_strings, option_string=None):
seen_actions.add(action)
argument_values = self._get_values(action, argument_strings)
# error if this argument is not allowed with other previously
# seen arguments, assuming that actions that use the default
# value don't really count as "present"
if argument_values is not action.default:
seen_non_default_actions.add(action)
for conflict_action in action_conflicts.get(action, []):
if conflict_action in seen_non_default_actions:
msg = _('not allowed with argument %s')
action_name = _get_action_name(conflict_action)
raise ArgumentError(action, msg % action_name)
# take the action if we didn't receive a SUPPRESS value
# (e.g. from a default)
if argument_values is not SUPPRESS:
action(self, namespace, argument_values, option_string)
# function to convert arg_strings into an optional action
def consume_optional(start_index):
# get the optional identified at this index
option_tuple = option_string_indices[start_index]
action, option_string, explicit_arg = option_tuple
# identify additional optionals in the same arg string
# (e.g. -xyz is the same as -x -y -z if no args are required)
match_argument = self._match_argument
action_tuples = []
while True:
# if we found no optional action, skip it
if action is None:
extras.append(arg_strings[start_index])
return start_index + 1
# if there is an explicit argument, try to match the
# optional's string arguments to only this
if explicit_arg is not None:
arg_count = match_argument(action, 'A')
# if the action is a single-dash option and takes no
# arguments, try to parse more single-dash options out
# of the tail of the option string
chars = self.prefix_chars
if arg_count == 0 and option_string[1] not in chars:
action_tuples.append((action, [], option_string))
char = option_string[0]
option_string = char + explicit_arg[0]
new_explicit_arg = explicit_arg[1:] or None
optionals_map = self._option_string_actions
if option_string in optionals_map:
action = optionals_map[option_string]
explicit_arg = new_explicit_arg
else:
msg = _('ignored explicit argument %r')
raise ArgumentError(action, msg % explicit_arg)
# if the action expect exactly one argument, we've
# successfully matched the option; exit the loop
elif arg_count == 1:
stop = start_index + 1
args = [explicit_arg]
action_tuples.append((action, args, option_string))
break
# error if a double-dash option did not use the
# explicit argument
else:
msg = _('ignored explicit argument %r')
raise ArgumentError(action, msg % explicit_arg)
# if there is no explicit argument, try to match the
# optional's string arguments with the following strings
# if successful, exit the loop
else:
start = start_index + 1
selected_patterns = arg_strings_pattern[start:]
arg_count = match_argument(action, selected_patterns)
stop = start + arg_count
args = arg_strings[start:stop]
action_tuples.append((action, args, option_string))
break
# add the Optional to the list and return the index at which
# the Optional's string args stopped
assert action_tuples
for action, args, option_string in action_tuples:
take_action(action, args, option_string)
return stop
# the list of Positionals left to be parsed; this is modified
# by consume_positionals()
positionals = self._get_positional_actions()
# function to convert arg_strings into positional actions
def consume_positionals(start_index):
# match as many Positionals as possible
match_partial = self._match_arguments_partial
selected_pattern = arg_strings_pattern[start_index:]
arg_counts = match_partial(positionals, selected_pattern)
# slice off the appropriate arg strings for each Positional
# and add the Positional and its args to the list
for action, arg_count in zip(positionals, arg_counts):
args = arg_strings[start_index: start_index + arg_count]
start_index += arg_count
take_action(action, args)
# slice off the Positionals that we just parsed and return the
# index at which the Positionals' string args stopped
positionals[:] = positionals[len(arg_counts):]
return start_index
# consume Positionals and Optionals alternately, until we have
# passed the last option string
extras = []
start_index = 0
if option_string_indices:
max_option_string_index = max(option_string_indices)
else:
max_option_string_index = -1
while start_index <= max_option_string_index:
# consume any Positionals preceding the next option
next_option_string_index = min([
index
for index in option_string_indices
if index >= start_index])
if start_index != next_option_string_index:
positionals_end_index = consume_positionals(start_index)
# only try to parse the next optional if we didn't consume
# the option string during the positionals parsing
if positionals_end_index > start_index:
start_index = positionals_end_index
continue
else:
start_index = positionals_end_index
# if we consumed all the positionals we could and we're not
# at the index of an option string, there were extra arguments
if start_index not in option_string_indices:
strings = arg_strings[start_index:next_option_string_index]
extras.extend(strings)
start_index = next_option_string_index
# consume the next optional and any arguments for it
> start_index = consume_optional(start_index)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:2006:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
start_index = 3
def consume_optional(start_index):
# get the optional identified at this index
option_tuple = option_string_indices[start_index]
action, option_string, explicit_arg = option_tuple
# identify additional optionals in the same arg string
# (e.g. -xyz is the same as -x -y -z if no args are required)
match_argument = self._match_argument
action_tuples = []
while True:
# if we found no optional action, skip it
if action is None:
extras.append(arg_strings[start_index])
return start_index + 1
# if there is an explicit argument, try to match the
# optional's string arguments to only this
if explicit_arg is not None:
arg_count = match_argument(action, 'A')
# if the action is a single-dash option and takes no
# arguments, try to parse more single-dash options out
# of the tail of the option string
chars = self.prefix_chars
if arg_count == 0 and option_string[1] not in chars:
action_tuples.append((action, [], option_string))
char = option_string[0]
option_string = char + explicit_arg[0]
new_explicit_arg = explicit_arg[1:] or None
optionals_map = self._option_string_actions
if option_string in optionals_map:
action = optionals_map[option_string]
explicit_arg = new_explicit_arg
else:
msg = _('ignored explicit argument %r')
raise ArgumentError(action, msg % explicit_arg)
# if the action expect exactly one argument, we've
# successfully matched the option; exit the loop
elif arg_count == 1:
stop = start_index + 1
args = [explicit_arg]
action_tuples.append((action, args, option_string))
break
# error if a double-dash option did not use the
# explicit argument
else:
msg = _('ignored explicit argument %r')
raise ArgumentError(action, msg % explicit_arg)
# if there is no explicit argument, try to match the
# optional's string arguments with the following strings
# if successful, exit the loop
else:
start = start_index + 1
selected_patterns = arg_strings_pattern[start:]
arg_count = match_argument(action, selected_patterns)
stop = start + arg_count
args = arg_strings[start:stop]
action_tuples.append((action, args, option_string))
break
# add the Optional to the list and return the index at which
# the Optional's string args stopped
assert action_tuples
for action, args, option_string in action_tuples:
> take_action(action, args, option_string)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1946:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
action = _StoreAction(option_strings=['--excludeIfModifiedAfter'], dest='excludeIfModifiedAfter', nargs=None, const=None, defau...one, type=<function parse_millis_from_float_timestamp at 0x7ffff68b8820>, choices=None, help=None, metavar='TIMESTAMP')
argument_strings = ['1367700664.152']
option_string = '--excludeIfModifiedAfter'
def take_action(action, argument_strings, option_string=None):
seen_actions.add(action)
> argument_values = self._get_values(action, argument_strings)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1858:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = ArgumentParser(prog='b2 sync', usage=None, description='\n Copies multiple files from source to destination. Optiona...r uploading)\n', formatter_class=<class 'b2.arg_parser.RawTextHelpFormatter'>, conflict_handler='error', add_help=True)
action = _StoreAction(option_strings=['--excludeIfModifiedAfter'], dest='excludeIfModifiedAfter', nargs=None, const=None, defau...one, type=<function parse_millis_from_float_timestamp at 0x7ffff68b8820>, choices=None, help=None, metavar='TIMESTAMP')
arg_strings = ['1367700664.152']
def _get_values(self, action, arg_strings):
# for everything but PARSER, REMAINDER args, strip out first '--'
if action.nargs not in [PARSER, REMAINDER]:
try:
arg_strings.remove('--')
except ValueError:
pass
# optional argument produces a default when not present
if not arg_strings and action.nargs == OPTIONAL:
if action.option_strings:
value = action.const
else:
value = action.default
if isinstance(value, str):
value = self._get_value(action, value)
self._check_value(action, value)
# when nargs='*' on a positional, if there were no command-line
# args, use the default if it is anything other than None
elif (not arg_strings and action.nargs == ZERO_OR_MORE and
not action.option_strings):
if action.default is not None:
value = action.default
else:
value = arg_strings
self._check_value(action, value)
# single argument or optional argument produces a single value
elif len(arg_strings) == 1 and action.nargs in [None, OPTIONAL]:
arg_string, = arg_strings
> value = self._get_value(action, arg_string)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:2389:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = ArgumentParser(prog='b2 sync', usage=None, description='\n Copies multiple files from source to destination. Optiona...r uploading)\n', formatter_class=<class 'b2.arg_parser.RawTextHelpFormatter'>, conflict_handler='error', add_help=True)
action = _StoreAction(option_strings=['--excludeIfModifiedAfter'], dest='excludeIfModifiedAfter', nargs=None, const=None, defau...one, type=<function parse_millis_from_float_timestamp at 0x7ffff68b8820>, choices=None, help=None, metavar='TIMESTAMP')
arg_string = '1367700664.152'
def _get_value(self, action, arg_string):
type_func = self._registry_get('type', action.type, action.type)
if not callable(type_func):
msg = _('%r is not callable')
raise ArgumentError(action, msg % type_func)
# convert the value to the appropriate type
try:
result = type_func(arg_string)
# ArgumentTypeErrors indicate errors
except ArgumentTypeError:
name = getattr(action.type, '__name__', repr(action.type))
msg = str(_sys.exc_info()[1])
raise ArgumentError(action, msg)
# TypeErrors or ValueErrors also indicate errors
except (TypeError, ValueError):
name = getattr(action.type, '__name__', repr(action.type))
args = {'type': name, 'value': arg_string}
msg = _('invalid %(type)s value: %(value)r')
> raise ArgumentError(action, msg % args)
E argparse.ArgumentError: argument --excludeIfModifiedAfter: invalid parse_millis_from_float_timestamp value: '1367700664.152'
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:2435: ArgumentError
During handling of the above exception, another exception occurred:
self = <test.unit.test_console_tool.TestConsoleTool testMethod=test_sync_exclude_if_modified_after_in_range>
def test_sync_exclude_if_modified_after_in_range(self):
self._authorize_account()
self._create_my_bucket()
with TempDir() as temp_dir:
for file, mtime in (('test.txt', 1367900664.152), ('test2.txt', 1367600664.152)):
self._make_local_file(temp_dir, file)
path = os.path.join(temp_dir, file)
os.utime(path, (mtime, mtime))
expected_stdout = '''
upload test2.txt
'''
command = [
'sync', '--threads', '1', '--noProgress', '--excludeIfModifiedAfter',
'1367700664.152', temp_dir, 'b2://my-bucket'
]
> self._run_command(command, expected_stdout, '', 0)
test/unit/test_console_tool.py:1310:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test/unit/test_console_tool.py:1559: in _run_command
actual_status = console_tool.run_command(['b2'] + argv)
b2/console_tool.py:1513: in run_command
args = b2_command.get_parser().parse_args(argv[1:])
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1768: in parse_args
args, argv = self.parse_known_args(args, namespace)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1800: in parse_known_args
namespace, args = self._parse_known_args(args, namespace)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1988: in _parse_known_args
positionals_end_index = consume_positionals(start_index)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1965: in consume_positionals
take_action(action, args)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1874: in take_action
action(self, namespace, argument_values, option_string)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1159: in __call__
subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:1807: in parse_known_args
self.error(str(err))
b2/arg_parser.py:65: in error
self.exit(2, '\n%(prog)s: error: %(message)s\n' % args)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = ArgumentParser(prog='b2 sync', usage=None, description='\n Copies multiple files from source to destination. Optiona...r uploading)\n', formatter_class=<class 'b2.arg_parser.RawTextHelpFormatter'>, conflict_handler='error', add_help=True)
status = 2
message = "\nb2 sync: error: argument --excludeIfModifiedAfter: invalid parse_millis_from_float_timestamp value: '1367700664.152'\n"
def exit(self, status=0, message=None):
if message:
self._print_message(message, _sys.stderr)
> _sys.exit(status)
E SystemExit: 2
/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/argparse.py:2508: SystemExit
----------------------------- Captured stdout call -----------------------------
b2 sync [-h] [--noProgress] [--dryRun] [--allowEmptySource]
[--excludeAllSymlinks] [--threads THREADS]
[--compareVersions {none,modTime,size}] [--compareThreshold MILLIS]
[--excludeRegex REGEX] [--includeRegex REGEX]
[--excludeDirRegex REGEX] [--excludeIfModifiedAfter TIMESTAMP]
[--skipNewer | --replaceNewer] [--delete | --keepDays DAYS]
source destination
Copies multiple files from source to destination. Optionally
deletes or hides destination files that the source does not have.
The synchronizer can copy files:
- From a B2 bucket to a local destination.
- From a local source to a B2 bucket.
- From one B2 bucket to another.
- Between different folders in the same B2 bucket.
Use "b2://<bucketName>/<prefix>" for B2 paths, e.g. "b2://my-bucket-name/a/path/prefix/".
Progress is displayed on the console unless '--noProgress' is
specified. A list of actions taken is always printed.
Specify '--dryRun' to simulate the actions that would be taken.
To allow sync to run when the source directory is empty, potentially
deleting all files in a bucket, specify '--allowEmptySource'.
The default is to fail when the specified source directory doesn't exist
or is empty. (This check only applies to version 1.0 and later.)
Users with high-performance networks, or file sets with very small
files, will benefit from multi-threaded uploads. The default number
of threads is 10. Experiment with the --threads parameter if the
default is not working well.
Users with low-performance networks may benefit from reducing the
number of threads. Using just one thread will minimize the impact
on other users of the network.
Note that using multiple threads will usually be detrimental to
the other users on your network.
You can specify --excludeRegex to selectively ignore files that
match the given pattern. Ignored files will not copy during
the sync operation. The pattern is a regular expression
that is tested against the full path of each file.
You can specify --includeRegex to selectively override ignoring
files that match the given --excludeRegex pattern by an
--includeRegex pattern. Similarly to --excludeRegex, the pattern
is a regular expression that is tested against the full path
of each file.
Note that --includeRegex cannot be used without --excludeRegex.
You can specify --excludeAllSymlinks to skip symlinks when
syncing from a local source.
When a directory is excluded by using --excludeDirRegex, all of
the files within it are excluded, even if they match an --includeRegex
pattern. This means that there is no need to look inside excluded
directories, and you can exclude directories containing files for which
you don't have read permission and avoid getting errors.
The --excludeDirRegex is a regular expression that is tested against
the full path of each directory. The path being matched does not have
a trailing '/', so don't include on in your regular expression.
Multiple regex rules can be applied by supplying them as pipe
delimited instructions. Note that the regex for this command
is Python regex. Reference: https://docs.python.org/2/library/re.html.
Regular expressions are considered a match if they match a substring
starting at the first character. ".*e" will match "hello". This is
not ideal, but we will maintain this behavior for compatibility.
If you want to match the entire path, put a "$" at the end of the
regex, such as ".*llo$".
You can specify --excludeIfModifiedAfter to selectively ignore file versions
(including hide markers) which were synced after given time (for local source)
or ignore only specific file versions (for b2 source).
Ignored files or file versions will not be taken for consideration during sync.
The time should be given as a seconds timestamp (e.g. "1367900664")
If you need milliseconds precision, put it after the comma (e.g. "1367900664.152")
Files are considered to be the same if they have the same name
and modification time. This behaviour can be changed using the
--compareVersions option. Possible values are:
'none': Comparison using the file name only
'modTime': Comparison using the modification time (default)
'size': Comparison using the file size
A future enhancement may add the ability to compare the SHA1 checksum
of the files.
Fuzzy comparison of files based on modTime or size can be enabled by
specifying the --compareThreshold option. This will treat modTimes
(in milliseconds) or sizes (in bytes) as the same if they are within
the comparison threshold. Files that match, within the threshold, will
not be synced. Specifying --verbose and --dryRun can be useful to
determine comparison value differences.
When a destination file is present that is not in the source, the
default is to leave it there. Specifying --delete means to delete
destination files that are not in the source.
When the destination is B2, you have the option of leaving older
versions in place. Specifying --keepDays will delete any older
versions more than the given number of days old, based on the
modification time of the file. This option is not available when
the destination is a local folder.
Files at the source that have a newer modification time are always
copied to the destination. If the destination file is newer, the
default is to report an error and stop. But with --skipNewer set,
those files will just be skipped. With --replaceNewer set, the
old file from the source will replace the newer one in the destination.
To make the destination exactly match the source, use:
b2 sync --delete --replaceNewer ... ...
WARNING: Using '--delete' deletes files! We recommend not using it.
If you use --keepDays instead, you will have some time to recover your
files if you discover they are missing on the source end.
To make the destination match the source, but retain previous versions
for 30 days:
b2 sync --keepDays 30 --replaceNewer ... b2://...
Example of sync being used with excludeRegex. This will ignore .DS_Store files
and .Spotlight-V100 folders
b2 sync -excludeRegex '(.*\.DS_Store)|(.*\.Spotlight-V100)' ... b2://...
Requires capabilities: listFiles, readFiles (for downloading), writeFiles (for uploading)
positional arguments:
source
destination
optional arguments:
-h, --help show this help message and exit
--noProgress
--dryRun
--allowEmptySource
--excludeAllSymlinks
--threads THREADS
--compareVersions {none,modTime,size}
--compareThreshold MILLIS
--excludeRegex REGEX
--includeRegex REGEX
--excludeDirRegex REGEX
--excludeIfModifiedAfter TIMESTAMP
--skipNewer
--replaceNewer
--delete
--keepDays DAYS
----------------------------- Captured stderr call -----------------------------
b2 sync: error: argument --excludeIfModifiedAfter: invalid parse_millis_from_float_timestamp value: '1367700664.152'
=============================== warnings summary ===============================
b2/console_tool.py:1143
/build/source/b2/console_tool.py:1143: DeprecationWarning: invalid escape sequence \.
"""
-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED test/unit/test_arg_parser.py::TestCustomArgTypes::test_parse_millis_from_float_timestamp
FAILED test/unit/test_console_tool.py::TestConsoleTool::test_sync_exclude_if_modified_after_exact
FAILED test/unit/test_console_tool.py::TestConsoleTool::test_sync_exclude_if_modified_after_in_range
============ 3 failed, 47 passed, 2 deselected, 1 warning in 8.37s =============
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment