dfarrell07/logs.txt

## logs.txt
2021-07-20T04:04:54.6954582Z Found online and idle hosted runner(s) in the current repository's organization account that matches the required labels: 'ubuntu-latest'
2021-07-20T04:04:54.6954633Z Waiting for a hosted runner in 'organization' to pick this job...
2021-07-20T04:05:04.1866667Z Current runner version: '2.278.0'
2021-07-20T04:05:04.1900245Z ##[group]Operating System
2021-07-20T04:05:04.1901350Z Ubuntu
2021-07-20T04:05:04.1901832Z 20.04.2
2021-07-20T04:05:04.1902345Z LTS
2021-07-20T04:05:04.1902848Z ##[endgroup]
2021-07-20T04:05:04.1910199Z ##[group]Virtual Environment
2021-07-20T04:05:04.1911045Z Environment: ubuntu-20.04
2021-07-20T04:05:04.1911857Z Version: 20210712.0
2021-07-20T04:05:04.1912911Z Included Software: https://github.com/actions/virtual-environments/blob/ubuntu20/20210712.0/images/linux/Ubuntu2004-README.md
2021-07-20T04:05:04.1916702Z Image Release: https://github.com/actions/virtual-environments/releases/tag/ubuntu20%2F20210712.0
2021-07-20T04:05:04.1917787Z ##[endgroup]
2021-07-20T04:05:04.1920298Z ##[group]GITHUB_TOKEN Permissions
2021-07-20T04:05:04.1921979Z Actions: write
2021-07-20T04:05:04.1922622Z Checks: write
2021-07-20T04:05:04.1923445Z Contents: write
2021-07-20T04:05:04.1924060Z Deployments: write
2021-07-20T04:05:04.1924982Z Discussions: write
2021-07-20T04:05:04.1925629Z Issues: write
2021-07-20T04:05:04.1926180Z Metadata: read
2021-07-20T04:05:04.1926839Z Packages: write
2021-07-20T04:05:04.1927576Z PullRequests: write
2021-07-20T04:05:04.1928287Z RepositoryProjects: write
2021-07-20T04:05:04.1928951Z SecurityEvents: write
2021-07-20T04:05:04.1929809Z Statuses: write
2021-07-20T04:05:04.1930520Z ##[endgroup]
2021-07-20T04:05:04.1934704Z Prepare workflow directory
2021-07-20T04:05:04.2698792Z Prepare all required actions
2021-07-20T04:05:04.2711940Z Getting action download info
2021-07-20T04:05:04.6654220Z Download action repository 'actions/checkout@v2'
2021-07-20T04:05:06.7060360Z Download action repository 'submariner-io/shipyard@devel'
2021-07-20T04:05:07.2673159Z ##[group]Run actions/checkout@v2
2021-07-20T04:05:07.2674146Z with:
2021-07-20T04:05:07.2674843Z   repository: submariner-io/submariner-operator
2021-07-20T04:05:07.2676078Z   token: ***
2021-07-20T04:05:07.2676600Z   ssh-strict: true
2021-07-20T04:05:07.2677264Z   persist-credentials: true
2021-07-20T04:05:07.2677980Z   clean: true
2021-07-20T04:05:07.2678576Z   fetch-depth: 1
2021-07-20T04:05:07.2679085Z   lfs: false
2021-07-20T04:05:07.2679639Z   submodules: false
2021-07-20T04:05:07.2680592Z ##[endgroup]
2021-07-20T04:05:07.5096198Z Syncing repository: submariner-io/submariner-operator
2021-07-20T04:05:07.5097717Z ##[group]Getting Git version info
2021-07-20T04:05:07.5099245Z Working directory is '/home/runner/work/submariner-operator/submariner-operator'
2021-07-20T04:05:07.5220009Z [command]/usr/bin/git version
2021-07-20T04:05:07.5357456Z git version 2.32.0
2021-07-20T04:05:07.5381678Z ##[endgroup]
2021-07-20T04:05:07.5389864Z Deleting the contents of '/home/runner/work/submariner-operator/submariner-operator'
2021-07-20T04:05:07.5395822Z ##[group]Initializing the repository
2021-07-20T04:05:07.5401420Z [command]/usr/bin/git init /home/runner/work/submariner-operator/submariner-operator
2021-07-20T04:05:07.5480027Z hint: Using 'master' as the name for the initial branch. This default branch name
2021-07-20T04:05:08.2135843Z hint: is subject to change. To configure the initial branch name to use in all
2021-07-20T04:05:08.2137278Z hint: of your new repositories, which will suppress this warning, call:
2021-07-20T04:05:08.2138046Z hint:
2021-07-20T04:05:08.2139495Z hint: 	git config --global init.defaultBranch <name>
2021-07-20T04:05:08.2140389Z hint:
2021-07-20T04:05:08.2141400Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
2021-07-20T04:05:08.2146647Z hint: 'development'. The just-created branch can be renamed via this command:
2021-07-20T04:05:08.2147509Z hint:
2021-07-20T04:05:08.2148363Z hint: 	git branch -m <name>
2021-07-20T04:05:08.2149735Z Initialized empty Git repository in /home/runner/work/submariner-operator/submariner-operator/.git/
2021-07-20T04:05:08.2151513Z [command]/usr/bin/git remote add origin https://github.com/submariner-io/submariner-operator
2021-07-20T04:05:08.2152601Z ##[endgroup]
2021-07-20T04:05:08.2153387Z ##[group]Disabling automatic garbage collection
2021-07-20T04:05:08.2154758Z [command]/usr/bin/git config --local gc.auto 0
2021-07-20T04:05:08.2155727Z ##[endgroup]
2021-07-20T04:05:08.2167434Z ##[group]Setting up auth
2021-07-20T04:05:08.2168711Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2021-07-20T04:05:08.2171237Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :
2021-07-20T04:05:08.2173118Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2021-07-20T04:05:08.2175496Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :
2021-07-20T04:05:08.2177802Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***
2021-07-20T04:05:08.2178837Z ##[endgroup]
2021-07-20T04:05:08.2179492Z ##[group]Fetching the repository
2021-07-20T04:05:08.2181340Z [command]/usr/bin/git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +0105e4d08ef7236073547f1383566fc76c548f6d:refs/remotes/origin/devel
2021-07-20T04:05:08.2182894Z remote: Enumerating objects: 486, done.
2021-07-20T04:05:08.2183647Z remote: Counting objects:   0% (1/486)
2021-07-20T04:05:08.2184459Z remote: Counting objects:   1% (5/486)
2021-07-20T04:05:08.2185140Z remote: Counting objects:   2% (10/486)
2021-07-20T04:05:08.2185819Z remote: Counting objects:   3% (15/486)
2021-07-20T04:05:08.2186699Z remote: Counting objects:   4% (20/486)
2021-07-20T04:05:08.2187438Z remote: Counting objects:   5% (25/486)
2021-07-20T04:05:08.2188078Z remote: Counting objects:   6% (30/486)
2021-07-20T04:05:08.2188758Z remote: Counting objects:   7% (35/486)
2021-07-20T04:05:08.2189690Z remote: Counting objects:   8% (39/486)
2021-07-20T04:05:08.2190373Z remote: Counting objects:   9% (44/486)
2021-07-20T04:05:08.2191013Z remote: Counting objects:  10% (49/486)
2021-07-20T04:05:08.2191698Z remote: Counting objects:  11% (54/486)
2021-07-20T04:05:08.2192578Z remote: Counting objects:  12% (59/486)
2021-07-20T04:05:08.2193272Z remote: Counting objects:  13% (64/486)
2021-07-20T04:05:08.2193984Z remote: Counting objects:  14% (69/486)
2021-07-20T04:05:08.2194660Z remote: Counting objects:  15% (73/486)
2021-07-20T04:05:08.2195387Z remote: Counting objects:  16% (78/486)
2021-07-20T04:05:08.2196227Z remote: Counting objects:  17% (83/486)
2021-07-20T04:05:08.2196964Z remote: Counting objects:  18% (88/486)
2021-07-20T04:05:08.2197625Z remote: Counting objects:  19% (93/486)
2021-07-20T04:05:08.2198327Z remote: Counting objects:  20% (98/486)
2021-07-20T04:05:08.2199028Z remote: Counting objects:  21% (103/486)
2021-07-20T04:05:08.2199773Z remote: Counting objects:  22% (107/486)
2021-07-20T04:05:08.2200501Z remote: Counting objects:  23% (112/486)
2021-07-20T04:05:08.2201160Z remote: Counting objects:  24% (117/486)
2021-07-20T04:05:08.2201864Z remote: Counting objects:  25% (122/486)
2021-07-20T04:05:08.2202525Z remote: Counting objects:  26% (127/486)
2021-07-20T04:05:08.2203445Z remote: Counting objects:  27% (132/486)
2021-07-20T04:05:08.2204122Z remote: Counting objects:  28% (137/486)
2021-07-20T04:05:08.2204877Z remote: Counting objects:  29% (141/486)
2021-07-20T04:05:08.2205613Z remote: Counting objects:  30% (146/486)
2021-07-20T04:05:08.2206298Z remote: Counting objects:  31% (151/486)
2021-07-20T04:05:08.2207192Z remote: Counting objects:  32% (156/486)
2021-07-20T04:05:08.2207912Z remote: Counting objects:  33% (161/486)
2021-07-20T04:05:08.2208647Z remote: Counting objects:  34% (166/486)
2021-07-20T04:05:08.2209465Z remote: Counting objects:  35% (171/486)
2021-07-20T04:05:08.2210196Z remote: Counting objects:  36% (175/486)
2021-07-20T04:05:08.2210857Z remote: Counting objects:  37% (180/486)
2021-07-20T04:05:08.2211564Z remote: Counting objects:  38% (185/486)
2021-07-20T04:05:08.2212250Z remote: Counting objects:  39% (190/486)
2021-07-20T04:05:08.2212959Z remote: Counting objects:  40% (195/486)
2021-07-20T04:05:08.2213680Z remote: Counting objects:  41% (200/486)
2021-07-20T04:05:08.2214337Z remote: Counting objects:  42% (205/486)
2021-07-20T04:05:08.2215052Z remote: Counting objects:  43% (209/486)
2021-07-20T04:05:08.2215757Z remote: Counting objects:  44% (214/486)
2021-07-20T04:05:08.2216669Z remote: Counting objects:  45% (219/486)
2021-07-20T04:05:08.2217566Z remote: Counting objects:  46% (224/486)
2021-07-20T04:05:08.2218299Z remote: Counting objects:  47% (229/486)
2021-07-20T04:05:08.2218991Z remote: Counting objects:  48% (234/486)
2021-07-20T04:05:08.2219754Z remote: Counting objects:  49% (239/486)
2021-07-20T04:05:08.2220549Z remote: Counting objects:  50% (243/486)
2021-07-20T04:05:08.2221239Z remote: Counting objects:  51% (248/486)
2021-07-20T04:05:08.2221973Z remote: Counting objects:  52% (253/486)
2021-07-20T04:05:08.2222659Z remote: Counting objects:  53% (258/486)
2021-07-20T04:05:08.2223416Z remote: Counting objects:  54% (263/486)
2021-07-20T04:05:08.2224103Z remote: Counting objects:  55% (268/486)
2021-07-20T04:05:08.2224970Z remote: Counting objects:  56% (273/486)
2021-07-20T04:05:08.2225686Z remote: Counting objects:  57% (278/486)
2021-07-20T04:05:08.2226424Z remote: Counting objects:  58% (282/486)
2021-07-20T04:05:08.2227181Z remote: Counting objects:  59% (287/486)
2021-07-20T04:05:08.2227882Z remote: Counting objects:  60% (292/486)
2021-07-20T04:05:08.2228635Z remote: Counting objects:  61% (297/486)
2021-07-20T04:05:08.2229319Z remote: Counting objects:  62% (302/486)
2021-07-20T04:05:08.2230048Z remote: Counting objects:  63% (307/486)
2021-07-20T04:05:08.2230803Z remote: Counting objects:  64% (312/486)
2021-07-20T04:05:08.2231583Z remote: Counting objects:  65% (316/486)
2021-07-20T04:05:08.2232266Z remote: Counting objects:  66% (321/486)
2021-07-20T04:05:08.2232999Z remote: Counting objects:  67% (326/486)
2021-07-20T04:05:08.2234153Z remote: Counting objects:  68% (331/486)
2021-07-20T04:05:08.2234832Z remote: Counting objects:  69% (336/486)
2021-07-20T04:05:08.2235789Z remote: Counting objects:  70% (341/486)
2021-07-20T04:05:08.2236454Z remote: Counting objects:  71% (346/486)
2021-07-20T04:05:08.2237693Z remote: Counting objects:  72% (350/486)
2021-07-20T04:05:08.2238424Z remote: Counting objects:  73% (355/486)
2021-07-20T04:05:08.2239132Z remote: Counting objects:  74% (360/486)
2021-07-20T04:05:08.2239886Z remote: Counting objects:  75% (365/486)
2021-07-20T04:05:08.2240564Z remote: Counting objects:  76% (370/486)
2021-07-20T04:05:08.2241627Z remote: Counting objects:  77% (375/486)
2021-07-20T04:05:08.2242550Z remote: Counting objects:  78% (380/486)
2021-07-20T04:05:08.2243463Z remote: Counting objects:  79% (384/486)
2021-07-20T04:05:08.2244173Z remote: Counting objects:  80% (389/486)
2021-07-20T04:05:08.2244896Z remote: Counting objects:  81% (394/486)
2021-07-20T04:05:08.2245626Z remote: Counting objects:  82% (399/486)
2021-07-20T04:05:08.2246316Z remote: Counting objects:  83% (404/486)
2021-07-20T04:05:08.2247063Z remote: Counting objects:  84% (409/486)
2021-07-20T04:05:08.2247765Z remote: Counting objects:  85% (414/486)
2021-07-20T04:05:08.2248493Z remote: Counting objects:  86% (418/486)
2021-07-20T04:05:08.2249291Z remote: Counting objects:  87% (423/486)
2021-07-20T04:05:08.2250038Z remote: Counting objects:  88% (428/486)
2021-07-20T04:05:08.2250718Z remote: Counting objects:  89% (433/486)
2021-07-20T04:05:08.2251494Z remote: Counting objects:  90% (438/486)
2021-07-20T04:05:08.2252181Z remote: Counting objects:  91% (443/486)
2021-07-20T04:05:08.2252988Z remote: Counting objects:  92% (448/486)
2021-07-20T04:05:08.2253884Z remote: Counting objects:  93% (452/486)
2021-07-20T04:05:08.2254540Z remote: Counting objects:  94% (457/486)
2021-07-20T04:05:08.2255265Z remote: Counting objects:  95% (462/486)
2021-07-20T04:05:08.2255931Z remote: Counting objects:  96% (467/486)
2021-07-20T04:05:08.2256637Z remote: Counting objects:  97% (472/486)
2021-07-20T04:05:08.2257443Z remote: Counting objects:  98% (477/486)
2021-07-20T04:05:08.2258162Z remote: Counting objects:  99% (482/486)
2021-07-20T04:05:08.2258817Z remote: Counting objects: 100% (486/486)
2021-07-20T04:05:08.2259589Z remote: Counting objects: 100% (486/486), done.
2021-07-20T04:05:08.2260539Z remote: Compressing objects:   0% (1/353)
2021-07-20T04:05:08.2261287Z remote: Compressing objects:   1% (4/353)
2021-07-20T04:05:08.2262068Z remote: Compressing objects:   2% (8/353)
2021-07-20T04:05:08.2262793Z remote: Compressing objects:   3% (11/353)
2021-07-20T04:05:08.2263754Z remote: Compressing objects:   4% (15/353)
2021-07-20T04:05:08.2264450Z remote: Compressing objects:   5% (18/353)
2021-07-20T04:05:08.2265206Z remote: Compressing objects:   6% (22/353)
2021-07-20T04:05:08.2265996Z remote: Compressing objects:   7% (25/353)
2021-07-20T04:05:08.2266781Z remote: Compressing objects:   8% (29/353)
2021-07-20T04:05:08.2267559Z remote: Compressing objects:   9% (32/353)
2021-07-20T04:05:08.2268276Z remote: Compressing objects:  10% (36/353)
2021-07-20T04:05:08.2269045Z remote: Compressing objects:  11% (39/353)
2021-07-20T04:05:08.2271046Z remote: Compressing objects:  12% (43/353)
2021-07-20T04:05:08.2271922Z remote: Compressing objects:  13% (46/353)
2021-07-20T04:05:08.2272709Z remote: Compressing objects:  14% (50/353)
2021-07-20T04:05:08.2273528Z remote: Compressing objects:  15% (53/353)
2021-07-20T04:05:08.2274411Z remote: Compressing objects:  16% (57/353)
2021-07-20T04:05:08.2544747Z remote: Compressing objects:  17% (61/353)
2021-07-20T04:05:08.2545477Z remote: Compressing objects:  18% (64/353)
2021-07-20T04:05:08.2546213Z remote: Compressing objects:  19% (68/353)
2021-07-20T04:05:08.2546915Z remote: Compressing objects:  20% (71/353)
2021-07-20T04:05:08.2547595Z remote: Compressing objects:  21% (75/353)
2021-07-20T04:05:08.2548286Z remote: Compressing objects:  22% (78/353)
2021-07-20T04:05:08.2548973Z remote: Compressing objects:  23% (82/353)
2021-07-20T04:05:08.2549679Z remote: Compressing objects:  24% (85/353)
2021-07-20T04:05:08.2550357Z remote: Compressing objects:  25% (89/353)
2021-07-20T04:05:08.2551044Z remote: Compressing objects:  26% (92/353)
2021-07-20T04:05:08.2551725Z remote: Compressing objects:  27% (96/353)
2021-07-20T04:05:08.2552417Z remote: Compressing objects:  28% (99/353)
2021-07-20T04:05:08.2553114Z remote: Compressing objects:  29% (103/353)
2021-07-20T04:05:08.2553873Z remote: Compressing objects:  30% (106/353)
2021-07-20T04:05:08.2554729Z remote: Compressing objects:  31% (110/353)
2021-07-20T04:05:08.2555817Z remote: Compressing objects:  32% (113/353)
2021-07-20T04:05:08.2556529Z remote: Compressing objects:  33% (117/353)
2021-07-20T04:05:08.2557382Z remote: Compressing objects:  34% (121/353)
2021-07-20T04:05:08.2558089Z remote: Compressing objects:  35% (124/353)
2021-07-20T04:05:08.2558778Z remote: Compressing objects:  36% (128/353)
2021-07-20T04:05:08.2559750Z remote: Compressing objects:  37% (131/353)
2021-07-20T04:05:08.2560461Z remote: Compressing objects:  38% (135/353)
2021-07-20T04:05:08.2561151Z remote: Compressing objects:  39% (138/353)
2021-07-20T04:05:08.2561846Z remote: Compressing objects:  40% (142/353)
2021-07-20T04:05:08.2562523Z remote: Compressing objects:  41% (145/353)
2021-07-20T04:05:08.2563210Z remote: Compressing objects:  42% (149/353)
2021-07-20T04:05:08.2563892Z remote: Compressing objects:  43% (152/353)
2021-07-20T04:05:08.2564576Z remote: Compressing objects:  44% (156/353)
2021-07-20T04:05:08.2565268Z remote: Compressing objects:  45% (159/353)
2021-07-20T04:05:08.2566026Z remote: Compressing objects:  46% (163/353)
2021-07-20T04:05:08.2566710Z remote: Compressing objects:  47% (166/353)
2021-07-20T04:05:08.2567385Z remote: Compressing objects:  48% (170/353)
2021-07-20T04:05:08.2568084Z remote: Compressing objects:  49% (173/353)
2021-07-20T04:05:08.2569355Z remote: Compressing objects:  50% (177/353)
2021-07-20T04:05:08.2570017Z remote: Compressing objects:  51% (181/353)
2021-07-20T04:05:08.2570668Z remote: Compressing objects:  52% (184/353)
2021-07-20T04:05:08.2571554Z remote: Compressing objects:  53% (188/353)
2021-07-20T04:05:08.2572230Z remote: Compressing objects:  54% (191/353)
2021-07-20T04:05:08.2573541Z remote: Compressing objects:  55% (195/353)
2021-07-20T04:05:08.2574262Z remote: Compressing objects:  56% (198/353)
2021-07-20T04:05:08.2574939Z remote: Compressing objects:  57% (202/353)
2021-07-20T04:05:08.2575807Z remote: Compressing objects:  58% (205/353)
2021-07-20T04:05:08.2576679Z remote: Compressing objects:  59% (209/353)
2021-07-20T04:05:08.2577759Z remote: Compressing objects:  60% (212/353)
2021-07-20T04:05:08.2578422Z remote: Compressing objects:  61% (216/353)
2021-07-20T04:05:08.2579104Z remote: Compressing objects:  62% (219/353)
2021-07-20T04:05:08.2579760Z remote: Compressing objects:  63% (223/353)
2021-07-20T04:05:08.2580432Z remote: Compressing objects:  64% (226/353)
2021-07-20T04:05:08.2581100Z remote: Compressing objects:  65% (230/353)
2021-07-20T04:05:08.2581754Z remote: Compressing objects:  66% (233/353)
2021-07-20T04:05:08.2582418Z remote: Compressing objects:  67% (237/353)
2021-07-20T04:05:08.2583070Z remote: Compressing objects:  68% (241/353)
2021-07-20T04:05:08.2583736Z remote: Compressing objects:  69% (244/353)
2021-07-20T04:05:08.2584402Z remote: Compressing objects:  70% (248/353)
2021-07-20T04:05:08.2585072Z remote: Compressing objects:  71% (251/353)
2021-07-20T04:05:08.2585931Z remote: Compressing objects:  72% (255/353)
2021-07-20T04:05:08.2586769Z remote: Compressing objects:  73% (258/353)
2021-07-20T04:05:08.2587446Z remote: Compressing objects:  74% (262/353)
2021-07-20T04:05:08.2588117Z remote: Compressing objects:  75% (265/353)
2021-07-20T04:05:08.2588788Z remote: Compressing objects:  76% (269/353)
2021-07-20T04:05:08.2589444Z remote: Compressing objects:  77% (272/353)
2021-07-20T04:05:08.2590112Z remote: Compressing objects:  78% (276/353)
2021-07-20T04:05:08.2590773Z remote: Compressing objects:  79% (279/353)
2021-07-20T04:05:08.2591484Z remote: Compressing objects:  80% (283/353)
2021-07-20T04:05:08.2592159Z remote: Compressing objects:  81% (286/353)
2021-07-20T04:05:08.2592837Z remote: Compressing objects:  82% (290/353)
2021-07-20T04:05:08.2593522Z remote: Compressing objects:  83% (293/353)
2021-07-20T04:05:08.2594352Z remote: Compressing objects:  84% (297/353)
2021-07-20T04:05:08.2595002Z remote: Compressing objects:  85% (301/353)
2021-07-20T04:05:08.2595811Z remote: Compressing objects:  86% (304/353)
2021-07-20T04:05:08.2596824Z remote: Compressing objects:  87% (308/353)
2021-07-20T04:05:08.2597814Z remote: Compressing objects:  88% (311/353)
2021-07-20T04:05:08.2598460Z remote: Compressing objects:  89% (315/353)
2021-07-20T04:05:08.2599092Z remote: Compressing objects:  90% (318/353)
2021-07-20T04:05:08.2599740Z remote: Compressing objects:  91% (322/353)
2021-07-20T04:05:08.2617976Z remote: Compressing objects:  92% (325/353)
2021-07-20T04:05:08.2619031Z remote: Compressing objects:  93% (329/353)
2021-07-20T04:05:08.2619731Z remote: Compressing objects:  94% (332/353)
2021-07-20T04:05:08.2620743Z remote: Compressing objects:  95% (336/353)
2021-07-20T04:05:08.2621393Z remote: Compressing objects:  96% (339/353)
2021-07-20T04:05:08.2622031Z remote: Compressing objects:  97% (343/353)
2021-07-20T04:05:08.2622701Z remote: Compressing objects:  98% (346/353)
2021-07-20T04:05:08.2623351Z remote: Compressing objects:  99% (350/353)
2021-07-20T04:05:08.2624018Z remote: Compressing objects: 100% (353/353)
2021-07-20T04:05:08.2624845Z remote: Compressing objects: 100% (353/353), done.
2021-07-20T04:05:08.2625841Z Receiving objects:   0% (1/486)
2021-07-20T04:05:08.2626392Z Receiving objects:   1% (5/486)
2021-07-20T04:05:08.2626959Z Receiving objects:   2% (10/486)
2021-07-20T04:05:08.2627688Z Receiving objects:   3% (15/486)
2021-07-20T04:05:08.2628276Z Receiving objects:   4% (20/486)
2021-07-20T04:05:08.2628999Z Receiving objects:   5% (25/486)
2021-07-20T04:05:08.2629564Z Receiving objects:   6% (30/486)
2021-07-20T04:05:08.2630306Z Receiving objects:   7% (35/486)
2021-07-20T04:05:08.2631103Z Receiving objects:   8% (39/486)
2021-07-20T04:05:08.2631746Z Receiving objects:   9% (44/486)
2021-07-20T04:05:08.2632328Z Receiving objects:  10% (49/486)
2021-07-20T04:05:08.2632925Z Receiving objects:  11% (54/486)
2021-07-20T04:05:08.2633661Z Receiving objects:  12% (59/486)
2021-07-20T04:05:08.2634234Z Receiving objects:  13% (64/486)
2021-07-20T04:05:08.2634804Z Receiving objects:  14% (69/486)
2021-07-20T04:05:08.2635370Z Receiving objects:  15% (73/486)
2021-07-20T04:05:08.2636063Z Receiving objects:  16% (78/486)
2021-07-20T04:05:08.2636633Z Receiving objects:  17% (83/486)
2021-07-20T04:05:08.2637184Z Receiving objects:  18% (88/486)
2021-07-20T04:05:08.2637748Z Receiving objects:  19% (93/486)
2021-07-20T04:05:08.2638303Z Receiving objects:  20% (98/486)
2021-07-20T04:05:08.2639048Z Receiving objects:  21% (103/486)
2021-07-20T04:05:08.2639802Z Receiving objects:  22% (107/486)
2021-07-20T04:05:08.2640357Z Receiving objects:  23% (112/486)
2021-07-20T04:05:08.2640935Z Receiving objects:  24% (117/486)
2021-07-20T04:05:08.2641673Z Receiving objects:  25% (122/486)
2021-07-20T04:05:08.2642257Z Receiving objects:  26% (127/486)
2021-07-20T04:05:08.2643213Z Receiving objects:  27% (132/486)
2021-07-20T04:05:08.2643968Z Receiving objects:  28% (137/486)
2021-07-20T04:05:08.2644719Z Receiving objects:  29% (141/486)
2021-07-20T04:05:08.2645316Z Receiving objects:  30% (146/486)
2021-07-20T04:05:08.2645891Z Receiving objects:  31% (151/486)
2021-07-20T04:05:08.2646476Z Receiving objects:  32% (156/486)
2021-07-20T04:05:08.2647050Z Receiving objects:  33% (161/486)
2021-07-20T04:05:08.2647629Z Receiving objects:  34% (166/486)
2021-07-20T04:05:08.2648205Z Receiving objects:  35% (171/486)
2021-07-20T04:05:08.2648791Z Receiving objects:  36% (175/486)
2021-07-20T04:05:08.2649365Z Receiving objects:  37% (180/486)
2021-07-20T04:05:08.2649957Z Receiving objects:  38% (185/486)
2021-07-20T04:05:08.2650547Z Receiving objects:  39% (190/486)
2021-07-20T04:05:08.2651127Z Receiving objects:  40% (195/486)
2021-07-20T04:05:08.2651726Z Receiving objects:  41% (200/486)
2021-07-20T04:05:08.2652300Z Receiving objects:  42% (205/486)
2021-07-20T04:05:08.2652890Z Receiving objects:  43% (209/486)
2021-07-20T04:05:08.2653467Z Receiving objects:  44% (214/486)
2021-07-20T04:05:08.2654153Z Receiving objects:  45% (219/486)
2021-07-20T04:05:08.2654870Z Receiving objects:  46% (224/486)
2021-07-20T04:05:08.2655460Z Receiving objects:  47% (229/486)
2021-07-20T04:05:08.2656036Z Receiving objects:  48% (234/486)
2021-07-20T04:05:08.2656629Z Receiving objects:  49% (239/486)
2021-07-20T04:05:08.2657425Z Receiving objects:  50% (243/486)
2021-07-20T04:05:08.2658016Z Receiving objects:  51% (248/486)
2021-07-20T04:05:08.2658607Z Receiving objects:  52% (253/486)
2021-07-20T04:05:08.2659332Z Receiving objects:  53% (258/486)
2021-07-20T04:05:08.2659904Z Receiving objects:  54% (263/486)
2021-07-20T04:05:08.2660468Z Receiving objects:  55% (268/486)
2021-07-20T04:05:08.2661036Z Receiving objects:  56% (273/486)
2021-07-20T04:05:08.2661604Z Receiving objects:  57% (278/486)
2021-07-20T04:05:08.2662178Z Receiving objects:  58% (282/486)
2021-07-20T04:05:08.2662918Z Receiving objects:  59% (287/486)
2021-07-20T04:05:08.2663521Z Receiving objects:  60% (292/486)
2021-07-20T04:05:08.2664095Z Receiving objects:  61% (297/486)
2021-07-20T04:05:08.2664682Z Receiving objects:  62% (302/486)
2021-07-20T04:05:08.2665284Z Receiving objects:  63% (307/486)
2021-07-20T04:05:08.2665859Z Receiving objects:  64% (312/486)
2021-07-20T04:05:08.2666452Z Receiving objects:  65% (316/486)
2021-07-20T04:05:08.2667033Z Receiving objects:  66% (321/486)
2021-07-20T04:05:08.2667633Z Receiving objects:  67% (326/486)
2021-07-20T04:05:08.2668211Z Receiving objects:  68% (331/486)
2021-07-20T04:05:08.2668796Z Receiving objects:  69% (336/486)
2021-07-20T04:05:08.2669372Z Receiving objects:  70% (341/486)
2021-07-20T04:05:08.2669963Z Receiving objects:  71% (346/486)
2021-07-20T04:05:08.2670540Z Receiving objects:  72% (350/486)
2021-07-20T04:05:08.2671242Z Receiving objects:  73% (355/486)
2021-07-20T04:05:08.2671856Z Receiving objects:  74% (360/486)
2021-07-20T04:05:08.2672621Z Receiving objects:  75% (365/486)
2021-07-20T04:05:08.2675076Z Receiving objects:  76% (370/486)
2021-07-20T04:05:08.2678271Z Receiving objects:  77% (375/486)
2021-07-20T04:05:08.2679929Z Receiving objects:  78% (380/486)
2021-07-20T04:05:08.2680585Z Receiving objects:  79% (384/486)
2021-07-20T04:05:08.2681155Z Receiving objects:  80% (389/486)
2021-07-20T04:05:08.2683393Z Receiving objects:  81% (394/486)
2021-07-20T04:05:08.2686826Z Receiving objects:  82% (399/486)
2021-07-20T04:05:08.2687518Z Receiving objects:  83% (404/486)
2021-07-20T04:05:08.2688115Z Receiving objects:  84% (409/486)
2021-07-20T04:05:08.2688854Z Receiving objects:  85% (414/486)
2021-07-20T04:05:08.2689408Z Receiving objects:  86% (418/486)
2021-07-20T04:05:08.2689980Z Receiving objects:  87% (423/486)
2021-07-20T04:05:08.2690540Z Receiving objects:  88% (428/486)
2021-07-20T04:05:08.2691130Z Receiving objects:  89% (433/486)
2021-07-20T04:05:08.2691689Z Receiving objects:  90% (438/486)
2021-07-20T04:05:08.2692261Z Receiving objects:  91% (443/486)
2021-07-20T04:05:08.2692817Z Receiving objects:  92% (448/486)
2021-07-20T04:05:08.2693394Z Receiving objects:  93% (452/486)
2021-07-20T04:05:08.2693949Z Receiving objects:  94% (457/486)
2021-07-20T04:05:08.2694528Z Receiving objects:  95% (462/486)
2021-07-20T04:05:08.2695261Z Receiving objects:  96% (467/486)
2021-07-20T04:05:08.2696289Z Receiving objects:  97% (472/486)
2021-07-20T04:05:08.2698039Z remote: Total 486 (delta 154), reused 305 (delta 87), pack-reused 0
2021-07-20T04:05:08.2698762Z Receiving objects:  98% (477/486)
2021-07-20T04:05:08.2699358Z Receiving objects:  99% (482/486)
2021-07-20T04:05:08.2699938Z Receiving objects: 100% (486/486)
2021-07-20T04:05:08.2700604Z Receiving objects: 100% (486/486), 349.02 KiB | 2.73 MiB/s, done.
2021-07-20T04:05:08.2701557Z Resolving deltas:   0% (0/154)
2021-07-20T04:05:08.2702091Z Resolving deltas:   1% (2/154)
2021-07-20T04:05:08.2702623Z Resolving deltas:   2% (4/154)
2021-07-20T04:05:08.2703158Z Resolving deltas:   3% (5/154)
2021-07-20T04:05:08.2703691Z Resolving deltas:   4% (7/154)
2021-07-20T04:05:08.2704212Z Resolving deltas:   5% (8/154)
2021-07-20T04:05:08.2705123Z Resolving deltas:   6% (10/154)
2021-07-20T04:05:08.2705682Z Resolving deltas:   7% (11/154)
2021-07-20T04:05:08.2706454Z Resolving deltas:   8% (13/154)
2021-07-20T04:05:08.2707012Z Resolving deltas:   9% (14/154)
2021-07-20T04:05:08.2707586Z Resolving deltas:  10% (16/154)
2021-07-20T04:05:08.2708145Z Resolving deltas:  11% (17/154)
2021-07-20T04:05:08.2708718Z Resolving deltas:  12% (19/154)
2021-07-20T04:05:08.2709274Z Resolving deltas:  13% (21/154)
2021-07-20T04:05:08.2709848Z Resolving deltas:  14% (23/154)
2021-07-20T04:05:08.2710560Z Resolving deltas:  15% (24/154)
2021-07-20T04:05:08.2711118Z Resolving deltas:  16% (25/154)
2021-07-20T04:05:08.2711668Z Resolving deltas:  17% (27/154)
2021-07-20T04:05:08.2712203Z Resolving deltas:  18% (28/154)
2021-07-20T04:05:08.2712764Z Resolving deltas:  19% (30/154)
2021-07-20T04:05:08.2713303Z Resolving deltas:  20% (31/154)
2021-07-20T04:05:08.2713862Z Resolving deltas:  21% (33/154)
2021-07-20T04:05:08.2714402Z Resolving deltas:  22% (34/154)
2021-07-20T04:05:08.2714952Z Resolving deltas:  23% (36/154)
2021-07-20T04:05:08.2715502Z Resolving deltas:  24% (37/154)
2021-07-20T04:05:08.2716054Z Resolving deltas:  25% (39/154)
2021-07-20T04:05:08.2716598Z Resolving deltas:  26% (41/154)
2021-07-20T04:05:08.2717148Z Resolving deltas:  27% (42/154)
2021-07-20T04:05:08.2717687Z Resolving deltas:  28% (44/154)
2021-07-20T04:05:08.2718239Z Resolving deltas:  29% (45/154)
2021-07-20T04:05:08.2718791Z Resolving deltas:  30% (47/154)
2021-07-20T04:05:08.2719511Z Resolving deltas:  31% (48/154)
2021-07-20T04:05:08.2720091Z Resolving deltas:  32% (50/154)
2021-07-20T04:05:08.2720648Z Resolving deltas:  33% (51/154)
2021-07-20T04:05:08.2721221Z Resolving deltas:  34% (53/154)
2021-07-20T04:05:08.2722039Z Resolving deltas:  35% (54/154)
2021-07-20T04:05:08.2722615Z Resolving deltas:  36% (56/154)
2021-07-20T04:05:08.2723356Z Resolving deltas:  37% (57/154)
2021-07-20T04:05:08.2723930Z Resolving deltas:  38% (59/154)
2021-07-20T04:05:08.2724489Z Resolving deltas:  39% (61/154)
2021-07-20T04:05:08.2725063Z Resolving deltas:  40% (62/154)
2021-07-20T04:05:08.2725646Z Resolving deltas:  41% (64/154)
2021-07-20T04:05:08.2726208Z Resolving deltas:  42% (65/154)
2021-07-20T04:05:08.2726785Z Resolving deltas:  43% (67/154)
2021-07-20T04:05:08.2727343Z Resolving deltas:  44% (68/154)
2021-07-20T04:05:08.2727914Z Resolving deltas:  45% (70/154)
2021-07-20T04:05:08.2728474Z Resolving deltas:  46% (71/154)
2021-07-20T04:05:08.2729056Z Resolving deltas:  47% (73/154)
2021-07-20T04:05:08.2729614Z Resolving deltas:  48% (74/154)
2021-07-20T04:05:08.2730300Z Resolving deltas:  49% (76/154)
2021-07-20T04:05:08.2730856Z Resolving deltas:  50% (77/154)
2021-07-20T04:05:08.2731428Z Resolving deltas:  51% (79/154)
2021-07-20T04:05:08.2731997Z Resolving deltas:  52% (81/154)
2021-07-20T04:05:08.2732570Z Resolving deltas:  53% (82/154)
2021-07-20T04:05:08.2733142Z Resolving deltas:  54% (84/154)
2021-07-20T04:05:08.2733697Z Resolving deltas:  55% (85/154)
2021-07-20T04:05:08.2734270Z Resolving deltas:  56% (87/154)
2021-07-20T04:05:08.2734831Z Resolving deltas:  57% (88/154)
2021-07-20T04:05:08.2735573Z Resolving deltas:  58% (90/154)
2021-07-20T04:05:08.2736111Z Resolving deltas:  59% (91/154)
2021-07-20T04:05:08.2736658Z Resolving deltas:  60% (93/154)
2021-07-20T04:05:08.2737367Z Resolving deltas:  61% (94/154)
2021-07-20T04:05:08.2738207Z Resolving deltas:  62% (96/154)
2021-07-20T04:05:08.2738926Z Resolving deltas:  63% (98/154)
2021-07-20T04:05:08.2739483Z Resolving deltas:  64% (99/154)
2021-07-20T04:05:08.2740028Z Resolving deltas:  65% (101/154)
2021-07-20T04:05:08.2740589Z Resolving deltas:  66% (102/154)
2021-07-20T04:05:08.2741212Z Resolving deltas:  67% (104/154)
2021-07-20T04:05:08.2741757Z Resolving deltas:  68% (105/154)
2021-07-20T04:05:08.2742325Z Resolving deltas:  69% (107/154)
2021-07-20T04:05:08.2742869Z Resolving deltas:  70% (108/154)
2021-07-20T04:05:08.2743427Z Resolving deltas:  71% (110/154)
2021-07-20T04:05:08.2743970Z Resolving deltas:  72% (111/154)
2021-07-20T04:05:08.2744534Z Resolving deltas:  73% (113/154)
2021-07-20T04:05:08.2745075Z Resolving deltas:  74% (114/154)
2021-07-20T04:05:08.2745778Z Resolving deltas:  75% (116/154)
2021-07-20T04:05:08.2746323Z Resolving deltas:  76% (118/154)
2021-07-20T04:05:08.2746879Z Resolving deltas:  77% (119/154)
2021-07-20T04:05:08.2747440Z Resolving deltas:  78% (121/154)
2021-07-20T04:05:08.2747981Z Resolving deltas:  79% (122/154)
2021-07-20T04:05:08.2748536Z Resolving deltas:  80% (124/154)
2021-07-20T04:05:08.2749076Z Resolving deltas:  81% (125/154)
2021-07-20T04:05:08.2749627Z Resolving deltas:  82% (127/154)
2021-07-20T04:05:08.2750174Z Resolving deltas:  83% (128/154)
2021-07-20T04:05:08.2750732Z Resolving deltas:  84% (130/154)
2021-07-20T04:05:08.2751282Z Resolving deltas:  85% (131/154)
2021-07-20T04:05:08.2751837Z Resolving deltas:  86% (133/154)
2021-07-20T04:05:08.2752592Z Resolving deltas:  87% (134/154)
2021-07-20T04:05:08.2753177Z Resolving deltas:  88% (136/154)
2021-07-20T04:05:08.2753741Z Resolving deltas:  89% (138/154)
2021-07-20T04:05:08.2754319Z Resolving deltas:  90% (139/154)
2021-07-20T04:05:08.2754904Z Resolving deltas:  91% (141/154)
2021-07-20T04:05:08.2755596Z Resolving deltas:  92% (142/154)
2021-07-20T04:05:08.2756502Z Resolving deltas:  93% (144/154)
2021-07-20T04:05:08.2757066Z Resolving deltas:  94% (145/154)
2021-07-20T04:05:08.2757686Z Resolving deltas:  95% (147/154)
2021-07-20T04:05:08.2758245Z Resolving deltas:  96% (148/154)
2021-07-20T04:05:08.2758817Z Resolving deltas:  97% (150/154)
2021-07-20T04:05:08.2759376Z Resolving deltas:  98% (151/154)
2021-07-20T04:05:08.2759955Z Resolving deltas:  99% (153/154)
2021-07-20T04:05:08.2760668Z Resolving deltas: 100% (154/154)
2021-07-20T04:05:08.2761241Z Resolving deltas: 100% (154/154), done.
2021-07-20T04:05:08.2762663Z From https://github.com/submariner-io/submariner-operator
2021-07-20T04:05:08.2764396Z  * [new ref]         0105e4d08ef7236073547f1383566fc76c548f6d -> origin/devel
2021-07-20T04:05:08.2765232Z ##[endgroup]
2021-07-20T04:05:08.2765852Z ##[group]Determining the checkout info
2021-07-20T04:05:08.2766465Z ##[endgroup]
2021-07-20T04:05:08.2767031Z ##[group]Checking out the ref
2021-07-20T04:05:08.2768137Z [command]/usr/bin/git checkout --progress --force -B devel refs/remotes/origin/devel
2021-07-20T04:05:08.2769156Z Switched to a new branch 'devel'
2021-07-20T04:05:08.2770290Z Branch 'devel' set up to track remote branch 'devel' from 'origin'.
2021-07-20T04:05:08.2770961Z ##[endgroup]
2021-07-20T04:05:08.2771885Z [command]/usr/bin/git log -1 --format='%H'
2021-07-20T04:05:08.2772771Z '0105e4d08ef7236073547f1383566fc76c548f6d'
2021-07-20T04:05:08.2866926Z ##[group]Run submariner-io/shipyard/gh-actions/upgrade-e2e@devel
2021-07-20T04:05:08.2867902Z ##[endgroup]
2021-07-20T04:05:08.3104633Z ##[group]Reclaiming free space
2021-07-20T04:05:09.6155811Z Filesystem      Size  Used Avail Use% Mounted on
2021-07-20T04:05:09.6157916Z /dev/root        84G   42G   43G  50% /
2021-07-20T04:05:09.6158742Z devtmpfs        3.4G     0  3.4G   0% /dev
2021-07-20T04:05:09.6159963Z tmpfs           3.4G   12K  3.4G   1% /dev/shm
2021-07-20T04:05:09.6160710Z tmpfs           696M  1.1M  695M   1% /run
2021-07-20T04:05:09.6161331Z tmpfs           5.0M     0  5.0M   0% /run/lock
2021-07-20T04:05:09.6162172Z tmpfs           3.4G     0  3.4G   0% /sys/fs/cgroup
2021-07-20T04:05:09.6162758Z /dev/loop0       56M   56M     0 100% /snap/core18/2066
2021-07-20T04:05:09.6163275Z /dev/loop1       56M   56M     0 100% /snap/core18/2074
2021-07-20T04:05:09.6163749Z /dev/sda15      105M  7.9M   97M   8% /boot/efi
2021-07-20T04:05:09.6164311Z /dev/loop2       68M   68M     0 100% /snap/lxd/20326
2021-07-20T04:05:09.6164807Z /dev/loop3       33M   33M     0 100% /snap/snapd/12159
2021-07-20T04:05:09.6165285Z /dev/loop4       33M   33M     0 100% /snap/snapd/12398
2021-07-20T04:05:09.6165745Z /dev/sdb1        14G  4.1G  9.0G  32% /mnt
2021-07-20T04:05:09.6166836Z ##[endgroup]
2021-07-20T04:05:09.6341594Z ##[group]Disable swap
2021-07-20T04:05:09.6592360Z ##[endgroup]
2021-07-20T04:05:09.6739863Z ##[group]Report available RAM
2021-07-20T04:05:09.6764589Z               total        used        free      shared  buff/cache   available
2021-07-20T04:05:09.6765403Z Mem:          6.8Gi       500Mi       5.2Gi       7.0Mi       1.1Gi       6.0Gi
2021-07-20T04:05:09.6766020Z Swap:            0B          0B          0B
2021-07-20T04:05:09.6766483Z ##[endgroup]
2021-07-20T04:05:09.6946833Z Downloading Makefile.dapper
2021-07-20T04:05:09.8808876Z Downloading dapper
2021-07-20T04:05:10.0880377Z .dapper.tmp version v0.5.6
2021-07-20T04:05:10.0914974Z Downloading Dockerfile.dapper
2021-07-20T04:05:10.2545615Z docker network create -d bridge kind
2021-07-20T04:05:10.6022771Z 89b506689c924954e88b6b011210f7b22925aa3a577984d1330c4ffdc7a805ef
2021-07-20T04:05:10.6041759Z ./.dapper -m bind  make --debug=b upgrade-e2e
2021-07-20T04:05:10.8546270Z Sending build context to Docker daemon  7.517MB
2021-07-20T04:05:10.8547201Z
2021-07-20T04:05:10.8625669Z Step 1/8 : ARG BASE_BRANCH
2021-07-20T04:05:10.8632601Z Step 2/8 : FROM quay.io/submariner/shipyard-dapper-base:${BASE_BRANCH}
2021-07-20T04:05:12.6045078Z devel: Pulling from submariner/shipyard-dapper-base
2021-07-20T04:05:12.6056826Z b1495d80d526: Pulling fs layer
2021-07-20T04:05:12.6059012Z 436e487a3a14: Pulling fs layer
2021-07-20T04:05:12.6060358Z ca0647cd69c4: Pulling fs layer
2021-07-20T04:05:12.6061768Z 169bd7d49280: Pulling fs layer
2021-07-20T04:05:12.6062914Z 355db6a505b3: Pulling fs layer
2021-07-20T04:05:12.6064015Z 80e9cdf47183: Pulling fs layer
2021-07-20T04:05:12.6068808Z 5dc5d4a7886e: Pulling fs layer
2021-07-20T04:05:12.6070268Z bcc150802595: Pulling fs layer
2021-07-20T04:05:12.6071760Z 169bd7d49280: Waiting
2021-07-20T04:05:12.6073652Z 355db6a505b3: Waiting
2021-07-20T04:05:12.6074188Z 80e9cdf47183: Waiting
2021-07-20T04:05:12.6074709Z 5dc5d4a7886e: Waiting
2021-07-20T04:05:12.6075174Z bcc150802595: Waiting
2021-07-20T04:05:16.0089886Z ca0647cd69c4: Verifying Checksum
2021-07-20T04:05:16.0092664Z ca0647cd69c4: Download complete
2021-07-20T04:05:16.1411394Z b1495d80d526: Verifying Checksum
2021-07-20T04:05:16.1422079Z b1495d80d526: Download complete
2021-07-20T04:05:16.3661082Z 169bd7d49280: Verifying Checksum
2021-07-20T04:05:16.3662835Z 169bd7d49280: Download complete
2021-07-20T04:05:16.5038035Z 355db6a505b3: Verifying Checksum
2021-07-20T04:05:16.5038581Z 355db6a505b3: Download complete
2021-07-20T04:05:16.7946145Z 80e9cdf47183: Verifying Checksum
2021-07-20T04:05:16.7950321Z 80e9cdf47183: Download complete
2021-07-20T04:05:16.8781588Z 5dc5d4a7886e: Verifying Checksum
2021-07-20T04:05:16.8784016Z 5dc5d4a7886e: Download complete
2021-07-20T04:05:17.2152722Z bcc150802595: Verifying Checksum
2021-07-20T04:05:17.2153573Z bcc150802595: Download complete
2021-07-20T04:05:19.1109759Z b1495d80d526: Pull complete
2021-07-20T04:05:30.5999643Z 436e487a3a14: Verifying Checksum
2021-07-20T04:05:30.6023357Z 436e487a3a14: Download complete
2021-07-20T04:05:41.5182810Z 436e487a3a14: Pull complete
2021-07-20T04:05:43.0819999Z ca0647cd69c4: Pull complete
2021-07-20T04:05:43.1612902Z 169bd7d49280: Pull complete
2021-07-20T04:05:43.2393886Z 355db6a505b3: Pull complete
2021-07-20T04:05:43.4412184Z 80e9cdf47183: Pull complete
2021-07-20T04:05:43.5181774Z 5dc5d4a7886e: Pull complete
2021-07-20T04:05:43.6114787Z bcc150802595: Pull complete
2021-07-20T04:05:43.6171594Z Digest: sha256:afbc987524ae61c7701040f8e92be374f0dadcfe1b08cba635eb8d9e1fafdd61
2021-07-20T04:05:43.6197685Z Status: Downloaded newer image for quay.io/submariner/shipyard-dapper-base:devel
2021-07-20T04:05:43.6209749Z  ---> 8e970b865a56
2021-07-20T04:05:43.6210839Z Step 3/8 : ARG PROJECT
2021-07-20T04:05:43.6491812Z  ---> Running in 0a86ce1cbb79
2021-07-20T04:05:44.7492302Z Removing intermediate container 0a86ce1cbb79
2021-07-20T04:05:44.7493708Z  ---> d57065b9d674
2021-07-20T04:05:44.7495473Z Step 4/8 : ENV DAPPER_ENV="QUAY_USERNAME QUAY_PASSWORD CLUSTERS_ARGS DEPLOY_ARGS CLEANUP_ARGS E2E_ARGS RELEASE_ARGS MAKEFLAGS FOCUS SKIP PLUGIN E2E_TESTDIR"     DAPPER_SOURCE=/go/src/github.com/submariner-io/${PROJECT} DAPPER_DOCKER_SOCKET=true
2021-07-20T04:05:44.7695929Z  ---> Running in 44e7fb1151f1
2021-07-20T04:05:45.8136020Z Removing intermediate container 44e7fb1151f1
2021-07-20T04:05:45.8137549Z  ---> 66732cc433bf
2021-07-20T04:05:45.8138114Z Step 5/8 : ENV DAPPER_OUTPUT=${DAPPER_SOURCE}/output
2021-07-20T04:05:45.8397284Z  ---> Running in 95bdd1133e8d
2021-07-20T04:05:46.9021157Z Removing intermediate container 95bdd1133e8d
2021-07-20T04:05:46.9036465Z  ---> 80c1fbf21b4b
2021-07-20T04:05:46.9037059Z Step 6/8 : WORKDIR ${DAPPER_SOURCE}
2021-07-20T04:05:46.9251527Z  ---> Running in 1c980ab1a8c3
2021-07-20T04:05:48.0268346Z Removing intermediate container 1c980ab1a8c3
2021-07-20T04:05:48.0269963Z  ---> 4cc5ee346985
2021-07-20T04:05:48.0270509Z Step 7/8 : ENTRYPOINT ["/opt/shipyard/scripts/entry"]
2021-07-20T04:05:48.0499221Z  ---> Running in 999948c690f2
2021-07-20T04:05:49.1364235Z Removing intermediate container 999948c690f2
2021-07-20T04:05:49.1365450Z  ---> 66902d107889
2021-07-20T04:05:49.1365864Z Step 8/8 : CMD ["sh"]
2021-07-20T04:05:49.1731086Z  ---> Running in 36b0861eae35
2021-07-20T04:05:50.2601365Z Removing intermediate container 36b0861eae35
2021-07-20T04:05:50.2603116Z  ---> 09b1871361d8
2021-07-20T04:05:50.2755760Z Successfully built 09b1871361d8
2021-07-20T04:05:50.2793376Z Successfully tagged submariner-operator:devel
2021-07-20T04:05:50.8329849Z [36m[submariner-operator]$ trap chown -R 1001:121 . exit[0m
2021-07-20T04:05:50.8358837Z [36m[submariner-operator]$ mkdir -p bin dist output[0m
2021-07-20T04:05:50.8380496Z [36m[submariner-operator]$ make --debug=b upgrade-e2e[0m
2021-07-20T04:05:50.8387142Z GNU Make 4.3
2021-07-20T04:05:50.8394722Z Built for x86_64-redhat-linux-gnu
2021-07-20T04:05:50.8395557Z Copyright (C) 1988-2020 Free Software Foundation, Inc.
2021-07-20T04:05:50.8396315Z License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
2021-07-20T04:05:50.8397105Z This is free software: you are free to change and redistribute it.
2021-07-20T04:05:50.8397780Z There is NO WARRANTY, to the extent permitted by law.
2021-07-20T04:05:50.8398331Z Reading makefiles...
2021-07-20T04:05:50.9486132Z Makefile:94: warning: overriding recipe for target 'e2e'
2021-07-20T04:05:50.9487105Z /opt/shipyard/Makefile.inc:106: warning: ignoring old recipe for target 'e2e'
2021-07-20T04:05:51.1585013Z Updating makefiles....
2021-07-20T04:05:51.1596744Z Updating goal targets....
2021-07-20T04:05:51.1598922Z  File 'upgrade-e2e' does not exist.
2021-07-20T04:05:51.1601932Z    File 'deploy-latest' does not exist.
2021-07-20T04:05:51.1603029Z   Must remake target 'deploy-latest'.
2021-07-20T04:05:51.1604388Z curl -L get.submariner.io | VERSION=latest bash
2021-07-20T04:05:51.1705241Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2021-07-20T04:05:51.1716550Z                                  Dload  Upload   Total   Spent    Left  Speed
2021-07-20T04:05:51.1719653Z
2021-07-20T04:05:51.2131367Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
2021-07-20T04:05:51.2132297Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
2021-07-20T04:05:51.3462972Z
2021-07-20T04:05:51.3464378Z 100  4155    0  4155    0     0  23607      0 --:--:-- --:--:-- --:--:-- 23607
2021-07-20T04:05:51.5277759Z Installing subctl version latest
2021-07-20T04:05:51.5278310Z   OS detected:           linux
2021-07-20T04:05:51.5278810Z   Architecture detected: amd64
2021-07-20T04:05:51.5280574Z   Download URL:          https://github.com/submariner-io/releases/releases/download/v0.9.1/subctl-v0.9.1-linux-amd64.tar.xz
2021-07-20T04:05:51.5281430Z
2021-07-20T04:05:51.5281807Z Downloading...
2021-07-20T04:05:58.6867957Z subctl-v0.9.1-linux-amd64 has been installed as /root/.local/bin/subctl
2021-07-20T04:05:58.7263357Z This provides subctl version: v0.9.1
2021-07-20T04:05:58.7291129Z
2021-07-20T04:05:58.7292059Z please update your path (and consider adding it to ~/.profile):
2021-07-20T04:05:58.7292920Z     export PATH=$PATH:/root/.local/bin
2021-07-20T04:05:58.7293487Z
2021-07-20T04:05:58.7384378Z make -o images -o preload-images deploy SUBCTL=~/.local/bin/subctl DEV_VERSION=latest CUTTING_EDGE=latest VERSION=latest DEPLOY_ARGS="--cluster_settings /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings --image_tag=subctl" using=
2021-07-20T04:05:58.7417464Z GNU Make 4.3
2021-07-20T04:05:58.7418845Z Built for x86_64-redhat-linux-gnu
2021-07-20T04:05:58.7420004Z Copyright (C) 1988-2020 Free Software Foundation, Inc.
2021-07-20T04:05:58.7421171Z License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
2021-07-20T04:05:58.7422255Z This is free software: you are free to change and redistribute it.
2021-07-20T04:05:58.7423966Z There is NO WARRANTY, to the extent permitted by law.
2021-07-20T04:05:58.7424767Z Reading makefiles...
2021-07-20T04:05:58.7426023Z make[1]: Entering directory '/go/src/github.com/submariner-io/submariner-operator'
2021-07-20T04:05:58.8662815Z Makefile:94: warning: overriding recipe for target 'e2e'
2021-07-20T04:05:58.8675125Z /opt/shipyard/Makefile.inc:106: warning: ignoring old recipe for target 'e2e'
2021-07-20T04:05:59.0835318Z Updating makefiles....
2021-07-20T04:05:59.0842465Z Updating goal targets....
2021-07-20T04:05:59.0844954Z    File 'clusters' does not exist.
2021-07-20T04:05:59.0845721Z   Must remake target 'clusters'.
2021-07-20T04:05:59.0847156Z /opt/shipyard/scripts/clusters.sh --cluster_settings /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings
2021-07-20T04:05:59.0877208Z Downloading shflags 1.0.3
2021-07-20T04:05:59.0944153Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2021-07-20T04:05:59.0944895Z                                  Dload  Upload   Total   Spent    Left  Speed
2021-07-20T04:05:59.0945256Z
2021-07-20T04:05:59.1963035Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
2021-07-20T04:05:59.1964248Z 100 31091  100 31091    0     0   297k      0 --:--:-- --:--:-- --:--:--  300k
2021-07-20T04:05:59.2736277Z Running with: k8s_version=1.20.7, olm_version=0.14.1, olm=false, globalnet=false, registry_inmemory=true, cluster_settings=/go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings, timeout=5m
2021-07-20T04:05:59.2764329Z [36m[submariner-operator]$ source /opt/shipyard/scripts/lib/utils[0m
2021-07-20T04:05:59.2775276Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:05:59.2787333Z [36m[submariner-operator]$ script_name=utils[0m
2021-07-20T04:05:59.2798292Z [36m[submariner-operator]$ exec_name=clusters.sh[0m
2021-07-20T04:05:59.2817883Z [36m[submariner-operator]$ source /opt/shipyard/scripts/lib/cluster_settings[0m
2021-07-20T04:05:59.2830002Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:05:59.2841072Z [36m[submariner-operator]$ script_name=cluster_settings[0m
2021-07-20T04:05:59.2856087Z [36m[submariner-operator]$ exec_name=clusters.sh[0m
2021-07-20T04:05:59.2876389Z [36m[submariner-operator]$ broker=cluster1[0m
2021-07-20T04:05:59.2887530Z [36m[submariner-operator]$ declare -gA cluster_nodes[0m
2021-07-20T04:05:59.2899564Z [36m[submariner-operator]$ cluster_nodes[cluster1]=control-plane worker[0m
2021-07-20T04:05:59.2910940Z [36m[submariner-operator]$ cluster_nodes[cluster2]=control-plane worker[0m
2021-07-20T04:05:59.2922254Z [36m[submariner-operator]$ cluster_nodes[cluster3]=control-plane worker worker[0m
2021-07-20T04:05:59.2937771Z [36m[submariner-operator]$ declare -gA cluster_subm[0m
2021-07-20T04:05:59.2950957Z [36m[submariner-operator]$ cluster_subm[cluster1]=true[0m
2021-07-20T04:05:59.2963563Z [36m[submariner-operator]$ cluster_subm[cluster2]=true[0m
2021-07-20T04:05:59.2976018Z [36m[submariner-operator]$ cluster_subm[cluster3]=true[0m
2021-07-20T04:05:59.2991448Z [36m[submariner-operator]$ declare -gA cluster_cni[0m
2021-07-20T04:05:59.3004777Z [36m[submariner-operator]$ cluster_cni[cluster2]=weave[0m
2021-07-20T04:05:59.3019202Z [36m[submariner-operator]$ cluster_cni[cluster3]=weave[0m
2021-07-20T04:05:59.3033631Z [36m[submariner-operator]$ source /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings[0m
2021-07-20T04:05:59.3044162Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:05:59.3060203Z [36m[submariner-operator]$ script_name=cluster_settings[0m
2021-07-20T04:05:59.3071144Z [36m[submariner-operator]$ exec_name=clusters.sh[0m
2021-07-20T04:05:59.3093972Z [36m[submariner-operator]$ cluster_nodes[cluster1]=control-plane worker worker[0m
2021-07-20T04:05:59.3107762Z [36m[submariner-operator]$ cluster_nodes[cluster2]=control-plane worker worker[0m
2021-07-20T04:05:59.3141866Z [36m[submariner-operator]$ cat <<EOM
2021-07-20T04:05:59.3143481Z Cluster settings::
2021-07-20T04:05:59.3144366Z   broker - ${broker@Q}
2021-07-20T04:05:59.3144996Z   clusters - ${clusters[*]@Q}
2021-07-20T04:05:59.3145743Z   cni - $(typeset -p cluster_cni | cut -f 2- -d=)
2021-07-20T04:05:59.3146610Z   nodes per cluster - $(typeset -p cluster_nodes | cut -f 2- -d=)
2021-07-20T04:05:59.3147741Z   install submariner - $(typeset -p cluster_subm | cut -f 2- -d=)
2021-07-20T04:05:59.3148417Z EOM[0m
2021-07-20T04:05:59.3163933Z [36m[submariner-operator]$ typeset -p cluster_cni[0m
2021-07-20T04:05:59.3177988Z [36m[submariner-operator]$ cut -f 2- -d=[0m
2021-07-20T04:05:59.3210984Z [36m[submariner-operator]$ typeset -p cluster_nodes[0m
2021-07-20T04:05:59.3226194Z [36m[submariner-operator]$ cut -f 2- -d=[0m
2021-07-20T04:05:59.3256241Z [36m[submariner-operator]$ typeset -p cluster_subm[0m
2021-07-20T04:05:59.3270357Z [36m[submariner-operator]$ cut -f 2- -d=[0m
2021-07-20T04:05:59.3291783Z Cluster settings::
2021-07-20T04:05:59.3292640Z   broker - 'cluster1'
2021-07-20T04:05:59.3293365Z   clusters - 'cluster1' 'cluster2'
2021-07-20T04:05:59.3294147Z   cni - ([cluster2]="weave" [cluster1]="weave" )
2021-07-20T04:05:59.3295516Z   nodes per cluster - ([cluster3]="control-plane worker worker" [cluster2]="control-plane worker worker" [cluster1]="control-plane worker worker" )
2021-07-20T04:05:59.3296875Z   install submariner - ([cluster2]="true" [cluster1]="true" )
2021-07-20T04:05:59.3312550Z [36m[submariner-operator]$ rm -rf /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs[0m
2021-07-20T04:05:59.3337529Z [36m[submariner-operator]$ mkdir -p /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs[0m
2021-07-20T04:05:59.3359833Z [36m[submariner-operator]$ run_local_registry[0m
2021-07-20T04:05:59.3373178Z [36m[submariner-operator]$ run_local_registry[0m
2021-07-20T04:05:59.3386319Z [36m[submariner-operator]$ registry_running[0m
2021-07-20T04:05:59.3399182Z [36m[submariner-operator]$ registry_running[0m
2021-07-20T04:05:59.3411288Z [36m[submariner-operator]$ docker ps --filter name=^/?kind-registry$[0m
2021-07-20T04:05:59.3427013Z [36m[submariner-operator]$ grep kind-registry[0m
2021-07-20T04:05:59.6198843Z [36m[submariner-operator]$ return 0[0m
2021-07-20T04:05:59.6199938Z Deploying local registry kind-registry to serve images centrally.
2021-07-20T04:05:59.6212295Z [36m[submariner-operator]$ volume_flag=-v /dev/shm/kind-registry:/var/lib/registry[0m
2021-07-20T04:05:59.6223860Z [36m[submariner-operator]$ selinuxenabled[0m
2021-07-20T04:05:59.6249386Z [36m[submariner-operator]$ docker run -d -v /dev/shm/kind-registry:/var/lib/registry -p 127.0.0.1:5000:5000 --restart=always --name kind-registry registry:2[0m
2021-07-20T04:05:59.8941072Z Unable to find image 'registry:2' locally
2021-07-20T04:06:01.2083935Z 2: Pulling from library/registry
2021-07-20T04:06:01.5400699Z ddad3d7c1e96: Already exists
2021-07-20T04:06:01.5428883Z 6eda6749503f: Pulling fs layer
2021-07-20T04:06:01.5429498Z 363ab70c2143: Pulling fs layer
2021-07-20T04:06:01.5430246Z 5b94580856e6: Pulling fs layer
2021-07-20T04:06:01.5430709Z 12008541203a: Pulling fs layer
2021-07-20T04:06:01.5431127Z 12008541203a: Waiting
2021-07-20T04:06:01.9125284Z 6eda6749503f: Download complete
2021-07-20T04:06:02.0072931Z 5b94580856e6: Verifying Checksum
2021-07-20T04:06:02.0073936Z 5b94580856e6: Download complete
2021-07-20T04:06:02.0095159Z 363ab70c2143: Verifying Checksum
2021-07-20T04:06:02.0095680Z 363ab70c2143: Download complete
2021-07-20T04:06:02.3071148Z 12008541203a: Verifying Checksum
2021-07-20T04:06:02.3072079Z 12008541203a: Download complete
2021-07-20T04:06:02.3185725Z 6eda6749503f: Pull complete
2021-07-20T04:06:02.6614954Z 363ab70c2143: Pull complete
2021-07-20T04:06:02.7373558Z 5b94580856e6: Pull complete
2021-07-20T04:06:02.8098322Z 12008541203a: Pull complete
2021-07-20T04:06:02.8163700Z Digest: sha256:aba2bfe9f0cff1ac0618ec4a54bfefb2e685bbac67c8ebaf3b6405929b3e616f
2021-07-20T04:06:02.8183285Z Status: Downloaded newer image for registry:2
2021-07-20T04:06:02.8435924Z 0b7e3cfde0fe12b57501b9df8b93d185f3296381e9296294bded34713b3813dd
2021-07-20T04:06:03.1900432Z [36m[submariner-operator]$ docker network connect kind kind-registry[0m
2021-07-20T04:06:03.5877796Z [36m[submariner-operator]$ declare_cidrs[0m
2021-07-20T04:06:03.5893079Z [36m[submariner-operator]$ declare_cidrs[0m
2021-07-20T04:06:03.5894190Z [36m[submariner-operator]$ declare -gA cluster_CIDRs service_CIDRs global_CIDRs[0m
2021-07-20T04:06:03.5900148Z [36m[submariner-operator]$ i=1[0m
2021-07-20T04:06:03.5904407Z [36m[submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1[0m
2021-07-20T04:06:03.5919304Z [36m[submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1[0m
2021-07-20T04:06:03.5931103Z [36m[submariner-operator]$ [cluster1] cluster_CIDRs[cluster1]=10.1.0.0/16[0m
2021-07-20T04:06:03.5942403Z [36m[submariner-operator]$ [cluster1] service_CIDRs[cluster1]=100.1.0.0/16[0m
2021-07-20T04:06:03.5952411Z [36m[submariner-operator]$ [cluster1] i=$(("$i"+1))[0m
2021-07-20T04:06:03.5962091Z [36m[submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2[0m
2021-07-20T04:06:03.5974578Z [36m[submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2[0m
2021-07-20T04:06:03.5985725Z [36m[submariner-operator]$ [cluster2] cluster_CIDRs[cluster2]=10.2.0.0/16[0m
2021-07-20T04:06:03.5998807Z [36m[submariner-operator]$ [cluster2] service_CIDRs[cluster2]=100.2.0.0/16[0m
2021-07-20T04:06:03.6011672Z [36m[submariner-operator]$ [cluster2] i=$(("$i"+1))[0m
2021-07-20T04:06:03.6028895Z [36m[submariner-operator]$ [cluster2] run_all_clusters with_retries 3 create_kind_cluster[0m
2021-07-20T04:06:03.6032317Z [36m[submariner-operator]$ [cluster2] wait 288[0m
2021-07-20T04:06:03.6047027Z [36m[submariner-operator]$ [cluster2] run_all_clusters with_retries 3 create_kind_cluster[0m
2021-07-20T04:06:03.6060511Z [36m[submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 with_retries 3 create_kind_cluster[0m
2021-07-20T04:06:03.6074085Z [36m[submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 with_retries 3 create_kind_cluster[0m
2021-07-20T04:06:03.6090701Z [36m[submariner-operator]$ [cluster2] declare -A pids[0m
2021-07-20T04:06:03.6109405Z [36m[submariner-operator]$ [cluster2] eval echo cluster1 cluster2[0m
2021-07-20T04:06:03.6131376Z [36m[submariner-operator]$ [cluster1] pids[cluster1]=297[0m
2021-07-20T04:06:03.6134322Z [36m[submariner-operator]$ [cluster1] set -o pipefail[0m
2021-07-20T04:06:03.6147287Z [36m[submariner-operator]$ [cluster1] with_retries 3 create_kind_cluster[0m
2021-07-20T04:06:03.6163406Z [36m[submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /[0m
2021-07-20T04:06:03.6171431Z [36m[submariner-operator]$ [cluster2] pids[cluster2]=301[0m
2021-07-20T04:06:03.6180130Z [36m[submariner-operator]$ [cluster2] set -o pipefail[0m
2021-07-20T04:06:03.6185353Z [36m[submariner-operator]$ [cluster2] wait 301[0m
2021-07-20T04:06:03.6201725Z [36m[submariner-operator]$ [cluster2] with_retries 3 create_kind_cluster[0m
2021-07-20T04:06:03.6217674Z [36m[submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /[0m
2021-07-20T04:08:09.5378330Z [36m[submariner-operator]$ [cluster2] with_retries[0m
2021-07-20T04:08:09.5380056Z [36m[submariner-operator]$ [cluster2] iteration=1[0m
2021-07-20T04:08:09.5381439Z [36m[submariner-operator]$ [cluster2] wait 320[0m
2021-07-20T04:08:09.5382412Z [36m[submariner-operator]$ [cluster2] create_kind_cluster[0m
2021-07-20T04:08:09.5383642Z [36m[submariner-operator]$ [cluster2] create_kind_cluster[0m
2021-07-20T04:08:09.5385916Z [36m[submariner-operator]$ [cluster2] export KUBECONFIG=/go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:08:09.5388019Z [36m[submariner-operator]$ [cluster2] rm -f /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:08:09.5389600Z [36m[submariner-operator]$ [cluster2] kind get clusters[0m
2021-07-20T04:08:09.5391088Z [36m[submariner-operator]$ [cluster2] grep -q ^cluster2$[0m
2021-07-20T04:08:09.5391780Z [cluster2] No kind clusters found.
2021-07-20T04:08:09.5392454Z [cluster2] Creating KIND cluster...
2021-07-20T04:08:09.5393423Z [36m[submariner-operator]$ [cluster2] generate_cluster_yaml[0m
2021-07-20T04:08:09.5394479Z [36m[submariner-operator]$ [cluster2] generate_cluster_yaml[0m
2021-07-20T04:08:09.5395978Z [36m[submariner-operator]$ [cluster2] disable_cni=true[0m
2021-07-20T04:08:09.5396913Z [36m[submariner-operator]$ [cluster2] nodes=
2021-07-20T04:08:09.5398332Z [cluster2] - role: control-plane[0m
2021-07-20T04:08:09.5399323Z [36m[submariner-operator]$ [cluster2] nodes=
2021-07-20T04:08:09.5400322Z [cluster2] - role: control-plane
2021-07-20T04:08:09.5401534Z [cluster2] - role: worker[0m
2021-07-20T04:08:09.5402349Z [36m[submariner-operator]$ [cluster2] nodes=
2021-07-20T04:08:09.5403191Z [cluster2] - role: control-plane
2021-07-20T04:08:09.5403918Z [cluster2] - role: worker
2021-07-20T04:08:09.5404581Z [cluster2] - role: worker[0m
2021-07-20T04:08:09.5405838Z [cluster2] [36m[0m
2021-07-20T04:08:09.5406516Z [cluster2] [36m[0m
2021-07-20T04:08:09.5407342Z [36m[submariner-operator]$ [cluster2] eval "echo \"$(cat $1)\""[0m
2021-07-20T04:08:09.5409113Z [36m[submariner-operator]$ [cluster2] cat /opt/shipyard/scripts/resources/kind-cluster-config.yaml[0m
2021-07-20T04:08:09.5410848Z [36m[submariner-operator]$ [cluster2] image_flag=--image=kindest/node:v1.20.7[0m
2021-07-20T04:08:09.5411965Z [36m[submariner-operator]$ [cluster2] kind version[0m
2021-07-20T04:08:09.5412637Z [cluster2] kind v0.11.1 go1.16.4 linux/amd64
2021-07-20T04:08:09.5414097Z [36m[submariner-operator]$ [cluster2] cat /opt/shipyard/scripts/resources/cluster2-config.yaml[0m
2021-07-20T04:08:09.5415492Z [cluster2] ---
2021-07-20T04:08:09.5415965Z [cluster2] kind: Cluster
2021-07-20T04:08:09.5416804Z [cluster2] apiVersion: kind.x-k8s.io/v1alpha4
2021-07-20T04:08:09.5417600Z [cluster2] networking:
2021-07-20T04:08:09.5418196Z [cluster2]   disableDefaultCNI: true
2021-07-20T04:08:09.5418941Z [cluster2]   podSubnet: 10.2.0.0/16
2021-07-20T04:08:09.5419462Z [cluster2]   serviceSubnet: 100.2.0.0/16
2021-07-20T04:08:09.5420122Z [cluster2] containerdConfigPatches:
2021-07-20T04:08:09.5420878Z [cluster2]   - |-
2021-07-20T04:08:09.5421691Z [cluster2]     [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"]
2021-07-20T04:08:09.5422892Z [cluster2]       endpoint = ["http://kind-registry:5000"]
2021-07-20T04:08:09.5423810Z [cluster2] kubeadmConfigPatches:
2021-07-20T04:08:09.5424518Z [cluster2]   - |
2021-07-20T04:08:09.5425176Z [cluster2]     apiVersion: kubeadm.k8s.io/v1beta2
2021-07-20T04:08:09.5426039Z [cluster2]     kind: ClusterConfiguration
2021-07-20T04:08:09.5426644Z [cluster2]     metadata:
2021-07-20T04:08:09.5427113Z [cluster2]       name: config
2021-07-20T04:08:09.5427608Z [cluster2]     networking:
2021-07-20T04:08:09.5428127Z [cluster2]       podSubnet: 10.2.0.0/16
2021-07-20T04:08:09.5428671Z [cluster2]       serviceSubnet: 100.2.0.0/16
2021-07-20T04:08:09.5429285Z [cluster2]       dnsDomain: cluster2.local
2021-07-20T04:08:09.5429957Z [cluster2] nodes:
2021-07-20T04:08:09.5430664Z [cluster2] - role: control-plane
2021-07-20T04:08:09.5431334Z [cluster2] - role: worker
2021-07-20T04:08:09.5432370Z [cluster2] - role: worker
2021-07-20T04:08:09.5433830Z [36m[submariner-operator]$ [cluster2] kind create cluster --image=kindest/node:v1.20.7 --name=cluster2 --config=/opt/shipyard/scripts/resources/cluster2-config.yaml[0m
2021-07-20T04:08:09.5435037Z [cluster2] Creating cluster "cluster2" ...
2021-07-20T04:08:09.5450210Z [cluster2]  • Ensuring node image (kindest/node:v1.20.7) 🖼  ...
2021-07-20T04:08:09.5451363Z [cluster2]  ✓ Ensuring node image (kindest/node:v1.20.7) 🖼
2021-07-20T04:08:09.5452438Z [cluster2]  • Preparing nodes 📦 📦 📦   ...
2021-07-20T04:08:09.5453185Z [cluster2]  ✓ Preparing nodes 📦 📦 📦
2021-07-20T04:08:09.5454167Z [cluster2]  • Writing configuration 📜  ...
2021-07-20T04:08:09.5454970Z [cluster2]  ✓ Writing configuration 📜
2021-07-20T04:08:09.5455783Z [cluster2]  • Starting control-plane 🕹️  ...
2021-07-20T04:08:09.5456583Z [cluster2]  ✓ Starting control-plane 🕹️
2021-07-20T04:08:09.5457571Z [cluster2]  • Installing StorageClass 💾  ...
2021-07-20T04:08:09.5458676Z [cluster2]  ✓ Installing StorageClass 💾
2021-07-20T04:08:09.5459433Z [cluster2]  • Joining worker nodes 🚜  ...
2021-07-20T04:08:09.5466004Z [cluster2]  ✓ Joining worker nodes 🚜
2021-07-20T04:08:09.5466828Z [cluster2] Set kubectl context to "kind-cluster2"
2021-07-20T04:08:09.5467658Z [cluster2] You can now use your cluster with:
2021-07-20T04:08:09.5468134Z [cluster2]
2021-07-20T04:08:09.5468941Z [cluster2] kubectl cluster-info --context kind-cluster2
2021-07-20T04:08:09.5469515Z [cluster2]
2021-07-20T04:08:09.5470566Z [cluster2] Not sure what to do next? 😅  Check out https://kind.sigs.k8s.io/docs/user/quick-start/
2021-07-20T04:08:09.5472022Z [36m[submariner-operator]$ [cluster2] kind_fixup_config[0m
2021-07-20T04:08:09.5473069Z [36m[submariner-operator]$ [cluster2] kind_fixup_config[0m
2021-07-20T04:08:09.5475983Z [36m[submariner-operator]$ [cluster2] docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} cluster2-control-plane[0m
2021-07-20T04:08:09.5477463Z [36m[submariner-operator]$ [cluster2] head -n 1[0m
2021-07-20T04:08:12.2597917Z [36m[sub[36m[submariner-operator]$ [cluster1] with_retries[0m
2021-07-20T04:08:12.2599004Z [36m[submariner-operator]$ [cluster1] iteration=1[0m
2021-07-20T04:08:12.2599916Z [36m[submariner-operator]$ [cluster1] wait 317[0m
2021-07-20T04:08:12.2600835Z [36m[submariner-operator]$ [cluster1] create_kind_cluster[0m
2021-07-20T04:08:12.2601814Z [36m[submariner-operator]$ [cluster1] create_kind_cluster[0m
2021-07-20T04:08:12.2603986Z [36m[submariner-operator]$ [cluster1] export KUBECONFIG=/go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:08:12.2606680Z [36m[submariner-operator]$ [cluster1] rm -f /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:08:12.2608332Z [36m[submariner-operator]$ [cluster1] kind get clusters[0m
2021-07-20T04:08:12.2609670Z [36m[submariner-operator]$ [cluster1] grep -q ^cluster1$[0m
2021-07-20T04:08:12.2610349Z [cluster1] No kind clusters found.
2021-07-20T04:08:12.2610875Z [cluster1] Creating KIND cluster...
2021-07-20T04:08:12.2611772Z [36m[submariner-operator]$ [cluster1] generate_cluster_yaml[0m
2021-07-20T04:08:12.2612844Z [36m[submariner-operator]$ [cluster1] generate_cluster_yaml[0m
2021-07-20T04:08:12.2614430Z [36m[submariner-operator]$ [cluster1] disable_cni=true[0m
2021-07-20T04:08:12.2615514Z [36m[submariner-operator]$ [cluster1] nodes=
2021-07-20T04:08:12.2616313Z [cluster1] - role: control-plane[0m
2021-07-20T04:08:12.2617630Z [36m[submariner-operator]$ [cluster1] nodes=
2021-07-20T04:08:12.2618423Z [cluster1] - role: control-plane
2021-07-20T04:08:12.2619262Z [cluster1] - role: worker[0m
2021-07-20T04:08:12.2620372Z [36m[submariner-operator]$ [cluster1] nodes=
2021-07-20T04:08:12.2621337Z [cluster1] - role: control-plane
2021-07-20T04:08:12.2622155Z [cluster1] - role: worker
2021-07-20T04:08:12.2622796Z [cluster1] - role: worker[0m
2021-07-20T04:08:12.2623718Z [cluster1] [36m[0m
2021-07-20T04:08:12.2624436Z [cluster1] [36m[0m
2021-07-20T04:08:12.2625372Z [36m[submariner-operator]$ [cluster1] eval "echo \"$(cat $1)\""[0m
2021-07-20T04:08:12.2626974Z [36m[submariner-operator]$ [cluster1] cat /opt/shipyard/scripts/resources/kind-cluster-config.yaml[0m
2021-07-20T04:08:12.2628287Z [36m[submariner-operator]$ [cluster1] image_flag=--image=kindest/node:v1.20.7[0m
2021-07-20T04:08:12.2629283Z [36m[submariner-operator]$ [cluster1] kind version[0m
2021-07-20T04:08:12.2629936Z [cluster1] kind v0.11.1 go1.16.4 linux/amd64
2021-07-20T04:08:12.2631007Z [36m[submariner-operator]$ [cluster1] cat /opt/shipyard/scripts/resources/cluster1-config.yaml[0m
2021-07-20T04:08:12.2632068Z [cluster1] ---
2021-07-20T04:08:12.2632479Z [cluster1] kind: Cluster
2021-07-20T04:08:12.2633215Z [cluster1] apiVersion: kind.x-k8s.io/v1alpha4
2021-07-20T04:08:12.2633773Z [cluster1] networking:
2021-07-20T04:08:12.2634483Z [cluster1]   disableDefaultCNI: true
2021-07-20T04:08:12.2635036Z [cluster1]   podSubnet: 10.1.0.0/16
2021-07-20T04:08:12.2637839Z [cluster1]   serviceSubnet: 100.1.0.0/16
2021-07-20T04:08:12.2639044Z [cluster1] containerdConfigPatches:
2021-07-20T04:08:12.2640233Z [cluster1]   - |-
2021-07-20T04:08:12.2640981Z [cluster1]     [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"]
2021-07-20T04:08:12.2642290Z [cluster1]       endpoint = ["http://kind-registry:5000"]
2021-07-20T04:08:12.2643674Z [cluster1] kubeadmConfigPatches:
2021-07-20T04:08:12.2644562Z [cluster1]   - |
2021-07-20T04:08:12.2646628Z [cluster1]     apiVersion: kubeadm.k8s.io/v1beta2
2021-07-20T04:08:12.2647326Z [cluster1]     kind: ClusterConfiguration
2021-07-20T04:08:12.2648105Z [cluster1]     metadata:
2021-07-20T04:08:12.2649532Z [cluster1]       name: config
2021-07-20T04:08:12.2650040Z [cluster1]     networking:
2021-07-20T04:08:12.2650559Z [cluster1]       podSubnet: 10.1.0.0/16
2021-07-20T04:08:12.2651106Z [cluster1]       serviceSubnet: 100.1.0.0/16
2021-07-20T04:08:12.2651742Z [cluster1]       dnsDomain: cluster1.local
2021-07-20T04:08:12.2652429Z [cluster1] nodes:
2021-07-20T04:08:12.2653376Z [cluster1] - role: control-plane
2021-07-20T04:08:12.2654049Z [cluster1] - role: worker
2021-07-20T04:08:12.2654684Z [cluster1] - role: worker
2021-07-20T04:08:12.2656313Z [36m[submariner-operator]$ [cluster1] kind create cluster --image=kindest/node:v1.20.7 --name=cluster1 --config=/opt/shipyard/scripts/resources/cluster1-config.yaml[0m
2021-07-20T04:08:12.2663353Z [cluster1] Creating cluster "cluster1" ...
2021-07-20T04:08:12.2664525Z [cluster1]  • Ensuring node image (kindest/node:v1.20.7) 🖼  ...
2021-07-20T04:08:12.2665515Z [cluster1]  ✓ Ensuring node image (kindest/node:v1.20.7) 🖼
2021-07-20T04:08:12.2666444Z [cluster1]  • Preparing nodes 📦 📦 📦   ...
2021-07-20T04:08:12.2667634Z [cluster1]  ✓ Preparing nodes 📦 📦 📦
2021-07-20T04:08:12.2668467Z [cluster1]  • Writing configuration 📜  ...
2021-07-20T04:08:12.2669234Z [cluster1]  ✓ Writing configuration 📜
2021-07-20T04:08:12.2670092Z [cluster1]  • Starting control-plane 🕹️  ...
2021-07-20T04:08:12.2670915Z [cluster1]  ✓ Starting control-plane 🕹️
2021-07-20T04:08:12.2671722Z [cluster1]  • Installing StorageClass 💾  ...
2021-07-20T04:08:12.2672533Z [cluster1]  ✓ Installing StorageClass 💾
2021-07-20T04:08:12.2673298Z [cluster1]  • Joining worker nodes 🚜  ...
2021-07-20T04:08:12.2674023Z [cluster1]  ✓ Joining worker nodes 🚜
2021-07-20T04:08:12.2674817Z [cluster1] Set kubectl context to "kind-cluster1"
2021-07-20T04:08:12.2675498Z [cluster1] You can now use your cluster with:
2021-07-20T04:08:12.2675974Z [cluster1]
2021-07-20T04:08:12.2676763Z [cluster1] kubectl cluster-info --context kind-cluster1
2021-07-20T04:08:12.2677338Z [cluster1]
2021-07-20T04:08:12.2677947Z [cluster1] Have a nice day! 👋
2021-07-20T04:08:12.2678792Z [36m[submariner-operator]$ [cluster1] kind_fixup_config[0m
2021-07-20T04:08:12.2679781Z [36m[submariner-operator]$ [cluster1] kind_fixup_config[0m
2021-07-20T04:08:12.2681309Z [36m[submariner-operator]$ [cluster1] docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} cluster1-control-plane[0m
2021-07-20T04:08:12.2683106Z [36m[submariner-operator]$ [cluster1] head -n 1[0m
2021-07-20T04:08:51.4696307Z [36m[submariner-operator]$ [cluster1] sed -i -- s/server: .*/server: https:\mariner-operator]$ [cluster2] sed -i -- s/server: .*/server: https:\/\/172.18.0.7:6443/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:08:51.4699755Z [36m[submariner-operator]$ [cluster2] sed -i -- s/user: kind-.*/user: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:08:51.4701789Z [36m[submariner-operator]$ [cluster2] sed -i -- s/name: kind-.*/name: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:08:51.4704052Z [36m[submariner-operator]$ [cluster2] sed -i -- s/cluster: kind-.*/cluster: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:08:51.4706385Z [36m[submariner-operator]$ [cluster2] sed -i -- s/current-context: .*/current-context: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:08:51.4708370Z [36m[submariner-operator]$ [cluster2] chmod a+r /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:08:51.4709653Z [36m[submariner-operator]$ [cluster2] wait 1552[0m
2021-07-20T04:08:51.4710597Z [36m[submariner-operator]$ [cluster2] deploy_cluster_capabilities[0m
2021-07-20T04:08:51.4711621Z [36m[submariner-operator]$ [cluster2] deploy_cluster_capabilities[0m
2021-07-20T04:08:51.4713315Z [36m[submariner-operator]$ [cluster2] deploy_cni[0m
2021-07-20T04:08:51.4714310Z [36m[submariner-operator]$ [cluster2] deploy_cni[0m
2021-07-20T04:08:51.4715732Z [36m[submariner-operator]$ [cluster2] eval deploy_weave_cni[0m
2021-07-20T04:08:51.4716682Z [36m[submariner-operator]$ [cluster2] deploy_weave_cni[0m
2021-07-20T04:08:51.4717581Z [36m[submariner-operator]$ [cluster2] deploy_weave_cni[0m
2021-07-20T04:08:51.4718207Z [cluster2] Applying weave network...
2021-07-20T04:08:51.4719467Z [36m[submariner-operator]$ [cluster2] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.20.7&env.IPALLOC_RANGE=10.2.0.0/16[0m
2021-07-20T04:08:51.4721354Z [36m[submariner-operator]$ [cluster2] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.20.7&env.IPALLOC_RANGE=10.2.0.0/16[0m
2021-07-20T04:08:51.4723206Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.20.7&env.IPALLOC_RANGE=10.2.0.0/16[0m
2021-07-20T04:08:51.4725191Z [cluster2] serviceaccount/weave-net created
2021-07-20T04:08:51.4726506Z [cluster2] clusterrole.rbac.authorization.k8s.io/weave-net created
2021-07-20T04:08:51.4728239Z [cluster2] clusterrolebinding.rbac.authorization.k8s.io/weave-net created
2021-07-20T04:08:51.4729644Z [cluster2] role.rbac.authorization.k8s.io/weave-net created
2021-07-20T04:08:51.4731091Z [cluster2] rolebinding.rbac.authorization.k8s.io/weave-net created
2021-07-20T04:08:51.4733222Z [cluster2] daemonset.apps/weave-net created
2021-07-20T04:08:51.4734309Z [cluster2] Waiting for weave-net pods to be ready...
2021-07-20T04:08:51.4736505Z [36m[submariner-operator]$ [cluster2] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m[0m
2021-07-20T04:08:51.4738512Z [36m[submariner-operator]$ [cluster2] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m[0m
2021-07-20T04:08:51.4741205Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m[0m
2021-07-20T04:08:51.4742832Z [cluster2] pod/weave-net-m5vwl condition met
2021-07-20T04:08:51.4743656Z [cluster2] pod/weave-net-z7xsn condition met
2021-07-20T04:08:51.4744720Z [cluster2] pod/weave-net-zj887 condition met
2021-07-20T04:08:51.4745749Z [cluster2] Waiting for core-dns deployment to be ready...
2021-07-20T04:08:51.4747490Z [36m[submariner-operator]$ [cluster2] kubectl -n kube-system rollout status deploy/coredns --timeout=5m[0m
2021-07-20T04:08:51.4749800Z [36m[submariner-operator]$ [cluster2] kubectl -n kube-system rollout status deploy/coredns --timeout=5m[0m
2021-07-20T04:08:51.4751320Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 -n kube-system rollout status deploy/coredns --timeout=5m[0m
2021-07-20T04:08:51.4752496Z [cluster2] Waiting for deployment "coredns" rollout to finish: 0 of 2 updated replicas are available...
2021-07-20T04:08:51.4753523Z [cluster2] Waiting for deployment "coredns" rollout to finish: 1 of 2 updated replicas are available...
2021-07-20T04:08:51.4754382Z [cluster2] deployment "coredns" successfully rolled out
2021-07-20T04:08:51.4755301Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:08:51.4778883Z [36m[submariner-operator]$ [cluster2] wait 297[0m
2021-07-20T04:08:55.3996073Z /\/172.18.0.8:6443/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:08:55.3998263Z [36m[submariner-operator]$ [cluster1] sed -i -- s/user: kind-.*/user: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:08:55.4000456Z [36m[submariner-operator]$ [cluster1] sed -i -- s/name: kind-.*/name: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:08:55.4002844Z [36m[submariner-operator]$ [cluster1] sed -i -- s/cluster: kind-.*/cluster: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:08:55.4005705Z [36m[submariner-operator]$ [cluster1] sed -i -- s/current-context: .*/current-context: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:08:55.4008490Z [36m[submariner-operator]$ [cluster1] chmod a+r /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:08:55.4010433Z [36m[submariner-operator]$ [cluster1] wait 1645[0m
2021-07-20T04:08:55.4011483Z [36m[submariner-operator]$ [cluster1] deploy_cluster_capabilities[0m
2021-07-20T04:08:55.4012802Z [36m[submariner-operator]$ [cluster1] deploy_cluster_capabilities[0m
2021-07-20T04:08:55.4013867Z [36m[submariner-operator]$ [cluster1] deploy_cni[0m
2021-07-20T04:08:55.4014780Z [36m[submariner-operator]$ [cluster1] deploy_cni[0m
2021-07-20T04:08:55.4015726Z [36m[submariner-operator]$ [cluster1] eval deploy_weave_cni[0m
2021-07-20T04:08:55.4019516Z [36m[submariner-operator]$ [cluster1] deploy_weave_cni[0m
2021-07-20T04:08:55.4021003Z [36m[submariner-operator]$ [cluster1] deploy_weave_cni[0m
2021-07-20T04:08:55.4021961Z [cluster1] Applying weave network...
2021-07-20T04:08:55.4023381Z [36m[submariner-operator]$ [cluster1] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.20.7&env.IPALLOC_RANGE=10.1.0.0/16[0m
2021-07-20T04:08:55.4025363Z [36m[submariner-operator]$ [cluster1] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.20.7&env.IPALLOC_RANGE=10.1.0.0/16[0m
2021-07-20T04:08:55.4027893Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.20.7&env.IPALLOC_RANGE=10.1.0.0/16[0m
2021-07-20T04:08:55.4030094Z [cluster1] serviceaccount/weave-net created
2021-07-20T04:08:55.4031242Z [cluster1] clusterrole.rbac.authorization.k8s.io/weave-net created
2021-07-20T04:08:55.4033985Z [cluster1] clusterrolebinding.rbac.authorization.k8s.io/weave-net created
2021-07-20T04:08:55.4036103Z [cluster1] role.rbac.authorization.k8s.io/weave-net created
2021-07-20T04:08:55.4037923Z [cluster1] rolebinding.rbac.authorization.k8s.io/weave-net created
2021-07-20T04:08:55.4039453Z [cluster1] daemonset.apps/weave-net created
2021-07-20T04:08:55.4040299Z [cluster1] Waiting for weave-net pods to be ready...
2021-07-20T04:08:55.4041472Z [36m[submariner-operator]$ [cluster1] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m[0m
2021-07-20T04:08:55.4042928Z [36m[submariner-operator]$ [cluster1] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m[0m
2021-07-20T04:08:55.4044518Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m[0m
2021-07-20T04:08:55.4045702Z [cluster1] pod/weave-net-7hdpl condition met
2021-07-20T04:08:55.4046520Z [cluster1] pod/weave-net-n2mvk condition met
2021-07-20T04:08:55.4047317Z [cluster1] pod/weave-net-sww6t condition met
2021-07-20T04:08:55.4048171Z [cluster1] Waiting for core-dns deployment to be ready...
2021-07-20T04:08:55.4049440Z [36m[submariner-operator]$ [cluster1] kubectl -n kube-system rollout status deploy/coredns --timeout=5m[0m
2021-07-20T04:08:55.4050730Z [36m[submariner-operator]$ [cluster1] kubectl -n kube-system rollout status deploy/coredns --timeout=5m[0m
2021-07-20T04:08:55.4052118Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 -n kube-system rollout status deploy/coredns --timeout=5m[0m
2021-07-20T04:08:55.4053387Z [cluster1] Waiting for deployment "coredns" rollout to finish: 0 of 2 updated replicas are available...
2021-07-20T04:08:55.4054542Z [cluster1] Waiting for deployment "coredns" rollout to finish: 1 of 2 updated replicas are available...
2021-07-20T04:08:55.4055344Z [cluster1] deployment "coredns" successfully rolled out
2021-07-20T04:08:55.4056209Z [36m[submariner-operator]$ [cluster1] return 0[0m
2021-07-20T04:08:55.4094102Z [36m[submariner-operator]$ [cluster2] print_clusters_message[0m
2021-07-20T04:08:55.4121167Z [36m[submariner-operator]$ [cluster2] print_clusters_message[0m
2021-07-20T04:08:55.4139406Z [36m[submariner-operator]$ [cluster2] cat <<EOM
2021-07-20T04:08:55.4140220Z Your virtual cluster(s) are deployed and working properly and can be accessed with:
2021-07-20T04:08:55.4140725Z
2021-07-20T04:08:55.4142080Z export KUBECONFIG=\$(find \$(git rev-parse --show-toplevel)/output/kubeconfigs/ -type f -printf %p:)
2021-07-20T04:08:55.4142729Z
2021-07-20T04:08:55.4143499Z $ kubectl config use-context cluster1 # or cluster2, cluster3..
2021-07-20T04:08:55.4143982Z
2021-07-20T04:08:55.4144874Z To clean everthing up, just run: make clean-clusters
2021-07-20T04:08:55.4145654Z EOM[0m
2021-07-20T04:08:55.4160178Z Your virtual cluster(s) are deployed and working properly and can be accessed with:
2021-07-20T04:08:55.4160687Z
2021-07-20T04:08:55.4162108Z export KUBECONFIG=$(find $(git rev-parse --show-toplevel)/output/kubeconfigs/ -type f -printf %p:)
2021-07-20T04:08:55.4162757Z
2021-07-20T04:08:55.4163561Z $ kubectl config use-context cluster1 # or cluster2, cluster3..
2021-07-20T04:08:55.4164067Z
2021-07-20T04:08:55.4164956Z To clean everthing up, just run: make clean-clusters
2021-07-20T04:08:55.4170321Z   Successfully remade target file 'clusters'.
2021-07-20T04:08:55.4171215Z    File 'bin/subctl' does not exist.
2021-07-20T04:08:55.4172225Z      File 'bin/subctl-latest-linux-amd64' does not exist.
2021-07-20T04:08:55.4173309Z        File 'generate-embeddedyamls' does not exist.
2021-07-20T04:08:55.4174507Z          File 'generate' does not exist.
2021-07-20T04:08:55.4176421Z            File '/go/src/github.com/submariner-io/submariner-operator/bin/controller-gen' does not exist.
2021-07-20T04:08:55.4179115Z              File 'vendor/modules.txt' does not exist.
2021-07-20T04:08:55.4179895Z             Must remake target 'vendor/modules.txt'.
2021-07-20T04:08:55.4180400Z go mod download
2021-07-20T04:09:58.7378901Z go mod vendor
2021-07-20T04:10:01.4268616Z go mod tidy
2021-07-20T04:10:02.2561130Z             Successfully remade target file 'vendor/modules.txt'.
2021-07-20T04:10:02.2562518Z           Must remake target '/go/src/github.com/submariner-io/submariner-operator/bin/controller-gen'.
2021-07-20T04:10:02.2569255Z mkdir -p /go/src/github.com/submariner-io/submariner-operator/bin
2021-07-20T04:10:02.2631478Z go build -o /go/src/github.com/submariner-io/submariner-operator/bin/controller-gen sigs.k8s.io/controller-tools/cmd/controller-gen
2021-07-20T04:10:25.4366988Z           Successfully remade target file '/go/src/github.com/submariner-io/submariner-operator/bin/controller-gen'.
2021-07-20T04:10:25.4368274Z         Must remake target 'generate'.
2021-07-20T04:10:25.4614486Z /go/src/github.com/submariner-io/submariner-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt,year=2021" paths="./..."
2021-07-20T04:10:39.6323170Z         Successfully remade target file 'generate'.
2021-07-20T04:10:39.6325524Z          File 'pkg/subctl/operator/common/embeddedyamls/yamls.go' does not exist.
2021-07-20T04:10:39.6327467Z            File 'deploy/crds/submariner.io_servicediscoveries.yaml' does not exist.
2021-07-20T04:10:39.6343174Z           Must remake target 'deploy/crds/submariner.io_servicediscoveries.yaml'.
2021-07-20T04:10:39.6345660Z /go/src/github.com/submariner-io/submariner-operator/bin/controller-gen "crd:crdVersions=v1,trivialVersions=false" paths="./..." output:crd:artifacts:config=deploy/crds
2021-07-20T04:10:42.4094164Z           Successfully remade target file 'deploy/crds/submariner.io_servicediscoveries.yaml'.
2021-07-20T04:10:42.4098204Z            File 'deploy/submariner/crds/submariner.io_clusters.yaml' does not exist.
2021-07-20T04:10:42.4099812Z           Must remake target 'deploy/submariner/crds/submariner.io_clusters.yaml'.
2021-07-20T04:10:42.4102578Z cd vendor/github.com/submariner-io/submariner && /go/src/github.com/submariner-io/submariner-operator/bin/controller-gen "crd:crdVersions=v1,trivialVersions=false" paths="./..." output:crd:artifacts:config=../../../../deploy/submariner/crds
2021-07-20T04:10:43.9846438Z           Successfully remade target file 'deploy/submariner/crds/submariner.io_clusters.yaml'.
2021-07-20T04:10:43.9954987Z         Must remake target 'pkg/subctl/operator/common/embeddedyamls/yamls.go'.
2021-07-20T04:10:43.9956288Z go generate pkg/subctl/operator/common/embeddedyamls/generate.go
2021-07-20T04:10:44.7786248Z Generating yamls.go
2021-07-20T04:10:44.7791815Z deploy/crds/submariner.io_brokers.yaml
2021-07-20T04:10:44.7797406Z deploy/crds/submariner.io_submariners.yaml
2021-07-20T04:10:44.7832462Z deploy/crds/submariner.io_servicediscoveries.yaml
2021-07-20T04:10:44.7838515Z deploy/submariner/crds/submariner.io_clusters.yaml
2021-07-20T04:10:44.7844426Z deploy/submariner/crds/submariner.io_endpoints.yaml
2021-07-20T04:10:44.7849380Z deploy/submariner/crds/submariner.io_gateways.yaml
2021-07-20T04:10:44.7856532Z deploy/submariner/crds/submariner.io_clusterglobalegressips.yaml
2021-07-20T04:10:44.7864586Z deploy/submariner/crds/submariner.io_globalegressips.yaml
2021-07-20T04:10:44.7875828Z deploy/submariner/crds/submariner.io_globalingressips.yaml
2021-07-20T04:10:44.7889193Z deploy/mcsapi/crds/multicluster.x_k8s.io_serviceexports.yaml
2021-07-20T04:10:44.7901429Z deploy/mcsapi/crds/multicluster.x_k8s.io_serviceimports.yaml
2021-07-20T04:10:44.7911610Z config/broker/broker-admin/service_account.yaml
2021-07-20T04:10:44.7916517Z config/broker/broker-admin/role.yaml
2021-07-20T04:10:44.7920841Z config/broker/broker-admin/role_binding.yaml
2021-07-20T04:10:44.7925743Z config/broker/broker-client/service_account.yaml
2021-07-20T04:10:44.7931078Z config/broker/broker-client/role.yaml
2021-07-20T04:10:44.7933998Z config/broker/broker-client/role_binding.yaml
2021-07-20T04:10:44.7940639Z config/rbac/submariner-operator/service_account.yaml
2021-07-20T04:10:44.7943414Z config/rbac/submariner-operator/role.yaml
2021-07-20T04:10:44.7946868Z config/rbac/submariner-operator/role_binding.yaml
2021-07-20T04:10:44.7951976Z config/rbac/submariner-operator/cluster_role.yaml
2021-07-20T04:10:44.7956511Z config/rbac/submariner-operator/cluster_role_binding.yaml
2021-07-20T04:10:44.7960235Z config/rbac/submariner-gateway/service_account.yaml
2021-07-20T04:10:44.7965221Z config/rbac/submariner-gateway/role.yaml
2021-07-20T04:10:44.7969768Z config/rbac/submariner-gateway/role_binding.yaml
2021-07-20T04:10:44.7972483Z config/rbac/submariner-gateway/cluster_role.yaml
2021-07-20T04:10:44.7979635Z config/rbac/submariner-gateway/cluster_role_binding.yaml
2021-07-20T04:10:44.7988041Z config/rbac/submariner-route-agent/service_account.yaml
2021-07-20T04:10:44.7989617Z config/rbac/submariner-route-agent/role.yaml
2021-07-20T04:10:44.7990864Z config/rbac/submariner-route-agent/role_binding.yaml
2021-07-20T04:10:44.7992237Z config/rbac/submariner-route-agent/cluster_role.yaml
2021-07-20T04:10:44.7998108Z config/rbac/submariner-route-agent/cluster_role_binding.yaml
2021-07-20T04:10:44.8002715Z config/rbac/submariner-globalnet/service_account.yaml
2021-07-20T04:10:44.8006567Z config/rbac/submariner-globalnet/role.yaml
2021-07-20T04:10:44.8010537Z config/rbac/submariner-globalnet/role_binding.yaml
2021-07-20T04:10:44.8023668Z config/rbac/submariner-globalnet/cluster_role.yaml
2021-07-20T04:10:44.8027491Z config/rbac/submariner-globalnet/cluster_role_binding.yaml
2021-07-20T04:10:44.8031022Z config/rbac/lighthouse-agent/service_account.yaml
2021-07-20T04:10:44.8038594Z config/rbac/lighthouse-agent/cluster_role.yaml
2021-07-20T04:10:44.8043334Z config/rbac/lighthouse-agent/cluster_role_binding.yaml
2021-07-20T04:10:44.8046019Z config/rbac/lighthouse-coredns/service_account.yaml
2021-07-20T04:10:44.8049878Z config/rbac/lighthouse-coredns/cluster_role.yaml
2021-07-20T04:10:44.8058335Z config/rbac/lighthouse-coredns/cluster_role_binding.yaml
2021-07-20T04:10:44.8059880Z config/rbac/networkplugin_syncer/service_account.yaml
2021-07-20T04:10:44.8066113Z config/rbac/networkplugin_syncer/cluster_role.yaml
2021-07-20T04:10:44.8070956Z config/rbac/networkplugin_syncer/cluster_role_binding.yaml
2021-07-20T04:10:44.8075844Z config/rbac/submariner-metrics-reader/role.yaml
2021-07-20T04:10:44.8078486Z config/rbac/submariner-metrics-reader/role_binding.yaml
2021-07-20T04:10:44.8125167Z         Successfully remade target file 'pkg/subctl/operator/common/embeddedyamls/yamls.go'.
2021-07-20T04:10:44.8126448Z       Must remake target 'generate-embeddedyamls'.
2021-07-20T04:10:44.8127561Z       Successfully remade target file 'generate-embeddedyamls'.
2021-07-20T04:10:44.8390780Z     Must remake target 'bin/subctl-latest-linux-amd64'.
2021-07-20T04:10:44.8391753Z mkdir -p bin
2021-07-20T04:10:44.8415771Z target=bin/subctl-latest-linux-amd64; \
2021-07-20T04:10:44.8416803Z target=${target%.exe}; \
2021-07-20T04:10:44.8418077Z components=($(echo ${target//-/ })); \
2021-07-20T04:10:44.8418881Z GOOS=${components[-2]}; \
2021-07-20T04:10:44.8419610Z GOARCH=${components[-1]}; \
2021-07-20T04:10:44.8420185Z export GOARCH GOOS; \
2021-07-20T04:10:44.8420898Z /opt/shipyard/scripts/compile.sh \
2021-07-20T04:10:44.8422373Z 	--ldflags "-X github.com/submariner-io/submariner-operator/pkg/version.Version=latest \
2021-07-20T04:10:44.8424544Z 		   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=${DEFAULT_IMAGE_VERSION#v}" \
2021-07-20T04:10:44.8426477Z 	--noupx bin/subctl-latest-linux-amd64 ./pkg/subctl/main.go
2021-07-20T04:10:44.9117686Z [36m[submariner-operator]$ mkdir -p bin[0m
2021-07-20T04:10:44.9135277Z Building 'bin/subctl-latest-linux-amd64' (ldflags: $'-X github.com/submariner-io/submariner-operator/pkg/version.Version=latest \t\t   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=devel')
2021-07-20T04:10:44.9146387Z [36m[submariner-operator]$ ldflags=-s -w -X github.com/submariner-io/submariner-operator/pkg/version.Version=latest 		   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=devel[0m
2021-07-20T04:10:44.9164474Z [36m[submariner-operator]$ CGO_ENABLED=0 go build -trimpath -ldflags -s -w -X github.com/submariner-io/submariner-operator/pkg/version.Version=latest 		   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=devel -o bin/subctl-latest-linux-amd64 ./pkg/subctl/main.go[0m
2021-07-20T04:12:58.5432166Z     Successfully remade target file 'bin/subctl-latest-linux-amd64'.
2021-07-20T04:12:58.5433458Z   Must remake target 'bin/subctl'.
2021-07-20T04:12:58.5434562Z ln -sf subctl-latest-linux-amd64 bin/subctl
2021-07-20T04:12:58.5502453Z   Successfully remade target file 'bin/subctl'.
2021-07-20T04:12:58.5503959Z  Prerequisite 'clusters' of target 'deploy' does not exist.
2021-07-20T04:12:58.5505270Z  Prerequisite 'bin/subctl' is newer than target 'deploy'.
2021-07-20T04:12:58.5506130Z Must remake target 'deploy'.
2021-07-20T04:12:58.5508148Z /opt/shipyard/scripts/deploy.sh --cluster_settings /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings --image_tag=subctl --cluster_settings /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings
2021-07-20T04:12:58.6451506Z Running with: globalnet='false', deploytool='operator', deploytool_broker_args='', deploytool_submariner_args='', cluster_settings='/go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings', timeout=5m, image_tag=subctl, cable_driver=libreswan, service_discovery=false
2021-07-20T04:12:58.6479266Z [36m[submariner-operator]$ source /opt/shipyard/scripts/lib/utils[0m
2021-07-20T04:12:58.6498687Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:12:58.6509682Z [36m[submariner-operator]$ script_name=utils[0m
2021-07-20T04:12:58.6523468Z [36m[submariner-operator]$ exec_name=deploy.sh[0m
2021-07-20T04:12:58.6544507Z [36m[submariner-operator]$ source /opt/shipyard/scripts/lib/deploy_funcs[0m
2021-07-20T04:12:58.6562505Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:12:58.6578303Z [36m[submariner-operator]$ script_name=deploy_funcs[0m
2021-07-20T04:12:58.6602858Z [36m[submariner-operator]$ exec_name=deploy.sh[0m
2021-07-20T04:12:58.6629032Z [36m[submariner-operator]$ source /opt/shipyard/scripts/lib/cluster_settings[0m
2021-07-20T04:12:58.6643388Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:12:58.6659271Z [36m[submariner-operator]$ script_name=cluster_settings[0m
2021-07-20T04:12:58.6672862Z [36m[submariner-operator]$ exec_name=deploy.sh[0m
2021-07-20T04:12:58.6693597Z [36m[submariner-operator]$ broker=cluster1[0m
2021-07-20T04:12:58.6709874Z [36m[submariner-operator]$ declare -gA cluster_nodes[0m
2021-07-20T04:12:58.6722373Z [36m[submariner-operator]$ cluster_nodes[cluster1]=control-plane worker[0m
2021-07-20T04:12:58.6733171Z [36m[submariner-operator]$ cluster_nodes[cluster2]=control-plane worker[0m
2021-07-20T04:12:58.6744335Z [36m[submariner-operator]$ cluster_nodes[cluster3]=control-plane worker worker[0m
2021-07-20T04:12:58.6758620Z [36m[submariner-operator]$ declare -gA cluster_subm[0m
2021-07-20T04:12:58.6769664Z [36m[submariner-operator]$ cluster_subm[cluster1]=true[0m
2021-07-20T04:12:58.6782893Z [36m[submariner-operator]$ cluster_subm[cluster2]=true[0m
2021-07-20T04:12:58.6803144Z [36m[submariner-operator]$ cluster_subm[cluster3]=true[0m
2021-07-20T04:12:58.6816873Z [36m[submariner-operator]$ declare -gA cluster_cni[0m
2021-07-20T04:12:58.6830928Z [36m[submariner-operator]$ cluster_cni[cluster2]=weave[0m
2021-07-20T04:12:58.6842368Z [36m[submariner-operator]$ cluster_cni[cluster3]=weave[0m
2021-07-20T04:12:58.6855846Z [36m[submariner-operator]$ source /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings[0m
2021-07-20T04:12:58.6870233Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:12:58.6883741Z [36m[submariner-operator]$ script_name=cluster_settings[0m
2021-07-20T04:12:58.6895591Z [36m[submariner-operator]$ exec_name=deploy.sh[0m
2021-07-20T04:12:58.6917534Z [36m[submariner-operator]$ cluster_nodes[cluster1]=control-plane worker worker[0m
2021-07-20T04:12:58.6929692Z [36m[submariner-operator]$ cluster_nodes[cluster2]=control-plane worker worker[0m
2021-07-20T04:12:58.6963711Z [36m[submariner-operator]$ declare_cidrs[0m
2021-07-20T04:12:58.6975775Z [36m[submariner-operator]$ declare_cidrs[0m
2021-07-20T04:12:58.6991766Z [36m[submariner-operator]$ declare -gA cluster_CIDRs service_CIDRs global_CIDRs[0m
2021-07-20T04:12:58.7005030Z [36m[submariner-operator]$ i=1[0m
2021-07-20T04:12:58.7021116Z [36m[submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1[0m
2021-07-20T04:12:58.7035563Z [36m[submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1[0m
2021-07-20T04:12:58.7052062Z [36m[submariner-operator]$ [cluster1] cluster_CIDRs[cluster1]=10.1.0.0/16[0m
2021-07-20T04:12:58.7085637Z [36m[submariner-operator]$ [cluster1] service_CIDRs[cluster1]=100.1.0.0/16[0m
2021-07-20T04:12:58.7104114Z [36m[submariner-operator]$ [cluster1] i=$(("$i"+1))[0m
2021-07-20T04:12:58.7121404Z [36m[submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2[0m
2021-07-20T04:12:58.7133736Z [36m[submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2[0m
2021-07-20T04:12:58.7179491Z [36m[submariner-operator]$ [cluster2] cluster_CIDRs[cluster2]=10.2.0.0/16[0m
2021-07-20T04:12:58.7193071Z [36m[submariner-operator]$ [cluster2] service_CIDRs[cluster2]=100.2.0.0/16[0m
2021-07-20T04:12:58.7212497Z [36m[submariner-operator]$ [cluster2] i=$(("$i"+1))[0m
2021-07-20T04:12:58.7268762Z [36m[submariner-operator]$ [cluster2] declare_kubeconfig[0m
2021-07-20T04:12:58.7288063Z [36m[submariner-operator]$ [cluster2] declare_kubeconfig[0m
2021-07-20T04:12:58.7304759Z [36m[submariner-operator]$ [cluster2] source /opt/shipyard/scripts/lib/kubecfg[0m
2021-07-20T04:12:58.7325184Z [36m[submariner-operator]$ [cluster2] export KUBECONFIG[0m
2021-07-20T04:12:58.7340446Z [36m[submariner-operator]$ [cluster2] KUBECONFIG=$(find "${DAPPER_OUTPUT:-output}"/kubeconfigs -type f -printf %p: || :)[0m
2021-07-20T04:12:58.7360972Z [36m[submariner-operator]$ [cluster2] find /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs -type f -printf %p:[0m
2021-07-20T04:12:58.7403634Z [36m[submariner-operator]$ [cluster2] bash -c curl -Ls https://get.submariner.io | VERSION=latest DESTDIR=/go/bin bash[0m
2021-07-20T04:12:59.1788733Z Installing subctl version latest
2021-07-20T04:12:59.1789713Z   OS detected:           linux
2021-07-20T04:12:59.1792739Z   Architecture detected: amd64
2021-07-20T04:12:59.1795790Z   Download URL:          https://github.com/submariner-io/releases/releases/download/v0.9.1/subctl-v0.9.1-linux-amd64.tar.xz
2021-07-20T04:12:59.1799574Z
2021-07-20T04:12:59.1803574Z Downloading...
2021-07-20T04:13:00.6054368Z subctl-v0.9.1-linux-amd64 has been installed as /go/bin/subctl
2021-07-20T04:13:00.6567492Z This provides subctl version: v0.9.1
2021-07-20T04:13:00.6597479Z
2021-07-20T04:13:00.6600704Z please update your path (and consider adding it to ~/.profile):
2021-07-20T04:13:00.6601484Z     export PATH=$PATH:/go/bin
2021-07-20T04:13:00.6602077Z
2021-07-20T04:13:00.6743376Z [36m[submariner-operator]$ [cluster2] load_deploytool operator[0m
2021-07-20T04:13:00.6758221Z [36m[submariner-operator]$ [cluster2] load_deploytool operator[0m
2021-07-20T04:13:00.6764271Z Will deploy submariner using operator
2021-07-20T04:13:00.6778232Z [36m[submariner-operator]$ [cluster2] . /opt/shipyard/scripts/lib/deploy_operator[0m
2021-07-20T04:13:00.6794259Z [36m[submariner-operator]$ [cluster2] . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:13:00.6807750Z [36m[submariner-operator]$ [cluster2] script_name=deploy_operator[0m
2021-07-20T04:13:00.6824372Z [36m[submariner-operator]$ [cluster2] exec_name=deploy.sh[0m
2021-07-20T04:13:00.6846491Z [36m[submariner-operator]$ [cluster2] deploytool_prereqs[0m
2021-07-20T04:13:00.6859295Z [36m[submariner-operator]$ [cluster2] deploytool_prereqs[0m
2021-07-20T04:13:00.6872238Z [36m[0m
2021-07-20T04:13:00.6889407Z [36m[submariner-operator]$ [cluster2] /root/.local/bin/subctl version[0m
2021-07-20T04:13:00.8814474Z subctl version: v0.9.1
2021-07-20T04:13:00.8854686Z [36m[submariner-operator]$ [cluster2] run_if_defined pre_deploy[0m
2021-07-20T04:13:00.8868185Z [36m[submariner-operator]$ [cluster2] run_if_defined pre_deploy[0m
2021-07-20T04:13:00.8891470Z [36m[submariner-operator]$ [cluster2] type -t pre_deploy[0m
2021-07-20T04:13:00.8909680Z [36m[submariner-operator]$ [cluster2] grep -q function[0m
2021-07-20T04:13:00.8941575Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:13:00.8955358Z [36m[submariner-operator]$ [cluster2] run_subm_clusters prepare_cluster[0m
2021-07-20T04:13:00.8969846Z [36m[submariner-operator]$ [cluster2] run_subm_clusters prepare_cluster[0m
2021-07-20T04:13:00.8987957Z [36m[submariner-operator]$ [cluster2] declare -a subm_clusters[0m
2021-07-20T04:13:00.9033249Z [36m[submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 prepare_cluster[0m
2021-07-20T04:13:00.9050458Z [36m[submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 prepare_cluster[0m
2021-07-20T04:13:00.9068290Z [36m[submariner-operator]$ [cluster2] declare -A pids[0m
2021-07-20T04:13:00.9090408Z [36m[submariner-operator]$ [cluster2] eval echo cluster1 cluster2[0m
2021-07-20T04:13:00.9114432Z [36m[submariner-operator]$ [cluster1] set -o pipefail[0m
2021-07-20T04:13:00.9126403Z [36m[submariner-operator]$ [cluster1] prepare_cluster[0m
2021-07-20T04:13:00.9132575Z [36m[submariner-operator]$ [cluster1] pids[cluster1]=6303[0m
2021-07-20T04:13:00.9155586Z [36m[submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /[0m
2021-07-20T04:13:00.9160360Z [36m[submariner-operator]$ [cluster2] pids[cluster2]=6310[0m
2021-07-20T04:13:00.9185099Z [36m[submariner-operator]$ [cluster2] wait 6310[0m
2021-07-20T04:13:00.9198858Z [36m[submariner-operator]$ [cluster2] set -o pipefail[0m
2021-07-20T04:13:00.9211783Z [36m[submariner-operator]$ [cluster2] prepare_cluster[0m
2021-07-20T04:13:00.9233827Z [36m[submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /[0m
2021-07-20T04:13:01.3597838Z [36m[submariner-operator]$ [cluster1] prepare_cluster[0m
2021-07-20T04:13:01.3598943Z [36m[submariner-operator]$ [cluster1] add_subm_gateway_label[0m
2021-07-20T04:13:01.3599973Z [36m[submariner-operator]$ [cluster1] add_subm_gateway_label[0m
2021-07-20T04:13:01.3601288Z [36m[submariner-operator]$ [cluster1] kubectl label node cluster1-worker submariner.io/gateway=true --overwrite[0m
2021-07-20T04:13:01.3603309Z [36m[submariner-operator]$ [cluster1] kubectl label node cluster1-worker submariner.io/gateway=true --overwrite[0m
2021-07-20T04:13:01.3605109Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 label node cluster1-worker submariner.io/gateway=true --overwrite[0m
2021-07-20T04:13:01.3606550Z [cluster1] node/cluster1-worker labeled
2021-07-20T04:13:01.3685034Z [36m[submariner-operator]$ [cluster2] prepare_cluster[0m
2021-07-20T04:13:01.3686414Z [36m[submariner-operator]$ [cluster2] add_subm_gateway_label[0m
2021-07-20T04:13:01.3687700Z [36m[submariner-operator]$ [cluster2] add_subm_gateway_label[0m
2021-07-20T04:13:01.3689030Z [36m[submariner-operator]$ [cluster2] kubectl label node cluster2-worker submariner.io/gateway=true --overwrite[0m
2021-07-20T04:13:01.3690637Z [36m[submariner-operator]$ [cluster2] kubectl label node cluster2-worker submariner.io/gateway=true --overwrite[0m
2021-07-20T04:13:01.3694371Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 label node cluster2-worker submariner.io/gateway=true --overwrite[0m
2021-07-20T04:13:01.3696253Z [cluster2] node/cluster2-worker labeled
2021-07-20T04:13:01.3717828Z [36m[submariner-operator]$ [cluster2] wait 6303[0m
2021-07-20T04:13:01.3730703Z [36m[submariner-operator]$ [cluster2] with_context cluster1 setup_broker[0m
2021-07-20T04:13:01.3743886Z [36m[submariner-operator]$ [cluster2] with_context cluster1 setup_broker[0m
2021-07-20T04:13:01.3756953Z [36m[submariner-operator]$ [cluster1] setup_broker[0m
2021-07-20T04:13:01.3768316Z [36m[submariner-operator]$ [cluster1] setup_broker[0m
2021-07-20T04:13:01.3774402Z Installing broker...
2021-07-20T04:13:01.3786658Z [36m[submariner-operator]$ [cluster1] subctlrepver=[0m
2021-07-20T04:13:01.3807649Z [36m[submariner-operator]$ [cluster1] cd /go/src/github.com/submariner-io/submariner-operator/output[0m
2021-07-20T04:13:01.3825369Z [36m[output]$ [cluster1] /root/.local/bin/subctl deploy-broker --kubeconfig /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:13:01.4365568Z  • Setting up broker RBAC  ...
2021-07-20T04:13:01.5388445Z  ✓ Setting up broker RBAC
2021-07-20T04:13:01.5389379Z  • Deploying the Submariner operator  ...
2021-07-20T04:13:12.0813541Z  ✓ Deploying the Submariner operator
2021-07-20T04:13:12.0815173Z  ✓ Created operator CRDs
2021-07-20T04:13:12.0817745Z  ✓ Created operator namespace: submariner-operator
2021-07-20T04:13:12.0819038Z  ✓ Created operator service account and role
2021-07-20T04:13:12.0820251Z  ✓ Created lighthouse service account and role
2021-07-20T04:13:12.0821313Z  ✓ Created Lighthouse service accounts and roles
2021-07-20T04:13:12.0822331Z  ✓ Deployed the operator successfully
2021-07-20T04:13:12.0823407Z  • Deploying the broker  ...
2021-07-20T04:13:12.2174751Z  ✓ Deploying the broker
2021-07-20T04:13:12.2175595Z  ✓ The broker has been deployed
2021-07-20T04:13:12.2176332Z  • Creating broker-info.subm file  ...
2021-07-20T04:13:12.2398780Z  ✓ Creating broker-info.subm file
2021-07-20T04:13:12.2399941Z  ✓ A new IPsec PSK will be generated for broker-info.subm
2021-07-20T04:13:12.2462124Z [36m[submariner-operator]$ [cluster2] install_subm_all_clusters[0m
2021-07-20T04:13:12.2477180Z [36m[submariner-operator]$ [cluster2] install_subm_all_clusters[0m
2021-07-20T04:13:12.2492347Z [36m[submariner-operator]$ [cluster2] run_subm_clusters subctl_install_subm[0m
2021-07-20T04:13:12.2507552Z [36m[submariner-operator]$ [cluster2] run_subm_clusters subctl_install_subm[0m
2021-07-20T04:13:12.2522498Z [36m[submariner-operator]$ [cluster2] declare -a subm_clusters[0m
2021-07-20T04:13:12.2582383Z [36m[submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 subctl_install_subm[0m
2021-07-20T04:13:12.2598144Z [36m[submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 subctl_install_subm[0m
2021-07-20T04:13:12.2614336Z [36m[submariner-operator]$ [cluster2] declare -A pids[0m
2021-07-20T04:13:12.2679787Z [36m[submariner-operator]$ [cluster2] eval echo cluster1 cluster2[0m
2021-07-20T04:13:12.2707443Z [36m[submariner-operator]$ [cluster1] set -o pipefail[0m
2021-07-20T04:13:12.2719901Z [36m[submariner-operator]$ [cluster1] subctl_install_subm[0m
2021-07-20T04:13:12.2725509Z [36m[submariner-operator]$ [cluster1] pids[cluster1]=6373[0m
2021-07-20T04:13:12.2820336Z [36m[submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /[0m
2021-07-20T04:13:12.2828458Z [36m[submariner-operator]$ [cluster2] pids[cluster2]=6380[0m
2021-07-20T04:13:12.2873613Z [36m[submariner-operator]$ [cluster2] set -o pipefail[0m
2021-07-20T04:13:12.2892100Z [36m[submariner-operator]$ [cluster2] wait 6380[0m
2021-07-20T04:13:12.2920201Z [36m[submariner-operator]$ [cluster2] subctl_install_subm[0m
2021-07-20T04:13:12.2940743Z [36m[submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /[0m
2021-07-20T04:13:23.6083263Z [36m[submariner-operator]$ [cluster2] subctl_install_subm[0m
2021-07-20T04:13:23.6084609Z [36m[submariner-operator]$ [cluster2] subctlrepver=[0m
2021-07-20T04:13:23.6085791Z [36m[submariner-operator]$ [cluster2] PASS_CIDR_ARGS=false[0m
2021-07-20T04:13:23.6088347Z [36m[submariner-operator]$ [cluster2] /root/.local/bin/subctl join --kubeconfig /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2 --clusterid cluster2 --nattport 4500 --ikeport 500 --colorcodes blue --globalnet-cidr  --natt=false --cable-driver libreswan /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm[0m
2021-07-20T04:13:23.6091337Z [cluster2] * /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm says broker is at: https://172.18.0.8:6443
2021-07-20T04:13:23.6092534Z [cluster2] * There are 1 labeled nodes in the cluster:
2021-07-20T04:13:23.6093326Z [cluster2]   - cluster2-worker
2021-07-20T04:13:23.6094100Z [cluster2]  • Discovering network details  ...
2021-07-20T04:13:23.6094719Z [cluster2]     Discovered network details:
2021-07-20T04:13:23.6095770Z [cluster2]         Network plugin:  weave-net
2021-07-20T04:13:23.6096359Z [cluster2]         Service CIDRs:   [100.2.0.0/16]
2021-07-20T04:13:23.6097027Z [cluster2]         Cluster CIDRs:   [10.2.0.0/16]
2021-07-20T04:13:23.6098223Z [cluster2]  ✓ Discovering network details
2021-07-20T04:13:23.6099038Z [cluster2]  • Discovering multi cluster details  ...
2021-07-20T04:13:23.6099890Z [cluster2]  • Validating Globalnet configurations  ...
2021-07-20T04:13:23.6100772Z [cluster2]  ✓ Validating Globalnet configurations
2021-07-20T04:13:23.6101646Z [cluster2]  ✓ Discovering multi cluster details
2021-07-20T04:13:23.6102477Z [cluster2]  • Deploying the Submariner operator  ...
2021-07-20T04:13:23.6103300Z [cluster2]  ✓ Deploying the Submariner operator
2021-07-20T04:13:23.6104068Z [cluster2]  ✓ Created operator CRDs
2021-07-20T04:13:23.6105409Z [cluster2]  ✓ Created operator namespace: submariner-operator
2021-07-20T04:13:23.6106443Z [cluster2]  ✓ Created operator service account and role
2021-07-20T04:13:23.6107417Z [cluster2]  ✓ Created lighthouse service account and role
2021-07-20T04:13:23.6108668Z [cluster2]  ✓ Created Lighthouse service accounts and roles
2021-07-20T04:13:23.6109716Z [cluster2]  ✓ Deployed the operator successfully
2021-07-20T04:13:23.6110511Z [cluster2]  • Creating SA for cluster  ...
2021-07-20T04:13:23.6111411Z [cluster2]  ✓ Creating SA for cluster
2021-07-20T04:13:23.6112253Z [cluster2]  • Deploying Submariner  ...
2021-07-20T04:13:23.6113029Z [cluster2]  ✓ Deploying Submariner
2021-07-20T04:13:23.6114726Z [cluster2]  ✓ Submariner is up and running
2021-07-20T04:13:23.6138088Z [36m[submariner-operator]$ [cluster2] wait 6373[0m
2021-07-20T04:13:23.7471510Z [36m[submariner-operator]$ [cluster1] subctl_install_subm[0m
2021-07-20T04:13:23.7472628Z [36m[submariner-operator]$ [cluster1] subctlrepver=[0m
2021-07-20T04:13:23.7473699Z [36m[submariner-operator]$ [cluster1] PASS_CIDR_ARGS=false[0m
2021-07-20T04:13:23.7476922Z [36m[submariner-operator]$ [cluster1] /root/.local/bin/subctl join --kubeconfig /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1 --clusterid cluster1 --nattport 4500 --ikeport 500 --colorcodes blue --globalnet-cidr  --natt=false --cable-driver libreswan /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm[0m
2021-07-20T04:13:23.7479970Z [cluster1] * /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm says broker is at: https://172.18.0.8:6443
2021-07-20T04:13:23.7481249Z [cluster1] * There are 1 labeled nodes in the cluster:
2021-07-20T04:13:23.7482115Z [cluster1]   - cluster1-worker
2021-07-20T04:13:23.7482967Z [cluster1]  • Discovering network details  ...
2021-07-20T04:13:23.7483628Z [cluster1]     Discovered network details:
2021-07-20T04:13:23.7484446Z [cluster1]         Network plugin:  weave-net
2021-07-20T04:13:23.7485046Z [cluster1]         Service CIDRs:   [100.1.0.0/16]
2021-07-20T04:13:23.7485716Z [cluster1]         Cluster CIDRs:   [10.1.0.0/16]
2021-07-20T04:13:23.7486506Z [cluster1]  ✓ Discovering network details
2021-07-20T04:13:23.7487338Z [cluster1]  • Discovering multi cluster details  ...
2021-07-20T04:13:23.7488234Z [cluster1]  • Validating Globalnet configurations  ...
2021-07-20T04:13:23.7489133Z [cluster1]  ✓ Validating Globalnet configurations
2021-07-20T04:13:23.7490001Z [cluster1]  ✓ Discovering multi cluster details
2021-07-20T04:13:23.7490848Z [cluster1]  • Deploying the Submariner operator  ...
2021-07-20T04:13:23.7491719Z [cluster1]  ✓ Deploying the Submariner operator
2021-07-20T04:13:23.7493064Z [cluster1]  ✓ Created Lighthouse service accounts and roles
2021-07-20T04:13:23.7493926Z [cluster1]  • Creating SA for cluster  ...
2021-07-20T04:13:23.7494690Z [cluster1]  ✓ Creating SA for cluster
2021-07-20T04:13:23.7495450Z [cluster1]  • Deploying Submariner  ...
2021-07-20T04:13:23.7496228Z [cluster1]  ✓ Deploying Submariner
2021-07-20T04:13:23.7497636Z [cluster1]  ✓ Submariner is up and running
2021-07-20T04:13:23.7524831Z [36m[submariner-operator]$ [cluster2] with_context cluster2 with_retries 30 verify_gw_status[0m
2021-07-20T04:13:23.7552484Z [36m[submariner-operator]$ [cluster2] with_context cluster2 with_retries 30 verify_gw_status[0m
2021-07-20T04:13:23.7567991Z [36m[submariner-operator]$ [cluster2] with_retries 30 verify_gw_status[0m
2021-07-20T04:13:23.7648105Z [36m[submariner-operator]$ [cluster2] with_retries[0m
2021-07-20T04:13:23.7663396Z [36m[submariner-operator]$ [cluster2] iteration=1[0m
2021-07-20T04:13:23.7689367Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:13:23.7690658Z [36m[submariner-operator]$ [cluster2] wait 6415[0m
2021-07-20T04:13:23.7707128Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:13:23.7781497Z [36m[submariner-operator]$ [cluster2] sleep_duration=6[0m
2021-07-20T04:13:23.7799157Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:13:23.7820495Z [36m[submariner-operator]$ [cluster2] grep the server could not find the requested resource[0m
2021-07-20T04:13:23.9459852Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:13:23.9478325Z [36m[submariner-operator]$ [cluster2] grep connected[0m
2021-07-20T04:13:24.0689578Z  • Showing Connections  ...
2021-07-20T04:13:24.0925485Z panic: runtime error: invalid memory address or nil pointer dereference
2021-07-20T04:13:24.0926448Z [signal SIGSEGV: segmentation violation code=0x1 addr=0x60 pc=0x1a02ca9]
2021-07-20T04:13:24.0926938Z
2021-07-20T04:13:24.0927323Z goroutine 1 [running]:
2021-07-20T04:13:24.0929447Z github.com/submariner-io/submariner-operator/pkg/subctl/cmd/show.getConnectionsStatus(0xc000114050, 0xc000538ea0)
2021-07-20T04:13:24.0931034Z 	github.com/submariner-io/submariner-operator/pkg/subctl/cmd/show/connections.go:65 +0x89
2021-07-20T04:13:24.0932571Z github.com/submariner-io/submariner-operator/pkg/subctl/cmd/show.showConnections(0xc000114050, 0xc0002cf208)
2021-07-20T04:13:24.0934293Z 	github.com/submariner-io/submariner-operator/pkg/subctl/cmd/show/connections.go:132 +0x3f
2021-07-20T04:13:24.0935879Z github.com/submariner-io/submariner-operator/pkg/subctl/cmd.ExecuteMultiCluster(0x1fe2f30)
2021-07-20T04:13:24.0937568Z 	github.com/submariner-io/submariner-operator/pkg/subctl/cmd/execute.go:103 +0x1a7
2021-07-20T04:13:24.0939492Z github.com/submariner-io/submariner-operator/pkg/subctl/cmd/show.init.1.func1(0xc000220580, 0x30541c8, 0x0, 0x0)
2021-07-20T04:13:24.0941452Z 	github.com/submariner-io/submariner-operator/pkg/subctl/cmd/show/connections.go:49 +0x2d
2021-07-20T04:13:24.0942487Z github.com/spf13/cobra.(*Command).execute(0xc000220580, 0x30541c8, 0x0, 0x0, 0xc000220580, 0x30541c8)
2021-07-20T04:13:24.0943253Z 	github.com/spf13/cobra@v1.2.0/command.go:863 +0x2c2
2021-07-20T04:13:24.0943996Z github.com/spf13/cobra.(*Command).ExecuteC(0x2ff79a0, 0x30256c0, 0x0, 0xc000060778)
2021-07-20T04:13:24.0944720Z 	github.com/spf13/cobra@v1.2.0/command.go:977 +0x375
2021-07-20T04:13:24.0945417Z github.com/spf13/cobra.(*Command).Execute(...)
2021-07-20T04:13:24.0946042Z 	github.com/spf13/cobra@v1.2.0/command.go:905
2021-07-20T04:13:24.0947147Z github.com/submariner-io/submariner-operator/pkg/subctl/cmd.Execute(...)
2021-07-20T04:13:24.0948384Z 	github.com/submariner-io/submariner-operator/pkg/subctl/cmd/root.go:58
2021-07-20T04:13:24.0949095Z main.main()
2021-07-20T04:13:24.0949914Z 	command-line-arguments/main.go:31 +0x32
2021-07-20T04:13:24.0985086Z iter: 1. Clusters not yet connected. sleeping for 6 secs
2021-07-20T04:13:24.0999219Z [36m[submariner-operator]$ [cluster2] sleep 6[0m
2021-07-20T04:13:30.1078336Z [36m[submariner-operator]$ [cluster2] return 1[0m
2021-07-20T04:13:30.1116526Z [36m[submariner-operator]$ [cluster2] wait 6441[0m
2021-07-20T04:13:30.1129381Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:13:30.1163606Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:13:30.1179889Z [36m[submariner-operator]$ [cluster2] sleep_duration=6[0m
2021-07-20T04:13:30.1194608Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:13:30.1397944Z [36m[submariner-operator]$ [cluster2] grep the server could not find the requested resource[0m
2021-07-20T04:13:30.4691989Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:13:30.4710823Z [36m[submariner-operator]$ [cluster2] grep connected[0m
2021-07-20T04:13:30.8971188Z  • Showing Connections  ...
2021-07-20T04:13:31.0359284Z  ✗ Showing Connections
2021-07-20T04:13:31.0360076Z  ✗ There are no gateways detected
2021-07-20T04:13:31.2267074Z  • Showing Connections  ...
2021-07-20T04:13:31.4112482Z  ✗ Showing Connections
2021-07-20T04:13:31.4113318Z  ✗ There are no gateways detected
2021-07-20T04:13:31.4142631Z iter: 2. Clusters not yet connected. sleeping for 6 secs
2021-07-20T04:13:31.4158637Z [36m[submariner-operator]$ [cluster2] sleep 6[0m
2021-07-20T04:13:37.4192128Z [36m[submariner-operator]$ [cluster2] return 1[0m
2021-07-20T04:13:37.4246425Z [36m[submariner-operator]$ [cluster2] wait 6469[0m
2021-07-20T04:13:37.4263546Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:13:37.4378449Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:13:37.4447606Z [36m[submariner-operator]$ [cluster2] sleep_duration=6[0m
2021-07-20T04:13:37.4468988Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:13:37.4489541Z [36m[submariner-operator]$ [cluster2] grep the server could not find the requested resource[0m
2021-07-20T04:13:37.7659699Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:13:37.7677239Z [36m[submariner-operator]$ [cluster2] grep connected[0m
2021-07-20T04:13:37.9923390Z  • Showing Connections  ...
2021-07-20T04:13:37.9966187Z  ✗ Showing Connections
2021-07-20T04:13:37.9966998Z  ✗ There are no gateways detected
2021-07-20T04:13:38.0080003Z  • Showing Connections  ...
2021-07-20T04:13:38.0566061Z  ✗ Showing Connections
2021-07-20T04:13:38.0570366Z  ✗ There are no gateways detected
2021-07-20T04:13:38.0626930Z iter: 3. Clusters not yet connected. sleeping for 6 secs
2021-07-20T04:13:38.0640438Z [36m[submariner-operator]$ [cluster2] sleep 6[0m
2021-07-20T04:13:44.0667924Z [36m[submariner-operator]$ [cluster2] return 1[0m
2021-07-20T04:13:44.0732094Z [36m[submariner-operator]$ [cluster2] wait 6496[0m
2021-07-20T04:13:44.0755851Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:13:44.0772747Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:13:44.0822368Z [36m[submariner-operator]$ [cluster2] sleep_duration=6[0m
2021-07-20T04:13:44.0886879Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:13:44.0927308Z [36m[submariner-operator]$ [cluster2] grep the server could not find the requested resource[0m
2021-07-20T04:13:45.2519270Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:13:45.2532546Z [36m[submariner-operator]$ [cluster2] grep connected[0m
2021-07-20T04:13:45.4722537Z  • Showing Connections  ...
2021-07-20T04:13:45.5241902Z  ✗ Showing Connections
2021-07-20T04:13:45.5242667Z  ✗ There are no gateways detected
2021-07-20T04:13:46.1373614Z  • Showing Connections  ...
2021-07-20T04:13:46.4165836Z  ✗ Showing Connections
2021-07-20T04:13:46.4166946Z  ✗ There are no gateways detected
2021-07-20T04:13:46.4227373Z iter: 4. Clusters not yet connected. sleeping for 6 secs
2021-07-20T04:13:46.4233074Z [36m[submariner-operator]$ [cluster2] sleep 6[0m
2021-07-20T04:13:52.4260910Z [36m[submariner-operator]$ [cluster2] return 1[0m
2021-07-20T04:13:52.4343169Z [36m[submariner-operator]$ [cluster2] wait 6523[0m
2021-07-20T04:13:52.4357744Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:13:52.4450409Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:13:52.4517904Z [36m[submariner-operator]$ [cluster2] sleep_duration=6[0m
2021-07-20T04:13:52.4565864Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:13:52.4607078Z [36m[submariner-operator]$ [cluster2] grep the server could not find the requested resource[0m
2021-07-20T04:13:52.9694777Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:13:52.9714809Z [36m[submariner-operator]$ [cluster2] grep connected[0m
2021-07-20T04:13:53.3276229Z  • Showing Connections  ...
2021-07-20T04:13:53.4683059Z  ✗ Showing Connections
2021-07-20T04:13:53.4687759Z  ✗ There are no gateways detected
2021-07-20T04:13:53.5446063Z  • Showing Connections  ...
2021-07-20T04:13:53.5508683Z  ✗ Showing Connections
2021-07-20T04:13:53.5513239Z  ✗ There are no gateways detected
2021-07-20T04:13:53.5543308Z iter: 5. Clusters not yet connected. sleeping for 6 secs
2021-07-20T04:13:53.5555933Z [36m[submariner-operator]$ [cluster2] sleep 6[0m
2021-07-20T04:13:59.5593344Z [36m[submariner-operator]$ [cluster2] return 1[0m
2021-07-20T04:13:59.5632956Z [36m[submariner-operator]$ [cluster2] wait 6550[0m
2021-07-20T04:13:59.5640333Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:13:59.5743227Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:13:59.5807432Z [36m[submariner-operator]$ [cluster2] sleep_duration=6[0m
2021-07-20T04:13:59.5819514Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:13:59.5839441Z [36m[submariner-operator]$ [cluster2] grep the server could not find the requested resource[0m
2021-07-20T04:14:00.5861314Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:14:00.5892067Z [36m[submariner-operator]$ [cluster2] grep connected[0m
2021-07-20T04:14:00.9021296Z  • Showing Connections  ...
2021-07-20T04:14:01.0454842Z  ✗ Showing Connections
2021-07-20T04:14:01.0457918Z  ✗ No connections found
2021-07-20T04:14:01.0975865Z  • Showing Connections  ...
2021-07-20T04:14:01.1148101Z  ✗ Showing Connections
2021-07-20T04:14:01.1148918Z  ✗ There are no gateways detected
2021-07-20T04:14:01.1176441Z iter: 6. Clusters not yet connected. sleeping for 6 secs
2021-07-20T04:14:01.1197944Z [36m[submariner-operator]$ [cluster2] sleep 6[0m
2021-07-20T04:14:07.1230998Z [36m[submariner-operator]$ [cluster2] return 1[0m
2021-07-20T04:14:07.1319853Z [36m[submariner-operator]$ [cluster2] wait 6577[0m
2021-07-20T04:14:07.1332444Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:14:07.1356790Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:14:07.1409967Z [36m[submariner-operator]$ [cluster2] sleep_duration=6[0m
2021-07-20T04:14:07.1444235Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:14:07.1514660Z [36m[submariner-operator]$ [cluster2] grep the server could not find the requested resource[0m
2021-07-20T04:14:07.4555410Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:14:07.4567845Z [36m[submariner-operator]$ [cluster2] grep connected[0m
2021-07-20T04:14:07.6124465Z  • Showing Connections  ...
2021-07-20T04:14:07.6167692Z  ✗ Showing Connections
2021-07-20T04:14:07.6168443Z  ✗ No connections found
2021-07-20T04:14:07.6493419Z  • Showing Connections  ...
2021-07-20T04:14:07.6601838Z  ✗ Showing Connections
2021-07-20T04:14:07.6602582Z  ✗ There are no gateways detected
2021-07-20T04:14:07.6630992Z iter: 7. Clusters not yet connected. sleeping for 6 secs
2021-07-20T04:14:07.6645034Z [36m[submariner-operator]$ [cluster2] sleep 6[0m
2021-07-20T04:14:13.6671432Z [36m[submariner-operator]$ [cluster2] return 1[0m
2021-07-20T04:14:13.6713794Z [36m[submariner-operator]$ [cluster2] wait 6603[0m
2021-07-20T04:14:13.6727032Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:14:13.6762612Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:14:13.6776646Z [36m[submariner-operator]$ [cluster2] sleep_duration=6[0m
2021-07-20T04:14:13.6794166Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:14:13.6878413Z [36m[submariner-operator]$ [cluster2] grep the server could not find the requested resource[0m
2021-07-20T04:14:13.8785956Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:14:13.8804026Z [36m[submariner-operator]$ [cluster2] grep connected[0m
2021-07-20T04:14:13.9957746Z  • Showing Connections  ...
2021-07-20T04:14:14.0011388Z  ✗ Showing Connections
2021-07-20T04:14:14.0012208Z  ✗ No connections found
2021-07-20T04:14:14.0283370Z  • Showing Connections  ...
2021-07-20T04:14:14.0315839Z  ✗ Showing Connections
2021-07-20T04:14:14.0316511Z  ✗ No connections found
2021-07-20T04:14:14.0353690Z iter: 8. Clusters not yet connected. sleeping for 6 secs
2021-07-20T04:14:14.0372812Z [36m[submariner-operator]$ [cluster2] sleep 6[0m
2021-07-20T04:14:20.0405077Z [36m[submariner-operator]$ [cluster2] return 1[0m
2021-07-20T04:14:20.0447228Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:14:20.0453059Z [36m[submariner-operator]$ [cluster2] wait 6630[0m
2021-07-20T04:14:20.0460779Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:14:20.0472733Z [36m[submariner-operator]$ [cluster2] sleep_duration=6[0m
2021-07-20T04:14:20.0483806Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:14:20.0502363Z [36m[submariner-operator]$ [cluster2] grep the server could not find the requested resource[0m
2021-07-20T04:14:20.1405005Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:14:20.1420860Z [36m[submariner-operator]$ [cluster2] grep connected[0m
2021-07-20T04:14:20.2014134Z  • Showing Connections  ...
2021-07-20T04:14:20.2046736Z  ✗ Showing Connections
2021-07-20T04:14:20.2047843Z  ✗ No connections found
2021-07-20T04:14:20.2091906Z  • Showing Connections  ...
2021-07-20T04:14:20.2115055Z  ✗ Showing Connections
2021-07-20T04:14:20.2116157Z  ✗ No connections found
2021-07-20T04:14:20.2146561Z iter: 9. Clusters not yet connected. sleeping for 6 secs
2021-07-20T04:14:20.2162530Z [36m[submariner-operator]$ [cluster2] sleep 6[0m
2021-07-20T04:14:26.2194999Z [36m[submariner-operator]$ [cluster2] return 1[0m
2021-07-20T04:14:26.2234003Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:14:26.2238244Z [36m[submariner-operator]$ [cluster2] wait 6656[0m
2021-07-20T04:14:26.2249355Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:14:26.2263043Z [36m[submariner-operator]$ [cluster2] sleep_duration=6[0m
2021-07-20T04:14:26.2274080Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:14:26.2289553Z [36m[submariner-operator]$ [cluster2] grep the server could not find the requested resource[0m
2021-07-20T04:14:26.3028655Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:14:26.3120508Z [36m[submariner-operator]$ [cluster2] grep connected[0m
2021-07-20T04:14:26.3671697Z  • Showing Connections  ...
2021-07-20T04:14:26.3710523Z  ✓ Showing Connections
2021-07-20T04:14:26.3922867Z  • Showing Connections  ...
2021-07-20T04:14:26.3948013Z  ✓ Showing Connections
2021-07-20T04:14:26.3971212Z cluster2-worker  cluster2  172.18.0.5  no   libreswan     100.2.0.0/16, 10.2.0.0/16  connected  272.638µs
2021-07-20T04:14:26.3973455Z cluster1-worker  cluster1  172.18.0.4  no   libreswan     100.1.0.0/16, 10.1.0.0/16  connected  190.065µs
2021-07-20T04:14:26.3991143Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:14:26.4004696Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:14:26.4018533Z [36m[submariner-operator]$ [cluster2] with_context cluster2 connectivity_tests cluster1[0m
2021-07-20T04:14:26.4032406Z [36m[submariner-operator]$ [cluster2] with_context cluster2 connectivity_tests cluster1[0m
2021-07-20T04:14:26.4062349Z [36m[submariner-operator]$ [cluster2] connectivity_tests cluster1[0m
2021-07-20T04:14:26.4082061Z [36m[submariner-operator]$ [cluster2] connectivity_tests[0m
2021-07-20T04:14:26.4096708Z [36m[submariner-operator]$ [cluster2] target_cluster=cluster1[0m
2021-07-20T04:14:26.4110030Z [36m[submariner-operator]$ [cluster2] import_image quay.io/submariner/nettest[0m
2021-07-20T04:14:26.4123733Z [36m[submariner-operator]$ [cluster2] import_image quay.io/submariner/nettest[0m
2021-07-20T04:14:26.4139998Z [36m[submariner-operator]$ [cluster2] docker tag quay.io/submariner/nettest:latest localhost:5000/nettest:local[0m
2021-07-20T04:14:26.7700584Z Error response from daemon: No such image: quay.io/submariner/nettest:latest
2021-07-20T04:14:26.7764057Z [36m[submariner-operator]$ [cluster2] docker pull quay.io/submariner/nettest:latest[0m
2021-07-20T04:14:28.4468397Z latest: Pulling from submariner/nettest
2021-07-20T04:14:28.4515209Z df20fa9351a1: Pulling fs layer
2021-07-20T04:14:28.4516058Z 741138c6c583: Pulling fs layer
2021-07-20T04:14:28.4517082Z d34ea735672c: Pulling fs layer
2021-07-20T04:14:28.4517677Z c91acd455ec9: Pulling fs layer
2021-07-20T04:14:28.4518177Z c91acd455ec9: Waiting
2021-07-20T04:14:28.8320853Z 741138c6c583: Verifying Checksum
2021-07-20T04:14:28.8321991Z 741138c6c583: Download complete
2021-07-20T04:14:28.8868063Z d34ea735672c: Verifying Checksum
2021-07-20T04:14:28.8868712Z d34ea735672c: Download complete
2021-07-20T04:14:28.9637489Z df20fa9351a1: Verifying Checksum
2021-07-20T04:14:28.9639575Z df20fa9351a1: Download complete
2021-07-20T04:14:29.2130072Z c91acd455ec9: Verifying Checksum
2021-07-20T04:14:29.2148478Z c91acd455ec9: Download complete
2021-07-20T04:14:29.3645473Z df20fa9351a1: Pull complete
2021-07-20T04:14:29.5063251Z 741138c6c583: Pull complete
2021-07-20T04:14:30.0802273Z d34ea735672c: Pull complete
2021-07-20T04:14:30.1800214Z c91acd455ec9: Pull complete
2021-07-20T04:14:30.1848216Z Digest: sha256:63f21b9d8065d60870c2bca980dbc4dc6358edf669661576b8cbee39e3024a2d
2021-07-20T04:14:30.1870858Z Status: Downloaded newer image for quay.io/submariner/nettest:latest
2021-07-20T04:14:30.1910024Z quay.io/submariner/nettest:latest
2021-07-20T04:14:30.1947005Z [36m[submariner-operator]$ [cluster2] docker tag quay.io/submariner/nettest:latest quay.io/submariner/nettest:latest[0m
2021-07-20T04:14:30.5153426Z [36m[submariner-operator]$ [cluster2] docker tag quay.io/submariner/nettest:latest localhost:5000/nettest:local[0m
2021-07-20T04:14:30.8407750Z [36m[submariner-operator]$ [cluster2] docker push localhost:5000/nettest:local[0m
2021-07-20T04:14:31.1411829Z The push refers to repository [localhost:5000/nettest]
2021-07-20T04:14:31.1809626Z 7497bfb0ae54: Preparing
2021-07-20T04:14:31.1810205Z 17de069c349e: Preparing
2021-07-20T04:14:31.1810658Z b85398c91342: Preparing
2021-07-20T04:14:31.1811970Z 50644c29ef5a: Preparing
2021-07-20T04:14:31.5401461Z b85398c91342: Pushed
2021-07-20T04:14:31.9863799Z 7497bfb0ae54: Pushed
2021-07-20T04:14:32.0089717Z 50644c29ef5a: Pushed
2021-07-20T04:14:33.0170464Z 17de069c349e: Pushed
2021-07-20T04:14:33.0464688Z local: digest: sha256:63f21b9d8065d60870c2bca980dbc4dc6358edf669661576b8cbee39e3024a2d size: 1155
2021-07-20T04:14:33.0549070Z [36m[submariner-operator]$ [cluster2] deploy_resource /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:14:33.0563666Z [36m[submariner-operator]$ [cluster2] deploy_resource /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:14:33.0581033Z [36m[submariner-operator]$ [cluster2] resource_name=$(basename "$resource_file" ".yaml")[0m
2021-07-20T04:14:33.0603318Z [36m[submariner-operator]$ [cluster2] basename /opt/shipyard/scripts/resources/netshoot.yaml .yaml[0m
2021-07-20T04:14:33.0637786Z [36m[submariner-operator]$ [cluster2] kubectl apply -f /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:14:33.0653355Z [36m[submariner-operator]$ [cluster2] kubectl apply -f /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:14:33.0669587Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 apply -f /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:14:33.6659136Z deployment.apps/netshoot created
2021-07-20T04:14:33.6775299Z Waiting for netshoot pods to be ready.
2021-07-20T04:14:33.6812163Z [36m[submariner-operator]$ [cluster2] kubectl rollout status deploy/netshoot --timeout=5m[0m
2021-07-20T04:14:33.6894249Z [36m[submariner-operator]$ [cluster2] kubectl rollout status deploy/netshoot --timeout=5m[0m
2021-07-20T04:14:33.6908089Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 rollout status deploy/netshoot --timeout=5m[0m
2021-07-20T04:14:34.2006242Z Waiting for deployment "netshoot" rollout to finish: 0 of 1 updated replicas are available...
2021-07-20T04:14:36.1680340Z deployment "netshoot" successfully rolled out
2021-07-20T04:14:36.1754657Z [36m[submariner-operator]$ [cluster2] with_context cluster1 deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:14:36.1772038Z [36m[submariner-operator]$ [cluster2] with_context cluster1 deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:14:36.1790965Z [36m[submariner-operator]$ [cluster1] deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:14:36.1805563Z [36m[submariner-operator]$ [cluster1] deploy_resource[0m
2021-07-20T04:14:36.1825162Z [36m[submariner-operator]$ [cluster1] resource_name=$(basename "$resource_file" ".yaml")[0m
2021-07-20T04:14:36.1846074Z [36m[submariner-operator]$ [cluster1] basename /opt/shipyard/scripts/resources/nginx-demo.yaml .yaml[0m
2021-07-20T04:14:36.1876935Z [36m[submariner-operator]$ [cluster1] kubectl apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:14:36.1896177Z [36m[submariner-operator]$ [cluster1] kubectl apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:14:36.1914447Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:14:36.9715127Z deployment.apps/nginx-demo created
2021-07-20T04:14:37.0004411Z service/nginx-demo created
2021-07-20T04:14:37.1328637Z serviceexport.multicluster.x-k8s.io/nginx-demo created
2021-07-20T04:14:37.1410559Z Waiting for nginx-demo pods to be ready.
2021-07-20T04:14:37.1429587Z [36m[submariner-operator]$ [cluster1] kubectl rollout status deploy/nginx-demo --timeout=5m[0m
2021-07-20T04:14:37.1440660Z [36m[submariner-operator]$ [cluster1] kubectl rollout status deploy/nginx-demo --timeout=5m[0m
2021-07-20T04:14:37.1452926Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 rollout status deploy/nginx-demo --timeout=5m[0m
2021-07-20T04:14:37.8081404Z Waiting for deployment "nginx-demo" rollout to finish: 0 of 2 updated replicas are available...
2021-07-20T04:14:45.8000190Z Waiting for deployment "nginx-demo" rollout to finish: 1 of 2 updated replicas are available...
2021-07-20T04:14:45.9445802Z deployment "nginx-demo" successfully rolled out
2021-07-20T04:14:45.9498204Z [36m[submariner-operator]$ [cluster2] netshoot_pod=$(kubectl get pods -l app=netshoot | awk 'FNR == 2 {print $1}')[0m
2021-07-20T04:14:45.9519107Z [36m[submariner-operator]$ [cluster2] kubectl get pods -l app=netshoot[0m
2021-07-20T04:14:45.9550554Z [36m[submariner-operator]$ [cluster2] awk FNR == 2 {print $1}[0m
2021-07-20T04:14:45.9588438Z [36m[submariner-operator]$ [cluster2] kubectl get pods -l app=netshoot[0m
2021-07-20T04:14:45.9606334Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pods -l app=netshoot[0m
2021-07-20T04:14:46.3194550Z [36m[submariner-operator]$ [cluster2] nginx_svc_ip=$(with_context "$target_cluster" get_svc_ip nginx-demo)[0m
2021-07-20T04:14:46.3216741Z [36m[submariner-operator]$ [cluster2] with_context cluster1 get_svc_ip nginx-demo[0m
2021-07-20T04:14:46.3229029Z [36m[submariner-operator]$ [cluster2] with_context cluster1 get_svc_ip nginx-demo[0m
2021-07-20T04:14:46.3244038Z [36m[submariner-operator]$ [cluster1] get_svc_ip nginx-demo[0m
2021-07-20T04:14:46.3256643Z [36m[submariner-operator]$ [cluster1] get_svc_ip[0m
2021-07-20T04:14:46.3272303Z [36m[submariner-operator]$ [cluster1] svc_ip=$(kubectl --context="$cluster" get svc -l "app=${svc_name}" | awk 'FNR == 2 {print $3}')[0m
2021-07-20T04:14:46.3291136Z [36m[submariner-operator]$ [cluster1] kubectl --context=cluster1 get svc -l app=nginx-demo[0m
2021-07-20T04:14:46.3310327Z [36m[submariner-operator]$ [cluster1] awk FNR == 2 {print $3}[0m
2021-07-20T04:14:46.3313977Z [36m[submariner-operator]$ [cluster1] kubectl --context=cluster1 get svc -l app=nginx-demo[0m
2021-07-20T04:14:46.3329254Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc -l app=nginx-demo[0m
2021-07-20T04:14:46.6389109Z [36m[submariner-operator]$ [cluster2] with_retries 5 test_connection netshoot-57677568b8-4zwwf 100.1.58.233[0m
2021-07-20T04:14:46.6401683Z [36m[submariner-operator]$ [cluster2] with_retries 5 test_connection netshoot-57677568b8-4zwwf 100.1.58.233[0m
2021-07-20T04:14:46.6415565Z [36m[submariner-operator]$ [cluster2] iteration=1[0m
2021-07-20T04:14:46.6443725Z [36m[submariner-operator]$ [cluster2] wait 6814[0m
2021-07-20T04:14:46.6450936Z [36m[submariner-operator]$ [cluster2] test_connection netshoot-57677568b8-4zwwf 100.1.58.233[0m
2021-07-20T04:14:46.6465515Z [36m[submariner-operator]$ [cluster2] test_connection[0m
2021-07-20T04:14:46.6495732Z Attempting connectivity between clusters - netshoot-57677568b8-4zwwf --> 100.1.58.233
2021-07-20T04:14:46.6558821Z [36m[submariner-operator]$ [cluster2] wait_time=30[0m
2021-07-20T04:14:46.6575560Z [36m[submariner-operator]$ [cluster2] start_time=$(date +%s)[0m
2021-07-20T04:14:46.6606746Z [36m[submariner-operator]$ [cluster2] date +%s[0m
2021-07-20T04:14:46.6642384Z [36m[submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-4zwwf -- curl --output /dev/null -m 30 --silent --head --fail 100.1.58.233[0m
2021-07-20T04:14:46.6660009Z [36m[submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-4zwwf -- curl --output /dev/null -m 30 --silent --head --fail 100.1.58.233[0m
2021-07-20T04:14:46.6678981Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 exec netshoot-57677568b8-4zwwf -- curl --output /dev/null -m 30 --silent --head --fail 100.1.58.233[0m
2021-07-20T04:14:47.1550921Z Connection test was successful!
2021-07-20T04:14:47.1569678Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:14:47.1582568Z [36m[submariner-operator]$ [cluster2] remove_resource /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:14:47.1596345Z [36m[submariner-operator]$ [cluster2] remove_resource /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:14:47.1610133Z [36m[submariner-operator]$ [cluster2] kubectl delete -f /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:14:47.1621700Z [36m[submariner-operator]$ [cluster2] kubectl delete -f /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:14:47.1640114Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 delete -f /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:14:47.5018225Z deployment.apps "netshoot" deleted
2021-07-20T04:14:47.5161114Z [36m[submariner-operator]$ [cluster2] with_context cluster1 remove_resource /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:14:47.5174987Z [36m[submariner-operator]$ [cluster2] with_context cluster1 remove_resource /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:14:47.5189264Z [36m[submariner-operator]$ [cluster1] remove_resource /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:14:47.5200391Z [36m[submariner-operator]$ [cluster1] remove_resource[0m
2021-07-20T04:14:47.5263118Z [36m[submariner-operator]$ [cluster1] kubectl delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:14:47.5277062Z [36m[submariner-operator]$ [cluster1] kubectl delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:14:47.5369667Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:14:47.8585743Z deployment.apps "nginx-demo" deleted
2021-07-20T04:14:47.9239877Z service "nginx-demo" deleted
2021-07-20T04:14:47.9602431Z serviceexport.multicluster.x-k8s.io "nginx-demo" deleted
2021-07-20T04:14:48.0385449Z [36m[submariner-operator]$ [cluster2] run_if_defined post_deploy[0m
2021-07-20T04:14:48.0399377Z [36m[submariner-operator]$ [cluster2] run_if_defined post_deploy[0m
2021-07-20T04:14:48.0418564Z [36m[submariner-operator]$ [cluster2] type -t post_deploy[0m
2021-07-20T04:14:48.0442983Z [36m[submariner-operator]$ [cluster2] grep -q function[0m
2021-07-20T04:14:48.0520501Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:14:48.0568738Z Successfully remade target file 'deploy'.
2021-07-20T04:14:48.0623350Z make[1]: Leaving directory '/go/src/github.com/submariner-io/submariner-operator'
2021-07-20T04:14:48.0692668Z   Successfully remade target file 'deploy-latest'.
2021-07-20T04:14:48.0695224Z      File 'clusters' does not exist.
2021-07-20T04:14:48.0695966Z     Must remake target 'clusters'.
2021-07-20T04:14:48.0697907Z /opt/shipyard/scripts/clusters.sh --cluster_settings /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings
2021-07-20T04:14:48.2978745Z Running with: k8s_version=1.20.7, olm_version=0.14.1, olm=false, globalnet=false, registry_inmemory=true, cluster_settings=/go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings, timeout=5m
2021-07-20T04:14:48.3000413Z [36m[submariner-operator]$ source /opt/shipyard/scripts/lib/utils[0m
2021-07-20T04:14:48.3020726Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:14:48.3075560Z [36m[submariner-operator]$ script_name=utils[0m
2021-07-20T04:14:48.3088319Z [36m[submariner-operator]$ exec_name=clusters.sh[0m
2021-07-20T04:14:48.3112575Z [36m[submariner-operator]$ source /opt/shipyard/scripts/lib/cluster_settings[0m
2021-07-20T04:14:48.3198378Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:14:48.3212854Z [36m[submariner-operator]$ script_name=cluster_settings[0m
2021-07-20T04:14:48.3250427Z [36m[submariner-operator]$ exec_name=clusters.sh[0m
2021-07-20T04:14:48.3297984Z [36m[submariner-operator]$ broker=cluster1[0m
2021-07-20T04:14:48.3318432Z [36m[submariner-operator]$ declare -gA cluster_nodes[0m
2021-07-20T04:14:48.3333466Z [36m[submariner-operator]$ cluster_nodes[cluster1]=control-plane worker[0m
2021-07-20T04:14:48.3503947Z [36m[submariner-operator]$ cluster_nodes[cluster2]=control-plane worker[0m
2021-07-20T04:14:48.3518649Z [36m[submariner-operator]$ cluster_nodes[cluster3]=control-plane worker worker[0m
2021-07-20T04:14:48.3532614Z [36m[submariner-operator]$ declare -gA cluster_subm[0m
2021-07-20T04:14:48.3548596Z [36m[submariner-operator]$ cluster_subm[cluster1]=true[0m
2021-07-20T04:14:48.3575436Z [36m[submariner-operator]$ cluster_subm[cluster2]=true[0m
2021-07-20T04:14:48.3677497Z [36m[submariner-operator]$ cluster_subm[cluster3]=true[0m
2021-07-20T04:14:48.3704703Z [36m[submariner-operator]$ declare -gA cluster_cni[0m
2021-07-20T04:14:48.3719648Z [36m[submariner-operator]$ cluster_cni[cluster2]=weave[0m
2021-07-20T04:14:48.3739977Z [36m[submariner-operator]$ cluster_cni[cluster3]=weave[0m
2021-07-20T04:14:48.3793252Z [36m[submariner-operator]$ source /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings[0m
2021-07-20T04:14:48.3810092Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:14:48.3826303Z [36m[submariner-operator]$ script_name=cluster_settings[0m
2021-07-20T04:14:48.3841194Z [36m[submariner-operator]$ exec_name=clusters.sh[0m
2021-07-20T04:14:48.3872099Z [36m[submariner-operator]$ cluster_nodes[cluster1]=control-plane worker worker[0m
2021-07-20T04:14:48.3902081Z [36m[submariner-operator]$ cluster_nodes[cluster2]=control-plane worker worker[0m
2021-07-20T04:14:48.4022521Z [36m[submariner-operator]$ cat <<EOM
2021-07-20T04:14:48.4023350Z Cluster settings::
2021-07-20T04:14:48.4023999Z   broker - ${broker@Q}
2021-07-20T04:14:48.4024644Z   clusters - ${clusters[*]@Q}
2021-07-20T04:14:48.4025410Z   cni - $(typeset -p cluster_cni | cut -f 2- -d=)
2021-07-20T04:14:48.4026293Z   nodes per cluster - $(typeset -p cluster_nodes | cut -f 2- -d=)
2021-07-20T04:14:48.4027359Z   install submariner - $(typeset -p cluster_subm | cut -f 2- -d=)
2021-07-20T04:14:48.4036447Z EOM[0m
2021-07-20T04:14:48.4072673Z [36m[submariner-operator]$ typeset -p cluster_cni[0m
2021-07-20T04:14:48.4109733Z [36m[submariner-operator]$ cut -f 2- -d=[0m
2021-07-20T04:14:48.4304719Z [36m[submariner-operator]$ typeset -p cluster_nodes[0m
2021-07-20T04:14:48.4322264Z [36m[submariner-operator]$ cut -f 2- -d=[0m
2021-07-20T04:14:48.4470215Z [36m[submariner-operator]$ typeset -p cluster_subm[0m
2021-07-20T04:14:48.4484247Z [36m[submariner-operator]$ cut -f 2- -d=[0m
2021-07-20T04:14:48.4576313Z Cluster settings::
2021-07-20T04:14:48.4577553Z   broker - 'cluster1'
2021-07-20T04:14:48.4578262Z   clusters - 'cluster1' 'cluster2'
2021-07-20T04:14:48.4579059Z   cni - ([cluster2]="weave" [cluster1]="weave" )
2021-07-20T04:14:48.4580442Z   nodes per cluster - ([cluster3]="control-plane worker worker" [cluster2]="control-plane worker worker" [cluster1]="control-plane worker worker" )
2021-07-20T04:14:48.4581918Z   install submariner - ([cluster2]="true" [cluster1]="true" )
2021-07-20T04:14:48.4594767Z [36m[submariner-operator]$ rm -rf /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs[0m
2021-07-20T04:14:48.4673232Z [36m[submariner-operator]$ mkdir -p /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs[0m
2021-07-20T04:14:48.4792910Z [36m[submariner-operator]$ run_local_registry[0m
2021-07-20T04:14:48.4819998Z [36m[submariner-operator]$ run_local_registry[0m
2021-07-20T04:14:48.4832164Z [36m[submariner-operator]$ registry_running[0m
2021-07-20T04:14:48.4864749Z [36m[submariner-operator]$ registry_running[0m
2021-07-20T04:14:48.4890045Z [36m[submariner-operator]$ docker ps --filter name=^/?kind-registry$[0m
2021-07-20T04:14:48.4938974Z [36m[submariner-operator]$ grep kind-registry[0m
2021-07-20T04:14:48.8407110Z 0b7e3cfde0fe   registry:2   "/entrypoint.sh /etc…"   8 minutes ago   Up 8 minutes   127.0.0.1:5000->5000/tcp   kind-registry
2021-07-20T04:14:48.8422065Z [36m[submariner-operator]$ return 0[0m
2021-07-20T04:14:48.8428060Z Local registry kind-registry already running.
2021-07-20T04:14:48.8439435Z [36m[submariner-operator]$ declare_cidrs[0m
2021-07-20T04:14:48.8451462Z [36m[submariner-operator]$ declare_cidrs[0m
2021-07-20T04:14:48.8464205Z [36m[submariner-operator]$ declare -gA cluster_CIDRs service_CIDRs global_CIDRs[0m
2021-07-20T04:14:48.8478191Z [36m[submariner-operator]$ i=1[0m
2021-07-20T04:14:48.8491948Z [36m[submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1[0m
2021-07-20T04:14:48.8505795Z [36m[submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1[0m
2021-07-20T04:14:48.8522448Z [36m[submariner-operator]$ [cluster1] cluster_CIDRs[cluster1]=10.1.0.0/16[0m
2021-07-20T04:14:48.8536370Z [36m[submariner-operator]$ [cluster1] service_CIDRs[cluster1]=100.1.0.0/16[0m
2021-07-20T04:14:48.8551734Z [36m[submariner-operator]$ [cluster1] i=$(("$i"+1))[0m
2021-07-20T04:14:48.8565657Z [36m[submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2[0m
2021-07-20T04:14:48.8581372Z [36m[submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2[0m
2021-07-20T04:14:48.8598475Z [36m[submariner-operator]$ [cluster2] cluster_CIDRs[cluster2]=10.2.0.0/16[0m
2021-07-20T04:14:48.8614190Z [36m[submariner-operator]$ [cluster2] service_CIDRs[cluster2]=100.2.0.0/16[0m
2021-07-20T04:14:48.8626827Z [36m[submariner-operator]$ [cluster2] i=$(("$i"+1))[0m
2021-07-20T04:14:48.8645531Z [36m[submariner-operator]$ [cluster2] wait 7024[0m
2021-07-20T04:14:48.8650846Z [36m[submariner-operator]$ [cluster2] run_all_clusters with_retries 3 create_kind_cluster[0m
2021-07-20T04:14:48.8665231Z [36m[submariner-operator]$ [cluster2] run_all_clusters with_retries 3 create_kind_cluster[0m
2021-07-20T04:14:48.8684511Z [36m[submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 with_retries 3 create_kind_cluster[0m
2021-07-20T04:14:48.8702282Z [36m[submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 with_retries 3 create_kind_cluster[0m
2021-07-20T04:14:48.8720024Z [36m[submariner-operator]$ [cluster2] declare -A pids[0m
2021-07-20T04:14:48.8751015Z [36m[submariner-operator]$ [cluster2] eval echo cluster1 cluster2[0m
2021-07-20T04:14:48.8788814Z [36m[submariner-operator]$ [cluster1] set -o pipefail[0m
2021-07-20T04:14:48.8800375Z [36m[submariner-operator]$ [cluster1] pids[cluster1]=7033[0m
2021-07-20T04:14:48.8804858Z [36m[submariner-operator]$ [cluster1] with_retries 3 create_kind_cluster[0m
2021-07-20T04:14:48.8826692Z [36m[submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /[0m
2021-07-20T04:14:48.8838522Z [36m[submariner-operator]$ [cluster2] pids[cluster2]=7038[0m
2021-07-20T04:14:48.8862196Z [36m[submariner-operator]$ [cluster2] wait 7038[0m
2021-07-20T04:14:48.8869586Z [36m[submariner-operator]$ [cluster2] set -o pipefail[0m
2021-07-20T04:14:48.8890499Z [36m[submariner-operator]$ [cluster2] with_retries 3 create_kind_cluster[0m
2021-07-20T04:14:48.8908671Z [36m[submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /[0m
2021-07-20T04:14:56.8308779Z [36m[submariner-operator]$ [cluster2] with_retries[0m
2021-07-20T04:14:56.8310420Z [36m[submariner-operator]$ [cluster2] iteration=1[0m
2021-07-20T04:14:56.8311653Z [36m[submariner-operator]$ [cluster2] wait 7060[0m
2021-07-20T04:14:56.8312792Z [36m[submariner-operator]$ [cluster2] create_kind_cluster[0m
2021-07-20T04:14:56.8314009Z [36m[submariner-operator]$ [cluster2] create_kind_cluster[0m
2021-07-20T04:14:56.8315802Z [36m[submariner-operator]$ [cluster2] export KUBECONFIG=/go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:14:56.8317999Z [36m[submariner-operator]$ [cluster2] rm -f /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:14:56.8320008Z [36m[submariner-operator]$ [cluster2] kind get clusters[0m
2021-07-20T04:14:56.8321263Z [36m[submariner-operator]$ [cluster2] grep -q ^cluster2$[0m
2021-07-20T04:14:56.8322200Z [cluster2] KIND cluster already exists, skipping its creation...
2021-07-20T04:14:56.8323405Z [36m[submariner-operator]$ [cluster2] kind export kubeconfig --name=cluster2[0m
2021-07-20T04:14:56.8324562Z [cluster2] Set kubectl context to "kind-cluster2"
2021-07-20T04:14:56.8325617Z [36m[submariner-operator]$ [cluster2] kind_fixup_config[0m
2021-07-20T04:14:56.8328616Z [36m[submariner-operator]$ [cluster2] wait 7033[0m
2021-07-20T04:14:56.8330137Z [36m[submariner-operator]$ [cluster2] kind_fixup_config[0m
2021-07-20T04:14:56.8332489Z [36m[submariner-operator]$ [cluster2] docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} cluster2-control-plane[0m
2021-07-20T04:14:56.8334175Z [36m[submariner-operator]$ [cluster2] head -n 1[0m
2021-07-20T04:14:56.8335889Z [36m[submariner-operator]$ [cluster2] sed -i -- s/server: .*/server: https:\/\/172.18.0.7:6443/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:14:56.8339032Z [36m[submariner-operator]$ [cluster2] sed -i -- s/user: kind-.*/user: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:14:56.8341578Z [36m[submariner-operator]$ [cluster2] sed -i -- s/name: kind-.*/name: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:14:56.8344526Z [36m[submariner-operator]$ [cluster2] sed -i -- s/cluster: kind-.*/cluster: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:14:56.8346927Z [36m[submariner-operator]$ [cluster2] sed -i -- s/current-context: .*/current-context: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:14:56.8349341Z [36m[submariner-operator]$ [cluster2] chmod a+r /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2[0m
2021-07-20T04:14:56.8350819Z [36m[submariner-operator]$ [cluster2] return[0m
2021-07-20T04:14:56.8351783Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:14:56.8825849Z [36m[submariner-operator]$ [cluster1] with_retries[0m
2021-07-20T04:14:56.8827077Z [36m[submariner-operator]$ [cluster1] iteration=1[0m
2021-07-20T04:14:56.8828058Z [36m[submariner-operator]$ [cluster1] wait 7049[0m
2021-07-20T04:14:56.8829372Z [36m[submariner-operator]$ [cluster1] create_kind_cluster[0m
2021-07-20T04:14:56.8830603Z [36m[submariner-operator]$ [cluster1] create_kind_cluster[0m
2021-07-20T04:14:56.8832283Z [36m[submariner-operator]$ [cluster1] export KUBECONFIG=/go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:14:56.8835453Z [36m[submariner-operator]$ [cluster1] rm -f /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:14:56.8837119Z [36m[submariner-operator]$ [cluster1] kind get clusters[0m
2021-07-20T04:14:56.8838151Z [36m[submariner-operator]$ [cluster1] grep -q ^cluster1$[0m
2021-07-20T04:14:56.8839008Z [cluster1] KIND cluster already exists, skipping its creation...
2021-07-20T04:14:56.8840310Z [36m[submariner-operator]$ [cluster1] kind export kubeconfig --name=cluster1[0m
2021-07-20T04:14:56.8841526Z [cluster1] Set kubectl context to "kind-cluster1"
2021-07-20T04:14:56.8842831Z [36m[submariner-operator]$ [cluster1] kind_fixup_config[0m
2021-07-20T04:14:56.8843839Z [36m[submariner-operator]$ [cluster1] kind_fixup_config[0m
2021-07-20T04:14:56.8845580Z [36m[submariner-operator]$ [cluster1] docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} cluster1-control-plane[0m
2021-07-20T04:14:56.8846966Z [36m[submariner-operator]$ [cluster1] head -n 1[0m
2021-07-20T04:14:56.8848887Z [36m[submariner-operator]$ [cluster1] sed -i -- s/server: .*/server: https:\/\/172.18.0.8:6443/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:14:56.8851183Z [36m[submariner-operator]$ [cluster1] sed -i -- s/user: kind-.*/user: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:14:56.8853377Z [36m[submariner-operator]$ [cluster1] sed -i -- s/name: kind-.*/name: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:14:56.8859035Z [36m[submariner-operator]$ [cluster1] sed -i -- s/cluster: kind-.*/cluster: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:14:56.8861668Z [36m[submariner-operator]$ [cluster1] sed -i -- s/current-context: .*/current-context: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:14:56.8864287Z [36m[submariner-operator]$ [cluster1] chmod a+r /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1[0m
2021-07-20T04:14:56.8865880Z [36m[submariner-operator]$ [cluster1] return[0m
2021-07-20T04:14:56.8866837Z [36m[submariner-operator]$ [cluster1] return 0[0m
2021-07-20T04:14:56.8867911Z [36m[submariner-operator]$ [cluster2] print_clusters_message[0m
2021-07-20T04:14:56.8877945Z [36m[submariner-operator]$ [cluster2] print_clusters_message[0m
2021-07-20T04:14:56.8897669Z [36m[submariner-operator]$ [cluster2] cat <<EOM
2021-07-20T04:14:56.8898520Z Your virtual cluster(s) are deployed and working properly and can be accessed with:
2021-07-20T04:14:56.8899045Z
2021-07-20T04:14:56.8900052Z export KUBECONFIG=\$(find \$(git rev-parse --show-toplevel)/output/kubeconfigs/ -type f -printf %p:)
2021-07-20T04:14:56.8900664Z
2021-07-20T04:14:56.8901596Z $ kubectl config use-context cluster1 # or cluster2, cluster3..
2021-07-20T04:14:56.8902099Z
2021-07-20T04:14:56.8903126Z To clean everthing up, just run: make clean-clusters
2021-07-20T04:14:56.8903953Z EOM[0m
2021-07-20T04:14:56.8912606Z Your virtual cluster(s) are deployed and working properly and can be accessed with:
2021-07-20T04:14:56.8913176Z
2021-07-20T04:14:56.8914286Z export KUBECONFIG=$(find $(git rev-parse --show-toplevel)/output/kubeconfigs/ -type f -printf %p:)
2021-07-20T04:14:56.8914922Z
2021-07-20T04:14:56.8915698Z $ kubectl config use-context cluster1 # or cluster2, cluster3..
2021-07-20T04:14:56.8916198Z
2021-07-20T04:14:56.8916928Z To clean everthing up, just run: make clean-clusters
2021-07-20T04:14:56.8922233Z     Successfully remade target file 'clusters'.
2021-07-20T04:14:56.8923124Z      File 'preload-images' does not exist.
2021-07-20T04:14:56.8923884Z        File 'images' does not exist.
2021-07-20T04:14:56.8929660Z          File 'package/.image.submariner-operator' does not exist.
2021-07-20T04:14:56.9010285Z find: 'bin/submariner-operator': No such file or directory
2021-07-20T04:14:57.2213196Z              File 'bin/submariner-operator' does not exist.
2021-07-20T04:14:57.2218273Z                File 'generate-embeddedyamls' does not exist.
2021-07-20T04:14:57.2219218Z                  File 'generate' does not exist.
2021-07-20T04:14:57.2219958Z                 Must remake target 'generate'.
2021-07-20T04:14:57.2244479Z /go/src/github.com/submariner-io/submariner-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt,year=2021" paths="./..."
2021-07-20T04:15:01.5335206Z                 Successfully remade target file 'generate'.
2021-07-20T04:15:01.5451227Z               Must remake target 'generate-embeddedyamls'.
2021-07-20T04:15:01.5452876Z               Successfully remade target file 'generate-embeddedyamls'.
2021-07-20T04:15:01.5454043Z             Must remake target 'bin/submariner-operator'.
2021-07-20T04:15:01.5454662Z /opt/shipyard/scripts/compile.sh \
2021-07-20T04:15:01.5456018Z --ldflags "-X=github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d" \
2021-07-20T04:15:01.5457822Z bin/submariner-operator main.go
2021-07-20T04:15:01.5973740Z [36m[submariner-operator]$ mkdir -p bin[0m
2021-07-20T04:15:01.5986295Z Building 'bin/submariner-operator' (ldflags: '-X=github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d')
2021-07-20T04:15:01.5995223Z [36m[submariner-operator]$ ldflags=-s -w -X=github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d[0m
2021-07-20T04:15:01.6005127Z [36m[submariner-operator]$ CGO_ENABLED=0 go build -trimpath -ldflags -s -w -X=github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d -o bin/submariner-operator main.go[0m
2021-07-20T04:15:21.4448322Z [36m[submariner-operator]$ upx bin/submariner-operator[0m
2021-07-20T04:15:29.2237678Z                        Ultimate Packer for eXecutables
2021-07-20T04:15:29.2239622Z                           Copyright (C) 1996 - 2020
2021-07-20T04:15:29.2240586Z UPX 3.96        Markus Oberhumer, Laszlo Molnar & John Reiser   Jan 23rd 2020
2021-07-20T04:15:29.2241217Z
2021-07-20T04:15:29.2241829Z         File size         Ratio      Format      Name
2021-07-20T04:15:29.2251587Z    --------------------   ------   -----------   -----------
2021-07-20T04:15:29.2252548Z   34459648 ->   9933112   28.83%   linux/amd64   submariner-operator
2021-07-20T04:15:29.2252959Z
2021-07-20T04:15:29.2253317Z Packed 1 file.
2021-07-20T04:15:29.2254165Z             Successfully remade target file 'bin/submariner-operator'.
2021-07-20T04:15:29.2256156Z            Prerequisite 'bin/submariner-operator' is newer than target 'package/Dockerfile.submariner-operator'.
2021-07-20T04:15:29.2258239Z           Must remake target 'package/Dockerfile.submariner-operator'.
2021-07-20T04:15:29.2259628Z           Successfully remade target file 'package/Dockerfile.submariner-operator'.
2021-07-20T04:15:29.2260852Z            File 'FORCE_IMAGE' does not exist.
2021-07-20T04:15:29.2261567Z           Must remake target 'FORCE_IMAGE'.
2021-07-20T04:15:29.2262368Z           Successfully remade target file 'FORCE_IMAGE'.
2021-07-20T04:15:29.2263353Z         Must remake target 'package/.image.submariner-operator'.
2021-07-20T04:15:29.2265249Z /opt/shipyard/scripts/build_image.sh -i submariner-operator -f package/Dockerfile.submariner-operator -h package/.image.submariner-operator
2021-07-20T04:15:29.2918831Z flags:WARN flag short name (h) already defined
2021-07-20T04:15:29.3545337Z [36m[submariner-operator]$ set -e[0m
2021-07-20T04:15:29.3558204Z [36m[submariner-operator]$ cache_image=quay.io/submariner/submariner-operator:devel[0m
2021-07-20T04:15:29.3572656Z [36m[submariner-operator]$ cache_flag=[0m
2021-07-20T04:15:29.3589386Z [36m[submariner-operator]$ cache_flag=--cache-from quay.io/submariner/submariner-operator:devel[0m
2021-07-20T04:15:29.3611667Z [36m[submariner-operator]$ docker image ls -q quay.io/submariner/submariner-operator:devel[0m
2021-07-20T04:15:29.6684615Z [36m[submariner-operator]$ docker pull quay.io/submariner/submariner-operator:devel[0m
2021-07-20T04:15:31.3084348Z devel: Pulling from submariner/submariner-operator
2021-07-20T04:15:31.3087560Z 9aabc6bb514c: Pulling fs layer
2021-07-20T04:15:31.8163373Z 9aabc6bb514c: Verifying Checksum
2021-07-20T04:15:31.8164783Z 9aabc6bb514c: Download complete
2021-07-20T04:15:32.0987457Z 9aabc6bb514c: Pull complete
2021-07-20T04:15:32.1043454Z Digest: sha256:4fc688208e3986e77644027577c7d3896e83b320cc64da4dcdbe1f94adcbda8b
2021-07-20T04:15:32.1081731Z Status: Downloaded newer image for quay.io/submariner/submariner-operator:devel
2021-07-20T04:15:32.1128382Z quay.io/submariner/submariner-operator:devel
2021-07-20T04:15:32.1153463Z [36m[submariner-operator]$ grep FROM package/Dockerfile.submariner-operator[0m
2021-07-20T04:15:32.1167823Z [36m[submariner-operator]$ cut -f2 -d [0m
2021-07-20T04:15:32.1176501Z [36m[submariner-operator]$ grep -v scratch[0m
2021-07-20T04:15:32.1209669Z [36m[submariner-operator]$ buildargs_flag=--build-arg BUILDKIT_INLINE_CACHE=1 --build-arg BASE_BRANCH=devel[0m
2021-07-20T04:15:32.1224084Z [36m[0m
2021-07-20T04:15:33.0223182Z [36m[submariner-operator]$ docker buildx use buildx_builder[0m
2021-07-20T04:15:33.8488501Z error: failed to find instance "buildx_builder": open /root/.docker/buildx/instances/buildx_builder: no such file or directory
2021-07-20T04:15:33.8569431Z [36m[submariner-operator]$ docker buildx create --name buildx_builder --use[0m
2021-07-20T04:15:34.7759316Z buildx_builder
2021-07-20T04:15:34.7854911Z [36m[submariner-operator]$ docker buildx build --load -t quay.io/submariner/submariner-operator:dev --cache-from quay.io/submariner/submariner-operator:devel -f package/Dockerfile.submariner-operator --iidfile package/.image.submariner-operator --platform linux/amd64 --build-arg BUILDKIT_INLINE_CACHE=1 --build-arg BASE_BRANCH=devel .[0m
2021-07-20T04:15:35.6115285Z #1 [internal] booting buildkit
2021-07-20T04:15:35.6116408Z #1 sha256:8355904eca4539a1f923b1a37ea33d993832dd4197472e87cfdbe2cb530944a3
2021-07-20T04:15:35.7613757Z #1 pulling image moby/buildkit:buildx-stable-1
2021-07-20T04:15:41.9219521Z #1 pulling image moby/buildkit:buildx-stable-1 6.2s done
2021-07-20T04:15:41.9222126Z #1 creating container buildx_buildkit_buildx_builder0
2021-07-20T04:15:42.4724616Z #1 creating container buildx_buildkit_buildx_builder0 0.6s done
2021-07-20T04:15:42.6232703Z #1 DONE 6.9s
2021-07-20T04:15:42.9236369Z
2021-07-20T04:15:42.9239038Z #2 [internal] load build definition from Dockerfile.submariner-operator
2021-07-20T04:15:42.9243962Z #2 sha256:3d1a84f5d8161e3453bdc74eee654c16a8d3818d302909f6d071f1ee916f345c
2021-07-20T04:15:42.9245649Z #2 transferring dockerfile: 230B done
2021-07-20T04:15:42.9247640Z #2 DONE 0.0s
2021-07-20T04:15:42.9248197Z
2021-07-20T04:15:42.9248616Z #3 [internal] load .dockerignore
2021-07-20T04:15:42.9250414Z #3 sha256:5ce998a3e6c798769b267e0b6f7ce37658af221575a4369de2a16380a61fb32b
2021-07-20T04:15:42.9251487Z #3 transferring context: 2B done
2021-07-20T04:15:42.9251900Z #3 DONE 0.0s
2021-07-20T04:15:42.9252145Z
2021-07-20T04:15:42.9252513Z #4 [internal] load build context
2021-07-20T04:15:42.9253430Z #4 sha256:a804ddf7302022b7fb38962afe2f3d557e26c4bc3d021c3d1bdba3b9ef53cc1c
2021-07-20T04:15:42.9254266Z #4 DONE 0.0s
2021-07-20T04:15:42.9254478Z
2021-07-20T04:15:42.9255522Z #6 importing cache manifest from quay.io/submariner/submariner-operator:devel
2021-07-20T04:15:42.9256741Z #6 sha256:db6e513456d7d63fd9e86cca8d8a83f1eb0b2eaefc7f50d06048094c1b3ec182
2021-07-20T04:15:44.1250312Z #6 DONE 1.2s
2021-07-20T04:15:44.1252472Z
2021-07-20T04:15:44.1253152Z #4 [internal] load build context
2021-07-20T04:15:44.1254359Z #4 sha256:a804ddf7302022b7fb38962afe2f3d557e26c4bc3d021c3d1bdba3b9ef53cc1c
2021-07-20T04:15:44.5038593Z #4 transferring context: 9.94MB 0.4s done
2021-07-20T04:15:44.5039174Z #4 DONE 0.4s
2021-07-20T04:15:44.5039431Z
2021-07-20T04:15:44.5040821Z #5 [1/1] COPY bin/submariner-operator /submariner-operator
2021-07-20T04:15:44.5042210Z #5 sha256:54ae7e6e858c9c6f08965ce6987b9f93e7a1461e49a9296d33b460321a9c25cc
2021-07-20T04:15:44.5043078Z #5 CACHED
2021-07-20T04:15:44.5043477Z
2021-07-20T04:15:44.5043912Z #7 exporting to oci image format
2021-07-20T04:15:44.5045097Z #7 sha256:69a2560eef4d3ece902c3b5149d142e9bd132f25db0bc9e35b94201534c415d2
2021-07-20T04:15:44.5046214Z #7 exporting layers done
2021-07-20T04:15:44.5047369Z #7 exporting manifest sha256:de9bbdc497e579259a39507af2f25bbb63c3b0a7ac43576fa382f07e1dd99e1f done
2021-07-20T04:15:44.5049107Z #7 exporting config sha256:2d7e862e7c9a7cf6d0fff86d715882397631fcc6b1ba617b2d8a8d78322e83b7
2021-07-20T04:15:44.6540779Z #7 exporting config sha256:2d7e862e7c9a7cf6d0fff86d715882397631fcc6b1ba617b2d8a8d78322e83b7 0.0s done
2021-07-20T04:15:44.8041215Z #7 ...
2021-07-20T04:15:44.8041645Z
2021-07-20T04:15:44.8042969Z #5 [1/1] COPY bin/submariner-operator /submariner-operator
2021-07-20T04:15:44.8044562Z #5 sha256:54ae7e6e858c9c6f08965ce6987b9f93e7a1461e49a9296d33b460321a9c25cc
2021-07-20T04:15:44.8046298Z #5 sha256:9aabc6bb514c8021810110e5fb3bafd3f119cb23f253d128b1b53e2bf65b8a9a 0B / 9.62MB 0.2s
2021-07-20T04:15:44.8047453Z #5 CACHED
2021-07-20T04:15:44.9543317Z
2021-07-20T04:15:44.9545853Z #5 [1/1] COPY bin/submariner-operator /submariner-operator
2021-07-20T04:15:44.9547439Z #5 sha256:54ae7e6e858c9c6f08965ce6987b9f93e7a1461e49a9296d33b460321a9c25cc
2021-07-20T04:15:44.9548617Z #5 CACHED
2021-07-20T04:15:44.9550195Z
2021-07-20T04:15:44.9551259Z #7 exporting to oci image format
2021-07-20T04:15:44.9552295Z #7 sha256:69a2560eef4d3ece902c3b5149d142e9bd132f25db0bc9e35b94201534c415d2
2021-07-20T04:15:45.1050171Z #7 ...
2021-07-20T04:15:45.1050570Z
2021-07-20T04:15:45.1051833Z #5 [1/1] COPY bin/submariner-operator /submariner-operator
2021-07-20T04:15:45.1106765Z #5 sha256:54ae7e6e858c9c6f08965ce6987b9f93e7a1461e49a9296d33b460321a9c25cc
2021-07-20T04:15:45.1108425Z #5 sha256:9aabc6bb514c8021810110e5fb3bafd3f119cb23f253d128b1b53e2bf65b8a9a 9.62MB / 9.62MB 0.5s done
2021-07-20T04:15:45.1109406Z #5 CACHED
2021-07-20T04:15:45.1109659Z
2021-07-20T04:15:45.1110111Z #7 exporting to oci image format
2021-07-20T04:15:45.1111090Z #7 sha256:69a2560eef4d3ece902c3b5149d142e9bd132f25db0bc9e35b94201534c415d2
2021-07-20T04:15:45.1112080Z #7 sending tarball
2021-07-20T04:15:45.3662214Z #7 ...
2021-07-20T04:15:45.3667948Z
2021-07-20T04:15:45.3675649Z #8 importing to docker
2021-07-20T04:15:45.3676591Z #8 sha256:f09bdff89f3289437023346be15e3ae733543babb4133143707fb6fa9265d297
2021-07-20T04:15:45.3677412Z #8 DONE 0.0s
2021-07-20T04:15:45.3677660Z
2021-07-20T04:15:45.3678330Z #7 exporting to oci image format
2021-07-20T04:15:45.3679318Z #7 sha256:69a2560eef4d3ece902c3b5149d142e9bd132f25db0bc9e35b94201534c415d2
2021-07-20T04:15:45.3684052Z #7 sending tarball 0.4s done
2021-07-20T04:15:45.3718361Z #7 DONE 0.9s
2021-07-20T04:15:45.3718631Z
2021-07-20T04:15:45.3718988Z #9 exporting cache
2021-07-20T04:15:45.3719867Z #9 sha256:2700d4ef94dee473593c5c614b55b2dedcca7893909811a8f2b48291a1f581e4
2021-07-20T04:15:45.3721191Z #9 preparing build cache for export done
2021-07-20T04:15:45.3721696Z #9 DONE 0.0s
2021-07-20T04:15:45.3845306Z [36m[submariner-operator]$ docker tag quay.io/submariner/submariner-operator:dev quay.io/submariner/submariner-operator:devel[0m
2021-07-20T04:15:46.3726854Z [36m[submariner-operator]$ mkdir -p bin[0m
2021-07-20T04:15:46.3728178Z         Successfully remade target file 'package/.image.submariner-operator'.
2021-07-20T04:15:46.3730123Z          File 'build' does not exist.
2021-07-20T04:15:46.3731460Z              File 'bin/subctl-devel-0105e4d-linux-amd64' does not exist.
2021-07-20T04:15:46.3733438Z             Must remake target 'bin/subctl-devel-0105e4d-linux-amd64'.
2021-07-20T04:15:46.3736080Z [36m[submariner-operator]$ ldflags=-s -w -X github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d 		   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=devel[0m
2021-07-20T04:15:46.3741124Z [36m[submariner-operator]$ CGO_ENABLED=0 go build -trimpath -ldflags -s -w -X github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d 		   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=devel -o bin/subctl-devel-0105e4d-linux-amd64 ./pkg/subctl/main.go[0m
2021-07-20T04:15:46.3743765Z mkdir -p bin
2021-07-20T04:15:46.3744785Z target=bin/subctl-devel-0105e4d-linux-amd64; \
2021-07-20T04:15:46.3745580Z target=${target%.exe}; \
2021-07-20T04:15:46.3747930Z components=($(echo ${target//-/ })); \
2021-07-20T04:15:46.3748831Z GOOS=${components[-2]}; \
2021-07-20T04:15:46.3749677Z GOARCH=${components[-1]}; \
2021-07-20T04:15:46.3750470Z export GOARCH GOOS; \
2021-07-20T04:15:46.3751087Z /opt/shipyard/scripts/compile.sh \
2021-07-20T04:15:46.3752588Z 	--ldflags "-X github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d \
2021-07-20T04:15:46.3754920Z 		   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=${DEFAULT_IMAGE_VERSION#v}" \
2021-07-20T04:15:46.3757134Z 	--noupx bin/subctl-devel-0105e4d-linux-amd64 ./pkg/subctl/main.go
2021-07-20T04:15:46.3760120Z Building 'bin/subctl-devel-0105e4d-linux-amd64' (ldflags: $'-X github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d \t\t   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=devel')
2021-07-20T04:15:49.7424964Z             Successfully remade target file 'bin/subctl-devel-0105e4d-linux-amd64'.
2021-07-20T04:15:49.7426443Z            Prerequisite 'bin/subctl-devel-0105e4d-linux-amd64' is newer than target 'bin/subctl'.
2021-07-20T04:15:49.7427610Z           Must remake target 'bin/subctl'.
2021-07-20T04:15:49.7428668Z ln -sf subctl-devel-0105e4d-linux-amd64 bin/subctl
2021-07-20T04:15:49.7490942Z           Successfully remade target file 'bin/subctl'.
2021-07-20T04:15:49.7491757Z         Must remake target 'build'.
2021-07-20T04:15:49.7492502Z         Successfully remade target file 'build'.
2021-07-20T04:15:49.7493439Z       Must remake target 'images'.
2021-07-20T04:15:49.7494228Z       Successfully remade target file 'images'.
2021-07-20T04:15:49.7495083Z     Must remake target 'preload-images'.
2021-07-20T04:15:49.7499920Z set -e;\
2021-07-20T04:15:49.7500448Z . /opt/shipyard/scripts/lib/deploy_funcs;\
2021-07-20T04:15:49.7501071Z . /opt/shipyard/scripts/lib/debug_functions;\
2021-07-20T04:15:49.7502564Z for image in submariner-operator submariner-gateway submariner-route-agent lighthouse-agent lighthouse-coredns; do\
2021-07-20T04:15:49.7504069Z 	import_image quay.io/submariner/${image};\
2021-07-20T04:15:49.7504557Z done
2021-07-20T04:15:49.7551504Z [36m[submariner-operator]$ import_image quay.io/submariner/submariner-operator[0m
2021-07-20T04:15:49.7569303Z [36m[submariner-operator]$ import_image quay.io/submariner/submariner-operator[0m
2021-07-20T04:15:49.7582066Z [36m[submariner-operator]$ docker tag quay.io/submariner/submariner-operator:dev localhost:5000/submariner-operator:local[0m
2021-07-20T04:15:50.0708686Z [36m[submariner-operator]$ docker push localhost:5000/submariner-operator:local[0m
2021-07-20T04:15:50.3529919Z The push refers to repository [localhost:5000/submariner-operator]
2021-07-20T04:15:50.3607424Z f9269f9cee12: Preparing
2021-07-20T04:15:51.0021337Z f9269f9cee12: Pushed
2021-07-20T04:15:51.0251902Z local: digest: sha256:4fc688208e3986e77644027577c7d3896e83b320cc64da4dcdbe1f94adcbda8b size: 528
2021-07-20T04:15:51.0292679Z [36m[submariner-operator]$ import_image quay.io/submariner/submariner-gateway[0m
2021-07-20T04:15:51.0305273Z [36m[submariner-operator]$ import_image quay.io/submariner/submariner-gateway[0m
2021-07-20T04:15:51.0356339Z [36m[submariner-operator]$ docker tag quay.io/submariner/submariner-gateway:dev localhost:5000/submariner-gateway:local[0m
2021-07-20T04:15:51.3337745Z Error response from daemon: No such image: quay.io/submariner/submariner-gateway:dev
2021-07-20T04:15:51.3380534Z [36m[submariner-operator]$ docker pull quay.io/submariner/submariner-gateway:devel[0m
2021-07-20T04:15:52.9982851Z devel: Pulling from submariner/submariner-gateway
2021-07-20T04:15:52.9983684Z 8c741e919b67: Pulling fs layer
2021-07-20T04:15:52.9984205Z 13495efbbac4: Pulling fs layer
2021-07-20T04:15:52.9984659Z 6df6df193f2d: Pulling fs layer
2021-07-20T04:15:52.9985141Z 3afbe9ee7387: Pulling fs layer
2021-07-20T04:15:52.9985587Z 3afbe9ee7387: Waiting
2021-07-20T04:15:53.3959102Z 13495efbbac4: Verifying Checksum
2021-07-20T04:15:53.3959745Z 13495efbbac4: Download complete
2021-07-20T04:15:53.6629361Z 6df6df193f2d: Verifying Checksum
2021-07-20T04:15:53.6630015Z 6df6df193f2d: Download complete
2021-07-20T04:15:53.8807511Z 3afbe9ee7387: Verifying Checksum
2021-07-20T04:15:53.8808309Z 3afbe9ee7387: Download complete
2021-07-20T04:15:53.9620696Z 8c741e919b67: Verifying Checksum
2021-07-20T04:15:53.9621619Z 8c741e919b67: Download complete
2021-07-20T04:15:57.7003050Z 8c741e919b67: Pull complete
2021-07-20T04:15:58.7691905Z 13495efbbac4: Pull complete
2021-07-20T04:16:00.1646375Z 6df6df193f2d: Pull complete
2021-07-20T04:16:00.4542134Z 3afbe9ee7387: Pull complete
2021-07-20T04:16:00.4595355Z Digest: sha256:16252ca036cdc08183e3368d617ba5b424b6b2c09a9103dce35a0044453bf895
2021-07-20T04:16:00.4632698Z Status: Downloaded newer image for quay.io/submariner/submariner-gateway:devel
2021-07-20T04:16:00.4640300Z quay.io/submariner/submariner-gateway:devel
2021-07-20T04:16:00.4698014Z [36m[submariner-operator]$ docker tag quay.io/submariner/submariner-gateway:devel quay.io/submariner/submariner-gateway:dev[0m
2021-07-20T04:16:00.7882495Z [36m[submariner-operator]$ docker tag quay.io/submariner/submariner-gateway:dev localhost:5000/submariner-gateway:local[0m
2021-07-20T04:16:01.0714212Z [36m[submariner-operator]$ docker push localhost:5000/submariner-gateway:local[0m
2021-07-20T04:16:01.3598540Z The push refers to repository [localhost:5000/submariner-gateway]
2021-07-20T04:16:01.3662635Z 71972a942ed4: Preparing
2021-07-20T04:16:01.3663202Z 3c7f2d60eacb: Preparing
2021-07-20T04:16:01.3663660Z 00c5f6897675: Preparing
2021-07-20T04:16:01.3664111Z b3b62c837788: Preparing
2021-07-20T04:16:02.4978503Z 00c5f6897675: Pushed
2021-07-20T04:16:03.2924878Z 71972a942ed4: Pushed
2021-07-20T04:16:07.6736110Z 3c7f2d60eacb: Pushed
2021-07-20T04:16:15.0015650Z b3b62c837788: Pushed
2021-07-20T04:16:15.0396354Z local: digest: sha256:16252ca036cdc08183e3368d617ba5b424b6b2c09a9103dce35a0044453bf895 size: 1159
2021-07-20T04:16:15.0442636Z [36m[submariner-operator]$ import_image quay.io/submariner/submariner-route-agent[0m
2021-07-20T04:16:15.0451483Z [36m[submariner-operator]$ import_image quay.io/submariner/submariner-route-agent[0m
2021-07-20T04:16:15.0471719Z [36m[submariner-operator]$ docker tag quay.io/submariner/submariner-route-agent:dev localhost:5000/submariner-route-agent:local[0m
2021-07-20T04:16:15.3256465Z Error response from daemon: No such image: quay.io/submariner/submariner-route-agent:dev
2021-07-20T04:16:15.3281871Z [36m[submariner-operator]$ docker pull quay.io/submariner/submariner-route-agent:devel[0m
2021-07-20T04:16:16.9445761Z devel: Pulling from submariner/submariner-route-agent
2021-07-20T04:16:16.9446627Z 8c741e919b67: Already exists
2021-07-20T04:16:16.9466379Z 13495efbbac4: Already exists
2021-07-20T04:16:16.9496885Z 70cc6422093d: Pulling fs layer
2021-07-20T04:16:16.9497874Z 7ba74d821748: Pulling fs layer
2021-07-20T04:16:16.9499784Z 62b0802bcf90: Pulling fs layer
2021-07-20T04:16:16.9500309Z 3e9c83283ee4: Pulling fs layer
2021-07-20T04:16:16.9500777Z 3e9c83283ee4: Waiting
2021-07-20T04:16:17.3393951Z 62b0802bcf90: Verifying Checksum
2021-07-20T04:16:17.3394700Z 62b0802bcf90: Download complete
2021-07-20T04:16:17.4756843Z 7ba74d821748: Verifying Checksum
2021-07-20T04:16:17.4757653Z 7ba74d821748: Download complete
2021-07-20T04:16:17.6532727Z 70cc6422093d: Verifying Checksum
2021-07-20T04:16:17.6533333Z 70cc6422093d: Download complete
2021-07-20T04:16:17.7451955Z 3e9c83283ee4: Verifying Checksum
2021-07-20T04:16:17.7452575Z 3e9c83283ee4: Download complete
2021-07-20T04:16:19.4121840Z 70cc6422093d: Pull complete
2021-07-20T04:16:19.6552206Z 7ba74d821748: Pull complete
2021-07-20T04:16:19.7631690Z 62b0802bcf90: Pull complete
2021-07-20T04:16:19.8537384Z 3e9c83283ee4: Pull complete
2021-07-20T04:16:19.8596440Z Digest: sha256:1b9fb51c2e4e74ff6e1f02bef98d8998d772ad01e73ece5e247462590d8c8c4c
2021-07-20T04:16:19.8629180Z Status: Downloaded newer image for quay.io/submariner/submariner-route-agent:devel
2021-07-20T04:16:19.8668335Z quay.io/submariner/submariner-route-agent:devel
2021-07-20T04:16:19.8736679Z [36m[submariner-operator]$ docker tag quay.io/submariner/submariner-route-agent:devel quay.io/submariner/submariner-route-agent:dev[0m
2021-07-20T04:16:20.1650439Z [36m[submariner-operator]$ docker tag quay.io/submariner/submariner-route-agent:dev localhost:5000/submariner-route-agent:local[0m
2021-07-20T04:16:20.5514033Z [36m[submariner-operator]$ docker push localhost:5000/submariner-route-agent:local[0m
2021-07-20T04:16:20.8338801Z The push refers to repository [localhost:5000/submariner-route-agent]
2021-07-20T04:16:20.8418743Z 1d810a4c0610: Preparing
2021-07-20T04:16:20.8419288Z 7a86b74623fc: Preparing
2021-07-20T04:16:20.8419741Z 26de773c4486: Preparing
2021-07-20T04:16:20.8423245Z 6931e591f8e2: Preparing
2021-07-20T04:16:20.8423720Z 00c5f6897675: Preparing
2021-07-20T04:16:20.8424154Z b3b62c837788: Preparing
2021-07-20T04:16:20.8457853Z b3b62c837788: Waiting
2021-07-20T04:16:21.0598990Z 00c5f6897675: Mounted from submariner-gateway
2021-07-20T04:16:22.3905955Z b3b62c837788: Mounted from submariner-gateway
2021-07-20T04:16:22.3906634Z 1d810a4c0610: Pushed
2021-07-20T04:16:22.3907038Z 7a86b74623fc: Pushed
2021-07-20T04:16:22.3907493Z 26de773c4486: Pushed
2021-07-20T04:16:25.9985226Z 6931e591f8e2: Pushed
2021-07-20T04:16:26.0176519Z local: digest: sha256:1b9fb51c2e4e74ff6e1f02bef98d8998d772ad01e73ece5e247462590d8c8c4c size: 1575
2021-07-20T04:16:26.0220624Z [36m[submariner-operator]$ import_image quay.io/submariner/lighthouse-agent[0m
2021-07-20T04:16:26.0234509Z [36m[submariner-operator]$ import_image quay.io/submariner/lighthouse-agent[0m
2021-07-20T04:16:26.0243183Z [36m[submariner-operator]$ docker tag quay.io/submariner/lighthouse-agent:dev localhost:5000/lighthouse-agent:local[0m
2021-07-20T04:16:26.2994613Z Error response from daemon: No such image: quay.io/submariner/lighthouse-agent:dev
2021-07-20T04:16:26.3025704Z [36m[submariner-operator]$ docker pull quay.io/submariner/lighthouse-agent:devel[0m
2021-07-20T04:16:27.9265545Z devel: Pulling from submariner/lighthouse-agent
2021-07-20T04:16:27.9266190Z ecd69139dc76: Pulling fs layer
2021-07-20T04:16:27.9266959Z 8261c1767de3: Pulling fs layer
2021-07-20T04:16:28.3099079Z ecd69139dc76: Verifying Checksum
2021-07-20T04:16:28.3100346Z ecd69139dc76: Download complete
2021-07-20T04:16:28.4079131Z 8261c1767de3: Verifying Checksum
2021-07-20T04:16:28.4080023Z 8261c1767de3: Download complete
2021-07-20T04:16:28.4750119Z ecd69139dc76: Pull complete
2021-07-20T04:16:28.7856088Z 8261c1767de3: Pull complete
2021-07-20T04:16:28.7931491Z Digest: sha256:93957d5e6a2d72de099dcf9f589a4321618b426900e681d3837a17c64c38d69f
2021-07-20T04:16:28.7958153Z Status: Downloaded newer image for quay.io/submariner/lighthouse-agent:devel
2021-07-20T04:16:28.7985893Z quay.io/submariner/lighthouse-agent:devel
2021-07-20T04:16:28.8019580Z [36m[submariner-operator]$ docker tag quay.io/submariner/lighthouse-agent:devel quay.io/submariner/lighthouse-agent:dev[0m
2021-07-20T04:16:29.0994559Z [36m[submariner-operator]$ docker tag quay.io/submariner/lighthouse-agent:dev localhost:5000/lighthouse-agent:local[0m
2021-07-20T04:16:29.3821751Z [36m[submariner-operator]$ docker push localhost:5000/lighthouse-agent:local[0m
2021-07-20T04:16:29.6704157Z The push refers to repository [localhost:5000/lighthouse-agent]
2021-07-20T04:16:29.6757673Z 83c79e393e11: Preparing
2021-07-20T04:16:29.6758220Z 0026d2f62bbf: Preparing
2021-07-20T04:16:29.7402953Z 0026d2f62bbf: Pushed
2021-07-20T04:16:30.2910246Z 83c79e393e11: Pushed
2021-07-20T04:16:30.3317917Z local: digest: sha256:93957d5e6a2d72de099dcf9f589a4321618b426900e681d3837a17c64c38d69f size: 735
2021-07-20T04:16:30.3436795Z [36m[submariner-operator]$ import_image quay.io/submariner/lighthouse-coredns[0m
2021-07-20T04:16:30.3474435Z [36m[submariner-operator]$ import_image quay.io/submariner/lighthouse-coredns[0m
2021-07-20T04:16:30.3490195Z [36m[submariner-operator]$ docker tag quay.io/submariner/lighthouse-coredns:dev localhost:5000/lighthouse-coredns:local[0m
2021-07-20T04:16:30.6210891Z Error response from daemon: No such image: quay.io/submariner/lighthouse-coredns:dev
2021-07-20T04:16:30.6245143Z [36m[submariner-operator]$ docker pull quay.io/submariner/lighthouse-coredns:devel[0m
2021-07-20T04:16:32.2361228Z devel: Pulling from submariner/lighthouse-coredns
2021-07-20T04:16:32.2362095Z 088e56360f76: Pulling fs layer
2021-07-20T04:16:32.2362733Z 18aed70882d2: Pulling fs layer
2021-07-20T04:16:32.6675678Z 088e56360f76: Verifying Checksum
2021-07-20T04:16:32.6749583Z 088e56360f76: Download complete
2021-07-20T04:16:32.7603706Z 18aed70882d2: Verifying Checksum
2021-07-20T04:16:32.7604680Z 18aed70882d2: Download complete
2021-07-20T04:16:32.8295369Z 088e56360f76: Pull complete
2021-07-20T04:16:33.1683677Z 18aed70882d2: Pull complete
2021-07-20T04:16:33.1730482Z Digest: sha256:92280c8723eb4400aeadedcb1885052b0d8344fc4c3efeee4cc91d10c5d29dd6
2021-07-20T04:16:33.1756340Z Status: Downloaded newer image for quay.io/submariner/lighthouse-coredns:devel
2021-07-20T04:16:33.1825813Z quay.io/submariner/lighthouse-coredns:devel
2021-07-20T04:16:33.1864511Z [36m[submariner-operator]$ docker tag quay.io/submariner/lighthouse-coredns:devel quay.io/submariner/lighthouse-coredns:dev[0m
2021-07-20T04:16:33.4654744Z [36m[submariner-operator]$ docker tag quay.io/submariner/lighthouse-coredns:dev localhost:5000/lighthouse-coredns:local[0m
2021-07-20T04:16:33.7605516Z [36m[submariner-operator]$ docker push localhost:5000/lighthouse-coredns:local[0m
2021-07-20T04:16:34.0460147Z The push refers to repository [localhost:5000/lighthouse-coredns]
2021-07-20T04:16:34.0512013Z 7fca351e51be: Preparing
2021-07-20T04:16:34.0512645Z 4fa0ece06e25: Preparing
2021-07-20T04:16:34.1192200Z 4fa0ece06e25: Pushed
2021-07-20T04:16:34.9022884Z 7fca351e51be: Pushed
2021-07-20T04:16:34.9260885Z local: digest: sha256:92280c8723eb4400aeadedcb1885052b0d8344fc4c3efeee4cc91d10c5d29dd6 size: 739
2021-07-20T04:16:34.9294960Z     Successfully remade target file 'preload-images'.
2021-07-20T04:16:34.9296148Z    Prerequisite 'clusters' of target 'deploy' does not exist.
2021-07-20T04:16:34.9343850Z    Prerequisite 'preload-images' of target 'deploy' does not exist.
2021-07-20T04:16:34.9345495Z    Prerequisite 'bin/subctl' is newer than target 'deploy'.
2021-07-20T04:16:34.9393553Z   Must remake target 'deploy'.
2021-07-20T04:16:34.9395564Z /opt/shipyard/scripts/deploy.sh --cluster_settings /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings
2021-07-20T04:16:35.0508007Z Running with: globalnet='false', deploytool='operator', deploytool_broker_args='', deploytool_submariner_args='', cluster_settings='/go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings', timeout=5m, image_tag=local, cable_driver=libreswan, service_discovery=false
2021-07-20T04:16:35.0533158Z [36m[submariner-operator]$ source /opt/shipyard/scripts/lib/utils[0m
2021-07-20T04:16:35.0544600Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:16:35.0556211Z [36m[submariner-operator]$ script_name=utils[0m
2021-07-20T04:16:35.0566312Z [36m[submariner-operator]$ exec_name=deploy.sh[0m
2021-07-20T04:16:35.0589408Z [36m[submariner-operator]$ source /opt/shipyard/scripts/lib/deploy_funcs[0m
2021-07-20T04:16:35.0599523Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:16:35.0614185Z [36m[submariner-operator]$ script_name=deploy_funcs[0m
2021-07-20T04:16:35.0628541Z [36m[submariner-operator]$ exec_name=deploy.sh[0m
2021-07-20T04:16:35.0648758Z [36m[submariner-operator]$ source /opt/shipyard/scripts/lib/cluster_settings[0m
2021-07-20T04:16:35.0662993Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:16:35.0677858Z [36m[submariner-operator]$ script_name=cluster_settings[0m
2021-07-20T04:16:35.0692809Z [36m[submariner-operator]$ exec_name=deploy.sh[0m
2021-07-20T04:16:35.0715865Z [36m[submariner-operator]$ broker=cluster1[0m
2021-07-20T04:16:35.0743473Z [36m[submariner-operator]$ declare -gA cluster_nodes[0m
2021-07-20T04:16:35.0769843Z [36m[submariner-operator]$ cluster_nodes[cluster1]=control-plane worker[0m
2021-07-20T04:16:35.0782785Z [36m[submariner-operator]$ cluster_nodes[cluster2]=control-plane worker[0m
2021-07-20T04:16:35.0794176Z [36m[submariner-operator]$ cluster_nodes[cluster3]=control-plane worker worker[0m
2021-07-20T04:16:35.0862389Z [36m[submariner-operator]$ declare -gA cluster_subm[0m
2021-07-20T04:16:35.0886633Z [36m[submariner-operator]$ cluster_subm[cluster1]=true[0m
2021-07-20T04:16:35.0899092Z [36m[submariner-operator]$ cluster_subm[cluster2]=true[0m
2021-07-20T04:16:35.0914726Z [36m[submariner-operator]$ cluster_subm[cluster3]=true[0m
2021-07-20T04:16:35.0929483Z [36m[submariner-operator]$ declare -gA cluster_cni[0m
2021-07-20T04:16:35.0942778Z [36m[submariner-operator]$ cluster_cni[cluster2]=weave[0m
2021-07-20T04:16:35.0955363Z [36m[submariner-operator]$ cluster_cni[cluster3]=weave[0m
2021-07-20T04:16:35.0969767Z [36m[submariner-operator]$ source /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings[0m
2021-07-20T04:16:35.0989223Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:16:35.1010412Z [36m[submariner-operator]$ script_name=cluster_settings[0m
2021-07-20T04:16:35.1024020Z [36m[submariner-operator]$ exec_name=deploy.sh[0m
2021-07-20T04:16:35.1050997Z [36m[submariner-operator]$ cluster_nodes[cluster1]=control-plane worker worker[0m
2021-07-20T04:16:35.1076474Z [36m[submariner-operator]$ cluster_nodes[cluster2]=control-plane worker worker[0m
2021-07-20T04:16:35.1178916Z [36m[submariner-operator]$ declare_cidrs[0m
2021-07-20T04:16:35.1191625Z [36m[submariner-operator]$ declare_cidrs[0m
2021-07-20T04:16:35.1230937Z [36m[submariner-operator]$ declare -gA cluster_CIDRs service_CIDRs global_CIDRs[0m
2021-07-20T04:16:35.1244504Z [36m[submariner-operator]$ i=1[0m
2021-07-20T04:16:35.1259086Z [36m[submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1[0m
2021-07-20T04:16:35.1272165Z [36m[submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1[0m
2021-07-20T04:16:35.1293267Z [36m[submariner-operator]$ [cluster1] cluster_CIDRs[cluster1]=10.1.0.0/16[0m
2021-07-20T04:16:35.1307194Z [36m[submariner-operator]$ [cluster1] service_CIDRs[cluster1]=100.1.0.0/16[0m
2021-07-20T04:16:35.1322043Z [36m[submariner-operator]$ [cluster1] i=$(("$i"+1))[0m
2021-07-20T04:16:35.1335751Z [36m[submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2[0m
2021-07-20T04:16:35.1349249Z [36m[submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2[0m
2021-07-20T04:16:35.1375914Z [36m[submariner-operator]$ [cluster2] cluster_CIDRs[cluster2]=10.2.0.0/16[0m
2021-07-20T04:16:35.1394392Z [36m[submariner-operator]$ [cluster2] service_CIDRs[cluster2]=100.2.0.0/16[0m
2021-07-20T04:16:35.1416966Z [36m[submariner-operator]$ [cluster2] i=$(("$i"+1))[0m
2021-07-20T04:16:35.1431871Z [36m[submariner-operator]$ [cluster2] declare_kubeconfig[0m
2021-07-20T04:16:35.1445347Z [36m[submariner-operator]$ [cluster2] declare_kubeconfig[0m
2021-07-20T04:16:35.1459471Z [36m[submariner-operator]$ [cluster2] source /opt/shipyard/scripts/lib/kubecfg[0m
2021-07-20T04:16:35.1481113Z [36m[submariner-operator]$ [cluster2] export KUBECONFIG[0m
2021-07-20T04:16:35.1497802Z [36m[submariner-operator]$ [cluster2] KUBECONFIG=$(find "${DAPPER_OUTPUT:-output}"/kubeconfigs -type f -printf %p: || :)[0m
2021-07-20T04:16:35.1534623Z [36m[submariner-operator]$ [cluster2] find /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs -type f -printf %p:[0m
2021-07-20T04:16:35.1640285Z [36m[submariner-operator]$ [cluster2] bash -c curl -Ls https://get.submariner.io | VERSION=devel DESTDIR=/go/bin bash[0m
2021-07-20T04:16:35.3576905Z Installing subctl version devel
2021-07-20T04:16:35.3577929Z   OS detected:           linux
2021-07-20T04:16:35.3578563Z   Architecture detected: amd64
2021-07-20T04:16:35.3580445Z   Download URL:          https://github.com/submariner-io/submariner-operator/releases/download/subctl-devel/subctl-devel-linux-amd64.tar.xz
2021-07-20T04:16:35.3581417Z
2021-07-20T04:16:35.3581764Z Downloading...
2021-07-20T04:16:36.9060468Z subctl-devel-linux-amd64 has been installed as /go/bin/subctl
2021-07-20T04:16:36.9505344Z This provides subctl version: devel
2021-07-20T04:16:36.9534699Z
2021-07-20T04:16:36.9536770Z please update your path (and consider adding it to ~/.profile):
2021-07-20T04:16:36.9537888Z     export PATH=$PATH:/go/bin
2021-07-20T04:16:36.9538287Z
2021-07-20T04:16:36.9716924Z [36m[submariner-operator]$ [cluster2] load_deploytool operator[0m
2021-07-20T04:16:36.9741044Z [36m[submariner-operator]$ [cluster2] load_deploytool operator[0m
2021-07-20T04:16:36.9748064Z Will deploy submariner using operator
2021-07-20T04:16:36.9766374Z [36m[submariner-operator]$ [cluster2] . /opt/shipyard/scripts/lib/deploy_operator[0m
2021-07-20T04:16:36.9782613Z [36m[submariner-operator]$ [cluster2] . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:16:36.9815237Z [36m[submariner-operator]$ [cluster2] script_name=deploy_operator[0m
2021-07-20T04:16:36.9846457Z [36m[submariner-operator]$ [cluster2] exec_name=deploy.sh[0m
2021-07-20T04:16:36.9867302Z [36m[submariner-operator]$ [cluster2] deploytool_prereqs[0m
2021-07-20T04:16:36.9885391Z [36m[submariner-operator]$ [cluster2] deploytool_prereqs[0m
2021-07-20T04:16:36.9902883Z [36m[0m
2021-07-20T04:16:36.9918795Z [36m[submariner-operator]$ [cluster2] subctl version[0m
2021-07-20T04:16:37.0156524Z subctl version: devel-0105e4d
2021-07-20T04:16:37.0215731Z [36m[submariner-operator]$ [cluster2] run_if_defined pre_deploy[0m
2021-07-20T04:16:37.0230164Z [36m[submariner-operator]$ [cluster2] run_if_defined pre_deploy[0m
2021-07-20T04:16:37.0280387Z [36m[submariner-operator]$ [cluster2] type -t pre_deploy[0m
2021-07-20T04:16:37.0342081Z [36m[submariner-operator]$ [cluster2] grep -q function[0m
2021-07-20T04:16:37.0394335Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:16:37.0417743Z [36m[submariner-operator]$ [cluster2] run_subm_clusters prepare_cluster[0m
2021-07-20T04:16:37.0472362Z [36m[submariner-operator]$ [cluster2] run_subm_clusters prepare_cluster[0m
2021-07-20T04:16:37.0473839Z [36m[submariner-operator]$ [cluster2] declare -a subm_clusters[0m
2021-07-20T04:16:37.0522589Z [36m[submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 prepare_cluster[0m
2021-07-20T04:16:37.0525216Z [36m[submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 prepare_cluster[0m
2021-07-20T04:16:37.0526755Z [36m[submariner-operator]$ [cluster2] declare -A pids[0m
2021-07-20T04:16:37.0581999Z [36m[submariner-operator]$ [cluster2] eval echo cluster1 cluster2[0m
2021-07-20T04:16:37.0583824Z [36m[submariner-operator]$ [cluster1] pids[cluster1]=8679[0m
2021-07-20T04:16:37.0601161Z [36m[submariner-operator]$ [cluster1] set -o pipefail[0m
2021-07-20T04:16:37.0607110Z [36m[submariner-operator]$ [cluster2] pids[cluster2]=8681[0m
2021-07-20T04:16:37.0616184Z [36m[submariner-operator]$ [cluster1] prepare_cluster[0m
2021-07-20T04:16:37.0634635Z [36m[submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /[0m
2021-07-20T04:16:37.0646877Z [36m[submariner-operator]$ [cluster2] wait 8681[0m
2021-07-20T04:16:37.0655507Z [36m[submariner-operator]$ [cluster2] set -o pipefail[0m
2021-07-20T04:16:37.0674653Z [36m[submariner-operator]$ [cluster2] prepare_cluster[0m
2021-07-20T04:16:37.0694962Z [36m[submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /[0m
2021-07-20T04:16:37.4752513Z [36m[submariner-operator]$ [cluster1] prepare_cluster[0m
2021-07-20T04:16:37.4754266Z [36m[submariner-operator]$ [cluster1] add_subm_gateway_label[0m
2021-07-20T04:16:37.4755656Z [36m[submariner-operator]$ [cluster1] add_subm_gateway_label[0m
2021-07-20T04:16:37.4757252Z [36m[submariner-operator]$ [cluster1] kubectl label node cluster1-worker submariner.io/gateway=true --overwrite[0m
2021-07-20T04:16:37.4759338Z [36m[submariner-operator]$ [cluster1] kubectl label node cluster1-worker submariner.io/gateway=true --overwrite[0m
2021-07-20T04:16:37.4762457Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 label node cluster1-worker submariner.io/gateway=true --overwrite[0m
2021-07-20T04:16:37.4764513Z [cluster1] node/cluster1-worker not labeled
2021-07-20T04:16:37.5141235Z [36m[submariner-operator]$ [cluster2] prepare_cluster[0m
2021-07-20T04:16:37.5142451Z [36m[submariner-operator]$ [cluster2] add_subm_gateway_label[0m
2021-07-20T04:16:37.5143715Z [36m[submariner-operator]$ [cluster2] add_subm_gateway_label[0m
2021-07-20T04:16:37.5145318Z [36m[submariner-operator]$ [cluster2] kubectl label node cluster2-worker submariner.io/gateway=true --overwrite[0m
2021-07-20T04:16:37.5146977Z [36m[submariner-operator]$ [cluster2] kubectl label node cluster2-worker submariner.io/gateway=true --overwrite[0m
2021-07-20T04:16:37.5148694Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 label node cluster2-worker submariner.io/gateway=true --overwrite[0m
2021-07-20T04:16:37.5149984Z [cluster2] node/cluster2-worker not labeled
2021-07-20T04:16:37.5174148Z [36m[submariner-operator]$ [cluster2] wait 8679[0m
2021-07-20T04:16:37.5188328Z [36m[submariner-operator]$ [cluster2] with_context cluster1 setup_broker[0m
2021-07-20T04:16:37.5201119Z [36m[submariner-operator]$ [cluster2] with_context cluster1 setup_broker[0m
2021-07-20T04:16:37.5216720Z [36m[submariner-operator]$ [cluster1] setup_broker[0m
2021-07-20T04:16:37.5235643Z [36m[submariner-operator]$ [cluster1] setup_broker[0m
2021-07-20T04:16:37.5241930Z Installing broker...
2021-07-20T04:16:37.5254840Z [36m[submariner-operator]$ [cluster1] subctlrepver=[0m
2021-07-20T04:16:37.5268679Z [36m[submariner-operator]$ [cluster1] subctlrepver=--repository localhost:5000 --version local[0m
2021-07-20T04:16:37.5290787Z [36m[submariner-operator]$ [cluster1] cd /go/src/github.com/submariner-io/submariner-operator/output[0m
2021-07-20T04:16:37.5322162Z [36m[output]$ [cluster1] subctl deploy-broker --kubeconfig /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1 --repository localhost:5000 --version local[0m
2021-07-20T04:16:37.5585884Z  • Setting up broker RBAC  ...
2021-07-20T04:16:37.6234742Z  ✓ Setting up broker RBAC
2021-07-20T04:16:37.6235920Z  • Deploying the Submariner operator  ...
2021-07-20T04:16:44.0369141Z  ✓ Deploying the Submariner operator
2021-07-20T04:16:44.0370329Z  ✓ Created operator service account and role
2021-07-20T04:16:44.0371222Z  ✓ Created Lighthouse service accounts and roles
2021-07-20T04:16:44.0372512Z  • Deploying the broker  ...
2021-07-20T04:16:45.1659411Z  ✓ Deploying the broker
2021-07-20T04:16:45.1660288Z  ✓ The broker has been deployed
2021-07-20T04:16:45.1661063Z  • Creating broker-info.subm file  ...
2021-07-20T04:16:45.1983266Z  ✓ Creating broker-info.subm file
2021-07-20T04:16:45.1984664Z  ✓ Backed up previous broker-info.subm to broker-info.subm.2021-07-20T04_16_45Z
2021-07-20T04:16:45.1985881Z  ⚠ Reusing IPsec PSK from existing broker-info.subm
2021-07-20T04:16:45.2033402Z [36m[submariner-operator]$ [cluster2] install_subm_all_clusters[0m
2021-07-20T04:16:45.2046765Z [36m[submariner-operator]$ [cluster2] install_subm_all_clusters[0m
2021-07-20T04:16:45.2060123Z [36m[submariner-operator]$ [cluster2] run_subm_clusters subctl_install_subm[0m
2021-07-20T04:16:45.2097728Z [36m[submariner-operator]$ [cluster2] run_subm_clusters subctl_install_subm[0m
2021-07-20T04:16:45.2110413Z [36m[submariner-operator]$ [cluster2] declare -a subm_clusters[0m
2021-07-20T04:16:45.2223507Z [36m[submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 subctl_install_subm[0m
2021-07-20T04:16:45.2246726Z [36m[submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 subctl_install_subm[0m
2021-07-20T04:16:45.2263948Z [36m[submariner-operator]$ [cluster2] declare -A pids[0m
2021-07-20T04:16:45.2342882Z [36m[submariner-operator]$ [cluster2] eval echo cluster1 cluster2[0m
2021-07-20T04:16:45.8935572Z [36m[submariner-operator]$ [cluster1] pids[cluster1]=8744[0m
2021-07-20T04:16:45.8944483Z [36m[submariner-operator]$ [cluster1] set -o pipefail[0m
2021-07-20T04:16:45.8945808Z [36m[submariner-operator]$ [cluster2] pids[cluster2]=8747[0m
2021-07-20T04:16:45.8948482Z [36m[submariner-operator]$ [cluster2] set -o pipefail[0m
2021-07-20T04:16:45.8950116Z [36m[submariner-operator]$ [cluster2] subctl_install_subm[0m
2021-07-20T04:16:45.8951820Z [36m[submariner-operator]$ [cluster1] subctl_install_subm[0m
2021-07-20T04:16:45.8953441Z [36m[submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /[0m
2021-07-20T04:16:45.8954687Z [36m[submariner-operator]$ [cluster2] wait 8747[0m
2021-07-20T04:16:45.8955882Z [36m[submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /[0m
2021-07-20T04:17:02.0094004Z [36m[submariner-operator]$ [cluster2] subctl_install_subm[0m
2021-07-20T04:17:02.0095570Z [36m[submariner-operator]$ [cluster2] subctlrepver=[0m
2021-07-20T04:17:02.0097638Z [36m[submariner-operator]$ [cluster2] subctlrepver=--repository localhost:5000 --version local[0m
2021-07-20T04:17:02.0099907Z [36m[submariner-operator]$ [cluster2] PASS_CIDR_ARGS=false[0m
2021-07-20T04:17:02.0103544Z [36m[submariner-operator]$ [cluster2] subctl join --kubeconfig /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2 --clusterid cluster2 --repository localhost:5000 --version local --nattport 4500 --ikeport 500 --colorcodes blue --globalnet-cidr  --natt=false --cable-driver libreswan /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm[0m
2021-07-20T04:17:02.0107503Z [cluster2] * /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm says broker is at: https://172.18.0.8:6443
2021-07-20T04:17:02.0109217Z [cluster2] * There are 1 labeled nodes in the cluster:
2021-07-20T04:17:02.0110269Z [cluster2]   - cluster2-worker
2021-07-20T04:17:02.0111479Z [cluster2]  • Discovering network details  ...
2021-07-20T04:17:02.0113225Z [cluster2]         Network plugin:  weave-net
2021-07-20T04:17:02.0115943Z [cluster2]         Service CIDRs:   [100.2.0.0/16]
2021-07-20T04:17:02.0118064Z [cluster2]         Cluster CIDRs:   [10.2.0.0/16]
2021-07-20T04:17:02.0122118Z [cluster2]  ✓ Discovering network details
2021-07-20T04:17:02.0122983Z [cluster2]  • Discovering multi cluster details  ...
2021-07-20T04:17:02.0124201Z [cluster2]  • Validating Globalnet configurations  ...
2021-07-20T04:17:02.0125049Z [cluster2]  ✓ Validating Globalnet configurations
2021-07-20T04:17:02.0125887Z [cluster2]  ✓ Discovering multi cluster details
2021-07-20T04:17:02.0126687Z [cluster2]  • Deploying the Submariner operator  ...
2021-07-20T04:17:02.0128160Z [cluster2]  ✓ Deploying the Submariner operator
2021-07-20T04:17:02.0129529Z [cluster2]  ✓ Created operator service account and role
2021-07-20T04:17:02.0130601Z [cluster2]  ✓ Created Lighthouse service accounts and roles
2021-07-20T04:17:02.0131457Z [cluster2]  • Creating SA for cluster  ...
2021-07-20T04:17:02.0132509Z [cluster2]  ✓ Creating SA for cluster
2021-07-20T04:17:02.0133567Z [cluster2]  • Deploying Submariner  ...
2021-07-20T04:17:02.0134306Z [cluster2]  ✓ Deploying Submariner
2021-07-20T04:17:02.0135360Z [cluster2]  ✓ Submariner is up and running
2021-07-20T04:17:02.0157423Z [36m[submariner-operator]$ [cluster2] wait 8744[0m
2021-07-20T04:17:05.1664193Z [36m[submariner-operator]$ [cluster1] subctl_install_subm[0m
2021-07-20T04:17:05.1665892Z [36m[submariner-operator]$ [cluster1] subctlrepver=[0m
2021-07-20T04:17:05.1667481Z [36m[submariner-operator]$ [cluster1] subctlrepver=--repository localhost:5000 --version local[0m
2021-07-20T04:17:05.1668833Z [36m[submariner-operator]$ [cluster1] PASS_CIDR_ARGS=false[0m
2021-07-20T04:17:05.1672363Z [36m[submariner-operator]$ [cluster1] subctl join --kubeconfig /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1 --clusterid cluster1 --repository localhost:5000 --version local --nattport 4500 --ikeport 500 --colorcodes blue --globalnet-cidr  --natt=false --cable-driver libreswan /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm[0m
2021-07-20T04:17:05.1676027Z [cluster1] * /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm says broker is at: https://172.18.0.8:6443
2021-07-20T04:17:05.1677254Z [cluster1] * There are 1 labeled nodes in the cluster:
2021-07-20T04:17:05.1678365Z [cluster1]   - cluster1-worker
2021-07-20T04:17:05.1679334Z [cluster1]  • Discovering network details  ...
2021-07-20T04:17:05.1680965Z [cluster1]         Network plugin:  weave-net
2021-07-20T04:17:05.1681871Z [cluster1]         Service CIDRs:   [100.1.0.0/16]
2021-07-20T04:17:05.1683344Z [cluster1]         Cluster CIDRs:   [10.1.0.0/16]
2021-07-20T04:17:05.1684555Z [cluster1]  ✓ Discovering network details
2021-07-20T04:17:05.1685365Z [cluster1]  • Discovering multi cluster details  ...
2021-07-20T04:17:05.1686239Z [cluster1]  • Validating Globalnet configurations  ...
2021-07-20T04:17:05.1687112Z [cluster1]  ✓ Validating Globalnet configurations
2021-07-20T04:17:05.1687982Z [cluster1]  ✓ Discovering multi cluster details
2021-07-20T04:17:05.1688809Z [cluster1]  • Deploying the Submariner operator  ...
2021-07-20T04:17:05.1690464Z [cluster1]  ✓ Deploying the Submariner operator
2021-07-20T04:17:05.1691426Z [cluster1]  ✓ Created Lighthouse service accounts and roles
2021-07-20T04:17:05.1692306Z [cluster1]  • Creating SA for cluster  ...
2021-07-20T04:17:05.1693083Z [cluster1]  ✓ Creating SA for cluster
2021-07-20T04:17:05.1694223Z [cluster1]  • Deploying Submariner  ...
2021-07-20T04:17:05.1694967Z [cluster1]  ✓ Deploying Submariner
2021-07-20T04:17:05.1695699Z [cluster1]  ✓ Submariner is up and running
2021-07-20T04:17:05.1733653Z [36m[submariner-operator]$ [cluster2] with_context cluster2 with_retries 30 verify_gw_status[0m
2021-07-20T04:17:05.1748099Z [36m[submariner-operator]$ [cluster2] with_context cluster2 with_retries 30 verify_gw_status[0m
2021-07-20T04:17:05.1762362Z [36m[submariner-operator]$ [cluster2] with_retries 30 verify_gw_status[0m
2021-07-20T04:17:05.1772934Z [36m[submariner-operator]$ [cluster2] with_retries[0m
2021-07-20T04:17:05.1785060Z [36m[submariner-operator]$ [cluster2] iteration=1[0m
2021-07-20T04:17:05.1883229Z [36m[submariner-operator]$ [cluster2] wait 8787[0m
2021-07-20T04:17:05.1896314Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:17:05.1906632Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:17:05.1916033Z [36m[submariner-operator]$ [cluster2] sleep_duration=6[0m
2021-07-20T04:17:05.1953813Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:17:05.1968500Z [36m[submariner-operator]$ [cluster2] grep the server could not find the requested resource[0m
2021-07-20T04:17:05.4908912Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:17:05.4924013Z [36m[submariner-operator]$ [cluster2] grep connected[0m
2021-07-20T04:17:05.6203249Z  • Showing Connections  ...
2021-07-20T04:17:05.6349260Z  ✗ Showing Connections
2021-07-20T04:17:05.6350129Z  ✗ There are no gateways detected
2021-07-20T04:17:05.6965340Z  • Showing Connections  ...
2021-07-20T04:17:05.7195958Z  ✗ Showing Connections
2021-07-20T04:17:05.7196808Z  ✗ There are no gateways detected
2021-07-20T04:17:05.7220155Z iter: 1. Clusters not yet connected. sleeping for 6 secs
2021-07-20T04:17:05.7232470Z [36m[submariner-operator]$ [cluster2] sleep 6[0m
2021-07-20T04:17:11.7268838Z [36m[submariner-operator]$ [cluster2] return 1[0m
2021-07-20T04:17:11.7342749Z [36m[submariner-operator]$ [cluster2] wait 8815[0m
2021-07-20T04:17:11.7361799Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:17:11.7394376Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:17:11.7415438Z [36m[submariner-operator]$ [cluster2] sleep_duration=6[0m
2021-07-20T04:17:11.7476699Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:17:11.7527777Z [36m[submariner-operator]$ [cluster2] grep the server could not find the requested resource[0m
2021-07-20T04:17:13.7898533Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:17:13.7911367Z [36m[submariner-operator]$ [cluster2] grep connected[0m
2021-07-20T04:17:14.6946133Z  • Showing Connections  ...
2021-07-20T04:17:14.9614212Z  ✗ Showing Connections
2021-07-20T04:17:14.9617417Z  ✗ There are no gateways detected
2021-07-20T04:17:15.1078688Z  • Showing Connections  ...
2021-07-20T04:17:15.1532669Z  ✗ Showing Connections
2021-07-20T04:17:15.1533654Z  ✗ There are no gateways detected
2021-07-20T04:17:15.1560956Z iter: 2. Clusters not yet connected. sleeping for 6 secs
2021-07-20T04:17:15.1575381Z [36m[submariner-operator]$ [cluster2] sleep 6[0m
2021-07-20T04:17:21.1667258Z [36m[submariner-operator]$ [cluster2] return 1[0m
2021-07-20T04:17:21.1704930Z [36m[submariner-operator]$ [cluster2] wait 8841[0m
2021-07-20T04:17:21.1729277Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:17:21.1754473Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:17:21.1792641Z [36m[submariner-operator]$ [cluster2] sleep_duration=6[0m
2021-07-20T04:17:21.1813441Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:17:21.1863599Z [36m[submariner-operator]$ [cluster2] grep the server could not find the requested resource[0m
2021-07-20T04:17:21.8603937Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:17:21.8623659Z [36m[submariner-operator]$ [cluster2] grep connected[0m
2021-07-20T04:17:22.2447038Z  • Showing Connections  ...
2021-07-20T04:17:22.2566634Z  ✗ Showing Connections
2021-07-20T04:17:22.2567406Z  ✗ There are no gateways detected
2021-07-20T04:17:22.3361793Z  • Showing Connections  ...
2021-07-20T04:17:22.3635290Z  ✗ Showing Connections
2021-07-20T04:17:22.3640134Z  ✗ No connections found
2021-07-20T04:17:22.3667558Z iter: 3. Clusters not yet connected. sleeping for 6 secs
2021-07-20T04:17:22.3682407Z [36m[submariner-operator]$ [cluster2] sleep 6[0m
2021-07-20T04:17:28.3709972Z [36m[submariner-operator]$ [cluster2] return 1[0m
2021-07-20T04:17:28.3761274Z [36m[submariner-operator]$ [cluster2] wait 8868[0m
2021-07-20T04:17:28.3767865Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:17:28.3782359Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:17:28.3807674Z [36m[submariner-operator]$ [cluster2] sleep_duration=6[0m
2021-07-20T04:17:28.3859193Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:17:28.9067294Z [36m[submariner-operator]$ [cluster2] grep the server could not find the requested resource[0m
2021-07-20T04:17:28.9649575Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:17:28.9671593Z [36m[submariner-operator]$ [cluster2] grep connected[0m
2021-07-20T04:17:29.3156684Z  • Showing Connections  ...
2021-07-20T04:17:29.3988561Z  ✗ Showing Connections
2021-07-20T04:17:29.3993976Z  ✗ No connections found
2021-07-20T04:17:29.4328100Z  • Showing Connections  ...
2021-07-20T04:17:29.4404384Z  ✓ Showing Connections
2021-07-20T04:17:29.4434870Z iter: 4. Clusters not yet connected. sleeping for 6 secs
2021-07-20T04:17:29.4448947Z [36m[submariner-operator]$ [cluster2] sleep 6[0m
2021-07-20T04:17:35.4484890Z [36m[submariner-operator]$ [cluster2] return 1[0m
2021-07-20T04:17:35.4538973Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:17:35.4545317Z [36m[submariner-operator]$ [cluster2] wait 8894[0m
2021-07-20T04:17:35.4554115Z [36m[submariner-operator]$ [cluster2] verify_gw_status[0m
2021-07-20T04:17:35.4568560Z [36m[submariner-operator]$ [cluster2] sleep_duration=6[0m
2021-07-20T04:17:35.4583672Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:17:35.4603360Z [36m[submariner-operator]$ [cluster2] grep the server could not find the requested resource[0m
2021-07-20T04:17:35.5446121Z [36m[submariner-operator]$ [cluster2] subctl show connections[0m
2021-07-20T04:17:35.5461894Z [36m[submariner-operator]$ [cluster2] grep connected[0m
2021-07-20T04:17:35.6009599Z  • Showing Connections  ...
2021-07-20T04:17:35.6040876Z  ✓ Showing Connections
2021-07-20T04:17:35.6160821Z  • Showing Connections  ...
2021-07-20T04:17:35.6185580Z  ✓ Showing Connections
2021-07-20T04:17:35.6210850Z cluster2-worker  cluster2  172.18.0.5  no   libreswan     100.2.0.0/16, 10.2.0.0/16  connected  0s
2021-07-20T04:17:35.6212361Z cluster1-worker  cluster1  172.18.0.4  no   libreswan     100.1.0.0/16, 10.1.0.0/16  connected  379.598µs
2021-07-20T04:17:35.6229287Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:17:35.6250002Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:17:35.6264687Z [36m[submariner-operator]$ [cluster2] with_context cluster2 connectivity_tests cluster1[0m
2021-07-20T04:17:35.6282775Z [36m[submariner-operator]$ [cluster2] with_context cluster2 connectivity_tests cluster1[0m
2021-07-20T04:17:35.6298122Z [36m[submariner-operator]$ [cluster2] connectivity_tests cluster1[0m
2021-07-20T04:17:35.6313467Z [36m[submariner-operator]$ [cluster2] connectivity_tests[0m
2021-07-20T04:17:35.6325364Z [36m[submariner-operator]$ [cluster2] target_cluster=cluster1[0m
2021-07-20T04:17:35.6339233Z [36m[submariner-operator]$ [cluster2] import_image quay.io/submariner/nettest[0m
2021-07-20T04:17:35.6357305Z [36m[submariner-operator]$ [cluster2] import_image quay.io/submariner/nettest[0m
2021-07-20T04:17:35.6379692Z [36m[submariner-operator]$ [cluster2] docker tag quay.io/submariner/nettest:dev localhost:5000/nettest:local[0m
2021-07-20T04:17:35.9266932Z Error response from daemon: No such image: quay.io/submariner/nettest:dev
2021-07-20T04:17:35.9353647Z [36m[submariner-operator]$ [cluster2] docker pull quay.io/submariner/nettest:devel[0m
2021-07-20T04:17:37.5866028Z devel: Pulling from submariner/nettest
2021-07-20T04:17:37.5867091Z 5843afab3874: Already exists
2021-07-20T04:17:37.5901504Z 07557957ddf1: Pulling fs layer
2021-07-20T04:17:37.5903241Z 3e381c3e8fb7: Pulling fs layer
2021-07-20T04:17:37.5904263Z 9eef9f7a1e2d: Pulling fs layer
2021-07-20T04:17:37.9526881Z 9eef9f7a1e2d: Download complete
2021-07-20T04:17:37.9883847Z 07557957ddf1: Verifying Checksum
2021-07-20T04:17:37.9884542Z 07557957ddf1: Download complete
2021-07-20T04:17:38.0189817Z 3e381c3e8fb7: Verifying Checksum
2021-07-20T04:17:38.0190799Z 3e381c3e8fb7: Download complete
2021-07-20T04:17:38.1372516Z 07557957ddf1: Pull complete
2021-07-20T04:17:38.6628415Z 3e381c3e8fb7: Pull complete
2021-07-20T04:17:38.7828547Z 9eef9f7a1e2d: Pull complete
2021-07-20T04:17:38.7880578Z Digest: sha256:97257f0ca58fda5e76ba72feed36f899d870c0811b0ada92a9abfeea2027ee88
2021-07-20T04:17:38.7902745Z Status: Downloaded newer image for quay.io/submariner/nettest:devel
2021-07-20T04:17:38.7918078Z quay.io/submariner/nettest:devel
2021-07-20T04:17:38.7994100Z [36m[submariner-operator]$ [cluster2] docker tag quay.io/submariner/nettest:devel quay.io/submariner/nettest:dev[0m
2021-07-20T04:17:39.1130691Z [36m[submariner-operator]$ [cluster2] docker tag quay.io/submariner/nettest:dev localhost:5000/nettest:local[0m
2021-07-20T04:17:39.4104264Z [36m[submariner-operator]$ [cluster2] docker push localhost:5000/nettest:local[0m
2021-07-20T04:17:39.7363436Z The push refers to repository [localhost:5000/nettest]
2021-07-20T04:17:39.7411755Z 84184d8131fb: Preparing
2021-07-20T04:17:39.7412270Z c277b396665a: Preparing
2021-07-20T04:17:39.7412698Z 135742dfad9e: Preparing
2021-07-20T04:17:39.7413177Z 72e830a4dff5: Preparing
2021-07-20T04:17:40.5026816Z 135742dfad9e: Pushed
2021-07-20T04:17:40.5714761Z 84184d8131fb: Pushed
2021-07-20T04:17:40.5878180Z 72e830a4dff5: Pushed
2021-07-20T04:17:41.2410083Z c277b396665a: Pushed
2021-07-20T04:17:41.2630449Z local: digest: sha256:97257f0ca58fda5e76ba72feed36f899d870c0811b0ada92a9abfeea2027ee88 size: 1155
2021-07-20T04:17:41.2747401Z [36m[submariner-operator]$ [cluster2] deploy_resource /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:17:41.2793851Z [36m[submariner-operator]$ [cluster2] deploy_resource /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:17:41.2809036Z [36m[submariner-operator]$ [cluster2] resource_name=$(basename "$resource_file" ".yaml")[0m
2021-07-20T04:17:41.2829948Z [36m[submariner-operator]$ [cluster2] basename /opt/shipyard/scripts/resources/netshoot.yaml .yaml[0m
2021-07-20T04:17:41.2854513Z [36m[submariner-operator]$ [cluster2] kubectl apply -f /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:17:41.2870643Z [36m[submariner-operator]$ [cluster2] kubectl apply -f /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:17:41.2886498Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 apply -f /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:17:41.8363681Z deployment.apps/netshoot created
2021-07-20T04:17:41.8465274Z Waiting for netshoot pods to be ready.
2021-07-20T04:17:41.8482410Z [36m[submariner-operator]$ [cluster2] kubectl rollout status deploy/netshoot --timeout=5m[0m
2021-07-20T04:17:41.8501607Z [36m[submariner-operator]$ [cluster2] kubectl rollout status deploy/netshoot --timeout=5m[0m
2021-07-20T04:17:41.8518124Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 rollout status deploy/netshoot --timeout=5m[0m
2021-07-20T04:17:42.2434551Z Waiting for deployment "netshoot" rollout to finish: 0 of 1 updated replicas are available...
2021-07-20T04:17:42.8267923Z deployment "netshoot" successfully rolled out
2021-07-20T04:17:42.8330283Z [36m[submariner-operator]$ [cluster2] with_context cluster1 deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:17:42.8344765Z [36m[submariner-operator]$ [cluster2] with_context cluster1 deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:17:42.8363368Z [36m[submariner-operator]$ [cluster1] deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:17:42.8383864Z [36m[submariner-operator]$ [cluster1] deploy_resource[0m
2021-07-20T04:17:42.8402849Z [36m[submariner-operator]$ [cluster1] resource_name=$(basename "$resource_file" ".yaml")[0m
2021-07-20T04:17:42.8425855Z [36m[submariner-operator]$ [cluster1] basename /opt/shipyard/scripts/resources/nginx-demo.yaml .yaml[0m
2021-07-20T04:17:42.8449601Z [36m[submariner-operator]$ [cluster1] kubectl apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:17:42.8465685Z [36m[submariner-operator]$ [cluster1] kubectl apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:17:42.8482339Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:17:43.4966926Z deployment.apps/nginx-demo created
2021-07-20T04:17:43.5411602Z service/nginx-demo created
2021-07-20T04:17:43.6769617Z serviceexport.multicluster.x-k8s.io/nginx-demo created
2021-07-20T04:17:43.6829675Z Waiting for nginx-demo pods to be ready.
2021-07-20T04:17:43.6846075Z [36m[submariner-operator]$ [cluster1] kubectl rollout status deploy/nginx-demo --timeout=5m[0m
2021-07-20T04:17:43.6864942Z [36m[submariner-operator]$ [cluster1] kubectl rollout status deploy/nginx-demo --timeout=5m[0m
2021-07-20T04:17:43.6883851Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 rollout status deploy/nginx-demo --timeout=5m[0m
2021-07-20T04:17:44.4479922Z Waiting for deployment "nginx-demo" rollout to finish: 0 of 2 updated replicas are available...
2021-07-20T04:17:45.4939112Z Waiting for deployment "nginx-demo" rollout to finish: 1 of 2 updated replicas are available...
2021-07-20T04:17:45.6645912Z deployment "nginx-demo" successfully rolled out
2021-07-20T04:17:45.6712801Z [36m[submariner-operator]$ [cluster2] netshoot_pod=$(kubectl get pods -l app=netshoot | awk 'FNR == 2 {print $1}')[0m
2021-07-20T04:17:45.6733534Z [36m[submariner-operator]$ [cluster2] kubectl get pods -l app=netshoot[0m
2021-07-20T04:17:45.6757256Z [36m[submariner-operator]$ [cluster2] awk FNR == 2 {print $1}[0m
2021-07-20T04:17:45.6826493Z [36m[submariner-operator]$ [cluster2] kubectl get pods -l app=netshoot[0m
2021-07-20T04:17:45.6838839Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pods -l app=netshoot[0m
2021-07-20T04:17:46.0485207Z [36m[submariner-operator]$ [cluster2] nginx_svc_ip=$(with_context "$target_cluster" get_svc_ip nginx-demo)[0m
2021-07-20T04:17:46.0508007Z [36m[submariner-operator]$ [cluster2] with_context cluster1 get_svc_ip nginx-demo[0m
2021-07-20T04:17:46.0524545Z [36m[submariner-operator]$ [cluster2] with_context cluster1 get_svc_ip nginx-demo[0m
2021-07-20T04:17:46.0542960Z [36m[submariner-operator]$ [cluster1] get_svc_ip nginx-demo[0m
2021-07-20T04:17:46.0557343Z [36m[submariner-operator]$ [cluster1] get_svc_ip[0m
2021-07-20T04:17:46.0575534Z [36m[submariner-operator]$ [cluster1] svc_ip=$(kubectl --context="$cluster" get svc -l "app=${svc_name}" | awk 'FNR == 2 {print $3}')[0m
2021-07-20T04:17:46.0595800Z [36m[submariner-operator]$ [cluster1] kubectl --context=cluster1 get svc -l app=nginx-demo[0m
2021-07-20T04:17:46.0620536Z [36m[submariner-operator]$ [cluster1] awk FNR == 2 {print $3}[0m
2021-07-20T04:17:46.0629248Z [36m[submariner-operator]$ [cluster1] kubectl --context=cluster1 get svc -l app=nginx-demo[0m
2021-07-20T04:17:46.0644685Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc -l app=nginx-demo[0m
2021-07-20T04:17:46.3709782Z [36m[submariner-operator]$ [cluster2] with_retries 5 test_connection netshoot-57677568b8-699x7 100.1.106.86[0m
2021-07-20T04:17:46.3727186Z [36m[submariner-operator]$ [cluster2] with_retries 5 test_connection netshoot-57677568b8-699x7 100.1.106.86[0m
2021-07-20T04:17:46.3747359Z [36m[submariner-operator]$ [cluster2] iteration=1[0m
2021-07-20T04:17:46.3776333Z [36m[submariner-operator]$ [cluster2] test_connection netshoot-57677568b8-699x7 100.1.106.86[0m
2021-07-20T04:17:46.3781994Z [36m[submariner-operator]$ [cluster2] wait 9047[0m
2021-07-20T04:17:46.3789523Z [36m[submariner-operator]$ [cluster2] test_connection[0m
2021-07-20T04:17:46.3797700Z Attempting connectivity between clusters - netshoot-57677568b8-699x7 --> 100.1.106.86
2021-07-20T04:17:46.3812061Z [36m[submariner-operator]$ [cluster2] wait_time=30[0m
2021-07-20T04:17:46.3825933Z [36m[submariner-operator]$ [cluster2] start_time=$(date +%s)[0m
2021-07-20T04:17:46.3845723Z [36m[submariner-operator]$ [cluster2] date +%s[0m
2021-07-20T04:17:46.3877748Z [36m[submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-699x7 -- curl --output /dev/null -m 30 --silent --head --fail 100.1.106.86[0m
2021-07-20T04:17:46.3891060Z [36m[submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-699x7 -- curl --output /dev/null -m 30 --silent --head --fail 100.1.106.86[0m
2021-07-20T04:17:46.3904608Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 exec netshoot-57677568b8-699x7 -- curl --output /dev/null -m 30 --silent --head --fail 100.1.106.86[0m
2021-07-20T04:17:46.7881363Z Connection test was successful!
2021-07-20T04:17:46.7904811Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:17:46.7922115Z [36m[submariner-operator]$ [cluster2] remove_resource /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:17:46.7938174Z [36m[submariner-operator]$ [cluster2] remove_resource /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:17:46.7957394Z [36m[submariner-operator]$ [cluster2] kubectl delete -f /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:17:46.7974220Z [36m[submariner-operator]$ [cluster2] kubectl delete -f /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:17:46.7990648Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 delete -f /opt/shipyard/scripts/resources/netshoot.yaml[0m
2021-07-20T04:17:47.1062487Z deployment.apps "netshoot" deleted
2021-07-20T04:17:47.1254854Z [36m[submariner-operator]$ [cluster2] with_context cluster1 remove_resource /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:17:47.1304820Z [36m[submariner-operator]$ [cluster2] with_context cluster1 remove_resource /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:17:47.1318292Z [36m[submariner-operator]$ [cluster1] remove_resource /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:17:47.1405348Z [36m[submariner-operator]$ [cluster1] remove_resource[0m
2021-07-20T04:17:47.1420239Z [36m[submariner-operator]$ [cluster1] kubectl delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:17:47.1432273Z [36m[submariner-operator]$ [cluster1] kubectl delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:17:47.1492587Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml[0m
2021-07-20T04:17:47.4616465Z deployment.apps "nginx-demo" deleted
2021-07-20T04:17:47.5270684Z service "nginx-demo" deleted
2021-07-20T04:17:47.5470218Z serviceexport.multicluster.x-k8s.io "nginx-demo" deleted
2021-07-20T04:17:47.5710946Z [36m[submariner-operator]$ [cluster2] run_if_defined post_deploy[0m
2021-07-20T04:17:47.5728242Z [36m[submariner-operator]$ [cluster2] run_if_defined post_deploy[0m
2021-07-20T04:17:47.5745161Z [36m[submariner-operator]$ [cluster2] type -t post_deploy[0m
2021-07-20T04:17:47.5769941Z [36m[submariner-operator]$ [cluster2] grep -q function[0m
2021-07-20T04:17:47.5835267Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:17:47.5858206Z   Successfully remade target file 'deploy'.
2021-07-20T04:17:47.5864064Z    File 'e2e' does not exist.
2021-07-20T04:17:47.5864949Z   Must remake target 'e2e'.
2021-07-20T04:17:47.5869727Z scripts/kind-e2e/e2e.sh
2021-07-20T04:17:47.5989976Z [36m[submariner-operator]$ source /opt/shipyard/scripts/lib/utils[0m
2021-07-20T04:17:47.6038279Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:17:47.6051544Z [36m[submariner-operator]$ script_name=utils[0m
2021-07-20T04:17:47.6121595Z [36m[submariner-operator]$ exec_name=e2e.sh[0m
2021-07-20T04:17:47.6172892Z [36m[submariner-operator]$ source /opt/shipyard/scripts/lib/deploy_funcs[0m
2021-07-20T04:17:47.6187784Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:17:47.6211948Z [36m[submariner-operator]$ script_name=deploy_funcs[0m
2021-07-20T04:17:47.6230113Z [36m[submariner-operator]$ exec_name=e2e.sh[0m
2021-07-20T04:17:47.6285219Z [36m[submariner-operator]$ source /opt/shipyard/scripts/lib/deploy_operator[0m
2021-07-20T04:17:47.6308324Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:17:47.6416395Z [36m[submariner-operator]$ script_name=deploy_operator[0m
2021-07-20T04:17:47.6440687Z [36m[submariner-operator]$ exec_name=e2e.sh[0m
2021-07-20T04:17:47.6567408Z [36m[submariner-operator]$ source /opt/shipyard/scripts/lib/cluster_settings[0m
2021-07-20T04:17:47.6666515Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:17:47.6744512Z [36m[submariner-operator]$ script_name=cluster_settings[0m
2021-07-20T04:17:47.6765519Z [36m[submariner-operator]$ exec_name=e2e.sh[0m
2021-07-20T04:17:47.6833567Z [36m[submariner-operator]$ broker=cluster1[0m
2021-07-20T04:17:47.6846839Z [36m[submariner-operator]$ declare -gA cluster_nodes[0m
2021-07-20T04:17:47.6861620Z [36m[submariner-operator]$ cluster_nodes[cluster1]=control-plane worker[0m
2021-07-20T04:17:47.6875669Z [36m[submariner-operator]$ cluster_nodes[cluster2]=control-plane worker[0m
2021-07-20T04:17:47.6895713Z [36m[submariner-operator]$ cluster_nodes[cluster3]=control-plane worker worker[0m
2021-07-20T04:17:47.6933936Z [36m[submariner-operator]$ declare -gA cluster_subm[0m
2021-07-20T04:17:47.6977716Z [36m[submariner-operator]$ cluster_subm[cluster1]=true[0m
2021-07-20T04:17:47.6998497Z [36m[submariner-operator]$ cluster_subm[cluster2]=true[0m
2021-07-20T04:17:47.7026044Z [36m[submariner-operator]$ cluster_subm[cluster3]=true[0m
2021-07-20T04:17:47.7054234Z [36m[submariner-operator]$ declare -gA cluster_cni[0m
2021-07-20T04:17:47.7114492Z [36m[submariner-operator]$ cluster_cni[cluster2]=weave[0m
2021-07-20T04:17:47.7130667Z [36m[submariner-operator]$ cluster_cni[cluster3]=weave[0m
2021-07-20T04:17:47.7142853Z [36m[submariner-operator]$ source /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings[0m
2021-07-20T04:17:47.7152598Z [36m[submariner-operator]$ . /opt/shipyard/scripts/lib/source_only[0m
2021-07-20T04:17:47.7163284Z [36m[submariner-operator]$ script_name=cluster_settings[0m
2021-07-20T04:17:47.7175344Z [36m[submariner-operator]$ exec_name=e2e.sh[0m
2021-07-20T04:17:47.7283886Z [36m[submariner-operator]$ cluster_nodes[cluster1]=control-plane worker worker[0m
2021-07-20T04:17:47.7294743Z [36m[submariner-operator]$ cluster_nodes[cluster2]=control-plane worker worker[0m
2021-07-20T04:17:47.7352944Z [36m[submariner-operator]$ timeout=2m[0m
2021-07-20T04:17:47.7389817Z [36m[submariner-operator]$ declare_kubeconfig[0m
2021-07-20T04:17:47.7422538Z [36m[submariner-operator]$ declare_kubeconfig[0m
2021-07-20T04:17:47.7432567Z [36m[submariner-operator]$ source /opt/shipyard/scripts/lib/kubecfg[0m
2021-07-20T04:17:47.7459165Z [36m[submariner-operator]$ export KUBECONFIG[0m
2021-07-20T04:17:47.7470576Z [36m[submariner-operator]$ KUBECONFIG=$(find "${DAPPER_OUTPUT:-output}"/kubeconfigs -type f -printf %p: || :)[0m
2021-07-20T04:17:47.7500496Z [36m[submariner-operator]$ find /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs -type f -printf %p:[0m
2021-07-20T04:17:47.7568885Z [36m[submariner-operator]$ . /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/lib_operator_verify_subm.sh[0m
2021-07-20T04:17:47.7617664Z [36m[submariner-operator]$ create_subm_vars[0m
2021-07-20T04:17:47.7657476Z [36m[submariner-operator]$ create_subm_vars[0m
2021-07-20T04:17:47.7681777Z [36m[submariner-operator]$ deployment_name=submariner[0m
2021-07-20T04:17:47.7715030Z [36m[submariner-operator]$ operator_deployment_name=submariner-operator[0m
2021-07-20T04:17:47.7736264Z [36m[submariner-operator]$ gateway_deployment_name=submariner-gateway[0m
2021-07-20T04:17:47.7745869Z [36m[submariner-operator]$ routeagent_deployment_name=submariner-routeagent[0m
2021-07-20T04:17:47.7756229Z [36m[submariner-operator]$ broker_deployment_name=submariner-k8s-broker[0m
2021-07-20T04:17:47.7766450Z [36m[submariner-operator]$ globalnet_deployment_name=submariner-globalnet[0m
2021-07-20T04:17:47.7789486Z [36m[submariner-operator]$ declare_cidrs[0m
2021-07-20T04:17:47.7806385Z [36m[submariner-operator]$ declare_cidrs[0m
2021-07-20T04:17:47.7824162Z [36m[submariner-operator]$ declare -gA cluster_CIDRs service_CIDRs global_CIDRs[0m
2021-07-20T04:17:47.7835768Z [36m[submariner-operator]$ i=1[0m
2021-07-20T04:17:47.7860808Z [36m[submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1[0m
2021-07-20T04:17:47.7874182Z [36m[submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1[0m
2021-07-20T04:17:47.7891216Z [36m[submariner-operator]$ [cluster1] cluster_CIDRs[cluster1]=10.1.0.0/16[0m
2021-07-20T04:17:47.7910937Z [36m[submariner-operator]$ [cluster1] service_CIDRs[cluster1]=100.1.0.0/16[0m
2021-07-20T04:17:47.7948532Z [36m[submariner-operator]$ [cluster1] i=$(("$i"+1))[0m
2021-07-20T04:17:47.7969108Z [36m[submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2[0m
2021-07-20T04:17:47.7994987Z [36m[submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2[0m
2021-07-20T04:17:47.8010955Z [36m[submariner-operator]$ [cluster2] cluster_CIDRs[cluster2]=10.2.0.0/16[0m
2021-07-20T04:17:47.8040003Z [36m[submariner-operator]$ [cluster2] service_CIDRs[cluster2]=100.2.0.0/16[0m
2021-07-20T04:17:47.8057575Z [36m[submariner-operator]$ [cluster2] i=$(("$i"+1))[0m
2021-07-20T04:17:47.8071885Z [36m[submariner-operator]$ [cluster2] natEnabled=false[0m
2021-07-20T04:17:47.8085295Z [36m[submariner-operator]$ [cluster2] subm_gateway_image_repo=localhost:5000[0m
2021-07-20T04:17:47.8103442Z [36m[submariner-operator]$ [cluster2] subm_gateway_image_tag=local[0m
2021-07-20T04:17:47.8117398Z [36m[submariner-operator]$ [cluster2] subm_gateway_size=3[0m
2021-07-20T04:17:47.8131738Z [36m[submariner-operator]$ [cluster2] subm_colorcodes=blue[0m
2021-07-20T04:17:47.8150337Z [36m[submariner-operator]$ [cluster2] subm_debug=false[0m
2021-07-20T04:17:47.8168050Z [36m[submariner-operator]$ [cluster2] subm_broker=k8s[0m
2021-07-20T04:17:47.8184112Z [36m[submariner-operator]$ [cluster2] ce_ipsec_debug=false[0m
2021-07-20T04:17:47.8202752Z [36m[submariner-operator]$ [cluster2] ce_ipsec_ikeport=500[0m
2021-07-20T04:17:47.8219021Z [36m[submariner-operator]$ [cluster2] ce_ipsec_nattport=4500[0m
2021-07-20T04:17:47.8232585Z [36m[submariner-operator]$ [cluster2] subm_ns=submariner-operator[0m
2021-07-20T04:17:47.8249438Z [36m[submariner-operator]$ [cluster2] SUBMARINER_BROKER_NS=submariner-k8s-broker[0m
2021-07-20T04:17:47.8264248Z [36m[submariner-operator]$ [cluster2] with_context cluster1 broker_vars[0m
2021-07-20T04:17:47.8281578Z [36m[submariner-operator]$ [cluster2] with_context cluster1 broker_vars[0m
2021-07-20T04:17:47.8302690Z [36m[submariner-operator]$ [cluster1] broker_vars[0m
2021-07-20T04:17:47.8324425Z [36m[submariner-operator]$ [cluster1] broker_vars[0m
2021-07-20T04:17:47.8414463Z [36m[submariner-operator]$ [cluster1] SUBMARINER_BROKER_URL=$(kubectl -n default get endpoints kubernetes -o jsonpath="{.subsets[0].addresses[0].ip}:{.subsets[0].ports[?(@.name=='https')].port}")[0m
2021-07-20T04:17:47.8436351Z [36m[submariner-operator]$ [cluster1] kubectl -n default get endpoints kubernetes -o jsonpath={.subsets[0].addresses[0].ip}:{.subsets[0].ports[?(@.name=='https')].port}[0m
2021-07-20T04:17:47.8453947Z [36m[submariner-operator]$ [cluster1] kubectl -n default get endpoints kubernetes -o jsonpath={.subsets[0].addresses[0].ip}:{.subsets[0].ports[?(@.name=='https')].port}[0m
2021-07-20T04:17:47.8468389Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 -n default get endpoints kubernetes -o jsonpath={.subsets[0].addresses[0].ip}:{.subsets[0].ports[?(@.name=='https')].port}[0m
2021-07-20T04:17:48.3042125Z [36m[submariner-operator]$ [cluster1] SUBMARINER_BROKER_CA=$(kubectl -n ${SUBMARINER_BROKER_NS} get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='${SUBMARINER_BROKER_NS}-client')].data['ca\.crt']}")[0m
2021-07-20T04:17:48.3087520Z [36m[submariner-operator]$ [cluster1] kubectl -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data['ca\.crt']}[0m
2021-07-20T04:17:48.3115987Z [36m[submariner-operator]$ [cluster1] kubectl -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data['ca\.crt']}[0m
2021-07-20T04:17:48.3129544Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data['ca\.crt']}[0m
2021-07-20T04:17:48.8716255Z [36m[submariner-operator]$ [cluster1] SUBMARINER_BROKER_TOKEN=$(kubectl -n ${SUBMARINER_BROKER_NS} get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='${SUBMARINER_BROKER_NS}-client')].data.token}"|base64 --decode)[0m
2021-07-20T04:17:48.8737585Z [36m[submariner-operator]$ [cluster1] kubectl -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data.token}[0m
2021-07-20T04:17:48.8752257Z [36m[submariner-operator]$ [cluster1] base64 --decode[0m
2021-07-20T04:17:48.8759806Z [36m[submariner-operator]$ [cluster1] kubectl -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data.token}[0m
2021-07-20T04:17:48.8771256Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data.token}[0m
2021-07-20T04:17:49.1993568Z [36m[submariner-operator]$ [cluster2] with_context cluster1 verify_subm_broker_secrets[0m
2021-07-20T04:17:49.2009096Z [36m[submariner-operator]$ [cluster2] with_context cluster1 verify_subm_broker_secrets[0m
2021-07-20T04:17:49.2028490Z [36m[submariner-operator]$ [cluster1] verify_subm_broker_secrets[0m
2021-07-20T04:17:49.2044284Z [36m[submariner-operator]$ [cluster1] verify_subm_broker_secrets[0m
2021-07-20T04:17:49.2626105Z [36m[submariner-operator]$ [cluster1] verify_secrets submariner-k8s-broker submariner-k8s-broker-client LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3lNREEwTURjd05Gb1hEVE14TURjeE9EQTBNRGN3TkZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTDMvCm5MZDVnKzRsa2hja3FVZC9ZdHJ2VzZrN3FuM0NrWXlRSkEzL3hvVU1DY1BNUElzR0RySlNhK0tDdzVDVkY4ZE8KWWNQUStpU3ozOXBaNFc0M1dsUjhNVDZkUGx4VzhOVGJDWmFzK3RtKzBFOHE0ZlFpbEdIQWZNenUrYTJQdjRkUgpUTm9BZE9nRU9WcXZ6NExjNmhaUnhnUkFzeGExRHZieUVySXNleDNiU1RuREROb0svSkhQSnFNVkJ3bFNFZTVCCkErelFNV0IrOWJmeHloWmhBZGpMZ1pacm5VT3ZkREp2V0Mzay9TaEZSOWplMWZJRGVQaXE1YncrYWJ4c1lHRXMKaHEyZHFHa1B0bkFkN0laZTV1cnJLdVhicURNei9tOENDSWYyd2ovd05adnMxTkRWaEFrL1pndUxOSUt0Y0loLwp4cnk3YllEZW9rVUtCQ2hlS1pVQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZESjRwZkJmQ0s2Y2hCdmlINWtKQ2RBUmlEVWFNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTlRHZFgyMXZuYWNFY2J6MGlEMGxvRjlYbTcxVmxZRGJGR1h0VzF1a25nZ3R6c3BJOApOL0RGVWFsOU1RZmtmT05zUDVMaWRVSDlDRzRsc0Z4ekloWWxoOHUwMzNiNFVRbTJkN09lTUIwTGc3dnFkT01yCmxLQWZPYzhHa1ViemRQUWh2YTd2SDlkN3dzRWdtVFlzTGVqTEdMaG5sbFFTN0tIbVdLNTN2Qml6amtJamVyeWkKbE4rVEhTelIxbTVwME1QMk8xMmRxVXE5UHhUZHd4VzFkNVc4ZWZDT3VGM0wxdlVseVAyZXVWVXo4dWxGNGFlZApVOHVOVmhMK3NZZ3hVcE9KRVU0Z1hTMEpXYk9FSVpXaFRBUHpoTXRSb04rNjZCK2s0S3VxR2xkaHJZYlpOT0U1CkV2Z3J3a0Z0RUJ6WmRYNWNNMnZ4NE96Zzh4c1NEMHV5U3pRSAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==[0m
2021-07-20T04:17:49.3748214Z [36m[submariner-operator]$ [cluster1] verify_secrets submariner-k8s-broker submariner-k8s-broker-client LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3lNREEwTURjd05Gb1hEVE14TURjeE9EQTBNRGN3TkZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTDMvCm5MZDVnKzRsa2hja3FVZC9ZdHJ2VzZrN3FuM0NrWXlRSkEzL3hvVU1DY1BNUElzR0RySlNhK0tDdzVDVkY4ZE8KWWNQUStpU3ozOXBaNFc0M1dsUjhNVDZkUGx4VzhOVGJDWmFzK3RtKzBFOHE0ZlFpbEdIQWZNenUrYTJQdjRkUgpUTm9BZE9nRU9WcXZ6NExjNmhaUnhnUkFzeGExRHZieUVySXNleDNiU1RuREROb0svSkhQSnFNVkJ3bFNFZTVCCkErelFNV0IrOWJmeHloWmhBZGpMZ1pacm5VT3ZkREp2V0Mzay9TaEZSOWplMWZJRGVQaXE1YncrYWJ4c1lHRXMKaHEyZHFHa1B0bkFkN0laZTV1cnJLdVhicURNei9tOENDSWYyd2ovd05adnMxTkRWaEFrL1pndUxOSUt0Y0loLwp4cnk3YllEZW9rVUtCQ2hlS1pVQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZESjRwZkJmQ0s2Y2hCdmlINWtKQ2RBUmlEVWFNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTlRHZFgyMXZuYWNFY2J6MGlEMGxvRjlYbTcxVmxZRGJGR1h0VzF1a25nZ3R6c3BJOApOL0RGVWFsOU1RZmtmT05zUDVMaWRVSDlDRzRsc0Z4ekloWWxoOHUwMzNiNFVRbTJkN09lTUIwTGc3dnFkT01yCmxLQWZPYzhHa1ViemRQUWh2YTd2SDlkN3dzRWdtVFlzTGVqTEdMaG5sbFFTN0tIbVdLNTN2Qml6amtJamVyeWkKbE4rVEhTelIxbTVwME1QMk8xMmRxVXE5UHhUZHd4VzFkNVc4ZWZDT3VGM0wxdlVseVAyZXVWVXo4dWxGNGFlZApVOHVOVmhMK3NZZ3hVcE9KRVU0Z1hTMEpXYk9FSVpXaFRBUHpoTXRSb04rNjZCK2s0S3VxR2xkaHJZYlpOT0U1CkV2Z3J3a0Z0RUJ6WmRYNWNNMnZ4NE96Zzh4c1NEMHV5U3pRSAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==[0m
2021-07-20T04:17:49.4384543Z [36m[submariner-operator]$ [cluster1] wait_for_secrets submariner-k8s-broker[0m
2021-07-20T04:17:49.4386512Z [36m[submariner-operator]$ [cluster1] wait_for_secrets submariner-k8s-broker[0m
2021-07-20T04:17:49.4387938Z [36m[submariner-operator]$ [cluster1] kubectl get secret -n submariner-k8s-broker[0m
2021-07-20T04:17:49.4389296Z [36m[submariner-operator]$ [cluster1] grep -q submariner-[0m
2021-07-20T04:17:49.4391828Z [36m[submariner-operator]$ [cluster1] kubectl get secret -n submariner-k8s-broker[0m
2021-07-20T04:17:49.4394590Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get secret -n submariner-k8s-broker[0m
2021-07-20T04:17:49.6288439Z [36m[submariner-operator]$ [cluster1] return[0m
2021-07-20T04:17:49.6299406Z [36m[submariner-operator]$ [cluster1] kubectl get secrets -n submariner-k8s-broker[0m
2021-07-20T04:17:49.6328364Z [36m[submariner-operator]$ [cluster1] kubectl get secrets -n submariner-k8s-broker[0m
2021-07-20T04:17:49.6356842Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get secrets -n submariner-k8s-broker[0m
2021-07-20T04:17:49.9782094Z NAME                                       TYPE                                  DATA   AGE
2021-07-20T04:17:49.9802782Z cluster-cluster1-token-j5kx2               kubernetes.io/service-account-token   3      4m26s
2021-07-20T04:17:49.9815732Z cluster-cluster2-token-kkz78               kubernetes.io/service-account-token   3      4m26s
2021-07-20T04:17:49.9818908Z default-token-pjlxp                        kubernetes.io/service-account-token   3      4m48s
2021-07-20T04:17:49.9820688Z submariner-k8s-broker-admin-token-4f2x5    kubernetes.io/service-account-token   3      4m48s
2021-07-20T04:17:49.9822851Z submariner-k8s-broker-client-token-5649n   kubernetes.io/service-account-token   3      4m48s
2021-07-20T04:17:49.9886702Z [36m[submariner-operator]$ [cluster1] kubectl get secrets -n submariner-k8s-broker -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].metadata.name}[0m
2021-07-20T04:17:49.9902140Z [36m[submariner-operator]$ [cluster1] kubectl get secrets -n submariner-k8s-broker -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].metadata.name}[0m
2021-07-20T04:17:49.9919734Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get secrets -n submariner-k8s-broker -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].metadata.name}[0m
2021-07-20T04:17:50.2952317Z [36m[submariner-operator]$ [cluster1] json_file=/tmp/submariner-k8s-broker-client-token-5649n.cluster1.json[0m
2021-07-20T04:17:50.2969592Z [36m[submariner-operator]$ [cluster1] kubectl get secret submariner-k8s-broker-client-token-5649n -n submariner-k8s-broker -o json[0m
2021-07-20T04:17:50.2988021Z [36m[submariner-operator]$ [cluster1] tee /tmp/submariner-k8s-broker-client-token-5649n.cluster1.json[0m
2021-07-20T04:17:50.3014814Z [36m[submariner-operator]$ [cluster1] kubectl get secret submariner-k8s-broker-client-token-5649n -n submariner-k8s-broker -o json[0m
2021-07-20T04:17:50.3029082Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get secret submariner-k8s-broker-client-token-5649n -n submariner-k8s-broker -o json[0m
2021-07-20T04:17:50.6045467Z {
2021-07-20T04:17:50.6046389Z     "apiVersion": "v1",
2021-07-20T04:17:50.6046923Z     "data": {
2021-07-20T04:17:50.6652650Z         "ca.crt": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3lNREEwTURjd05Gb1hEVE14TURjeE9EQTBNRGN3TkZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTDMvCm5MZDVnKzRsa2hja3FVZC9ZdHJ2VzZrN3FuM0NrWXlRSkEzL3hvVU1DY1BNUElzR0RySlNhK0tDdzVDVkY4ZE8KWWNQUStpU3ozOXBaNFc0M1dsUjhNVDZkUGx4VzhOVGJDWmFzK3RtKzBFOHE0ZlFpbEdIQWZNenUrYTJQdjRkUgpUTm9BZE9nRU9WcXZ6NExjNmhaUnhnUkFzeGExRHZieUVySXNleDNiU1RuREROb0svSkhQSnFNVkJ3bFNFZTVCCkErelFNV0IrOWJmeHloWmhBZGpMZ1pacm5VT3ZkREp2V0Mzay9TaEZSOWplMWZJRGVQaXE1YncrYWJ4c1lHRXMKaHEyZHFHa1B0bkFkN0laZTV1cnJLdVhicURNei9tOENDSWYyd2ovd05adnMxTkRWaEFrL1pndUxOSUt0Y0loLwp4cnk3YllEZW9rVUtCQ2hlS1pVQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZESjRwZkJmQ0s2Y2hCdmlINWtKQ2RBUmlEVWFNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTlRHZFgyMXZuYWNFY2J6MGlEMGxvRjlYbTcxVmxZRGJGR1h0VzF1a25nZ3R6c3BJOApOL0RGVWFsOU1RZmtmT05zUDVMaWRVSDlDRzRsc0Z4ekloWWxoOHUwMzNiNFVRbTJkN09lTUIwTGc3dnFkT01yCmxLQWZPYzhHa1ViemRQUWh2YTd2SDlkN3dzRWdtVFlzTGVqTEdMaG5sbFFTN0tIbVdLNTN2Qml6amtJamVyeWkKbE4rVEhTelIxbTVwME1QMk8xMmRxVXE5UHhUZHd4VzFkNVc4ZWZDT3VGM0wxdlVseVAyZXVWVXo4dWxGNGFlZApVOHVOVmhMK3NZZ3hVcE9KRVU0Z1hTMEpXYk9FSVpXaFRBUHpoTXRSb04rNjZCK2s0S3VxR2xkaHJZYlpOT0U1CkV2Z3J3a0Z0RUJ6WmRYNWNNMnZ4NE96Zzh4c1NEMHV5U3pRSAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==",
2021-07-20T04:17:50.7235813Z [36m[submariner-operator]$ [cluster1] validate_equals .kind Secret[0m
2021-07-20T04:17:50.7236992Z [36m[submariner-operator]$ [cluster1] validate_equals .kind Secret[0m
2021-07-20T04:17:50.7239249Z [36m[submariner-operator]$ [cluster1] jq -r -M .kind /tmp/submariner-k8s-broker-client-token-5649n.cluster1.json[0m
2021-07-20T04:17:50.7241483Z [36m[submariner-operator]$ [cluster1] validate_equals .type kubernetes.io/service-account-token[0m
2021-07-20T04:17:50.7243080Z [36m[submariner-operator]$ [cluster1] validate_equals .type kubernetes.io/service-account-token[0m
2021-07-20T04:17:50.7245355Z [36m[submariner-operator]$ [cluster1] jq -r -M .type /tmp/submariner-k8s-broker-client-token-5649n.cluster1.json[0m
2021-07-20T04:17:50.7254753Z         "namespace": "c3VibWFyaW5lci1rOHMtYnJva2Vy",
2021-07-20T04:17:50.7764679Z         "token": "ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNkltZ3lkR2RGVVdwS1ZsWk9VbGhUVGtSQ1psVnVWbEpGTTBKVFoyNHlWRWxsWldJMmIxQTNZMjVWYWswaWZRLmV5SnBjM01pT2lKcmRXSmxjbTVsZEdWekwzTmxjblpwWTJWaFkyTnZkVzUwSWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXVZVzFsYzNCaFkyVWlPaUp6ZFdKdFlYSnBibVZ5TFdzNGN5MWljbTlyWlhJaUxDSnJkV0psY201bGRHVnpMbWx2TDNObGNuWnBZMlZoWTJOdmRXNTBMM05sWTNKbGRDNXVZVzFsSWpvaWMzVmliV0Z5YVc1bGNpMXJPSE10WW5KdmEyVnlMV05zYVdWdWRDMTBiMnRsYmkwMU5qUTViaUlzSW10MVltVnlibVYwWlhNdWFXOHZjMlZ5ZG1salpXRmpZMjkxYm5RdmMyVnlkbWxqWlMxaFkyTnZkVzUwTG01aGJXVWlPaUp6ZFdKdFlYSnBibVZ5TFdzNGN5MWljbTlyWlhJdFkyeHBaVzUwSWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXpaWEoyYVdObExXRmpZMjkxYm5RdWRXbGtJam9pTnpJd04yUTJNbU10TnpFeE1DMDBNR1ZtTFdFeVlURXROamN5WmpSa01qUXpZV0UzSWl3aWMzVmlJam9pYzNsemRHVnRPbk5sY25acFkyVmhZMk52ZFc1ME9uTjFZbTFoY21sdVpYSXRhemh6TFdKeWIydGxjanB6ZFdKdFlYSnBibVZ5TFdzNGN5MWljbTlyWlhJdFkyeHBaVzUwSW4wLkctZFE0T211N19McmpIcmVHQXdDWWF1azBjbVFIa0dHQ01RLXFUNmk1UVExSlpERHlHNjV2ZEt1R3V5NzRnMldOdk5DLWhBS09BUTQzaVJNc2N5cjlnZzluQ1hwR2d5N2ZzNjR0TE9hZjVOWG9rU0F4MThwdTZSVzBKazhfWUlhRkdxeU1oeDlLVHd5bVpBOHVWeFRGeDU0Znh2ZHczMTJnNlB2eVZwUlBKZ1VTeGJkZm9GUVhFS0piNjRGX3VNMHVpZmVrTkVkakRFUUNZM09BOUNiWnpwaEg1MGh2NTE3U2ZXbm1DU2x0WG1XQjN3cEhlZFdLdUdRR1lobUpqVmdkNjBORk44SlNxd1R3Rlc3SmdtRVJoTnpud2lEQVZaT1dYNElvRVZHY3RYaENEbVNSSEswREtDRDhNczRUWkRGM0p5Q3NBVlZ6VzVvUVFZUkFfbmQwQQ=="
2021-07-20T04:17:50.8312802Z [36m[submariner-operator]$ [cluster1] validate_equals .metadata.name submariner-k8s-broker-client-token-5649n[0m
2021-07-20T04:17:50.8315236Z [36m[submariner-operator]$ [cluster1] validate_equals .metadata.name submariner-k8s-broker-client-token-5649n[0m
2021-07-20T04:17:50.8317644Z [36m[submariner-operator]$ [cluster1] jq -r -M .metadata.name /tmp/submariner-k8s-broker-client-token-5649n.cluster1.json[0m
2021-07-20T04:17:50.8319795Z [36m[submariner-operator]$ [cluster1] validate_equals .metadata.namespace submariner-k8s-broker[0m
2021-07-20T04:17:50.8321427Z [36m[submariner-operator]$ [cluster1] validate_equals .metadata.namespace submariner-k8s-broker[0m
2021-07-20T04:17:50.8323781Z [36m[submariner-operator]$ [cluster1] jq -r -M .metadata.namespace /tmp/submariner-k8s-broker-client-token-5649n.cluster1.json[0m
2021-07-20T04:17:50.8325338Z     },
2021-07-20T04:17:50.8326092Z     "kind": "Secret",
2021-07-20T04:17:50.8326539Z     "metadata": {
2021-07-20T04:17:50.8330658Z         "annotations": {
2021-07-20T04:17:50.8332240Z             "kubernetes.io/service-account.name": "submariner-k8s-broker-client",
2021-07-20T04:17:50.8333660Z             "kubernetes.io/service-account.uid": "7207d62c-7110-40ef-a2a1-672f4d243aa7"
2021-07-20T04:17:50.8334543Z         },
2021-07-20T04:17:50.8335610Z         "creationTimestamp": "2021-07-20T04:13:01Z",
2021-07-20T04:17:50.8336244Z         "managedFields": [
2021-07-20T04:17:50.8336785Z             {
2021-07-20T04:17:50.8337914Z                 "apiVersion": "v1",
2021-07-20T04:17:50.8338807Z                 "fieldsType": "FieldsV1",
2021-07-20T04:17:50.8339358Z                 "fieldsV1": {
2021-07-20T04:17:50.8351306Z                     "f:data": {
2021-07-20T04:17:50.8352133Z                         ".": {},
2021-07-20T04:17:50.8352721Z                         "f:ca.crt": {},
2021-07-20T04:17:50.8353178Z                         "f:namespace": {},
2021-07-20T04:17:50.8353800Z                         "f:token": {}
2021-07-20T04:17:50.8354704Z                     },
2021-07-20T04:17:50.8355338Z                     "f:metadata": {
2021-07-20T04:17:50.8355804Z                         "f:annotations": {
2021-07-20T04:17:50.8356422Z                             ".": {},
2021-07-20T04:17:50.8357496Z                             "f:kubernetes.io/service-account.name": {},
2021-07-20T04:17:50.8358690Z                             "f:kubernetes.io/service-account.uid": {}
2021-07-20T04:17:50.8359406Z                         }
2021-07-20T04:17:50.8359867Z                     },
2021-07-20T04:17:50.8360344Z                     "f:type": {}
2021-07-20T04:17:50.8360748Z                 },
2021-07-20T04:17:50.8361996Z                 "manager": "kube-controller-manager",
2021-07-20T04:17:50.8362681Z                 "operation": "Update",
2021-07-20T04:17:50.8363614Z                 "time": "2021-07-20T04:13:01Z"
2021-07-20T04:17:50.8364424Z             }
2021-07-20T04:17:50.8364923Z         ],
2021-07-20T04:17:50.8366022Z         "name": "submariner-k8s-broker-client-token-5649n",
2021-07-20T04:17:50.8367630Z         "namespace": "submariner-k8s-broker",
2021-07-20T04:17:50.8368420Z         "resourceVersion": "1245",
2021-07-20T04:17:50.8369559Z         "uid": "3c4afde9-5f7e-4cda-9d10-1ed4cc3f6714"
2021-07-20T04:17:50.8370296Z     },
2021-07-20T04:17:50.8371200Z     "type": "kubernetes.io/service-account-token"
2021-07-20T04:17:50.8372893Z }
2021-07-20T04:17:50.8623311Z [36m[submariner-operator]$ [cluster1] jq -r -M .data["ca.crt"] /tmp/submariner-k8s-broker-client-token-5649n.cluster1.json[0m
2021-07-20T04:17:50.9077863Z [36m[submariner-operator]$ [cluster1] one_ca_crt_ok=true[0m
2021-07-20T04:17:50.9095449Z [36m[submariner-operator]$ [cluster2] run_subm_clusters verify_subm_deployed[0m
2021-07-20T04:17:50.9112248Z [36m[submariner-operator]$ [cluster2] run_subm_clusters verify_subm_deployed[0m
2021-07-20T04:17:50.9129020Z [36m[submariner-operator]$ [cluster2] declare -a subm_clusters[0m
2021-07-20T04:17:50.9171464Z [36m[submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 verify_subm_deployed[0m
2021-07-20T04:17:50.9187700Z [36m[submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 verify_subm_deployed[0m
2021-07-20T04:17:50.9205566Z [36m[submariner-operator]$ [cluster2] declare -A pids[0m
2021-07-20T04:17:50.9228939Z [36m[submariner-operator]$ [cluster2] eval echo cluster1 cluster2[0m
2021-07-20T04:17:50.9254108Z [36m[submariner-operator]$ [cluster1] set -o pipefail[0m
2021-07-20T04:17:50.9255267Z [36m[submariner-operator]$ [cluster1] pids[cluster1]=9296[0m
2021-07-20T04:17:50.9270856Z [36m[submariner-operator]$ [cluster1] verify_subm_deployed[0m
2021-07-20T04:17:50.9288642Z [36m[submariner-operator]$ [cluster2] set -o pipefail[0m
2021-07-20T04:17:50.9294901Z [36m[submariner-operator]$ [cluster2] pids[cluster2]=9300[0m
2021-07-20T04:17:50.9310946Z [36m[submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /[0m
2021-07-20T04:17:50.9322287Z [36m[submariner-operator]$ [cluster2] verify_subm_deployed[0m
2021-07-20T04:17:50.9332212Z [36m[submariner-operator]$ [cluster2] wait 9300[0m
2021-07-20T04:17:50.9353430Z [36m[submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /[0m
2021-07-20T04:17:51.7788314Z [36m[submariner-operator]$ [cluster1] verify_subm_deployed[0m
2021-07-20T04:17:51.7789429Z [36m[submariner-operator]$ [cluster1] verify_endpoints_crd[0m
2021-07-20T04:17:51.7790473Z [36m[submariner-operator]$ [cluster1] verify_endpoints_crd[0m
2021-07-20T04:17:51.7792089Z [36m[submariner-operator]$ [cluster1] validate_crd endpoints.submariner.io v1 Endpoint[0m
2021-07-20T04:17:51.7793935Z [36m[submariner-operator]$ [cluster1] validate_crd endpoints.submariner.io v1 Endpoint[0m
2021-07-20T04:17:51.7795383Z [36m[submariner-operator]$ [cluster1] kubectl get crds endpoints.submariner.io[0m
2021-07-20T04:17:51.7796935Z [36m[submariner-operator]$ [cluster1] kubectl get crds endpoints.submariner.io[0m
2021-07-20T04:17:51.7798538Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crds endpoints.submariner.io[0m
2021-07-20T04:17:51.7799468Z [cluster1] NAME                      CREATED AT
2021-07-20T04:17:51.7800770Z [cluster1] endpoints.submariner.io   2021-07-20T04:13:13Z
2021-07-20T04:17:51.7802227Z [36m[submariner-operator]$ [cluster1] json_file=/tmp/endpoints.submariner.io.cluster1.json[0m
2021-07-20T04:17:51.7803851Z [36m[submariner-operator]$ [cluster1] kubectl get crd endpoints.submariner.io -o json[0m
2021-07-20T04:17:51.7806808Z [36m[submariner-operator]$ [cluster1] kubectl get crd endpoints.submariner.io -o json[0m
2021-07-20T04:17:51.7808442Z [36m[submariner-operator]$ [cluster1] tee /tmp/endpoints.submariner.io.cluster1.json[0m
2021-07-20T04:17:51.7810654Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crd endpoints.submariner.io -o json[0m
2021-07-20T04:17:51.7811531Z [cluster1] {
2021-07-20T04:17:51.7812332Z [cluster1]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-20T04:17:51.7813189Z [cluster1]     "kind": "CustomResourceDefinition",
2021-07-20T04:17:51.7814051Z [cluster1]     "metadata": {
2021-07-20T04:17:51.7814566Z [cluster1]         "annotations": {
2021-07-20T04:17:51.7815739Z [cluster1]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-20T04:17:51.7816470Z [cluster1]         },
2021-07-20T04:17:51.7867112Z [cluster1]         "creationTimestamp": "2021-07-20T04:13:13Z",
2021-07-20T04:17:51.7867719Z [cluster1]         "generation": 1,
2021-07-20T04:17:51.7868221Z [cluster1]         "managedFields": [
2021-07-20T04:17:51.7868704Z [cluster1]             {
2021-07-20T04:17:51.7869288Z [cluster1]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-20T04:17:51.7869984Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-20T04:17:51.7870669Z [cluster1]                 "fieldsV1": {
2021-07-20T04:17:51.7871121Z [cluster1]                     "f:metadata": {
2021-07-20T04:17:51.7871611Z [cluster1]                         "f:annotations": {
2021-07-20T04:17:51.7872059Z [cluster1]                             ".": {},
2021-07-20T04:17:51.7872998Z [cluster1]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-20T04:17:51.7873844Z [cluster1]                         }
2021-07-20T04:17:51.7874252Z [cluster1]                     },
2021-07-20T04:17:51.7875037Z [cluster1]                     "f:spec": {
2021-07-20T04:17:51.7875866Z [cluster1]                         "f:conversion": {
2021-07-20T04:17:51.7876725Z [cluster1]                             ".": {},
2021-07-20T04:17:51.7877370Z [cluster1]                             "f:strategy": {}
2021-07-20T04:17:51.7877822Z [cluster1]                         },
2021-07-20T04:17:51.7878413Z [cluster1]                         "f:group": {},
2021-07-20T04:17:51.7879230Z [cluster1]                         "f:names": {
2021-07-20T04:17:51.7879693Z [cluster1]                             "f:kind": {},
2021-07-20T04:17:51.7880385Z [cluster1]                             "f:listKind": {},
2021-07-20T04:17:51.7880899Z [cluster1]                             "f:plural": {},
2021-07-20T04:17:51.7881534Z [cluster1]                             "f:singular": {}
2021-07-20T04:17:51.7882161Z [cluster1]                         },
2021-07-20T04:17:51.7882946Z [cluster1]                         "f:scope": {},
2021-07-20T04:17:51.7883588Z [cluster1]                         "f:versions": {}
2021-07-20T04:17:51.7884022Z [cluster1]                     },
2021-07-20T04:17:51.7884452Z [cluster1]                     "f:status": {
2021-07-20T04:17:51.7885122Z [cluster1]                         "f:acceptedNames": {
2021-07-20T04:17:51.7885669Z [cluster1]                             "f:kind": {},
2021-07-20T04:17:51.7887588Z [cluster1]                             "f:plural": {}
2021-07-20T04:17:51.7888334Z [cluster1]                         },
2021-07-20T04:17:51.7888788Z [cluster1]                         "f:conditions": {},
2021-07-20T04:17:51.7889343Z [cluster1]                         "f:storedVersions": {}
2021-07-20T04:17:51.7890154Z [cluster1]                     }
2021-07-20T04:17:51.7890561Z [cluster1]                 },
2021-07-20T04:17:51.7891562Z [cluster1]                 "manager": "submariner-operator",
2021-07-20T04:17:51.7892379Z [cluster1]                 "operation": "Update",
2021-07-20T04:17:51.7893125Z [cluster1]                 "time": "2021-07-20T04:13:20Z"
2021-07-20T04:17:51.7893720Z [cluster1]             }
2021-07-20T04:17:51.7894107Z [cluster1]         ],
2021-07-20T04:17:51.7894670Z [cluster1]         "name": "endpoints.submariner.io",
2021-07-20T04:17:51.7895738Z [cluster1]         "resourceVersion": "1431",
2021-07-20T04:17:51.7910647Z [cluster1]         "uid": "baaa86f5-e7f9-4541-8313-7b5c36bcfb5e"
2021-07-20T04:17:51.7911347Z [cluster1]     },
2021-07-20T04:17:51.7911753Z [cluster1]     "spec": {
2021-07-20T04:17:51.7912227Z [cluster1]         "conversion": {
2021-07-20T04:17:51.7912742Z [cluster1]             "strategy": "None"
2021-07-20T04:17:51.7913185Z [cluster1]         },
2021-07-20T04:17:51.7913682Z [cluster1]         "group": "submariner.io",
2021-07-20T04:17:51.7914392Z [cluster1]         "names": {
2021-07-20T04:17:51.7914879Z [cluster1]             "kind": "Endpoint",
2021-07-20T04:17:51.7915434Z [cluster1]             "listKind": "EndpointList",
2021-07-20T04:17:51.7916027Z [cluster1]             "plural": "endpoints",
2021-07-20T04:17:51.7916577Z [cluster1]             "singular": "endpoint"
2021-07-20T04:17:51.7917156Z [cluster1]         },
2021-07-20T04:17:51.7917636Z [cluster1]         "scope": "Namespaced",
2021-07-20T04:17:51.7918232Z [cluster1]         "versions": [
2021-07-20T04:17:51.7918694Z [cluster1]             {
2021-07-20T04:17:51.7920226Z [cluster1]                 "name": "v1",
2021-07-20T04:17:51.7920722Z [cluster1]                 "schema": {
2021-07-20T04:17:51.7921491Z [cluster1]                     "openAPIV3Schema": {
2021-07-20T04:17:51.7922600Z [cluster1]                         "properties": {
2021-07-20T04:17:51.7923132Z [cluster1]                             "apiVersion": {
2021-07-20T04:17:51.7925752Z [cluster1]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-20T04:17:51.7928636Z [cluster1]                                 "type": "string"
2021-07-20T04:17:51.7929140Z [cluster1]                             },
2021-07-20T04:17:51.7929611Z [cluster1]                             "kind": {
2021-07-20T04:17:51.7931882Z [cluster1]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-20T04:17:51.7933889Z [cluster1]                                 "type": "string"
2021-07-20T04:17:51.7934388Z [cluster1]                             },
2021-07-20T04:17:51.7934854Z [cluster1]                             "metadata": {
2021-07-20T04:17:51.7935376Z [cluster1]                                 "type": "object"
2021-07-20T04:17:51.7935854Z [cluster1]                             },
2021-07-20T04:17:51.7936289Z [cluster1]                             "spec": {
2021-07-20T04:17:51.7936807Z [cluster1]                                 "properties": {
2021-07-20T04:17:51.7937687Z [cluster1]                                     "backend": {
2021-07-20T04:17:51.7938250Z [cluster1]                                         "type": "string"
2021-07-20T04:17:51.7938728Z [cluster1]                                     },
2021-07-20T04:17:51.7939236Z [cluster1]                                     "backend_config": {
2021-07-20T04:17:51.7939897Z [cluster1]                                         "additionalProperties": {
2021-07-20T04:17:51.7940794Z [cluster1]                                             "type": "string"
2021-07-20T04:17:51.7941333Z [cluster1]                                         },
2021-07-20T04:17:51.7942167Z [cluster1]                                         "type": "object"
2021-07-20T04:17:51.7942855Z [cluster1]                                     },
2021-07-20T04:17:51.7943332Z [cluster1]                                     "cable_name": {
2021-07-20T04:17:51.7943864Z [cluster1]                                         "type": "string"
2021-07-20T04:17:51.7944349Z [cluster1]                                     },
2021-07-20T04:17:51.7944837Z [cluster1]                                     "cluster_id": {
2021-07-20T04:17:51.7945550Z [cluster1]                                         "type": "string"
2021-07-20T04:17:51.7946044Z [cluster1]                                     },
2021-07-20T04:17:51.7946590Z [cluster1]                                     "healthCheckIP": {
2021-07-20T04:17:51.7947325Z [cluster1]                                         "type": "string"
2021-07-20T04:17:51.7947954Z [cluster1]                                     },
2021-07-20T04:17:51.7948421Z [cluster1]                                     "hostname": {
2021-07-20T04:17:51.7949280Z [cluster1]                                         "type": "string"
2021-07-20T04:17:51.7949758Z [cluster1]                                     },
2021-07-20T04:17:51.7950247Z [cluster1]                                     "nat_enabled": {
2021-07-20T04:17:51.7950798Z [cluster1]                                         "type": "boolean"
2021-07-20T04:17:51.7952034Z [cluster1]                                     },
2021-07-20T04:17:51.7952735Z [cluster1]                                     "private_ip": {
2021-07-20T04:17:51.7953251Z [cluster1]                                         "type": "string"
2021-07-20T04:17:51.7953725Z [cluster1]                                     },
2021-07-20T04:17:51.7954172Z [cluster1]                                     "public_ip": {
2021-07-20T04:17:51.7954695Z [cluster1]                                         "type": "string"
2021-07-20T04:17:51.7955602Z [cluster1]                                     },
2021-07-20T04:17:51.7956244Z [cluster1]                                     "subnets": {
2021-07-20T04:17:51.7956781Z [cluster1]                                         "items": {
2021-07-20T04:17:51.7957601Z [cluster1]                                             "type": "string"
2021-07-20T04:17:51.7958276Z [cluster1]                                         },
2021-07-20T04:17:51.7958745Z [cluster1]                                         "type": "array"
2021-07-20T04:17:51.7959625Z [cluster1]                                     }
2021-07-20T04:17:51.7960651Z [cluster1]                                 },
2021-07-20T04:17:51.7961132Z [cluster1]                                 "required": [
2021-07-20T04:17:51.7962130Z [cluster1]                                     "backend",
2021-07-20T04:17:51.7963356Z [cluster1]                                     "cable_name",
2021-07-20T04:17:51.7964222Z [cluster1]                                     "cluster_id",
2021-07-20T04:17:51.8560637Z [cluster1]                           [36m[submariner-operator]$ [cluster2] verify_subm_deployed[0m
2021-07-20T04:17:51.8562486Z [36m[submariner-operator]$ [cluster2] verify_endpoints_crd[0m
2021-07-20T04:17:51.8563879Z [36m[submariner-operator]$ [cluster2] verify_endpoints_crd[0m
2021-07-20T04:17:51.8565415Z [36m[submariner-operator]$ [cluster2] validate_crd endpoints.submariner.io v1 Endpoint[0m
2021-07-20T04:17:51.8566880Z [36m[submariner-operator]$ [cluster2] validate_crd endpoints.submariner.io v1 Endpoint[0m
2021-07-20T04:17:51.8568234Z [36m[submariner-operator]$ [cluster2] kubectl get crds endpoints.submariner.io[0m
2021-07-20T04:17:51.8569643Z [36m[submariner-operator]$ [cluster2] kubectl get crds endpoints.submariner.io[0m
2021-07-20T04:17:51.8571294Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crds endpoints.submariner.io[0m
2021-07-20T04:17:51.8572243Z [cluster2] NAME                      CREATED AT
2021-07-20T04:17:51.8573260Z [cluster2] endpoints.submariner.io   2021-07-20T04:13:23Z
2021-07-20T04:17:51.8574604Z [36m[submariner-operator]$ [cluster2] json_file=/tmp/endpoints.submariner.io.cluster2.json[0m
2021-07-20T04:17:51.8576251Z [36m[submariner-operator]$ [cluster2] kubectl get crd endpoints.submariner.io -o json[0m
2021-07-20T04:17:51.8584182Z [36m[submariner-operator]$ [cluster2] kubectl get crd endpoints.submariner.io -o json[0m
2021-07-20T04:17:51.8586490Z [36m[submariner-operator]$ [cluster2] tee /tmp/endpoints.submariner.io.cluster2.json[0m
2021-07-20T04:17:51.8588107Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crd endpoints.submariner.io -o json[0m
2021-07-20T04:17:51.8589093Z [cluster2] {
2021-07-20T04:17:51.8589643Z [cluster2]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-20T04:17:51.8590413Z [cluster2]     "kind": "CustomResourceDefinition",
2021-07-20T04:17:51.8591219Z [cluster2]     "metadata": {
2021-07-20T04:17:51.8591868Z [cluster2]         "annotations": {
2021-07-20T04:17:51.8592853Z [cluster2]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-20T04:17:51.8593864Z [cluster2]         },
2021-07-20T04:17:51.8594996Z [cluster2]         "creationTimestamp": "2021-07-20T04:13:23Z",
2021-07-20T04:17:51.8595853Z [cluster2]         "generation": 1,
2021-07-20T04:17:51.8596553Z [cluster2]         "managedFields": [
2021-07-20T04:17:51.8597041Z [cluster2]             {
2021-07-20T04:17:51.8597792Z [cluster2]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-20T04:17:51.8598485Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-20T04:17:51.8599018Z [cluster2]                 "fieldsV1": {
2021-07-20T04:17:51.8599484Z [cluster2]                     "f:metadata": {
2021-07-20T04:17:51.8600181Z [cluster2]                         "f:annotations": {
2021-07-20T04:17:51.8600679Z [cluster2]                             ".": {},
2021-07-20T04:17:51.8601706Z [cluster2]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-20T04:17:51.8602450Z [cluster2]                         }
2021-07-20T04:17:51.8602887Z [cluster2]                     },
2021-07-20T04:17:51.8603332Z [cluster2]                     "f:spec": {
2021-07-20T04:17:51.8604005Z [cluster2]                         "f:conversion": {
2021-07-20T04:17:51.8604683Z [cluster2]                             ".": {},
2021-07-20T04:17:51.8605168Z [cluster2]                             "f:strategy": {}
2021-07-20T04:17:51.8605651Z [cluster2]                         },
2021-07-20T04:17:51.8606108Z [cluster2]                         "f:group": {},
2021-07-20T04:17:51.8606605Z [cluster2]                         "f:names": {
2021-07-20T04:17:51.8607085Z [cluster2]                             "f:kind": {},
2021-07-20T04:17:51.8607598Z [cluster2]                             "f:listKind": {},
2021-07-20T04:17:51.8608110Z [cluster2]                             "f:plural": {},
2021-07-20T04:17:51.8608645Z [cluster2]                             "f:singular": {}
2021-07-20T04:17:51.8609129Z [cluster2]                         },
2021-07-20T04:17:51.8609885Z [cluster2]                         "f:scope": {},
2021-07-20T04:17:51.8610364Z [cluster2]                         "f:versions": {}
2021-07-20T04:17:51.8610792Z [cluster2]                     },
2021-07-20T04:17:51.8611221Z [cluster2]                     "f:status": {
2021-07-20T04:17:51.8611871Z [cluster2]                         "f:acceptedNames": {
2021-07-20T04:17:51.8612410Z [cluster2]                             "f:kind": {},
2021-07-20T04:17:51.8612872Z [cluster2]                             "f:plural": {}
2021-07-20T04:17:51.8614718Z [cluster2]                         },
2021-07-20T04:17:51.8615216Z [cluster2]                         "f:conditions": {},
2021-07-20T04:17:51.8615819Z [cluster2]                         "f:storedVersions": {}
2021-07-20T04:17:51.8616349Z [cluster2]                     }
2021-07-20T04:17:51.8616772Z [cluster2]                 },
2021-07-20T04:17:51.8654427Z [cluster2]                 "manager": "submariner-operator",
2021-07-20T04:17:51.8655087Z [cluster2]                 "operation": "Update",
2021-07-20T04:17:51.8655823Z [cluster2]                 "time": "2021-07-20T04:13:32Z"
2021-07-20T04:17:51.8656253Z [cluster2]             }
2021-07-20T04:17:51.8656645Z [cluster2]         ],
2021-07-20T04:17:51.8657409Z [cluster2]         "name": "endpoints.submariner.io",
2021-07-20T04:17:51.8658121Z [cluster2]         "resourceVersion": "1427",
2021-07-20T04:17:51.8659186Z [cluster2]         "uid": "73591756-cdb4-4108-8c9a-7d09e21e752b"
2021-07-20T04:17:51.8659724Z [cluster2]     },
2021-07-20T04:17:51.8660121Z [cluster2]     "spec": {
2021-07-20T04:17:51.8660722Z [cluster2]         "conversion": {
2021-07-20T04:17:51.8661197Z [cluster2]             "strategy": "None"
2021-07-20T04:17:51.8661607Z [cluster2]         },
2021-07-20T04:17:51.8662155Z [cluster2]         "group": "submariner.io",
2021-07-20T04:17:51.8662828Z [cluster2]         "names": {
2021-07-20T04:17:51.8663427Z [cluster2]             "kind": "Endpoint",
2021-07-20T04:17:51.8663956Z [cluster2]             "listKind": "EndpointList",
2021-07-20T04:17:51.8664513Z [cluster2]             "plural": "endpoints",
2021-07-20T04:17:51.8665027Z [cluster2]             "singular": "endpoint"
2021-07-20T04:17:51.8665550Z [cluster2]         },
2021-07-20T04:17:51.8665982Z [cluster2]         "scope": "Namespaced",
2021-07-20T04:17:51.8666475Z [cluster2]         "versions": [
2021-07-20T04:17:51.8666890Z [cluster2]             {
2021-07-20T04:17:51.8667366Z [cluster2]                 "name": "v1",
2021-07-20T04:17:51.8667808Z [cluster2]                 "schema": {
2021-07-20T04:17:51.8668309Z [cluster2]                     "openAPIV3Schema": {
2021-07-20T04:17:51.8669059Z [cluster2]                         "properties": {
2021-07-20T04:17:51.8669569Z [cluster2]                             "apiVersion": {
2021-07-20T04:17:51.8672436Z [cluster2]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-20T04:17:51.8674187Z [cluster2]                                 "type": "string"
2021-07-20T04:17:51.8674638Z [cluster2]                             },
2021-07-20T04:17:51.8675088Z [cluster2]                             "kind": {
2021-07-20T04:17:51.8677209Z [cluster2]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-20T04:17:51.8679955Z [cluster2]                                 "type": "string"
2021-07-20T04:17:51.8680504Z [cluster2]                             },
2021-07-20T04:17:51.8681769Z [cluster2]                             "metadata": {
2021-07-20T04:17:51.8682331Z [cluster2]                                 "type": "object"
2021-07-20T04:17:51.8683032Z [cluster2]                             },
2021-07-20T04:17:51.8683508Z [cluster2]                             "spec": {
2021-07-20T04:17:51.8684353Z [cluster2]                                 "properties": {
2021-07-20T04:17:51.8728252Z [cluster2]                                     "backend": {
2021-07-20T04:17:51.8728972Z [cluster2]                                         "type": "string"
2021-07-20T04:17:51.8729665Z [cluster2]                                     },
2021-07-20T04:17:51.8730200Z [cluster2]                                     "backend_config": {
2021-07-20T04:17:51.8730875Z [cluster2]                                         "additionalProperties": {
2021-07-20T04:17:51.8731562Z [cluster2]                                             "type": "string"
2021-07-20T04:17:51.8732109Z [cluster2]                                         },
2021-07-20T04:17:51.8732760Z [cluster2]                                         "type": "object"
2021-07-20T04:17:51.8733275Z [cluster2]                                     },
2021-07-20T04:17:51.8733761Z [cluster2]                                     "cable_name": {
2021-07-20T04:17:51.8734312Z [cluster2]                                         "type": "string"
2021-07-20T04:17:51.8734815Z [cluster2]                                     },
2021-07-20T04:17:51.8735317Z [cluster2]                                     "cluster_id": {
2021-07-20T04:17:51.8736159Z [cluster2]                                         "type": "string"
2021-07-20T04:17:51.8737257Z [cluster2]                                     },
2021-07-20T04:17:51.8737988Z [cluster2]                                     "healthCheckIP": {
2021-07-20T04:17:51.8738574Z [cluster2]                                         "type": "string"
2021-07-20T04:17:51.8739238Z [cluster2]                                     },
2021-07-20T04:17:51.8739883Z [cluster2]                                     "hostname": {
2021-07-20T04:17:51.8740445Z [cluster2]                                         "type": "string"
2021-07-20T04:17:51.8740938Z [cluster2]                                     },
2021-07-20T04:17:51.8741589Z [cluster2]                                     "nat_enabled": {
2021-07-20T04:17:51.8742144Z [cluster2]                                         "type": "boolean"
2021-07-20T04:17:51.8742627Z [cluster2]                                     },
2021-07-20T04:17:51.8743123Z [cluster2]                                     "private_ip": {
2021-07-20T04:17:51.8743650Z [cluster2]                                         "type": "string"
2021-07-20T04:17:51.8744142Z [cluster2]                                     },
2021-07-20T04:17:51.8744607Z [cluster2]                                     "public_ip": {
2021-07-20T04:17:51.8745135Z [cluster2]                                         "type": "string"
2021-07-20T04:17:51.8745612Z [cluster2]                                     },
2021-07-20T04:17:51.8746095Z [cluster2]                                     "subnets": {
2021-07-20T04:17:51.8746599Z [cluster2]                                         "items": {
2021-07-20T04:17:51.8747107Z [cluster2]                                             "type": "string"
2021-07-20T04:17:51.8747606Z [cluster2]                                         },
2021-07-20T04:17:51.8748086Z [cluster2]                                         "type": "array"
2021-07-20T04:17:51.8748579Z [cluster2]                                     }
2021-07-20T04:17:51.8749180Z [cluster2]                                 },
2021-07-20T04:17:51.8749680Z [cluster2]                                 "required": [
2021-07-20T04:17:51.8750532Z [cluster2]                                     "backend",
2021-07-20T04:17:51.8751067Z [cluster2]                                     "cable_name",
2021-07-20T04:17:51.8751612Z [cluster2]                                     "cluster_id",
2021-07-20T04:17:52.5643112Z [cluster2]                                     "hostname",
2021-07-20T04:17:52.5644176Z [cluster1]                                     "nat_enabled",
2021-07-20T04:17:52.5644885Z [cluster1]                                     "private_ip",
2021-07-20T04:17:52.5645531Z [cluster1]                                     "public_ip",
2021-07-20T04:17:52.5646202Z [cluster1]                                     "subnets"
2021-07-20T04:17:52.5647021Z [cluster1]                                 ],
2021-07-20T04:17:52.5647857Z [cluster1]                                 "type": "object"
2021-07-20T04:17:52.5648442Z [cluster1]                             }
2021-07-20T04:17:52.5648956Z [cluster1]                         },
2021-07-20T04:17:52.5649525Z [cluster1]                         "required": [
2021-07-20T04:17:52.5650103Z [cluster1]                             "spec"
2021-07-20T04:17:52.5650642Z [cluster1]                         ],
2021-07-20T04:17:52.5651241Z [cluster1]                         "type": "object"
2021-07-20T04:17:52.5651793Z [cluster1]                     }
2021-07-20T04:17:52.5652308Z [cluster1]                 },
2021-07-20T04:17:52.5652860Z [cluster1]                 "served": true,
2021-07-20T04:17:52.5653473Z [cluster1]                 "storage": true
2021-07-20T04:17:52.5654007Z [cluster1]             }
2021-07-20T04:17:52.5654503Z [cluster1]         ]
2021-07-20T04:17:52.5654992Z [cluster1]     },
2021-07-20T04:17:52.5655677Z [cluster1]     "status": {
2021-07-20T04:17:52.5656300Z [cluster1]         "acceptedNames": {
2021-07-20T04:17:52.5657274Z [cluster1]             "kind": "Endpoint",
2021-07-20T04:17:52.5657999Z [cluster1]             "listKind": "EndpointList",
2021-07-20T04:17:52.5658698Z [cluster1]             "plural": "endpoints",
2021-07-20T04:17:52.5659348Z [cluster1]             "singular": "endpoint"
2021-07-20T04:17:52.5659932Z [cluster1]         },
2021-07-20T04:17:52.5660466Z [cluster1]         "conditions": [
2021-07-20T04:17:52.5661365Z [cluster1]             {
2021-07-20T04:17:52.5662823Z [cluster1]                 "lastTransitionTime": "2021-07-20T04:13:13Z",
2021-07-20T04:17:52.5663677Z [cluster1]                 "message": "no conflicts found",
2021-07-20T04:17:52.5664379Z [cluster1]                 "reason": "NoConflicts",
2021-07-20T04:17:52.5665009Z [cluster1]                 "status": "True",
2021-07-20T04:17:52.5665777Z [cluster1]                 "type": "NamesAccepted"
2021-07-20T04:17:52.5667705Z [cluster1]             },
2021-07-20T04:17:52.5668118Z [cluster1]             {
2021-07-20T04:17:52.5669167Z [cluster1]                 "lastTransitionTime": "2021-07-20T04:13:13Z",
2021-07-20T04:17:52.5669909Z [cluster1]                 "message": "the initial names have been accepted",
2021-07-20T04:17:52.5670687Z [cluster1]                 "reason": "InitialNamesAccepted",
2021-07-20T04:17:52.5671358Z [cluster1]                 "status": "True",
2021-07-20T04:17:52.5671898Z [cluster1]                 "type": "Established"
2021-07-20T04:17:52.5672378Z [cluster1]             }
2021-07-20T04:17:52.5672919Z [cluster1]         ],
2021-07-20T04:17:52.5673419Z [cluster1]         "storedVersions": [
2021-07-20T04:17:52.5673902Z [cluster1]             "v1"
2021-07-20T04:17:52.5674306Z [cluster1]         ]
2021-07-20T04:17:52.5674686Z [cluster1]     }
2021-07-20T04:17:52.5675075Z [cluster1] }
2021-07-20T04:17:52.5676187Z [36m[submariner-operator]$ [cluster1] validate_equals .metadata.name endpoints.submariner.io[0m
2021-07-20T04:17:52.5677917Z [36m[submariner-operator]$ [cluster1] validate_equals .metadata.name endpoints.submariner.io[0m
2021-07-20T04:17:52.5679704Z [36m[submariner-operator]$ [cluster1] jq -r -M .metadata.name /tmp/endpoints.submariner.io.cluster1.json[0m
2021-07-20T04:17:52.5681484Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced[0m
2021-07-20T04:17:52.5684006Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced[0m
2021-07-20T04:17:52.5685856Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.scope /tmp/endpoints.submariner.io.cluster1.json[0m
2021-07-20T04:17:52.5687416Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io[0m
2021-07-20T04:17:52.5688667Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io[0m
2021-07-20T04:17:52.5690466Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.group /tmp/endpoints.submariner.io.cluster1.json[0m
2021-07-20T04:17:52.5692516Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.names.kind Endpoint[0m
2021-07-20T04:17:52.5694005Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.names.kind Endpoint[0m
2021-07-20T04:17:52.5695688Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.names.kind /tmp/endpoints.submariner.io.cluster1.json[0m
2021-07-20T04:17:52.5697903Z [36m[submariner-operator]$ [cluster1] validate_equals .status.acceptedNames.kind Endpoint[0m
2021-07-20T04:17:52.5699491Z [36m[submariner-operator]$ [cluster1] validate_equals .status.acceptedNames.kind Endpoint[0m
2021-07-20T04:17:52.5701416Z [36m[submariner-operator]$ [cluster1] jq -r -M .status.acceptedNames.kind /tmp/endpoints.submariner.io.cluster1.json[0m
2021-07-20T04:17:52.5703057Z [36m[submariner-operator]$ [cluster1] verify_clusters_crd[0m
2021-07-20T04:17:52.5704117Z [36m[submariner-operator]$ [cluster1] verify_clusters_crd[0m
2021-07-20T04:17:52.5705367Z [36m[submariner-operator]$ [cluster1] validate_crd clusters.submariner.io v1 Cluster[0m
2021-07-20T04:17:52.5706802Z [36m[submariner-operator]$ [cluster1] validate_crd clusters.submariner.io v1 Cluster[0m
2021-07-20T04:17:52.5708367Z [36m[submariner-operator]$ [cluster1] kubectl get crds clusters.submariner.io[0m
2021-07-20T04:17:52.5710071Z [36m[submariner-operator]$ [cluster1] kubectl get crds clusters.submariner.io[0m
2021-07-20T04:17:52.5711593Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crds clusters.submariner.io[0m
2021-07-20T04:17:52.5712719Z [cluster1] NAME                     CREATED AT
2021-07-20T04:17:52.8908718Z [cluster1] clusters.submariner.io   20          "hostname",
2021-07-20T04:17:52.8909448Z [cluster2]                                     "nat_enabled",
2021-07-20T04:17:52.8909975Z [cluster2]                                     "private_ip",
2021-07-20T04:17:52.8910506Z [cluster2]                                     "public_ip",
2021-07-20T04:17:52.8911006Z [cluster2]                                     "subnets"
2021-07-20T04:17:52.8911513Z [cluster2]                                 ],
2021-07-20T04:17:52.8911988Z [cluster2]                                 "type": "object"
2021-07-20T04:17:52.8912467Z [cluster2]                             }
2021-07-20T04:17:52.8912878Z [cluster2]                         },
2021-07-20T04:17:52.8913343Z [cluster2]                         "required": [
2021-07-20T04:17:52.8913839Z [cluster2]                             "spec"
2021-07-20T04:17:52.8915611Z [cluster2]                         ],
2021-07-20T04:17:52.8916135Z [cluster2]                         "type": "object"
2021-07-20T04:17:52.8916603Z [cluster2]                     }
2021-07-20T04:17:52.8917214Z [cluster2]                 },
2021-07-20T04:17:52.8917677Z [cluster2]                 "served": true,
2021-07-20T04:17:52.8918350Z [cluster2]                 "storage": true
2021-07-20T04:17:52.8918787Z [cluster2]             }
2021-07-20T04:17:52.8919301Z [cluster2]         ]
2021-07-20T04:17:52.8919682Z [cluster2]     },
2021-07-20T04:17:52.8920105Z [cluster2]     "status": {
2021-07-20T04:17:52.8920593Z [cluster2]         "acceptedNames": {
2021-07-20T04:17:52.8921142Z [cluster2]             "kind": "Endpoint",
2021-07-20T04:17:52.8921716Z [cluster2]             "listKind": "EndpointList",
2021-07-20T04:17:52.8922541Z [cluster2]             "plural": "endpoints",
2021-07-20T04:17:52.8923132Z [cluster2]             "singular": "endpoint"
2021-07-20T04:17:52.8923762Z [cluster2]         },
2021-07-20T04:17:52.8924213Z [cluster2]         "conditions": [
2021-07-20T04:17:52.8924755Z [cluster2]             {
2021-07-20T04:17:52.8926003Z [cluster2]                 "lastTransitionTime": "2021-07-20T04:13:24Z",
2021-07-20T04:17:52.8926671Z [cluster2]                 "message": "no conflicts found",
2021-07-20T04:17:52.8927276Z [cluster2]                 "reason": "NoConflicts",
2021-07-20T04:17:52.8927801Z [cluster2]                 "status": "True",
2021-07-20T04:17:52.8928366Z [cluster2]                 "type": "NamesAccepted"
2021-07-20T04:17:52.8929100Z [cluster2]             },
2021-07-20T04:17:52.8929516Z [cluster2]             {
2021-07-20T04:17:52.8930348Z [cluster2]                 "lastTransitionTime": "2021-07-20T04:13:24Z",
2021-07-20T04:17:52.8931080Z [cluster2]                 "message": "the initial names have been accepted",
2021-07-20T04:17:52.8931855Z [cluster2]                 "reason": "InitialNamesAccepted",
2021-07-20T04:17:52.8932476Z [cluster2]                 "status": "True",
2021-07-20T04:17:52.8933016Z [cluster2]                 "type": "Established"
2021-07-20T04:17:52.8933496Z [cluster2]             }
2021-07-20T04:17:52.8934076Z [cluster2]         ],
2021-07-20T04:17:52.8934564Z [cluster2]         "storedVersions": [
2021-07-20T04:17:52.8935073Z [cluster2]             "v1"
2021-07-20T04:17:52.8935488Z [cluster2]         ]
2021-07-20T04:17:52.8935879Z [cluster2]     }
2021-07-20T04:17:52.8936280Z [cluster2] }
2021-07-20T04:17:52.8937718Z [36m[submariner-operator]$ [cluster2] validate_equals .metadata.name endpoints.submariner.io[0m
2021-07-20T04:17:52.8939488Z [36m[submariner-operator]$ [cluster2] validate_equals .metadata.name endpoints.submariner.io[0m
2021-07-20T04:17:52.8941396Z [36m[submariner-operator]$ [cluster2] jq -r -M .metadata.name /tmp/endpoints.submariner.io.cluster2.json[0m
2021-07-20T04:17:52.8943062Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced[0m
2021-07-20T04:17:52.8944349Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced[0m
2021-07-20T04:17:52.8946169Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.scope /tmp/endpoints.submariner.io.cluster2.json[0m
2021-07-20T04:17:52.8947793Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io[0m
2021-07-20T04:17:52.8949238Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io[0m
2021-07-20T04:17:52.8950814Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.group /tmp/endpoints.submariner.io.cluster2.json[0m
2021-07-20T04:17:52.8952362Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.names.kind Endpoint[0m
2021-07-20T04:17:52.8953630Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.names.kind Endpoint[0m
2021-07-20T04:17:52.8955252Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.names.kind /tmp/endpoints.submariner.io.cluster2.json[0m
2021-07-20T04:17:52.8957172Z [36m[submariner-operator]$ [cluster2] validate_equals .status.acceptedNames.kind Endpoint[0m
2021-07-20T04:17:52.8958732Z [36m[submariner-operator]$ [cluster2] validate_equals .status.acceptedNames.kind Endpoint[0m
2021-07-20T04:17:52.8960919Z [36m[submariner-operator]$ [cluster2] jq -r -M .status.acceptedNames.kind /tmp/endpoints.submariner.io.cluster2.json[0m
2021-07-20T04:17:52.8962644Z [36m[submariner-operator]$ [cluster2] verify_clusters_crd[0m
2021-07-20T04:17:52.8963669Z [36m[submariner-operator]$ [cluster2] verify_clusters_crd[0m
2021-07-20T04:17:52.8964874Z [36m[submariner-operator]$ [cluster2] validate_crd clusters.submariner.io v1 Cluster[0m
2021-07-20T04:17:52.8966309Z [36m[submariner-operator]$ [cluster2] validate_crd clusters.submariner.io v1 Cluster[0m
2021-07-20T04:17:52.8967847Z [36m[submariner-operator]$ [cluster2] kubectl get crds clusters.submariner.io[0m
2021-07-20T04:17:52.8969222Z [36m[submariner-operator]$ [cluster2] kubectl get crds clusters.submariner.io[0m
2021-07-20T04:17:52.8970827Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crds clusters.submariner.io[0m
2021-07-20T04:17:52.8971766Z [cluster2] NAME                     CREATED AT
2021-07-20T04:17:53.1087669Z [cluster2] clusters.submariner.io   2021-07-20T04:13:13Z
2021-07-20T04:17:53.1089104Z [36m[submariner-operator]$ [cluster1] json_file=/tmp/clusters.submariner.io.cluster1.json[0m
2021-07-20T04:17:53.1090658Z [36m[submariner-operator]$ [cluster1] kubectl get crd clusters.submariner.io -o json[0m
2021-07-20T04:17:53.1092401Z [36m[submariner-operator]$ [cluster1] tee /tmp/clusters.submariner.io.cluster1.json[0m
2021-07-20T04:17:53.1094243Z [36m[submariner-operator]$ [cluster1] kubectl get crd clusters.submariner.io -o json[0m
2021-07-20T04:17:53.1096001Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crd clusters.submariner.io -o json[0m
2021-07-20T04:17:53.1097282Z [cluster1] {
2021-07-20T04:17:53.1097891Z [cluster1]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-20T04:17:53.1098699Z [cluster1]     "kind": "CustomResourceDefinition",
2021-07-20T04:17:53.1099439Z [cluster1]     "metadata": {
2021-07-20T04:17:53.1100162Z [cluster1]         "annotations": {
2021-07-20T04:17:53.1101141Z [cluster1]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-20T04:17:53.1101830Z [cluster1]         },
2021-07-20T04:17:53.1102716Z [cluster1]         "creationTimestamp": "2021-07-20T04:13:13Z",
2021-07-20T04:17:53.1103494Z [cluster1]         "generation": 1,
2021-07-20T04:17:53.1104031Z [cluster1]         "managedFields": [
2021-07-20T04:17:53.1104635Z [cluster1]             {
2021-07-20T04:17:53.1105256Z [cluster1]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-20T04:17:53.1105990Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-20T04:17:53.1106543Z [cluster1]                 "fieldsV1": {
2021-07-20T04:17:53.1107214Z [cluster1]                     "f:metadata": {
2021-07-20T04:17:53.1107882Z [cluster1]                         "f:annotations": {
2021-07-20T04:17:53.1108340Z [cluster1]                             ".": {},
2021-07-20T04:17:53.1109463Z [cluster1]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-20T04:17:53.1110320Z [cluster1]                         }
2021-07-20T04:17:53.1110895Z [cluster1]                     },
2021-07-20T04:17:53.1111325Z [cluster1]                     "f:spec": {
2021-07-20T04:17:53.1111826Z [cluster1]                         "f:conversion": {
2021-07-20T04:17:53.1112302Z [cluster1]                             ".": {},
2021-07-20T04:17:53.1112791Z [cluster1]                             "f:strategy": {}
2021-07-20T04:17:53.1113260Z [cluster1]                         },
2021-07-20T04:17:53.1113701Z [cluster1]                         "f:group": {},
2021-07-20T04:17:53.1114181Z [cluster1]                         "f:names": {
2021-07-20T04:17:53.1114643Z [cluster1]                             "f:kind": {},
2021-07-20T04:17:53.1115322Z [cluster1]                             "f:listKind": {},
2021-07-20T04:17:53.1115993Z [cluster1]                             "f:plural": {},
2021-07-20T04:17:53.1116507Z [cluster1]                             "f:singular": {}
2021-07-20T04:17:53.1116957Z [cluster1]                         },
2021-07-20T04:17:53.1130212Z [cluster1]                         "f:scope": {},
2021-07-20T04:17:53.1130885Z [cluster1]                         "f:versions": {}
2021-07-20T04:17:53.1131374Z [cluster1]                     },
2021-07-20T04:17:53.1131937Z [cluster1]                     "f:status": {
2021-07-20T04:17:53.1132498Z [cluster1]                         "f:acceptedNames": {
2021-07-20T04:17:53.1133059Z [cluster1]                             "f:kind": {},
2021-07-20T04:17:53.1133554Z [cluster1]                             "f:plural": {}
2021-07-20T04:17:53.1134033Z [cluster1]                         },
2021-07-20T04:17:53.1134910Z [cluster1]                         "f:conditions": {},
2021-07-20T04:17:53.1136039Z [cluster1]                         "f:storedVersions": {}
2021-07-20T04:17:53.1136558Z [cluster1]                     }
2021-07-20T04:17:53.1136999Z [cluster1]                 },
2021-07-20T04:17:53.1138683Z [cluster1]                 "manager": "submariner-operator",
2021-07-20T04:17:53.1139397Z [cluster1]                 "operation": "Update",
2021-07-20T04:17:53.1140169Z [cluster1]                 "time": "2021-07-20T04:13:20Z"
2021-07-20T04:17:53.1141833Z [cluster1]             }
2021-07-20T04:17:53.1143188Z [cluster1]         ],
2021-07-20T04:17:53.1143767Z [cluster1]         "name": "clusters.submariner.io",
2021-07-20T04:17:53.1145262Z [cluster1]         "resourceVersion": "1430",
2021-07-20T04:17:53.1147285Z [cluster1]         "uid": "a7fe06b3-d1c1-4afc-95ed-bac15d8951fe"
2021-07-20T04:17:53.1148383Z [cluster1]     },
2021-07-20T04:17:53.1148792Z [cluster1]     "spec": {
2021-07-20T04:17:53.1149258Z [cluster1]         "conversion": {
2021-07-20T04:17:53.1149751Z [cluster1]             "strategy": "None"
2021-07-20T04:17:53.1150204Z [cluster1]         },
2021-07-20T04:17:53.1150704Z [cluster1]         "group": "submariner.io",
2021-07-20T04:17:53.1151218Z [cluster1]         "names": {
2021-07-20T04:17:53.1151692Z [cluster1]             "kind": "Cluster",
2021-07-20T04:17:53.1152230Z [cluster1]             "listKind": "ClusterList",
2021-07-20T04:17:53.1153195Z [cluster1]             "plural": "clusters",
2021-07-20T04:17:53.1153769Z [cluster1]             "singular": "cluster"
2021-07-20T04:17:53.1154244Z [cluster1]         },
2021-07-20T04:17:53.1154710Z [cluster1]         "scope": "Namespaced",
2021-07-20T04:17:53.1155230Z [cluster1]         "versions": [
2021-07-20T04:17:53.1156077Z [cluster1]             {
2021-07-20T04:17:53.1156520Z [cluster1]                 "name": "v1",
2021-07-20T04:17:53.1156968Z [cluster1]                 "schema": {
2021-07-20T04:17:53.1157522Z [cluster1]                     "openAPIV3Schema": {
2021-07-20T04:17:53.1159161Z [cluster1]                         "properties": {
2021-07-20T04:17:53.1159800Z [cluster1]                             "apiVersion": {
2021-07-20T04:17:53.1162476Z [cluster1]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-20T04:17:53.1164924Z [cluster1]                                 "type": "string"
2021-07-20T04:17:53.1165380Z [cluster1]                             },
2021-07-20T04:17:53.1165826Z [cluster1]                             "kind": {
2021-07-20T04:17:53.1168142Z [cluster1]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-20T04:17:53.1169820Z [cluster1]                                 "type": "string"
2021-07-20T04:17:53.1170297Z [cluster1]                             },
2021-07-20T04:17:53.1170764Z [cluster1]                             "metadata": {
2021-07-20T04:17:53.1171255Z [cluster1]                                 "type": "object"
2021-07-20T04:17:53.1171723Z [cluster1]                             },
2021-07-20T04:17:53.1172150Z [cluster1]                             "spec": {
2021-07-20T04:17:53.1172980Z [cluster1]                                 "properties": {
2021-07-20T04:17:53.1173497Z [cluster1]                                     "cluster_cidr": {
2021-07-20T04:17:53.1173998Z [cluster1]                                         "items": {
2021-07-20T04:17:53.1174608Z [cluster1]                                             "type": "string"
2021-07-20T04:17:53.1175078Z [cluster1]                                         },
2021-07-20T04:17:53.1175625Z [cluster1]                                         "type": "array"
2021-07-20T04:17:53.1176080Z [cluster1]                                     },
2021-07-20T04:17:53.1176549Z [cluster1]                                     "cluster_id": {
2021-07-20T04:17:53.1177208Z [cluster1]                                         "type": "string"
2021-07-20T04:17:53.1177706Z [cluster1]                                     },
2021-07-20T04:17:53.1178160Z [cluster1]                                     "color_codes": {
2021-07-20T04:17:53.1178657Z [cluster1]                                         "items": {
2021-07-20T04:17:53.1179296Z [cluster1]                                             "type": "string"
2021-07-20T04:17:53.1179786Z [cluster1]                                         },
2021-07-20T04:17:53.1180626Z [cluster1]                                         "type": "array"
2021-07-20T04:17:53.1181100Z [cluster1]                                     },
2021-07-20T04:17:53.1181737Z [cluster1]                                     "global_cidr": {
2021-07-20T04:17:53.1182212Z [cluster1]                                         "items": {
2021-07-20T04:17:53.1183130Z [cluster1]                                             "type": "string"
2021-07-20T04:17:53.1183811Z [cluster1]                                         },
2021-07-20T04:17:53.1184305Z [cluster1]                                         "type": "array"
2021-07-20T04:17:53.1184792Z [cluster1]                                     },
2021-07-20T04:17:53.1185456Z [cluster1]                                     "service_cidr": {
2021-07-20T04:17:53.1185990Z [cluster1]                                         "items": {
2021-07-20T04:17:53.1186850Z [cluster1]                                             "type": "string"
2021-07-20T04:17:53.1187484Z [cluster1]                                         },
2021-07-20T04:17:53.1187927Z [cluster1]                                         "type": "array"
2021-07-20T04:17:53.1188377Z [cluster1]                                     }
2021-07-20T04:17:53.1188772Z [cluster1]                                 },
2021-07-20T04:17:53.1189222Z [cluster1]                                 "required": [
2021-07-20T04:17:53.1190014Z [cluster1]                                     "cluster_cidr",
2021-07-20T04:17:53.1190506Z [cluster1]                                     "cluster_id",
2021-07-20T04:17:53.1191011Z [cluster1]                                     "color_codes",
2021-07-20T04:17:53.1191498Z [cluster1]                                     "global_cidr",
2021-07-20T04:17:53.1192017Z [cluster1]                                     "service_cidr"
2021-07-20T04:17:53.1192476Z [cluster1]                                 ],
2021-07-20T04:17:53.1192951Z [cluster1]                                 "type": "object"
2021-07-20T04:17:53.1193395Z [cluster1]                             }
2021-07-20T04:17:53.1193805Z [cluster1]                         },
2021-07-20T04:17:53.1194258Z [cluster1]                         "required": [
2021-07-20T04:17:53.1194708Z [cluster1]                             "spec"
2021-07-20T04:17:53.1195135Z [cluster1]                         ],
2021-07-20T04:17:53.1195577Z [cluster1]                         "type": "object"
2021-07-20T04:17:53.1196033Z [cluster1]                     }
2021-07-20T04:17:53.1196419Z [cluster1]                 },
2021-07-20T04:17:53.1196860Z [cluster1]                 "served": true,
2021-07-20T04:17:53.1197334Z [cluster1]                 "storage": true
2021-07-20T04:17:53.1197769Z [cluster1]             }
2021-07-20T04:17:53.1198139Z [cluster1]         ]
2021-07-20T04:17:53.1198516Z [cluster1]     },
2021-07-20T04:17:53.1198904Z [cluster1]     "status": {
2021-07-20T04:17:53.1199395Z [cluster1]         "acceptedNames": {
2021-07-20T04:17:53.1199968Z [cluster1]             "kind": "Cluster",
2021-07-20T04:17:53.1200490Z [cluster1]             "listKind": "ClusterList",
2021-07-20T04:17:53.1201045Z [cluster1]             "plural": "clusters",
2021-07-20T04:17:53.1201561Z [cluster1]             "singular": "cluster"
2021-07-20T04:17:53.1219096Z [cluster1]         },
2021-07-20T04:17:53.1219542Z [cluster1]         "conditions": [
2021-07-20T04:17:53.1219996Z [cluster1]             {
2021-07-20T04:17:53.1221036Z [cluster1]                 "lastTransitionTime": "2021-07-20T04:13:13Z",
2021-07-20T04:17:53.1221682Z [cluster1]                 "message": "no conflicts found",
2021-07-20T04:17:53.1222230Z [cluster1]                 "reason": "NoConflicts",
2021-07-20T04:17:53.1222746Z [cluster1]                 "status": "True",
2021-07-20T04:17:53.1223269Z [cluster1]                 "type": "NamesAccepted"
2021-07-20T04:17:53.3024366Z [cluste21-07-20T04:13:23Z
2021-07-20T04:17:53.3025965Z [36m[submariner-operator]$ [cluster2] json_file=/tmp/clusters.submariner.io.cluster2.json[0m
2021-07-20T04:17:53.3027903Z [36m[submariner-operator]$ [cluster2] kubectl get crd clusters.submariner.io -o json[0m
2021-07-20T04:17:53.3029112Z [36m[submariner-operator]$ [cluster2] kubectl get crd clusters.submariner.io -o json[0m
2021-07-20T04:17:53.3030429Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crd clusters.submariner.io -o json[0m
2021-07-20T04:17:53.3031866Z [36m[submariner-operator]$ [cluster2] tee /tmp/clusters.submariner.io.cluster2.json[0m
2021-07-20T04:17:53.3032683Z [cluster2] {
2021-07-20T04:17:53.3033198Z [cluster2]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-20T04:17:53.3033918Z [cluster2]     "kind": "CustomResourceDefinition",
2021-07-20T04:17:53.3034477Z [cluster2]     "metadata": {
2021-07-20T04:17:53.3034906Z [cluster2]         "annotations": {
2021-07-20T04:17:53.3035772Z [cluster2]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-20T04:17:53.3036396Z [cluster2]         },
2021-07-20T04:17:53.3037062Z [cluster2]         "creationTimestamp": "2021-07-20T04:13:23Z",
2021-07-20T04:17:53.3037578Z [cluster2]         "generation": 1,
2021-07-20T04:17:53.3038047Z [cluster2]         "managedFields": [
2021-07-20T04:17:53.3038451Z [cluster2]             {
2021-07-20T04:17:53.3038992Z [cluster2]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-20T04:17:53.3039603Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-20T04:17:53.3040691Z [cluster2]                 "fieldsV1": {
2021-07-20T04:17:53.3041127Z [cluster2]                     "f:metadata": {
2021-07-20T04:17:53.3041605Z [cluster2]                         "f:annotations": {
2021-07-20T04:17:53.3042402Z [cluster2]                             ".": {},
2021-07-20T04:17:53.3043378Z [cluster2]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-20T04:17:53.3044539Z [cluster2]                         }
2021-07-20T04:17:53.3044924Z [cluster2]                     },
2021-07-20T04:17:53.3045494Z [cluster2]                     "f:spec": {
2021-07-20T04:17:53.3046110Z [cluster2]                         "f:conversion": {
2021-07-20T04:17:53.3046538Z [cluster2]                             ".": {},
2021-07-20T04:17:53.3046947Z [cluster2]                             "f:strategy": {}
2021-07-20T04:17:53.3047357Z [cluster2]                         },
2021-07-20T04:17:53.3047740Z [cluster2]                         "f:group": {},
2021-07-20T04:17:53.3048343Z [cluster2]                         "f:names": {
2021-07-20T04:17:53.3049571Z [cluster2]                             "f:kind": {},
2021-07-20T04:17:53.3050085Z [cluster2]                             "f:listKind": {},
2021-07-20T04:17:53.3050615Z [cluster2]                             "f:plural": {},
2021-07-20T04:17:53.3051129Z [cluster2]                             "f:singular": {}
2021-07-20T04:17:53.3051612Z [cluster2]                         },
2021-07-20T04:17:53.3052234Z [cluster2]                         "f:scope": {},
2021-07-20T04:17:53.3052676Z [cluster2]                         "f:versions": {}
2021-07-20T04:17:53.3053070Z [cluster2]                     },
2021-07-20T04:17:53.3053465Z [cluster2]                     "f:status": {
2021-07-20T04:17:53.3053923Z [cluster2]                         "f:acceptedNames": {
2021-07-20T04:17:53.3054432Z [cluster2]                             "f:kind": {},
2021-07-20T04:17:53.3054865Z [cluster2]                             "f:plural": {}
2021-07-20T04:17:53.3055256Z [cluster2]                         },
2021-07-20T04:17:53.3055865Z [cluster2]                         "f:conditions": {},
2021-07-20T04:17:53.3056367Z [cluster2]                         "f:storedVersions": {}
2021-07-20T04:17:53.3056829Z [cluster2]                     }
2021-07-20T04:17:53.3057312Z [cluster2]                 },
2021-07-20T04:17:53.3058130Z [cluster2]                 "manager": "submariner-operator",
2021-07-20T04:17:53.3058856Z [cluster2]                 "operation": "Update",
2021-07-20T04:17:53.3059577Z [cluster2]                 "time": "2021-07-20T04:13:32Z"
2021-07-20T04:17:53.3060221Z [cluster2]             }
2021-07-20T04:17:53.3060584Z [cluster2]         ],
2021-07-20T04:17:53.3061327Z [cluster2]         "name": "clusters.submariner.io",
2021-07-20T04:17:53.3061969Z [cluster2]         "resourceVersion": "1426",
2021-07-20T04:17:53.3063004Z [cluster2]         "uid": "eb513d00-8926-45df-af26-0fd8a0144a00"
2021-07-20T04:17:53.3063683Z [cluster2]     },
2021-07-20T04:17:53.3064069Z [cluster2]     "spec": {
2021-07-20T04:17:53.3064678Z [cluster2]         "conversion": {
2021-07-20T04:17:53.3065177Z [cluster2]             "strategy": "None"
2021-07-20T04:17:53.3065602Z [cluster2]         },
2021-07-20T04:17:53.3066084Z [cluster2]         "group": "submariner.io",
2021-07-20T04:17:53.3066569Z [cluster2]         "names": {
2021-07-20T04:17:53.3067033Z [cluster2]             "kind": "Cluster",
2021-07-20T04:17:53.3067891Z [cluster2]             "listKind": "ClusterList",
2021-07-20T04:17:53.3068396Z [cluster2]             "plural": "clusters",
2021-07-20T04:17:53.3069072Z [cluster2]             "singular": "cluster"
2021-07-20T04:17:53.3069657Z [cluster2]         },
2021-07-20T04:17:53.3070279Z [cluster2]         "scope": "Namespaced",
2021-07-20T04:17:53.3070933Z [cluster2]         "versions": [
2021-07-20T04:17:53.3071536Z [cluster2]             {
2021-07-20T04:17:53.3071954Z [cluster2]                 "name": "v1",
2021-07-20T04:17:53.3072649Z [cluster2]                 "schema": {
2021-07-20T04:17:53.3073287Z [cluster2]                     "openAPIV3Schema": {
2021-07-20T04:17:53.3073998Z [cluster2]                         "properties": {
2021-07-20T04:17:53.3074666Z [cluster2]                             "apiVersion": {
2021-07-20T04:17:53.3077144Z [cluster2]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-20T04:17:53.3124139Z [cluster2]                                 "type": "string"
2021-07-20T04:17:53.3124619Z [cluster2]                             },
2021-07-20T04:17:53.3125035Z [cluster2]                             "kind": {
2021-07-20T04:17:53.3127234Z [cluster2]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-20T04:17:53.3129466Z [cluster2]                                 "type": "string"
2021-07-20T04:17:53.3130068Z [cluster2]                             },
2021-07-20T04:17:53.3130662Z [cluster2]                             "metadata": {
2021-07-20T04:17:53.3131323Z [cluster2]                                 "type": "object"
2021-07-20T04:17:53.3131940Z [cluster2]                             },
2021-07-20T04:17:53.3132370Z [cluster2]                             "spec": {
2021-07-20T04:17:53.3132839Z [cluster2]                                 "properties": {
2021-07-20T04:17:53.3133808Z [cluster2]                                     "cluster_cidr": {
2021-07-20T04:17:53.3134318Z [cluster2]                                         "items": {
2021-07-20T04:17:53.3134998Z [cluster2]                                             "type": "string"
2021-07-20T04:17:53.3135468Z [cluster2]                                         },
2021-07-20T04:17:53.3135946Z [cluster2]                                         "type": "array"
2021-07-20T04:17:53.3136419Z [cluster2]                                     },
2021-07-20T04:17:53.3136875Z [cluster2]                                     "cluster_id": {
2021-07-20T04:17:53.3138634Z [cluster2]                                         "type": "string"
2021-07-20T04:17:53.3139126Z [cluster2]                                     },
2021-07-20T04:17:53.3139598Z [cluster2]                                     "color_codes": {
2021-07-20T04:17:53.3140080Z [cluster2]                                         "items": {
2021-07-20T04:17:53.3140588Z [cluster2]                                             "type": "string"
2021-07-20T04:17:53.3141057Z [cluster2]                                         },
2021-07-20T04:17:53.3141534Z [cluster2]                                         "type": "array"
2021-07-20T04:17:53.3142012Z [cluster2]                                     },
2021-07-20T04:17:53.3142468Z [cluster2]                                     "global_cidr": {
2021-07-20T04:17:53.3142962Z [cluster2]                                         "items": {
2021-07-20T04:17:53.3143455Z [cluster2]                                             "type": "string"
2021-07-20T04:17:53.3143935Z [cluster2]                                         },
2021-07-20T04:17:53.3144402Z [cluster2]                                         "type": "array"
2021-07-20T04:17:53.3145061Z [cluster2]                                     },
2021-07-20T04:17:53.3145599Z [cluster2]                                     "service_cidr": {
2021-07-20T04:17:53.3146484Z [cluster2]                                         "items": {
2021-07-20T04:17:53.3147150Z [cluster2]                                             "type": "string"
2021-07-20T04:17:53.3147761Z [cluster2]                                         },
2021-07-20T04:17:53.3148332Z [cluster2]                                         "type": "array"
2021-07-20T04:17:53.3148775Z [cluster2]                                     }
2021-07-20T04:17:53.3150142Z [cluster2]                                 },
2021-07-20T04:17:53.3150702Z [cluster2]                                 "required": [
2021-07-20T04:17:53.3151262Z [cluster2]                                     "cluster_cidr",
2021-07-20T04:17:53.3151995Z [cluster2]                                     "cluster_id",
2021-07-20T04:17:53.3152533Z [cluster2]                                     "color_codes",
2021-07-20T04:17:53.3153076Z [cluster2]                                     "global_cidr",
2021-07-20T04:17:53.3153944Z [cluster2]                                     "service_cidr"
2021-07-20T04:17:53.3154427Z [cluster2]                                 ],
2021-07-20T04:17:53.3155048Z [cluster2]                                 "type": "object"
2021-07-20T04:17:53.3155504Z [cluster2]                             }
2021-07-20T04:17:53.3155896Z [cluster2]                         },
2021-07-20T04:17:53.3156522Z [cluster2]                         "required": [
2021-07-20T04:17:53.3156979Z [cluster2]                             "spec"
2021-07-20T04:17:53.3157409Z [cluster2]                         ],
2021-07-20T04:17:53.3157873Z [cluster2]                         "type": "object"
2021-07-20T04:17:53.3158307Z [cluster2]                     }
2021-07-20T04:17:53.3158707Z [cluster2]                 },
2021-07-20T04:17:53.3159139Z [cluster2]                 "served": true,
2021-07-20T04:17:53.3160468Z [cluster2]                 "storage": true
2021-07-20T04:17:53.3160910Z [cluster2]             }
2021-07-20T04:17:53.3161310Z [cluster2]         ]
2021-07-20T04:17:53.3161837Z [cluster2]     },
2021-07-20T04:17:53.3162247Z [cluster2]     "status": {
2021-07-20T04:17:53.3162721Z [cluster2]         "acceptedNames": {
2021-07-20T04:17:53.3163240Z [cluster2]             "kind": "Cluster",
2021-07-20T04:17:53.3163775Z [cluster2]             "listKind": "ClusterList",
2021-07-20T04:17:53.3164336Z [cluster2]             "plural": "clusters",
2021-07-20T04:17:53.3164868Z [cluster2]             "singular": "cluster"
2021-07-20T04:17:53.3165309Z [cluster2]         },
2021-07-20T04:17:53.3165750Z [cluster2]         "conditions": [
2021-07-20T04:17:53.3166263Z [cluster2]             {
2021-07-20T04:17:53.3167165Z [cluster2]                 "lastTransitionTime": "2021-07-20T04:13:23Z",
2021-07-20T04:17:53.3167967Z [cluster2]                 "message": "no conflicts found",
2021-07-20T04:17:53.3168572Z [cluster2]                 "reason": "NoConflicts",
2021-07-20T04:17:53.3169086Z [cluster2]                 "status": "True",
2021-07-20T04:17:53.3169633Z [cluster2]                 "type": "NamesAccepted"
2021-07-20T04:17:54.2794622Z [cluster1]             },
2021-07-20T04:17:54.2795109Z [cluster1]             {
2021-07-20T04:17:54.2796628Z [cluster1]                 "lastTransitionTime": "2021-07-20T04:13:13Z",
2021-07-20T04:17:54.2797441Z [cluster1]                 "message": "the initial names have been accepted",
2021-07-20T04:17:54.2798204Z [cluster1]                 "reason": "InitialNamesAccepted",
2021-07-20T04:17:54.2798850Z [cluster1]                 "status": "True",
2021-07-20T04:17:54.2799376Z [cluster1]                 "type": "Established"
2021-07-20T04:17:54.2799869Z [cluster1]             }
2021-07-20T04:17:54.2800268Z [cluster1]         ],
2021-07-20T04:17:54.2800742Z [cluster1]         "storedVersions": [
2021-07-20T04:17:54.2801253Z [cluster1]             "v1"
2021-07-20T04:17:54.2801642Z [cluster1]         ]
2021-07-20T04:17:54.2802206Z [cluster1]     }
2021-07-20T04:17:54.2802586Z [cluster1] }
2021-07-20T04:17:54.2803719Z [36m[submariner-operator]$ [cluster1] validate_equals .metadata.name clusters.submariner.io[0m
2021-07-20T04:17:54.2805211Z [36m[submariner-operator]$ [cluster1] validate_equals .metadata.name clusters.submariner.io[0m
2021-07-20T04:17:54.2806898Z [36m[submariner-operator]$ [cluster1] jq -r -M .metadata.name /tmp/clusters.submariner.io.cluster1.json[0m
2021-07-20T04:17:54.2809199Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced[0m
2021-07-20T04:17:54.2810672Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced[0m
2021-07-20T04:17:54.2812505Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.scope /tmp/clusters.submariner.io.cluster1.json[0m
2021-07-20T04:17:54.2814124Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io[0m
2021-07-20T04:17:54.2815355Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io[0m
2021-07-20T04:17:54.2816842Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.group /tmp/clusters.submariner.io.cluster1.json[0m
2021-07-20T04:17:54.2818938Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.names.kind Cluster[0m
2021-07-20T04:17:54.2820365Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.names.kind Cluster[0m
2021-07-20T04:17:54.2822052Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.names.kind /tmp/clusters.submariner.io.cluster1.json[0m
2021-07-20T04:17:54.2823937Z [36m[submariner-operator]$ [cluster1] validate_equals .status.acceptedNames.kind Cluster[0m
2021-07-20T04:17:54.2825418Z [36m[submariner-operator]$ [cluster1] validate_equals .status.acceptedNames.kind Cluster[0m
2021-07-20T04:17:54.2827255Z [36m[submariner-operator]$ [cluster1] jq -r -M .status.acceptedNames.kind /tmp/clusters.submariner.io.cluster1.json[0m
2021-07-20T04:17:54.2828791Z [36m[submariner-operator]$ [cluster1] verify_subm_crd[0m
2021-07-20T04:17:54.2829745Z [36m[submariner-operator]$ [cluster1] verify_subm_crd[0m
2021-07-20T04:17:54.2831041Z [36m[submariner-operator]$ [cluster1] validate_crd submariners.submariner.io v1alpha1 Submariner[0m
2021-07-20T04:17:54.2832592Z [36m[submariner-operator]$ [cluster1] validate_crd submariners.submariner.io v1alpha1 Submariner[0m
2021-07-20T04:17:54.2834073Z [36m[submariner-operator]$ [cluster1] kubectl get crds submariners.submariner.io[0m
2021-07-20T04:17:54.2835472Z [36m[submariner-operator]$ [cluster1] kubectl get crds submariners.submariner.io[0m
2021-07-20T04:17:54.2837016Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crds submariners.submariner.io[0m
2021-07-20T04:17:54.2837989Z [cluster1] NAME                        CREATED AT
2021-07-20T04:17:54.2838873Z [cluster1] submariners.submariner.io   2021-07-20T04:13:01Z
2021-07-20T04:17:54.2840497Z [36m[submariner-operator]$ [cluster1] json_file=/tmp/submariners.submariner.io.cluster1.json[0m
2021-07-20T04:17:54.2842377Z [36m[submariner-operator]$ [cluster1] kubectl get crd submariners.submariner.io -o json[0m
2021-07-20T04:17:54.2843887Z [36m[submariner-operator]$ [cluster1] kubectl get crd submariners.submariner.io -o json[0m
2021-07-20T04:17:54.2845648Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crd submariners.submariner.io -o json[0m
2021-07-20T04:17:54.2848900Z [36m[submariner-operator]$ [cluster1] tee /tmp/submariners.submariner.io.cluster1.json[0m
2021-07-20T04:17:54.2849936Z [cluster1] {
2021-07-20T04:17:54.2850692Z [cluster1]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-20T04:17:54.2851510Z [cluster1]     "kind": "CustomResourceDefinition",
2021-07-20T04:17:54.2852148Z [cluster1]     "metadata": {
2021-07-20T04:17:54.2852641Z [cluster1]         "annotations": {
2021-07-20T04:17:54.2853624Z [cluster1]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-20T04:17:54.2854351Z [cluster1]         },
2021-07-20T04:17:54.2855101Z [cluster1]         "creationTimestamp": "2021-07-20T04:13:01Z",
2021-07-20T04:17:54.2855711Z [cluster1]         "generation": 2,
2021-07-20T04:17:54.2856246Z [cluster1]         "managedFields": [
2021-07-20T04:17:54.2856707Z [cluster1]             {
2021-07-20T04:17:54.2857489Z [cluster1]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-20T04:17:54.2858200Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-20T04:17:54.2858931Z [cluster1]                 "fieldsV1": {
2021-07-20T04:17:54.2859417Z [cluster1]                     "f:metadata": {
2021-07-20T04:17:54.2859948Z [cluster1]                         "f:annotations": {
2021-07-20T04:17:54.2860425Z [cluster1]                             ".": {},
2021-07-20T04:17:54.2861436Z [cluster1]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-20T04:17:54.2862164Z [cluster1]                         }
2021-07-20T04:17:54.2862580Z [cluster1]                     },
2021-07-20T04:17:54.2863030Z [cluster1]                     "f:spec": {
2021-07-20T04:17:54.2863709Z [cluster1]                         "f:conversion": {
2021-07-20T04:17:54.2864375Z [cluster1]                             ".": {},
2021-07-20T04:17:54.2865032Z [cluster1]                             "f:strategy": {}
2021-07-20T04:17:54.2865532Z [cluster1]                         },
2021-07-20T04:17:54.2865987Z [cluster1]                         "f:group": {},
2021-07-20T04:17:54.2866490Z [cluster1]                         "f:names": {
2021-07-20T04:17:54.2866968Z [cluster1]                             "f:kind": {},
2021-07-20T04:17:54.2867626Z [cluster1]                             "f:listKind": {},
2021-07-20T04:17:54.2868139Z [cluster1]                             "f:plural": {},
2021-07-20T04:17:54.2868632Z [cluster1]                             "f:singular": {}
2021-07-20T04:17:54.2869096Z [cluster1]                         },
2021-07-20T04:17:54.2869542Z [cluster1]                         "f:scope": {},
2021-07-20T04:17:54.2870048Z [cluster1]                         "f:versions": {}
2021-07-20T04:17:54.2870500Z [cluster1]                     },
2021-07-20T04:17:54.2870950Z [cluster1]                     "f:status": {
2021-07-20T04:17:54.2871474Z [cluster1]                         "f:acceptedNames": {
2021-07-20T04:17:54.2872009Z [cluster1]                             "f:kind": {},
2021-07-20T04:17:54.2872481Z [cluster1]                             "f:plural": {}
2021-07-20T04:17:54.2872948Z [cluster1]                         },
2021-07-20T04:17:54.2873438Z [cluster1]                         "f:conditions": {},
2021-07-20T04:17:54.2873998Z [cluster1]                         "f:storedVersions": {}
2021-07-20T04:17:54.2874511Z [cluster1]                     }
2021-07-20T04:17:54.2875104Z [cluster1]                 },
2021-07-20T04:17:54.2875595Z [cluster1]                 "manager": "subctl",
2021-07-20T04:17:54.2876148Z [cluster1]                 "operation": "Update",
2021-07-20T04:17:54.2964776Z [cluster1]                 "time": "2021-07-20T04:13:12Z"
2021-07-20T04:17:54.2965484Z [cluster1]             }
2021-07-20T04:17:54.2966036Z [cluster1]         ],
2021-07-20T04:17:54.2966785Z [cluster1]         "name": "submariners.submariner.io",
2021-07-20T04:17:54.2967832Z [cluster1]         "resourceVersion": "2198",
2021-07-20T04:17:54.2969160Z [cluster1]         "uid": "1ae48f55-af25-4fbd-b936-e26df4b6d48b"
2021-07-20T04:17:54.2969902Z [cluster1]     },
2021-07-20T04:17:54.2970335Z [cluster1]     "spec": {
2021-07-20T04:17:54.2970915Z [cluster1]         "conversion": {
2021-07-20T04:17:54.2971549Z [cluster1]             "strategy": "None"
2021-07-20T04:17:54.2972097Z [cluster1]         },
2021-07-20T04:17:54.2972602Z [cluster1]         "group": "submariner.io",
2021-07-20T04:17:54.2973225Z [cluster1]         "names": {
2021-07-20T04:17:54.2973851Z [cluster1]             "kind": "Submariner",
2021-07-20T04:17:54.2974547Z [cluster1]             "listKind": "SubmarinerList",
2021-07-20T04:17:54.2975304Z [cluster1]             "plural": "submariners",
2021-07-20T04:17:54.2976010Z [cluster1]             "singular": "submariner"
2021-07-20T04:17:54.2976485Z [cluster1]         },
2021-07-20T04:17:54.2977313Z [cluster1]         "scope": "Namespaced",
2021-07-20T04:17:54.2977824Z [cluster1]         "versions": [
2021-07-20T04:17:54.2978394Z [cluster1]             {
2021-07-20T04:17:54.2978967Z [cluster1]                 "name": "v1alpha1",
2021-07-20T04:17:54.2979561Z [cluster1]                 "schema": {
2021-07-20T04:17:54.2980424Z [cluster1]                     "openAPIV3Schema": {
2021-07-20T04:17:54.2981525Z [cluster1]                         "description": "Submariner is the Schema for the submariners API",
2021-07-20T04:17:54.2982729Z [cluster1]                         "properties": {
2021-07-20T04:17:54.2983288Z [cluster1]                             "apiVersion": {
2021-07-20T04:17:54.2986105Z [cluster1]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-20T04:17:54.2988814Z [cluster1]                                 "type": "string"
2021-07-20T04:17:54.2989319Z [cluster1]                             },
2021-07-20T04:17:54.2989920Z [cluster1]                             "kind": {
2021-07-20T04:17:54.2992667Z [cluster1]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-20T04:17:54.2995171Z [cluster1]                                 "type": "string"
2021-07-20T04:17:54.2995939Z [cluster1]                             },
2021-07-20T04:17:54.2996397Z [cluster1]                             "metadata": {
2021-07-20T04:17:54.2997032Z [cluster1]                                 "type": "object"
2021-07-20T04:17:54.2998540Z [cluster1]                             },
2021-07-20T04:17:54.2999439Z [cluster1]                             "spec": {
2021-07-20T04:17:54.3000353Z [cluster1]                                 "description": "SubmarinerSpec defines the desired state of Submariner",
2021-07-20T04:17:54.3001594Z [cluster1]                                 "properties": {
2021-07-20T04:17:54.3002132Z [cluster1]                                     "broker": {
2021-07-20T04:17:54.3002768Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3003433Z [cluster1]                                     },
2021-07-20T04:17:54.3004139Z [cluster1]                                     "brokerK8sApiServer": {
2021-07-20T04:17:54.3004930Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3005438Z [cluster1]                                     },
2021-07-20T04:17:54.3006411Z [cluster1]                                     "brokerK8sApiServerToken": {
2021-07-20T04:17:54.3007075Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3007552Z [cluster1]                                     },
2021-07-20T04:17:54.3008043Z [cluster1]                                     "brokerK8sCA": {
2021-07-20T04:17:54.3008560Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3009977Z [cluster1]                                     },
2021-07-20T04:17:54.3010636Z [cluster1]                                     "brokerK8sRemoteNamespace": {
2021-07-20T04:17:54.3011315Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3011771Z [cluster1]                                     },
2021-07-20T04:17:54.3012268Z [cluster1]                                     "cableDriver": {
2021-07-20T04:17:54.3012823Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3013287Z [cluster1]                                     },
2021-07-20T04:17:54.3013794Z [cluster1]                                     "ceIPSecDebug": {
2021-07-20T04:17:54.3014340Z [cluster1]                                         "type": "boolean"
2021-07-20T04:17:54.3014827Z [cluster1]                                     },
2021-07-20T04:17:54.3015418Z [cluster1]                                     "ceIPSecForceUDPEncaps": {
2021-07-20T04:17:54.3016307Z [cluster1]                                         "type": "boolean"
2021-07-20T04:17:54.3017005Z [cluster1]                                     },
2021-07-20T04:17:54.3017888Z [cluster1]                                     "ceIPSecIKEPort": {
2021-07-20T04:17:54.3026168Z [cluster1]                                         "type": "integer"
2021-07-20T04:17:54.3026939Z [cluster1]                                     },
2021-07-20T04:17:54.3027507Z [cluster1]                                     "ceIPSecNATTPort": {
2021-07-20T04:17:54.3028152Z [cluster1]                                         "type": "integer"
2021-07-20T04:17:54.3028713Z [cluster1]                                     },
2021-07-20T04:17:54.3029273Z [cluster1]                                     "ceIPSecPSK": {
2021-07-20T04:17:54.3029822Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3030608Z [cluster1]                                     },
2021-07-20T04:17:54.3037316Z [cluster1]                                     "ceIPSecPreferredServer": {
2021-07-20T04:17:54.3038096Z [cluster1]                                         "type": "boolean"
2021-07-20T04:17:54.3038684Z [cluster1]                                     },
2021-07-20T04:17:54.3039210Z [cluster1]                                     "clusterCIDR": {
2021-07-20T04:17:54.3039830Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3040346Z [cluster1]                                     },
2021-07-20T04:17:54.3040927Z [cluster1]                                     "clusterID": {
2021-07-20T04:17:54.3041775Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3042287Z [cluster1]                                     },
2021-07-20T04:17:54.3042848Z [cluster1]                                     "colorCodes": {
2021-07-20T04:17:54.3043403Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3044967Z [cluster1]                                     },
2021-07-20T04:17:54.3062520Z [cluster1]                                     "connectionHealthCheck": {
2021-07-20T04:17:54.3063465Z [cluster1]                                         "properties": {
2021-07-20T04:17:54.3064023Z [cluster1]                                             "enabled": {
2021-07-20T04:17:54.3064749Z [cluster1]                                                 "type": "boolean"
2021-07-20T04:17:54.3065517Z [cluster1]                                             },
2021-07-20T04:17:54.3066109Z [cluster1]                                             "intervalSeconds": {
2021-07-20T04:17:54.3066931Z [cluster1]                                                 "description": "The interval at which health check pings are sent.",
2021-07-20T04:17:54.3068047Z [cluster1]                                                 "format": "int64",
2021-07-20T04:17:54.3068760Z [cluster1]                                                 "type": "integer"
2021-07-20T04:17:54.3069288Z [cluster1]                                             },
2021-07-20T04:17:54.3069862Z [cluster1]                                             "maxPacketLossCount": {
2021-07-20T04:17:54.3073744Z [cluster1]                                                 "description": "The maximum number of packets lost at which the health checker will mark the connection as down.",
2021-07-20T04:17:54.3074971Z [cluster1]                                                 "format": "int64",
2021-07-20T04:17:54.3075506Z [cluster1]                                                 "type": "integer"
2021-07-20T04:17:54.3076163Z [cluster1]                                             }
2021-07-20T04:17:54.3076735Z [cluster1]                                         },
2021-07-20T04:17:54.3077204Z [cluster1]                                         "type": "object"
2021-07-20T04:17:54.3077650Z [cluster1]                                     },
2021-07-20T04:17:54.3078200Z [cluster1]                                     "coreDNSCustomConfig": {
2021-07-20T04:17:54.3078983Z [cluster1]                                         "properties": {
2021-07-20T04:17:54.3079525Z [cluster1]                                             "configMapName": {
2021-07-20T04:17:54.3080077Z [cluster1]                                                 "type": "string"
2021-07-20T04:17:54.3086575Z [cluster1]                                             },
2021-07-20T04:17:54.3087058Z [cluster1]                                             "namespace": {
2021-07-20T04:17:54.3087781Z [cluster1]                                                 "type": "string"
2021-07-20T04:17:54.3088258Z [cluster1]                                             }
2021-07-20T04:17:54.3088699Z [cluster1]                                         },
2021-07-20T04:17:54.3089181Z [cluster1]                                         "type": "object"
2021-07-20T04:17:54.3089642Z [cluster1]                                     },
2021-07-20T04:17:54.3090581Z [cluster1]                                     "customDomains": {
2021-07-20T04:17:54.3091272Z [cluster1]                                         "items": {
2021-07-20T04:17:54.3091756Z [cluster1]                                             "type": "string"
2021-07-20T04:17:54.3092206Z [cluster1]                                         },
2021-07-20T04:17:54.3092666Z [cluster1]                                         "type": "array",
2021-07-20T04:17:54.3093722Z [cluster1]                                         "x-kubernetes-list-type": "set"
2021-07-20T04:17:54.3094316Z [cluster1]                                     },
2021-07-20T04:17:54.3094751Z [cluster1]                                     "debug": {
2021-07-20T04:17:54.3095531Z [cluster1]                                         "type": "boolean"
2021-07-20T04:17:54.3096015Z [cluster1]                                     },
2021-07-20T04:17:54.3096466Z [cluster1]                                     "globalCIDR": {
2021-07-20T04:17:54.3096972Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3098279Z [cluster1]                                     },
2021-07-20T04:17:54.3098987Z [cluster1]                                     "imageOverrides": {
2021-07-20T04:17:54.3102374Z [cluster1]                                         "additionalProperties": {
2021-07-20T04:17:54.3103011Z [cluster1]                                             "type": "string"
2021-07-20T04:17:54.3103478Z [cluster1]                                         },
2021-07-20T04:17:54.3105735Z [cluster1]                                         "type": "object"
2021-07-20T04:17:54.3106267Z [cluster1]                                     },
2021-07-20T04:17:54.3106839Z [cluster1]                                     "loadBalancerEnabled": {
2021-07-20T04:17:54.3107642Z [cluster1]                                         "type": "boolean"
2021-07-20T04:17:54.3108261Z [cluster1]                                     },
2021-07-20T04:17:54.3110216Z [cluster1]                                     "namespace": {
2021-07-20T04:17:54.3110980Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3111437Z [cluster1]                                     },
2021-07-20T04:17:54.3111922Z [cluster1]                                     "natEnabled": {
2021-07-20T04:17:54.3112628Z [cluster1]                                         "type": "boolean"
2021-07-20T04:17:54.3113288Z [cluster1]                                     },
2021-07-20T04:17:54.3114035Z [cluster1]                                     "repository": {
2021-07-20T04:17:54.3114534Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3114995Z [cluster1]                                     },
2021-07-20T04:17:54.3115472Z [cluster1]                                     "serviceCIDR": {
2021-07-20T04:17:54.3116391Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3116866Z [cluster1]                                     },
2021-07-20T04:17:54.3117632Z [cluster1]                                     "serviceDiscoveryEnabled": {
2021-07-20T04:17:54.3118673Z [cluster1]                                         "type": "boolean"
2021-07-20T04:17:54.3119404Z [cluster1]                                     },
2021-07-20T04:17:54.3119872Z [cluster1]                                     "version": {
2021-07-20T04:17:54.3120377Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3120841Z [cluster1]                                     }
2021-07-20T04:17:54.3121265Z [cluster1]                                 },
2021-07-20T04:17:54.3121715Z [cluster1]                                 "required": [
2021-07-20T04:17:54.3122212Z [cluster1]                                     "broker",
2021-07-20T04:17:54.3122972Z [cluster1]                                     "brokerK8sApiServer",
2021-07-20T04:17:54.3123703Z [cluster1]                                     "brokerK8sApiServerToken",
2021-07-20T04:17:54.3124587Z [cluster1]                                     "brokerK8sCA",
2021-07-20T04:17:54.3125284Z [cluster1]                                     "brokerK8sRemoteNamespace",
2021-07-20T04:17:54.3125979Z [cluster1]                                     "ceIPSecDebug",
2021-07-20T04:17:54.3126518Z [cluster1]                                     "ceIPSecPSK",
2021-07-20T04:17:54.3127055Z [cluster1]                                     "clusterCIDR",
2021-07-20T04:17:54.3127568Z [cluster1]                                     "clusterID",
2021-07-20T04:17:54.3128068Z [cluster1]                                     "debug",
2021-07-20T04:17:54.3128554Z [cluster1]                                     "namespace",
2021-07-20T04:17:54.3129409Z [cluster1]                                     "natEnabled",
2021-07-20T04:17:54.3130091Z [cluster1]                                     "serviceCIDR"
2021-07-20T04:17:54.3130558Z [cluster1]                                 ],
2021-07-20T04:17:54.3131014Z [cluster1]                                 "type": "object"
2021-07-20T04:17:54.3131453Z [cluster1]                             },
2021-07-20T04:17:54.3132304Z [cluster1]                             "status": {
2021-07-20T04:17:54.3133220Z [cluster1]                                 "description": "SubmarinerStatus defines the observed state of Submariner",
2021-07-20T04:17:54.3133985Z [cluster1]                                 "properties": {
2021-07-20T04:17:54.3134498Z [cluster1]                                     "clusterCIDR": {
2021-07-20T04:17:54.3135135Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3135925Z [cluster1]                                     },
2021-07-20T04:17:54.3136631Z [cluster1]                                     "clusterID": {
2021-07-20T04:17:54.3137329Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3137777Z [cluster1]                                     },
2021-07-20T04:17:54.3138244Z [cluster1]                                     "colorCodes": {
2021-07-20T04:17:54.3138747Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3139202Z [cluster1]                                     },
2021-07-20T04:17:54.3140059Z [cluster1]                                     "deploymentInfo": {
2021-07-20T04:17:54.3140653Z [cluster1]                                         "properties": {
2021-07-20T04:17:54.3141396Z [cluster1]                                             "cloudProvider": {
2021-07-20T04:17:54.3141951Z [cluster1]                                                 "type": "string"
2021-07-20T04:17:54.3142436Z [cluster1]                                             },
2021-07-20T04:17:54.3143933Z [cluster1]                                             "kubernetesType": {
2021-07-20T04:17:54.3144718Z [cluster1]                                                 "type": "string"
2021-07-20T04:17:54.3145444Z [cluster1]                                             },
2021-07-20T04:17:54.3146515Z [cluster1]                                             "kubernetesTypeVersion": {
2021-07-20T04:17:54.3147799Z [cluster1]                                                 "type": "string"
2021-07-20T04:17:54.3148606Z [cluster1]                                             },
2021-07-20T04:17:54.3151459Z [cluster1]                                             "kubernetesVersion": {
2021-07-20T04:17:54.3152105Z [cluster1]                                                 "type": "string"
2021-07-20T04:17:54.3152948Z [cluster1]                                             }
2021-07-20T04:17:54.3153370Z [cluster1]                                         },
2021-07-20T04:17:54.3153851Z [cluster1]                                         "type": "object"
2021-07-20T04:17:54.3155294Z [cluster1]                                     },
2021-07-20T04:17:54.3155927Z [cluster1]                                     "gatewayDaemonSetStatus": {
2021-07-20T04:17:54.3156598Z [cluster1]                                         "properties": {
2021-07-20T04:17:54.3157216Z [cluster1]                                             "lastResourceVersion": {
2021-07-20T04:17:54.3157851Z [cluster1]                                                 "type": "string"
2021-07-20T04:17:54.3158320Z [cluster1]                                             },
2021-07-20T04:17:54.3158969Z [cluster1]                                             "mismatchedContainerImages": {
2021-07-20T04:17:54.3159657Z [cluster1]                                                 "type": "boolean"
2021-07-20T04:17:54.3160158Z [cluster1]                                             },
2021-07-20T04:17:54.3160960Z [cluster1]                                             "nonReadyContainerStates": {
2021-07-20T04:17:54.3161632Z [cluster1]                                                 "items": {
2021-07-20T04:17:54.3162867Z [cluster1]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-20T04:17:54.3164385Z [cluster1]                                                     "properties": {
2021-07-20T04:17:54.3165259Z [cluster1]                                                         "running": {
2021-07-20T04:17:54.3166394Z [cluster1]                                                             "description": "Details about a running container",
2021-07-20T04:17:54.3167108Z [cluster1]                                                             "properties": {
2021-07-20T04:17:54.3168003Z [cluster1]                                                                 "startedAt": {
2021-07-20T04:17:54.3169204Z [cluster1]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-20T04:17:54.3170174Z [cluster1]                                                                     "format": "date-time",
2021-07-20T04:17:54.3170766Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3171273Z [cluster1]                                                                 }
2021-07-20T04:17:54.3171740Z [cluster1]                                                             },
2021-07-20T04:17:54.3172233Z [cluster1]                                                             "type": "object"
2021-07-20T04:17:54.3172740Z [cluster1]                                                         },
2021-07-20T04:17:54.3173254Z [cluster1]                                                         "terminated": {
2021-07-20T04:17:54.3173951Z [cluster1]                                                             "description": "Details about a terminated container",
2021-07-20T04:17:54.3174668Z [cluster1]                                                             "properties": {
2021-07-20T04:17:54.3175234Z [cluster1]                                                                 "containerID": {
2021-07-20T04:17:54.3176338Z [cluster1]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-20T04:17:54.3177955Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3178502Z [cluster1]                                                                 },
2021-07-20T04:17:54.3179104Z [cluster1]                                                                 "exitCode": {
2021-07-20T04:17:54.3179909Z [cluster1]                                                                     "description": "Exit status from the last termination of the container",
2021-07-20T04:17:54.3180872Z [cluster1]                                                                     "format": "int32",
2021-07-20T04:17:54.3181459Z [cluster1]                                                                     "type": "integer"
2021-07-20T04:17:54.3181986Z [cluster1]                                                                 },
2021-07-20T04:17:54.3182530Z [cluster1]                                                                 "finishedAt": {
2021-07-20T04:17:54.3183645Z [cluster1]                                                                     "description": "Time at which the container last terminated",
2021-07-20T04:17:54.3184706Z [cluster1]                                                                     "format": "date-time",
2021-07-20T04:17:54.3185494Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3186035Z [cluster1]                                                                 },
2021-07-20T04:17:54.3186619Z [cluster1]                                                                 "message": {
2021-07-20T04:17:54.3187431Z [cluster1]                                                                     "description": "Message regarding the last termination of the container",
2021-07-20T04:17:54.3188267Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3188822Z [cluster1]                                                                 },
2021-07-20T04:17:54.3189657Z [cluster1]                                                                 "reason": {
2021-07-20T04:17:54.3190405Z [cluster1]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-20T04:17:54.3191270Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3191804Z [cluster1]                                                                 },
2021-07-20T04:17:54.3192290Z [cluster1]                                                                 "signal": {
2021-07-20T04:17:54.3192995Z [cluster1]                                                                     "description": "Signal from the last termination of the container",
2021-07-20T04:17:54.3193935Z [cluster1]                                                                     "format": "int32",
2021-07-20T04:17:54.3194777Z [cluster1]                                                                     "type": "integer"
2021-07-20T04:17:54.3195332Z [cluster1]                                                                 },
2021-07-20T04:17:54.3196181Z [cluster1]                                                                 "startedAt": {
2021-07-20T04:17:54.3197192Z [cluster1]                                                                     "description": "Time at which previous execution of the container started",
2021-07-20T04:17:54.3199077Z [cluster1]                                                                     "format": "date-time",
2021-07-20T04:17:54.3200154Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3200941Z [cluster1]                                                                 }
2021-07-20T04:17:54.3201584Z [cluster1]                                                             },
2021-07-20T04:17:54.3202430Z [cluster1]                                                             "required": [
2021-07-20T04:17:54.3203023Z [cluster1]                                                                 "exitCode"
2021-07-20T04:17:54.3203552Z [cluster1]                                                             ],
2021-07-20T04:17:54.3204241Z [cluster1]                                                             "type": "object"
2021-07-20T04:17:54.3206089Z [cluster1]                                                         },
2021-07-20T04:17:54.3206857Z [cluster1]                                                         "waiting": {
2021-07-20T04:17:54.3207564Z [cluster1]                                                             "description": "Details about a waiting container",
2021-07-20T04:17:54.3208307Z [cluster1]                                                             "properties": {
2021-07-20T04:17:54.3209206Z [cluster1]                                                                 "message": {
2021-07-20T04:17:54.3210153Z [cluster1]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-20T04:17:54.3211167Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3211693Z [cluster1]                                                                 },
2021-07-20T04:17:54.3212216Z [cluster1]                                                                 "reason": {
2021-07-20T04:17:54.3212956Z [cluster1]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-20T04:17:54.3213688Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3214212Z [cluster1]                                                                 }
2021-07-20T04:17:54.3214682Z [cluster1]                                                             },
2021-07-20T04:17:54.3215203Z [cluster1]                                                             "type": "object"
2021-07-20T04:17:54.3215716Z [cluster1]                                                         }
2021-07-20T04:17:54.3216188Z [cluster1]                                                     },
2021-07-20T04:17:54.3216831Z [cluster1]                                                     "type": "object"
2021-07-20T04:17:54.3218233Z [cluster1]                                                 },
2021-07-20T04:17:54.3218754Z [cluster1]                                                 "type": "array"
2021-07-20T04:17:54.3219255Z [cluster1]                                             },
2021-07-20T04:17:54.3219752Z [cluster1]                                             "status": {
2021-07-20T04:17:54.3220572Z [cluster1]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-20T04:17:54.3221451Z [cluster1]                                                 "properties": {
2021-07-20T04:17:54.3222081Z [cluster1]                                                     "collisionCount": {
2021-07-20T04:17:54.3223455Z [cluster1]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-20T04:17:54.3224800Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3225396Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3225932Z [cluster1]                                                     },
2021-07-20T04:17:54.3226469Z [cluster1]                                                     "conditions": {
2021-07-20T04:17:54.3227933Z [cluster1]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-20T04:17:54.3228981Z [cluster1]                                                         "items": {
2021-07-20T04:17:54.3230175Z [cluster1]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-20T04:17:54.3231097Z [cluster1]                                                             "properties": {
2021-07-20T04:17:54.3231763Z [cluster1]                                                                 "lastTransitionTime": {
2021-07-20T04:17:54.3232649Z [cluster1]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-20T04:17:54.3234379Z [cluster1]                                                                     "format": "date-time",
2021-07-20T04:17:54.3236294Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3236847Z [cluster1]                                                                 },
2021-07-20T04:17:54.3237393Z [cluster1]                                                                 "message": {
2021-07-20T04:17:54.3238220Z [cluster1]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-20T04:17:54.3239063Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3239597Z [cluster1]                                                                 },
2021-07-20T04:17:54.3240277Z [cluster1]                                                                 "reason": {
2021-07-20T04:17:54.3241547Z [cluster1]                                                                     "description": "The reason for the condition's last transition.",
2021-07-20T04:17:54.3242305Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3242836Z [cluster1]                                                                 },
2021-07-20T04:17:54.3243348Z [cluster1]                                                                 "status": {
2021-07-20T04:17:54.3244556Z [cluster1]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-20T04:17:54.3245369Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3254835Z [cluster1]                                                                 },
2021-07-20T04:17:54.3255387Z [cluster1]                                                                 "type": {
2021-07-20T04:17:54.3256239Z [cluster1]                                                                     "description": "Type of DaemonSet condition.",
2021-07-20T04:17:54.3257372Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3258325Z [cluster1]                                                                 }
2021-07-20T04:17:54.3258781Z [cluster1]                                                             },
2021-07-20T04:17:54.3259287Z [cluster1]                                                             "required": [
2021-07-20T04:17:54.3259811Z [cluster1]                                                                 "status",
2021-07-20T04:17:54.3260324Z [cluster1]                                                                 "type"
2021-07-20T04:17:54.3260812Z [cluster1]                                                             ],
2021-07-20T04:17:54.3261308Z [cluster1]                                                             "type": "object"
2021-07-20T04:17:54.3261814Z [cluster1]                                                         },
2021-07-20T04:17:54.3262517Z [cluster1]                                                         "type": "array"
2021-07-20T04:17:54.3263016Z [cluster1]                                                     },
2021-07-20T04:17:54.3263628Z [cluster1]                                                     "currentNumberScheduled": {
2021-07-20T04:17:54.3265288Z [cluster1]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.3266939Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3267488Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3268003Z [cluster1]                                                     },
2021-07-20T04:17:54.3268625Z [cluster1]                                                     "desiredNumberScheduled": {
2021-07-20T04:17:54.3270165Z [cluster1]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.3271407Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3271949Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3272425Z [cluster1]                                                     },
2021-07-20T04:17:54.3273133Z [cluster1]                                                     "numberAvailable": {
2021-07-20T04:17:54.3274596Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-20T04:17:54.3275951Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3276656Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3277326Z [cluster1]                                                     },
2021-07-20T04:17:54.3278081Z [cluster1]                                                     "numberMisscheduled": {
2021-07-20T04:17:54.3279527Z [cluster1]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.3281419Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3282007Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3283580Z [cluster1]                                                     },
2021-07-20T04:17:54.3284120Z [cluster1]                                                     "numberReady": {
2021-07-20T04:17:54.3285257Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-20T04:17:54.3286322Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3286877Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3287370Z [cluster1]                                                     },
2021-07-20T04:17:54.3287936Z [cluster1]                                                     "numberUnavailable": {
2021-07-20T04:17:54.3289079Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-20T04:17:54.3290828Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3291577Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3292390Z [cluster1]                                                     },
2021-07-20T04:17:54.3292981Z [cluster1]                                                     "observedGeneration": {
2021-07-20T04:17:54.3293838Z [cluster1]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-20T04:17:54.3294624Z [cluster1]                                                         "format": "int64",
2021-07-20T04:17:54.3295336Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3295865Z [cluster1]                                                     },
2021-07-20T04:17:54.3296924Z [cluster1]                                                     "updatedNumberScheduled": {
2021-07-20T04:17:54.3297963Z [cluster1]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-20T04:17:54.3299238Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3299818Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3300346Z [cluster1]                                                     }
2021-07-20T04:17:54.3300796Z [cluster1]                                                 },
2021-07-20T04:17:54.3301298Z [cluster1]                                                 "required": [
2021-07-20T04:17:54.3301980Z [cluster1]                                                     "currentNumberScheduled",
2021-07-20T04:17:54.3302979Z [cluster1]                                                     "desiredNumberScheduled",
2021-07-20T04:17:54.3303759Z [cluster1]                                                     "numberMisscheduled",
2021-07-20T04:17:54.3304395Z [cluster1]                                                     "numberReady"
2021-07-20T04:17:54.3305100Z [cluster1]                                                 ],
2021-07-20T04:17:54.3305772Z [cluster1]                                                 "type": "object"
2021-07-20T04:17:54.3307207Z [cluster1]                                             }
2021-07-20T04:17:54.3308093Z [cluster1]                                         },
2021-07-20T04:17:54.3308569Z [cluster1]                                         "required": [
2021-07-20T04:17:54.3309276Z [cluster1]                                             "mismatchedContainerImages"
2021-07-20T04:17:54.3310151Z [cluster1]                                         ],
2021-07-20T04:17:54.3310655Z [cluster1]                                         "type": "object"
2021-07-20T04:17:54.3311140Z [cluster1]                                     },
2021-07-20T04:17:54.3311623Z [cluster1]                                     "gateways": {
2021-07-20T04:17:54.3312116Z [cluster1]                                         "items": {
2021-07-20T04:17:54.3312647Z [cluster1]                                             "properties": {
2021-07-20T04:17:54.3313218Z [cluster1]                                                 "connections": {
2021-07-20T04:17:54.3313756Z [cluster1]                                                     "items": {
2021-07-20T04:17:54.3314308Z [cluster1]                                                         "properties": {
2021-07-20T04:17:54.3315119Z [cluster1]                                                             "endpoint": {
2021-07-20T04:17:54.3315721Z [cluster1]                                                                 "properties": {
2021-07-20T04:17:54.3316343Z [cluster1]                                                                     "backend": {
2021-07-20T04:17:54.3317039Z [cluster1]                                                                         "type": "string"
2021-07-20T04:17:54.3317581Z [cluster1]                                                                     },
2021-07-20T04:17:54.3318115Z [cluster1]                                                                     "backend_config": {
2021-07-20T04:17:54.3318818Z [cluster1]                                                                         "additionalProperties": {
2021-07-20T04:17:54.3319499Z [cluster1]                                                                             "type": "string"
2021-07-20T04:17:54.3320041Z [cluster1]                                                                         },
2021-07-20T04:17:54.3320585Z [cluster1]                                                                         "type": "object"
2021-07-20T04:17:54.3321105Z [cluster1]                                                                     },
2021-07-20T04:17:54.3321635Z [cluster1]                                                                     "cable_name": {
2021-07-20T04:17:54.3322197Z [cluster1]                                                                         "type": "string"
2021-07-20T04:17:54.3323016Z [cluster1]                                                                     },
2021-07-20T04:17:54.3323531Z [cluster1]                                                                     "cluster_id": {
2021-07-20T04:17:54.3324117Z [cluster1]                                                                         "type": "string"
2021-07-20T04:17:54.3324651Z [cluster1]                                                                     },
2021-07-20T04:17:54.3325208Z [cluster1]                                                                     "healthCheckIP": {
2021-07-20T04:17:54.3326078Z [cluster1]                                                                         "type": "string"
2021-07-20T04:17:54.3326603Z [cluster1]                                                                     },
2021-07-20T04:17:54.3327134Z [cluster1]                                                                     "hostname": {
2021-07-20T04:17:54.3327710Z [cluster1]                                                                         "type": "string"
2021-07-20T04:17:54.3328222Z [cluster1]                                                                     },
2021-07-20T04:17:54.3328843Z [cluster1]                                                                     "nat_enabled": {
2021-07-20T04:17:54.3329936Z [cluster1]                                                                         "type": "boolean"
2021-07-20T04:17:54.3330664Z [cluster1]                                                                     },
2021-07-20T04:17:54.3331175Z [cluster1]                                                                     "private_ip": {
2021-07-20T04:17:54.3331736Z [cluster1]                                                                         "type": "string"
2021-07-20T04:17:54.3332262Z [cluster1]                                                                     },
2021-07-20T04:17:54.3332746Z [cluster1]                                                                     "public_ip": {
2021-07-20T04:17:54.3333505Z [cluster1]                                                                         "type": "string"
2021-07-20T04:17:54.3334183Z [cluster1]                                                                     },
2021-07-20T04:17:54.3334711Z [cluster1]                                                                     "subnets": {
2021-07-20T04:17:54.3335435Z [cluster1]                                                                         "items": {
2021-07-20T04:17:54.3336151Z [cluster1]                                                                             "type": "string"
2021-07-20T04:17:54.3336892Z [cluster1]                                                                         },
2021-07-20T04:17:54.3337708Z [cluster1]                                                                         "type": "array"
2021-07-20T04:17:54.3338365Z [cluster1]                                                                     }
2021-07-20T04:17:54.3338819Z [cluster1]                                                                 },
2021-07-20T04:17:54.3339333Z [cluster1]                                                                 "required": [
2021-07-20T04:17:54.3340071Z [cluster1]                                                                     "backend",
2021-07-20T04:17:54.3340616Z [cluster1]                                                                     "cable_name",
2021-07-20T04:17:54.3341174Z [cluster1]                                                                     "cluster_id",
2021-07-20T04:17:54.3341719Z [cluster1]                                                                     "hostname",
2021-07-20T04:17:54.3342282Z [cluster1]                                                                     "nat_enabled",
2021-07-20T04:17:54.3342856Z [cluster1]                                                                     "private_ip",
2021-07-20T04:17:54.3343401Z [cluster1]                                                                     "public_ip",
2021-07-20T04:17:54.3344342Z [cluster1]                                                                     "subnets"
2021-07-20T04:17:54.3344853Z [cluster1]                                                                 ],
2021-07-20T04:17:54.3345565Z [cluster1]                                                                 "type": "object"
2021-07-20T04:17:54.3346465Z [cluster1]                                                             },
2021-07-20T04:17:54.3347041Z [cluster1]                                                             "latencyRTT": {
2021-07-20T04:17:54.3348084Z [cluster1]                                                                 "description": "LatencySpec describes the round trip time information for a packet between the gateway pods of two clusters.",
2021-07-20T04:17:54.3349447Z [cluster1]                                                                 "properties": {
2021-07-20T04:17:54.3351054Z [cluster1]                                                                     "average": {
2021-07-20T04:17:54.3351695Z [cluster1]                                                                         "type": "string"
2021-07-20T04:17:54.3352251Z [cluster1]                                                                     },
2021-07-20T04:17:54.3352946Z [cluster1]                                                                     "last": {
2021-07-20T04:17:54.3354785Z [cluster1]                                                                         "type": "string"
2021-07-20T04:17:54.3355344Z [cluster1]                                                                     },
2021-07-20T04:17:54.3355829Z [cluster1]                                                                     "max": {
2021-07-20T04:17:54.3356377Z [cluster1]                                                                         "type": "string"
2021-07-20T04:17:54.3356907Z [cluster1]                                                                     },
2021-07-20T04:17:54.3357412Z [cluster1]                                                                     "min": {
2021-07-20T04:17:54.3357964Z [cluster1]                                                                         "type": "string"
2021-07-20T04:17:54.3358653Z [cluster1]                                                                     },
2021-07-20T04:17:54.3359190Z [cluster1]                                                                     "stdDev": {
2021-07-20T04:17:54.3359754Z [cluster1]                                                                         "type": "string"
2021-07-20T04:17:54.3360312Z [cluster1]                                                                     }
2021-07-20T04:17:54.3360802Z [cluster1]                                                                 },
2021-07-20T04:17:54.3361495Z [cluster1]                                                                 "type": "object"
2021-07-20T04:17:54.3362041Z [cluster1]                                                             },
2021-07-20T04:17:54.3362551Z [cluster1]                                                             "status": {
2021-07-20T04:17:54.3363121Z [cluster1]                                                                 "type": "string"
2021-07-20T04:17:54.3363655Z [cluster1]                                                             },
2021-07-20T04:17:54.3365200Z [cluster1]                                                             "statusMessage": {
2021-07-20T04:17:54.3365869Z [cluster1]                                                                 "type": "string"
2021-07-20T04:17:54.3366399Z [cluster1]                                                             },
2021-07-20T04:17:54.3366934Z [cluster1]                                                             "usingIP": {
2021-07-20T04:17:54.3367760Z [cluster1]                                                                 "type": "string"
2021-07-20T04:17:54.3368329Z [cluster1]                                                             },
2021-07-20T04:17:54.3368851Z [cluster1]                                                             "usingNAT": {
2021-07-20T04:17:54.3369440Z [cluster1]                                                                 "type": "boolean"
2021-07-20T04:17:54.3370513Z [cluster1]                                                             }
2021-07-20T04:17:54.3370977Z [cluster1]                                                         },
2021-07-20T04:17:54.3371836Z [cluster1]                                                         "required": [
2021-07-20T04:17:54.3372590Z [cluster1]                                                             "endpoint",
2021-07-20T04:17:54.3373536Z [cluster1]                                                             "status",
2021-07-20T04:17:54.3374121Z [cluster1]                                                             "statusMessage"
2021-07-20T04:17:54.3374701Z [cluster1]                                                         ],
2021-07-20T04:17:54.3375401Z [cluster1]                                                         "type": "object"
2021-07-20T04:17:54.3376147Z [cluster1]                                                     },
2021-07-20T04:17:54.3376657Z [cluster1]                                                     "type": "array"
2021-07-20T04:17:54.3377764Z [cluster1]                                                 },
2021-07-20T04:17:54.3378587Z [cluster1]                                                 "haStatus": {
2021-07-20T04:17:54.3379126Z [cluster1]                                                     "type": "string"
2021-07-20T04:17:54.3379630Z [cluster1]                                                 },
2021-07-20T04:17:54.3380169Z [cluster1]                                                 "localEndpoint": {
2021-07-20T04:17:54.3380797Z [cluster1]                                                     "properties": {
2021-07-20T04:17:54.3381376Z [cluster1]                                                         "backend": {
2021-07-20T04:17:54.3381913Z [cluster1]                                                             "type": "string"
2021-07-20T04:17:54.3382733Z [cluster1]                                                         },
2021-07-20T04:17:54.3383252Z [cluster1]                                                         "backend_config": {
2021-07-20T04:17:54.3384235Z [cluster1]                                                             "additionalProperties": {
2021-07-20T04:17:54.3385113Z [cluster1]                                                                 "type": "string"
2021-07-20T04:17:54.3385646Z [cluster1]                                                             },
2021-07-20T04:17:54.3386190Z [cluster1]                                                             "type": "object"
2021-07-20T04:17:54.3386714Z [cluster1]                                                         },
2021-07-20T04:17:54.3387404Z [cluster1]                                                         "cable_name": {
2021-07-20T04:17:54.3388110Z [cluster1]                                                             "type": "string"
2021-07-20T04:17:54.3388813Z [cluster1]                                                         },
2021-07-20T04:17:54.3389340Z [cluster1]                                                         "cluster_id": {
2021-07-20T04:17:54.3389908Z [cluster1]                                                             "type": "string"
2021-07-20T04:17:54.3390446Z [cluster1]                                                         },
2021-07-20T04:17:54.3391009Z [cluster1]                                                         "healthCheckIP": {
2021-07-20T04:17:54.3391860Z [cluster1]                                                             "type": "string"
2021-07-20T04:17:54.3392521Z [cluster1]                                                         },
2021-07-20T04:17:54.3393041Z [cluster1]                                                         "hostname": {
2021-07-20T04:17:54.3393598Z [cluster1]                                                             "type": "string"
2021-07-20T04:17:54.3394104Z [cluster1]                                                         },
2021-07-20T04:17:54.3394702Z [cluster1]                                                         "nat_enabled": {
2021-07-20T04:17:54.3395265Z [cluster1]                                                             "type": "boolean"
2021-07-20T04:17:54.3395962Z [cluster1]                                                         },
2021-07-20T04:17:54.3396483Z [cluster1]                                                         "private_ip": {
2021-07-20T04:17:54.3397065Z [cluster1]                                                             "type": "string"
2021-07-20T04:17:54.3397597Z [cluster1]                                                         },
2021-07-20T04:17:54.3398110Z [cluster1]                                                         "public_ip": {
2021-07-20T04:17:54.3398689Z [cluster1]                                                             "type": "string"
2021-07-20T04:17:54.3399211Z [cluster1]                                                         },
2021-07-20T04:17:54.3399750Z [cluster1]                                                         "subnets": {
2021-07-20T04:17:54.3400282Z [cluster1]                                                             "items": {
2021-07-20T04:17:54.3400996Z [cluster1]                                                                 "type": "string"
2021-07-20T04:17:54.3401560Z [cluster1]                                                             },
2021-07-20T04:17:54.3402084Z [cluster1]                                                             "type": "array"
2021-07-20T04:17:54.3402626Z [cluster1]                                                         }
2021-07-20T04:17:54.3403097Z [cluster1]                                                     },
2021-07-20T04:17:54.3403633Z [cluster1]                                                     "required": [
2021-07-20T04:17:54.3404181Z [cluster1]                                                         "backend",
2021-07-20T04:17:54.3404744Z [cluster1]                                                         "cable_name",
2021-07-20T04:17:54.3405311Z [cluster1]                                                         "cluster_id",
2021-07-20T04:17:54.3405866Z [cluster1]                                                         "hostname",
2021-07-20T04:17:54.3406762Z [cluster1]                                                         "nat_enabled",
2021-07-20T04:17:54.3407330Z [cluster1]                                                         "private_ip",
2021-07-20T04:17:54.3407900Z [cluster1]                                                         "public_ip",
2021-07-20T04:17:54.3408441Z [cluster1]                                                         "subnets"
2021-07-20T04:17:54.3409336Z [cluster1]                                                     ],
2021-07-20T04:17:54.3410056Z [cluster1]                                                     "type": "object"
2021-07-20T04:17:54.3410887Z [cluster1]                                                 },
2021-07-20T04:17:54.3411445Z [cluster1]                                                 "statusFailure": {
2021-07-20T04:17:54.3412036Z [cluster1]                                                     "type": "string"
2021-07-20T04:17:54.3413814Z [cluster1]                                                 },
2021-07-20T04:17:54.3414895Z [cluster1]                                                 "version": {
2021-07-20T04:17:54.3415444Z [cluster1]                                                     "type": "string"
2021-07-20T04:17:54.3415952Z [cluster1]                                                 }
2021-07-20T04:17:54.3416398Z [cluster1]                                             },
2021-07-20T04:17:54.3418158Z [cluster1]                                             "required": [
2021-07-20T04:17:54.3418746Z [cluster1]                                                 "connections",
2021-07-20T04:17:54.3419325Z [cluster1]                                                 "haStatus",
2021-07-20T04:17:54.3419908Z [cluster1]                                                 "localEndpoint",
2021-07-20T04:17:54.3420548Z [cluster1]                                                 "statusFailure",
2021-07-20T04:17:54.3421130Z [cluster1]                                                 "version"
2021-07-20T04:17:54.3421640Z [cluster1]                                             ],
2021-07-20T04:17:54.3422156Z [cluster1]                                             "type": "object"
2021-07-20T04:17:54.3422668Z [cluster1]                                         },
2021-07-20T04:17:54.3423177Z [cluster1]                                         "type": "array"
2021-07-20T04:17:54.3423667Z [cluster1]                                     },
2021-07-20T04:17:54.3424190Z [cluster1]                                     "globalCIDR": {
2021-07-20T04:17:54.3424741Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3425249Z [cluster1]                                     },
2021-07-20T04:17:54.3425913Z [cluster1]                                     "globalnetDaemonSetStatus": {
2021-07-20T04:17:54.3426655Z [cluster1]                                         "properties": {
2021-07-20T04:17:54.3427480Z [cluster1]                                             "lastResourceVersion": {
2021-07-20T04:17:54.3428163Z [cluster1]                                                 "type": "string"
2021-07-20T04:17:54.3429218Z [cluster1]                                             },
2021-07-20T04:17:54.3429931Z [cluster1]                                             "mismatchedContainerImages": {
2021-07-20T04:17:54.3431442Z [cluster1]                                                 "type": "boolean"
2021-07-20T04:17:54.3431945Z [cluster1]                                             },
2021-07-20T04:17:54.3432608Z [cluster1]                                             "nonReadyContainerStates": {
2021-07-20T04:17:54.3433279Z [cluster1]                                                 "items": {
2021-07-20T04:17:54.3435357Z [cluster1]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-20T04:17:54.3436846Z [cluster1]                                                     "properties": {
2021-07-20T04:17:54.3437389Z [cluster1]                                                         "running": {
2021-07-20T04:17:54.3438104Z [cluster1]                                                             "description": "Details about a running container",
2021-07-20T04:17:54.3440678Z [cluster1]                                                             "properties": {
2021-07-20T04:17:54.3441718Z [cluster1]                                                                 "startedAt": {
2021-07-20T04:17:54.3443697Z [cluster1]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-20T04:17:54.3444911Z [cluster1]                                                                     "format": "date-time",
2021-07-20T04:17:54.3445895Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3446635Z [cluster1]                                                                 }
2021-07-20T04:17:54.3447275Z [cluster1]                                                             },
2021-07-20T04:17:54.3447830Z [cluster1]                                                             "type": "object"
2021-07-20T04:17:54.3448339Z [cluster1]                                                         },
2021-07-20T04:17:54.3448865Z [cluster1]                                                         "terminated": {
2021-07-20T04:17:54.3449770Z [cluster1]                                                             "description": "Details about a terminated container",
2021-07-20T04:17:54.3451020Z [cluster1]                                                             "properties": {
2021-07-20T04:17:54.3454835Z [cluster1]                                                                 "containerID": {
2021-07-20T04:17:54.3456147Z [cluster1]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-20T04:17:54.3457250Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3457812Z [cluster1]                                                                 },
2021-07-20T04:17:54.3458405Z [cluster1]                                                                 "exitCode": {
2021-07-20T04:17:54.3459279Z [cluster1]                                                                     "description": "Exit status from the last termination of the container",
2021-07-20T04:17:54.3460082Z [cluster1]                                                                     "format": "int32",
2021-07-20T04:17:54.3460690Z [cluster1]                                                                     "type": "integer"
2021-07-20T04:17:54.3461237Z [cluster1]                                                                 },
2021-07-20T04:17:54.3462190Z [cluster1]                                                                 "finishedAt": {
2021-07-20T04:17:54.3463009Z [cluster1]                                                                     "description": "Time at which the container last terminated",
2021-07-20T04:17:54.3464118Z [cluster1]                                                                     "format": "date-time",
2021-07-20T04:17:54.3464807Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3465684Z [cluster1]                                                                 },
2021-07-20T04:17:54.3466229Z [cluster1]                                                                 "message": {
2021-07-20T04:17:54.3467043Z [cluster1]                                                                     "description": "Message regarding the last termination of the container",
2021-07-20T04:17:54.3467890Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3468446Z [cluster1]                                                                 },
2021-07-20T04:17:54.3468961Z [cluster1]                                                                 "reason": {
2021-07-20T04:17:54.3469762Z [cluster1]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-20T04:17:54.3470865Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3471579Z [cluster1]                                                                 },
2021-07-20T04:17:54.3472118Z [cluster1]                                                                 "signal": {
2021-07-20T04:17:54.3472879Z [cluster1]                                                                     "description": "Signal from the last termination of the container",
2021-07-20T04:17:54.3473667Z [cluster1]                                                                     "format": "int32",
2021-07-20T04:17:54.3474256Z [cluster1]                                                                     "type": "integer"
2021-07-20T04:17:54.3474816Z [cluster1]                                                                 },
2021-07-20T04:17:54.3475376Z [cluster1]                                                                 "startedAt": {
2021-07-20T04:17:54.3476334Z [cluster1]                                                                     "description": "Time at which previous execution of the container started",
2021-07-20T04:17:54.3477699Z [cluster1]                                                                     "format": "date-time",
2021-07-20T04:17:54.3478320Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3478867Z [cluster1]                                                                 }
2021-07-20T04:17:54.3479360Z [cluster1]                                                             },
2021-07-20T04:17:54.3480135Z [cluster1]                                                             "required": [
2021-07-20T04:17:54.3480741Z [cluster1]                                                                 "exitCode"
2021-07-20T04:17:54.3481679Z [cluster1]                                                             ],
2021-07-20T04:17:54.3482235Z [cluster1]                                                             "type": "object"
2021-07-20T04:17:54.3483074Z [cluster1]                                                         },
2021-07-20T04:17:54.3483675Z [cluster1]                                                         "waiting": {
2021-07-20T04:17:54.3484381Z [cluster1]                                                             "description": "Details about a waiting container",
2021-07-20T04:17:54.3485131Z [cluster1]                                                             "properties": {
2021-07-20T04:17:54.3485853Z [cluster1]                                                                 "message": {
2021-07-20T04:17:54.3486981Z [cluster1]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-20T04:17:54.3489215Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3489757Z [cluster1]                                                                 },
2021-07-20T04:17:54.3490288Z [cluster1]                                                                 "reason": {
2021-07-20T04:17:54.3491063Z [cluster1]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-20T04:17:54.3492349Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3492877Z [cluster1]                                                                 }
2021-07-20T04:17:54.3493334Z [cluster1]                                                             },
2021-07-20T04:17:54.3493842Z [cluster1]                                                             "type": "object"
2021-07-20T04:17:54.3495042Z [cluster1]                                                         }
2021-07-20T04:17:54.3495498Z [cluster1]                                                     },
2021-07-20T04:17:54.3496241Z [cluster1]                                                     "type": "object"
2021-07-20T04:17:54.3496873Z [cluster1]                                                 },
2021-07-20T04:17:54.3498227Z [cluster1]                                                 "type": "array"
2021-07-20T04:17:54.3498891Z [cluster1]                                             },
2021-07-20T04:17:54.3499375Z [cluster1]                                             "status": {
2021-07-20T04:17:54.3500179Z [cluster1]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-20T04:17:54.3501021Z [cluster1]                                                 "properties": {
2021-07-20T04:17:54.3501626Z [cluster1]                                                     "collisionCount": {
2021-07-20T04:17:54.3503190Z [cluster1]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-20T04:17:54.3504514Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3505074Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3505780Z [cluster1]                                                     },
2021-07-20T04:17:54.3506317Z [cluster1]                                                     "conditions": {
2021-07-20T04:17:54.3508018Z [cluster1]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-20T04:17:54.3508888Z [cluster1]                                                         "items": {
2021-07-20T04:17:54.3509750Z [cluster1]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-20T04:17:54.3510660Z [cluster1]                                                             "properties": {
2021-07-20T04:17:54.3511853Z [cluster1]                                                                 "lastTransitionTime": {
2021-07-20T04:17:54.3512951Z [cluster1]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-20T04:17:54.3514126Z [cluster1]                                                                     "format": "date-time",
2021-07-20T04:17:54.3515162Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3515744Z [cluster1]                                                                 },
2021-07-20T04:17:54.3516267Z [cluster1]                                                                 "message": {
2021-07-20T04:17:54.3517081Z [cluster1]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-20T04:17:54.3518186Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3518705Z [cluster1]                                                                 },
2021-07-20T04:17:54.3519217Z [cluster1]                                                                 "reason": {
2021-07-20T04:17:54.3520329Z [cluster1]                                                                     "description": "The reason for the condition's last transition.",
2021-07-20T04:17:54.3521079Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3521611Z [cluster1]                                                                 },
2021-07-20T04:17:54.3522272Z [cluster1]                                                                 "status": {
2021-07-20T04:17:54.3523214Z [cluster1]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-20T04:17:54.3525310Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3526168Z [cluster1]                                                                 },
2021-07-20T04:17:54.3526679Z [cluster1]                                                                 "type": {
2021-07-20T04:17:54.3527361Z [cluster1]                                                                     "description": "Type of DaemonSet condition.",
2021-07-20T04:17:54.3528293Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3529003Z [cluster1]                                                                 }
2021-07-20T04:17:54.3529506Z [cluster1]                                                             },
2021-07-20T04:17:54.3530054Z [cluster1]                                                             "required": [
2021-07-20T04:17:54.3530604Z [cluster1]                                                                 "status",
2021-07-20T04:17:54.3531160Z [cluster1]                                                                 "type"
2021-07-20T04:17:54.3531663Z [cluster1]                                                             ],
2021-07-20T04:17:54.3532214Z [cluster1]                                                             "type": "object"
2021-07-20T04:17:54.3532737Z [cluster1]                                                         },
2021-07-20T04:17:54.3533276Z [cluster1]                                                         "type": "array"
2021-07-20T04:17:54.3533804Z [cluster1]                                                     },
2021-07-20T04:17:54.3534456Z [cluster1]                                                     "currentNumberScheduled": {
2021-07-20T04:17:54.3536246Z [cluster1]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.3537965Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3538693Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3539219Z [cluster1]                                                     },
2021-07-20T04:17:54.3539978Z [cluster1]                                                     "desiredNumberScheduled": {
2021-07-20T04:17:54.3541472Z [cluster1]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.3542782Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3543354Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3543885Z [cluster1]                                                     },
2021-07-20T04:17:54.3544609Z [cluster1]                                                     "numberAvailable": {
2021-07-20T04:17:54.3545859Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-20T04:17:54.3547636Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3548243Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3548767Z [cluster1]                                                     },
2021-07-20T04:17:54.3549341Z [cluster1]                                                     "numberMisscheduled": {
2021-07-20T04:17:54.3550680Z [cluster1]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.3552608Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3553636Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3554210Z [cluster1]                                                     },
2021-07-20T04:17:54.3554767Z [cluster1]                                                     "numberReady": {
2021-07-20T04:17:54.3555734Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-20T04:17:54.3556698Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3557455Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3557968Z [cluster1]                                                     },
2021-07-20T04:17:54.3558858Z [cluster1]                                                     "numberUnavailable": {
2021-07-20T04:17:54.3560078Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-20T04:17:54.3561202Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3561962Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3562489Z [cluster1]                                                     },
2021-07-20T04:17:54.3563598Z [cluster1]                                                     "observedGeneration": {
2021-07-20T04:17:54.3565286Z [cluster1]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-20T04:17:54.3566134Z [cluster1]                                                         "format": "int64",
2021-07-20T04:17:54.3566878Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3567409Z [cluster1]                                                     },
2021-07-20T04:17:54.3568382Z [cluster1]                                                     "updatedNumberScheduled": {
2021-07-20T04:17:54.3569705Z [cluster1]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-20T04:17:54.3570513Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3571070Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3571607Z [cluster1]                                                     }
2021-07-20T04:17:54.3572075Z [cluster1]                                                 },
2021-07-20T04:17:54.3572568Z [cluster1]                                                 "required": [
2021-07-20T04:17:54.3573918Z [cluster1]                                                     "currentNumberScheduled",
2021-07-20T04:17:54.3574747Z [cluster1]                                                     "desiredNumberScheduled",
2021-07-20T04:17:54.3576393Z [cluster1]                                                     "numberMisscheduled",
2021-07-20T04:17:54.3577254Z [cluster1]                                                     "numberReady"
2021-07-20T04:17:54.3577805Z [cluster1]                                                 ],
2021-07-20T04:17:54.3578321Z [cluster1]                                                 "type": "object"
2021-07-20T04:17:54.3578812Z [cluster1]                                             }
2021-07-20T04:17:54.3579437Z [cluster1]                                         },
2021-07-20T04:17:54.3580339Z [cluster1]                                         "required": [
2021-07-20T04:17:54.3581066Z [cluster1]                                             "mismatchedContainerImages"
2021-07-20T04:17:54.3581718Z [cluster1]                                         ],
2021-07-20T04:17:54.3582223Z [cluster1]                                         "type": "object"
2021-07-20T04:17:54.3582710Z [cluster1]                                     },
2021-07-20T04:17:54.3583284Z [cluster1]                                     "loadBalancerStatus": {
2021-07-20T04:17:54.3583919Z [cluster1]                                         "properties": {
2021-07-20T04:17:54.3584445Z [cluster1]                                             "status": {
2021-07-20T04:17:54.3585721Z [cluster1]                                                 "description": "LoadBalancerStatus represents the status of a load-balancer.",
2021-07-20T04:17:54.3587672Z [cluster1]                                                 "properties": {
2021-07-20T04:17:54.3588240Z [cluster1]                                                     "ingress": {
2021-07-20T04:17:54.3589906Z [cluster1]                                                         "description": "Ingress is a list containing ingress points for the load-balancer. Traffic intended for the service should be sent to these ingress points.",
2021-07-20T04:17:54.3592159Z [cluster1]                                                         "items": {
2021-07-20T04:17:54.3594201Z [cluster1]                                                             "description": "LoadBalancerIngress represents the status of a load-balancer ingress point: traffic intended for the service should be sent to an ingress point.",
2021-07-20T04:17:54.3595347Z [cluster1]                                                             "properties": {
2021-07-20T04:17:54.3595909Z [cluster1]                                                                 "hostname": {
2021-07-20T04:17:54.3597612Z [cluster1]                                                                     "description": "Hostname is set for load-balancer ingress points that are DNS based (typically AWS load-balancers)",
2021-07-20T04:17:54.3598571Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3599107Z [cluster1]                                                                 },
2021-07-20T04:17:54.3599925Z [cluster1]                                                                 "ip": {
2021-07-20T04:17:54.3601338Z [cluster1]                                                                     "description": "IP is set for load-balancer ingress points that are IP based (typically GCE or OpenStack load-balancers)",
2021-07-20T04:17:54.3602683Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3603373Z [cluster1]                                                                 }
2021-07-20T04:17:54.3604096Z [cluster1]                                                             },
2021-07-20T04:17:54.3604631Z [cluster1]                                                             "type": "object"
2021-07-20T04:17:54.3605498Z [cluster1]                                                         },
2021-07-20T04:17:54.3606009Z [cluster1]                                                         "type": "array"
2021-07-20T04:17:54.3606531Z [cluster1]                                                     }
2021-07-20T04:17:54.3607179Z [cluster1]                                                 },
2021-07-20T04:17:54.3607679Z [cluster1]                                                 "type": "object"
2021-07-20T04:17:54.3608531Z [cluster1]                                             }
2021-07-20T04:17:54.3608969Z [cluster1]                                         },
2021-07-20T04:17:54.3609476Z [cluster1]                                         "type": "object"
2021-07-20T04:17:54.3610134Z [cluster1]                                     },
2021-07-20T04:17:54.3610640Z [cluster1]                                     "natEnabled": {
2021-07-20T04:17:54.3611527Z [cluster1]                                         "type": "boolean"
2021-07-20T04:17:54.3612011Z [cluster1]                                     },
2021-07-20T04:17:54.3612540Z [cluster1]                                     "networkPlugin": {
2021-07-20T04:17:54.3613101Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3613589Z [cluster1]                                     },
2021-07-20T04:17:54.3614245Z [cluster1]                                     "routeAgentDaemonSetStatus": {
2021-07-20T04:17:54.3614973Z [cluster1]                                         "properties": {
2021-07-20T04:17:54.3615866Z [cluster1]                                             "lastResourceVersion": {
2021-07-20T04:17:54.3616538Z [cluster1]                                                 "type": "string"
2021-07-20T04:17:54.3617377Z [cluster1]                                             },
2021-07-20T04:17:54.3618077Z [cluster1]                                             "mismatchedContainerImages": {
2021-07-20T04:17:54.3618838Z [cluster1]                                                 "type": "boolean"
2021-07-20T04:17:54.3619360Z [cluster1]                                             },
2021-07-20T04:17:54.3620040Z [cluster1]                                             "nonReadyContainerStates": {
2021-07-20T04:17:54.3620728Z [cluster1]                                                 "items": {
2021-07-20T04:17:54.3622007Z [cluster1]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-20T04:17:54.3623323Z [cluster1]                                                     "properties": {
2021-07-20T04:17:54.3623896Z [cluster1]                                                         "running": {
2021-07-20T04:17:54.3624609Z [cluster1]                                                             "description": "Details about a running container",
2021-07-20T04:17:54.3625344Z [cluster1]                                                             "properties": {
2021-07-20T04:17:54.3625947Z [cluster1]                                                                 "startedAt": {
2021-07-20T04:17:54.3627259Z [cluster1]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-20T04:17:54.3628353Z [cluster1]                                                                     "format": "date-time",
2021-07-20T04:17:54.3628986Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3629524Z [cluster1]                                                                 }
2021-07-20T04:17:54.3630033Z [cluster1]                                                             },
2021-07-20T04:17:54.3630574Z [cluster1]                                                             "type": "object"
2021-07-20T04:17:54.3631109Z [cluster1]                                                         },
2021-07-20T04:17:54.3631661Z [cluster1]                                                         "terminated": {
2021-07-20T04:17:54.3632558Z [cluster1]                                                             "description": "Details about a terminated container",
2021-07-20T04:17:54.3633303Z [cluster1]                                                             "properties": {
2021-07-20T04:17:54.3633890Z [cluster1]                                                                 "containerID": {
2021-07-20T04:17:54.3635030Z [cluster1]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-20T04:17:54.3635969Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3636490Z [cluster1]                                                                 },
2021-07-20T04:17:54.3637022Z [cluster1]                                                                 "exitCode": {
2021-07-20T04:17:54.3637796Z [cluster1]                                                                     "description": "Exit status from the last termination of the container",
2021-07-20T04:17:54.3638577Z [cluster1]                                                                     "format": "int32",
2021-07-20T04:17:54.3639167Z [cluster1]                                                                     "type": "integer"
2021-07-20T04:17:54.3639699Z [cluster1]                                                                 },
2021-07-20T04:17:54.3640503Z [cluster1]                                                                 "finishedAt": {
2021-07-20T04:17:54.3641279Z [cluster1]                                                                     "description": "Time at which the container last terminated",
2021-07-20T04:17:54.3642370Z [cluster1]                                                                     "format": "date-time",
2021-07-20T04:17:54.3643004Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3643713Z [cluster1]                                                                 },
2021-07-20T04:17:54.3644238Z [cluster1]                                                                 "message": {
2021-07-20T04:17:54.3645226Z [cluster1]                                                                     "description": "Message regarding the last termination of the container",
2021-07-20T04:17:54.3646061Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3646595Z [cluster1]                                                                 },
2021-07-20T04:17:54.3647132Z [cluster1]                                                                 "reason": {
2021-07-20T04:17:54.3648078Z [cluster1]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-20T04:17:54.3648844Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3649479Z [cluster1]                                                                 },
2021-07-20T04:17:54.3649991Z [cluster1]                                                                 "signal": {
2021-07-20T04:17:54.3651151Z [cluster1]                                                                     "description": "Signal from the last termination of the container",
2021-07-20T04:17:54.3652130Z [cluster1]                                                                     "format": "int32",
2021-07-20T04:17:54.3653124Z [cluster1]                                                                     "type": "integer"
2021-07-20T04:17:54.3655965Z [cluster1]                                                                 },
2021-07-20T04:17:54.3657524Z [cluster1]                                                                 "startedAt": {
2021-07-20T04:17:54.3658381Z [cluster1]                                                                     "description": "Time at which previous execution of the container started",
2021-07-20T04:17:54.3660213Z [cluster1]                                                                     "format": "date-time",
2021-07-20T04:17:54.3661158Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3661729Z [cluster1]                                                                 }
2021-07-20T04:17:54.3663938Z [cluster1]                                                             },
2021-07-20T04:17:54.3664827Z [cluster1]                                                             "required": [
2021-07-20T04:17:54.3666536Z [cluster1]                                                                 "exitCode"
2021-07-20T04:17:54.3667102Z [cluster1]                                                             ],
2021-07-20T04:17:54.3667874Z [cluster1]                                                             "type": "object"
2021-07-20T04:17:54.3668421Z [cluster1]                                                         },
2021-07-20T04:17:54.3669036Z [cluster1]                                                         "waiting": {
2021-07-20T04:17:54.3669721Z [cluster1]                                                             "description": "Details about a waiting container",
2021-07-20T04:17:54.3670441Z [cluster1]                                                             "properties": {
2021-07-20T04:17:54.3671004Z [cluster1]                                                                 "message": {
2021-07-20T04:17:54.3672285Z [cluster1]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-20T04:17:54.3673586Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3674296Z [cluster1]                                                                 },
2021-07-20T04:17:54.3674825Z [cluster1]                                                                 "reason": {
2021-07-20T04:17:54.3675832Z [cluster1]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-20T04:17:54.3676952Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3677657Z [cluster1]                                                                 }
2021-07-20T04:17:54.3678125Z [cluster1]                                                             },
2021-07-20T04:17:54.3678649Z [cluster1]                                                             "type": "object"
2021-07-20T04:17:54.3679167Z [cluster1]                                                         }
2021-07-20T04:17:54.3679860Z [cluster1]                                                     },
2021-07-20T04:17:54.3680387Z [cluster1]                                                     "type": "object"
2021-07-20T04:17:54.3680917Z [cluster1]                                                 },
2021-07-20T04:17:54.3681582Z [cluster1]                                                 "type": "array"
2021-07-20T04:17:54.3682411Z [cluster1]                                             },
2021-07-20T04:17:54.3682923Z [cluster1]                                             "status": {
2021-07-20T04:17:54.3683749Z [cluster1]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-20T04:17:54.3684631Z [cluster1]                                                 "properties": {
2021-07-20T04:17:54.3685254Z [cluster1]                                                     "collisionCount": {
2021-07-20T04:17:54.3687162Z [cluster1]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-20T04:17:54.3688738Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3689480Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3690190Z [cluster1]                                                     },
2021-07-20T04:17:54.3690713Z [cluster1]                                                     "conditions": {
2021-07-20T04:17:54.3692077Z [cluster1]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-20T04:17:54.3693443Z [cluster1]                                                         "items": {
2021-07-20T04:17:54.3694481Z [cluster1]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-20T04:17:54.3695592Z [cluster1]                                                             "properties": {
2021-07-20T04:17:54.3696430Z [cluster1]                                                                 "lastTransitionTime": {
2021-07-20T04:17:54.3697667Z [cluster1]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-20T04:17:54.3699009Z [cluster1]                                                                     "format": "date-time",
2021-07-20T04:17:54.3699786Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3700354Z [cluster1]                                                                 },
2021-07-20T04:17:54.3700883Z [cluster1]                                                                 "message": {
2021-07-20T04:17:54.3701744Z [cluster1]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-20T04:17:54.3702613Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3703153Z [cluster1]                                                                 },
2021-07-20T04:17:54.3703686Z [cluster1]                                                                 "reason": {
2021-07-20T04:17:54.3704744Z [cluster1]                                                                     "description": "The reason for the condition's last transition.",
2021-07-20T04:17:54.3705535Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3706097Z [cluster1]                                                                 },
2021-07-20T04:17:54.3706609Z [cluster1]                                                                 "status": {
2021-07-20T04:17:54.3707399Z [cluster1]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-20T04:17:54.3708314Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3708884Z [cluster1]                                                                 },
2021-07-20T04:17:54.3709404Z [cluster1]                                                                 "type": {
2021-07-20T04:17:54.3710268Z [cluster1]                                                                     "description": "Type of DaemonSet condition.",
2021-07-20T04:17:54.3710979Z [cluster1]                                                                     "type": "string"
2021-07-20T04:17:54.3711500Z [cluster1]                                                                 }
2021-07-20T04:17:54.3711981Z [cluster1]                                                             },
2021-07-20T04:17:54.3712484Z [cluster1]                                                             "required": [
2021-07-20T04:17:54.3713033Z [cluster1]                                                                 "status",
2021-07-20T04:17:54.3713566Z [cluster1]                                                                 "type"
2021-07-20T04:17:54.3714045Z [cluster1]                                                             ],
2021-07-20T04:17:54.3714578Z [cluster1]                                                             "type": "object"
2021-07-20T04:17:54.3715088Z [cluster1]                                                         },
2021-07-20T04:17:54.3715610Z [cluster1]                                                         "type": "array"
2021-07-20T04:17:54.3716209Z [cluster1]                                                     },
2021-07-20T04:17:54.3716855Z [cluster1]                                                     "currentNumberScheduled": {
2021-07-20T04:17:54.3718272Z [cluster1]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.3719702Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3720295Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3720829Z [cluster1]                                                     },
2021-07-20T04:17:54.3721501Z [cluster1]                                                     "desiredNumberScheduled": {
2021-07-20T04:17:54.3723156Z [cluster1]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.3724480Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3725059Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3725573Z [cluster1]                                                     },
2021-07-20T04:17:54.3726450Z [cluster1]                                                     "numberAvailable": {
2021-07-20T04:17:54.3727660Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-20T04:17:54.3728784Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3729366Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3729873Z [cluster1]                                                     },
2021-07-20T04:17:54.3730661Z [cluster1]                                                     "numberMisscheduled": {
2021-07-20T04:17:54.3732162Z [cluster1]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.3734341Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3735245Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3735740Z [cluster1]                                                     },
2021-07-20T04:17:54.3736261Z [cluster1]                                                     "numberReady": {
2021-07-20T04:17:54.3737515Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-20T04:17:54.3738402Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3739137Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3739662Z [cluster1]                                                     },
2021-07-20T04:17:54.3740246Z [cluster1]                                                     "numberUnavailable": {
2021-07-20T04:17:54.3761508Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-20T04:17:54.3762826Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3763795Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3764452Z [cluster1]                                                     },
2021-07-20T04:17:54.3765074Z [cluster1]                                                     "observedGeneration": {
2021-07-20T04:17:54.3766101Z [cluster1]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-20T04:17:54.3767110Z [cluster1]                                                         "format": "int64",
2021-07-20T04:17:54.3768017Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3768530Z [cluster1]                                                     },
2021-07-20T04:17:54.3769270Z [cluster1]                                                     "updatedNumberScheduled": {
2021-07-20T04:17:54.3770183Z [cluster1]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-20T04:17:54.3770962Z [cluster1]                                                         "format": "int32",
2021-07-20T04:17:54.3771536Z [cluster1]                                                         "type": "integer"
2021-07-20T04:17:54.3772050Z [cluster1]                                                     }
2021-07-20T04:17:54.3772520Z [cluster1]                                                 },
2021-07-20T04:17:54.3773014Z [cluster1]                                                 "required": [
2021-07-20T04:17:54.3773698Z [cluster1]                                                     "currentNumberScheduled",
2021-07-20T04:17:54.3774509Z [cluster1]                                                     "desiredNumberScheduled",
2021-07-20T04:17:54.3775465Z [cluster1]                                                     "numberMisscheduled",
2021-07-20T04:17:54.3776150Z [cluster1]                                                     "numberReady"
2021-07-20T04:17:54.3776679Z [cluster1]                                                 ],
2021-07-20T04:17:54.3779332Z [cluster1]                                                 "type": "object"
2021-07-20T04:17:54.3779991Z [cluster1]                                             }
2021-07-20T04:17:54.3780447Z [cluster1]                                         },
2021-07-20T04:17:54.3781137Z [cluster1]                                         "required": [
2021-07-20T04:17:54.3781885Z [cluster1]                                             "mismatchedContainerImages"
2021-07-20T04:17:54.3782735Z [cluster1]                                         ],
2021-07-20T04:17:54.3783240Z [cluster1]                                         "type": "object"
2021-07-20T04:17:54.3783753Z [cluster1]                                     },
2021-07-20T04:17:54.3784265Z [cluster1]                                     "serviceCIDR": {
2021-07-20T04:17:54.3784847Z [cluster1]                                         "type": "string"
2021-07-20T04:17:54.3785342Z [cluster1]                                     }
2021-07-20T04:17:54.3785799Z [cluster1]                                 },
2021-07-20T04:17:54.3786283Z [cluster1]                                 "required": [
2021-07-20T04:17:54.3786834Z [cluster1]                                     "clusterID",
2021-07-20T04:17:54.3787807Z [cluster1]                                     "natEnabled"
2021-07-20T04:17:54.3788318Z [cluster1]                                 ],
2021-07-20T04:17:54.3788828Z [cluster1]                                 "type": "object"
2021-07-20T04:17:54.3789306Z [cluster1]                             }
2021-07-20T04:17:54.3789899Z [cluster1]                         },
2021-07-20T04:17:54.3790358Z [cluster1]                         "type": "object"
2021-07-20T04:17:54.3790830Z [cluster1]                     }
2021-07-20T04:17:54.3791225Z [cluster1]                 },
2021-07-20T04:17:54.3791829Z [cluster1]                 "served": true,
2021-07-20T04:17:54.3792327Z [cluster1]                 "storage": true,
2021-07-20T04:17:54.3792858Z [cluster1]                 "subresources": {
2021-07-20T04:17:54.3793370Z [cluster1]                     "status": {}
2021-07-20T04:17:54.3793791Z [cluster1]                 }
2021-07-20T04:17:54.3794198Z [cluster1]             }
2021-07-20T04:17:54.3794578Z [cluster1]         ]
2021-07-20T04:17:54.3794968Z [cluster1]     },
2021-07-20T04:17:54.3795374Z [cluster1]     "status": {
2021-07-20T04:17:54.3795879Z [cluster1]         "acceptedNames": {
2021-07-20T04:17:54.3796420Z [cluster1]             "kind": "Submariner",
2021-07-20T04:17:54.3797026Z [cluster1]             "listKind": "SubmarinerList",
2021-07-20T04:17:54.3797811Z [cluster1]             "plural": "submariners",
2021-07-20T04:17:54.3798842Z [cluster1]             "singular": "submariner"
2021-07-20T04:17:54.3799496Z [cluster1]         },
2021-07-20T04:17:54.3799974Z [cluster1]         "conditions": [
2021-07-20T04:17:54.3800440Z [cluster1]             {
2021-07-20T04:17:54.3801604Z [cluster1]                 "lastTransitionTime": "2021-07-20T04:13:01Z",
2021-07-20T04:17:54.5851766Z [cluster1]                 "message": "nr2]             },
2021-07-20T04:17:54.5916199Z [cluster2]             {
2021-07-20T04:17:54.5917476Z [cluster2]                 "lastTransitionTime": "2021-07-20T04:13:23Z",
2021-07-20T04:17:54.5918279Z [cluster2]                 "message": "the initial names have been accepted",
2021-07-20T04:17:54.5919046Z [cluster2]                 "reason": "InitialNamesAccepted",
2021-07-20T04:17:54.5919693Z [cluster2]                 "status": "True",
2021-07-20T04:17:54.5920232Z [cluster2]                 "type": "Established"
2021-07-20T04:17:54.5920698Z [cluster2]             }
2021-07-20T04:17:54.5921241Z [cluster2]         ],
2021-07-20T04:17:54.5921723Z [cluster2]         "storedVersions": [
2021-07-20T04:17:54.5922214Z [cluster2]             "v1"
2021-07-20T04:17:54.5922612Z [cluster2]         ]
2021-07-20T04:17:54.5923342Z [cluster2]     }
2021-07-20T04:17:54.5923958Z [cluster2] }
2021-07-20T04:17:54.5925237Z [36m[submariner-operator]$ [cluster2] validate_equals .metadata.name clusters.submariner.io[0m
2021-07-20T04:17:54.5926889Z [36m[submariner-operator]$ [cluster2] validate_equals .metadata.name clusters.submariner.io[0m
2021-07-20T04:17:54.5928685Z [36m[submariner-operator]$ [cluster2] jq -r -M .metadata.name /tmp/clusters.submariner.io.cluster2.json[0m
2021-07-20T04:17:54.5930452Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced[0m
2021-07-20T04:17:54.5931896Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced[0m
2021-07-20T04:17:54.5933434Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.scope /tmp/clusters.submariner.io.cluster2.json[0m
2021-07-20T04:17:54.5934954Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io[0m
2021-07-20T04:17:54.5936214Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io[0m
2021-07-20T04:17:54.5960156Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.group /tmp/clusters.submariner.io.cluster2.json[0m
2021-07-20T04:17:54.5961802Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.names.kind Cluster[0m
2021-07-20T04:17:54.5963022Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.names.kind Cluster[0m
2021-07-20T04:17:54.5964862Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.names.kind /tmp/clusters.submariner.io.cluster2.json[0m
2021-07-20T04:17:54.5966443Z [36m[submariner-operator]$ [cluster2] validate_equals .status.acceptedNames.kind Cluster[0m
2021-07-20T04:17:54.5968212Z [36m[submariner-operator]$ [cluster2] validate_equals .status.acceptedNames.kind Cluster[0m
2021-07-20T04:17:54.5970490Z [36m[submariner-operator]$ [cluster2] jq -r -M .status.acceptedNames.kind /tmp/clusters.submariner.io.cluster2.json[0m
2021-07-20T04:17:54.5972312Z [36m[submariner-operator]$ [cluster2] verify_subm_crd[0m
2021-07-20T04:17:54.5973635Z [36m[submariner-operator]$ [cluster2] verify_subm_crd[0m
2021-07-20T04:17:54.5974882Z [36m[submariner-operator]$ [cluster2] validate_crd submariners.submariner.io v1alpha1 Submariner[0m
2021-07-20T04:17:54.5977398Z [36m[submariner-operator]$ [cluster2] validate_crd submariners.submariner.io v1alpha1 Submariner[0m
2021-07-20T04:17:54.5979204Z [36m[submariner-operator]$ [cluster2] kubectl get crds submariners.submariner.io[0m
2021-07-20T04:17:54.5980690Z [36m[submariner-operator]$ [cluster2] kubectl get crds submariners.submariner.io[0m
2021-07-20T04:17:54.5982816Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crds submariners.submariner.io[0m
2021-07-20T04:17:54.5983908Z [cluster2] NAME                        CREATED AT
2021-07-20T04:17:54.5984730Z [cluster2] submariners.submariner.io   2021-07-20T04:13:12Z
2021-07-20T04:17:54.5986090Z [36m[submariner-operator]$ [cluster2] json_file=/tmp/submariners.submariner.io.cluster2.json[0m
2021-07-20T04:17:54.5988486Z [36m[submariner-operator]$ [cluster2] kubectl get crd submariners.submariner.io -o json[0m
2021-07-20T04:17:54.5990323Z [36m[submariner-operator]$ [cluster2] kubectl get crd submariners.submariner.io -o json[0m
2021-07-20T04:17:54.5991987Z [36m[submariner-operator]$ [cluster2] tee /tmp/submariners.submariner.io.cluster2.json[0m
2021-07-20T04:17:54.5993971Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crd submariners.submariner.io -o json[0m
2021-07-20T04:17:54.5995016Z [cluster2] {
2021-07-20T04:17:54.5995579Z [cluster2]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-20T04:17:54.5996790Z [cluster2]     "kind": "CustomResourceDefinition",
2021-07-20T04:17:54.5997401Z [cluster2]     "metadata": {
2021-07-20T04:17:54.5998061Z [cluster2]         "annotations": {
2021-07-20T04:17:54.5999167Z [cluster2]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-20T04:17:54.6000044Z [cluster2]         },
2021-07-20T04:17:54.6000793Z [cluster2]         "creationTimestamp": "2021-07-20T04:13:12Z",
2021-07-20T04:17:54.6001586Z [cluster2]         "generation": 2,
2021-07-20T04:17:54.6002137Z [cluster2]         "managedFields": [
2021-07-20T04:17:54.6002617Z [cluster2]             {
2021-07-20T04:17:54.6003406Z [cluster2]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-20T04:17:54.6004104Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-20T04:17:54.6005085Z [cluster2]                 "fieldsV1": {
2021-07-20T04:17:54.6005579Z [cluster2]                     "f:metadata": {
2021-07-20T04:17:54.6006091Z [cluster2]                         "f:annotations": {
2021-07-20T04:17:54.6006552Z [cluster2]                             ".": {},
2021-07-20T04:17:54.6007517Z [cluster2]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-20T04:17:54.6008208Z [cluster2]                         }
2021-07-20T04:17:54.6008765Z [cluster2]                     },
2021-07-20T04:17:54.6009205Z [cluster2]                     "f:spec": {
2021-07-20T04:17:54.6009754Z [cluster2]                         "f:conversion": {
2021-07-20T04:17:54.6010226Z [cluster2]                             ".": {},
2021-07-20T04:17:54.6010677Z [cluster2]                             "f:strategy": {}
2021-07-20T04:17:54.6011128Z [cluster2]                         },
2021-07-20T04:17:54.6011626Z [cluster2]                         "f:group": {},
2021-07-20T04:17:54.6012094Z [cluster2]                         "f:names": {
2021-07-20T04:17:54.6012539Z [cluster2]                             "f:kind": {},
2021-07-20T04:17:54.6013019Z [cluster2]                             "f:listKind": {},
2021-07-20T04:17:54.6013514Z [cluster2]                             "f:plural": {},
2021-07-20T04:17:54.6013986Z [cluster2]                             "f:singular": {}
2021-07-20T04:17:54.6014430Z [cluster2]                         },
2021-07-20T04:17:54.6015031Z [cluster2]                         "f:scope": {},
2021-07-20T04:17:54.6015815Z [cluster2]                         "f:versions": {}
2021-07-20T04:17:54.6016246Z [cluster2]                     },
2021-07-20T04:17:54.6016680Z [cluster2]                     "f:status": {
2021-07-20T04:17:54.6023235Z [cluster2]                         "f:acceptedNames": {
2021-07-20T04:17:54.6023819Z [cluster2]                             "f:kind": {},
2021-07-20T04:17:54.6024279Z [cluster2]                             "f:plural": {}
2021-07-20T04:17:54.6024727Z [cluster2]                         },
2021-07-20T04:17:54.6025349Z [cluster2]                         "f:conditions": {},
2021-07-20T04:17:54.6026016Z [cluster2]                         "f:storedVersions": {}
2021-07-20T04:17:54.6026494Z [cluster2]                     }
2021-07-20T04:17:54.6026863Z [cluster2]                 },
2021-07-20T04:17:54.6027304Z [cluster2]                 "manager": "subctl",
2021-07-20T04:17:54.6027808Z [cluster2]                 "operation": "Update",
2021-07-20T04:17:54.6028709Z [cluster2]                 "time": "2021-07-20T04:16:45Z"
2021-07-20T04:17:54.6029133Z [cluster2]             }
2021-07-20T04:17:54.6029702Z [cluster2]         ],
2021-07-20T04:17:54.6036015Z [cluster2]         "name": "submariners.submariner.io",
2021-07-20T04:17:54.6037035Z [cluster2]         "resourceVersion": "2125",
2021-07-20T04:17:54.6038201Z [cluster2]         "uid": "906e2794-fbc6-4e45-8501-c70941abaa5f"
2021-07-20T04:17:54.6039070Z [cluster2]     },
2021-07-20T04:17:54.6039777Z [cluster2]     "spec": {
2021-07-20T04:17:54.6040247Z [cluster2]         "conversion": {
2021-07-20T04:17:54.6040759Z [cluster2]             "strategy": "None"
2021-07-20T04:17:54.6041199Z [cluster2]         },
2021-07-20T04:17:54.6041694Z [cluster2]         "group": "submariner.io",
2021-07-20T04:17:54.6042195Z [cluster2]         "names": {
2021-07-20T04:17:54.6042688Z [cluster2]             "kind": "Submariner",
2021-07-20T04:17:54.6043495Z [cluster2]             "listKind": "SubmarinerList",
2021-07-20T04:17:54.6045048Z [cluster2]             "plural": "submariners",
2021-07-20T04:17:54.6045859Z [cluster2]             "singular": "submariner"
2021-07-20T04:17:54.6046334Z [cluster2]         },
2021-07-20T04:17:54.6048187Z [cluster2]         "scope": "Namespaced",
2021-07-20T04:17:54.6048817Z [cluster2]         "versions": [
2021-07-20T04:17:54.6049573Z [cluster2]             {
2021-07-20T04:17:54.6050002Z [cluster2]                 "name": "v1alpha1",
2021-07-20T04:17:54.6050469Z [cluster2]                 "schema": {
2021-07-20T04:17:54.6051422Z [cluster2]                     "openAPIV3Schema": {
2021-07-20T04:17:54.6052218Z [cluster2]                         "description": "Submariner is the Schema for the submariners API",
2021-07-20T04:17:54.6053090Z [cluster2]                         "properties": {
2021-07-20T04:17:54.6053635Z [cluster2]                             "apiVersion": {
2021-07-20T04:17:54.6056155Z [cluster2]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-20T04:17:54.6058237Z [cluster2]                                 "type": "string"
2021-07-20T04:17:54.6059548Z [cluster2]                             },
2021-07-20T04:17:54.6060226Z [cluster2]                             "kind": {
2021-07-20T04:17:54.6062733Z [cluster2]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-20T04:17:54.6064692Z [cluster2]                                 "type": "string"
2021-07-20T04:17:54.6065191Z [cluster2]                             },
2021-07-20T04:17:54.6065673Z [cluster2]                             "metadata": {
2021-07-20T04:17:54.6066380Z [cluster2]                                 "type": "object"
2021-07-20T04:17:54.6066859Z [cluster2]                             },
2021-07-20T04:17:54.6067326Z [cluster2]                             "spec": {
2021-07-20T04:17:54.6068396Z [cluster2]                                 "description": "SubmarinerSpec defines the desired state of Submariner",
2021-07-20T04:17:54.6069504Z [cluster2]                                 "properties": {
2021-07-20T04:17:54.6070239Z [cluster2]                                     "broker": {
2021-07-20T04:17:54.6070744Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6071615Z [cluster2]                                     },
2021-07-20T04:17:54.6072182Z [cluster2]                                     "brokerK8sApiServer": {
2021-07-20T04:17:54.6073128Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6073603Z [cluster2]                                     },
2021-07-20T04:17:54.6074390Z [cluster2]                                     "brokerK8sApiServerToken": {
2021-07-20T04:17:54.6075036Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6075505Z [cluster2]                                     },
2021-07-20T04:17:54.6075990Z [cluster2]                                     "brokerK8sCA": {
2021-07-20T04:17:54.6076672Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6078767Z [cluster2]                                     },
2021-07-20T04:17:54.6080138Z [cluster2]                                     "brokerK8sRemoteNamespace": {
2021-07-20T04:17:54.6081059Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6081553Z [cluster2]                                     },
2021-07-20T04:17:54.6082075Z [cluster2]                                     "cableDriver": {
2021-07-20T04:17:54.6082801Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6083282Z [cluster2]                                     },
2021-07-20T04:17:54.6083804Z [cluster2]                                     "ceIPSecDebug": {
2021-07-20T04:17:54.6084360Z [cluster2]                                         "type": "boolean"
2021-07-20T04:17:54.6084859Z [cluster2]                                     },
2021-07-20T04:17:54.6085467Z [cluster2]                                     "ceIPSecForceUDPEncaps": {
2021-07-20T04:17:54.6086296Z [cluster2]                                         "type": "boolean"
2021-07-20T04:17:54.6086810Z [cluster2]                                     },
2021-07-20T04:17:54.6087730Z [cluster2]                                     "ceIPSecIKEPort": {
2021-07-20T04:17:54.6088625Z [cluster2]                                         "type": "integer"
2021-07-20T04:17:54.6089288Z [cluster2]                                     },
2021-07-20T04:17:54.6089858Z [cluster2]                                     "ceIPSecNATTPort": {
2021-07-20T04:17:54.6090460Z [cluster2]                                         "type": "integer"
2021-07-20T04:17:54.6090956Z [cluster2]                                     },
2021-07-20T04:17:54.6091628Z [cluster2]                                     "ceIPSecPSK": {
2021-07-20T04:17:54.6092198Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6092841Z [cluster2]                                     },
2021-07-20T04:17:54.6093486Z [cluster2]                                     "ceIPSecPreferredServer": {
2021-07-20T04:17:54.6094178Z [cluster2]                                         "type": "boolean"
2021-07-20T04:17:54.6094659Z [cluster2]                                     },
2021-07-20T04:17:54.6095163Z [cluster2]                                     "clusterCIDR": {
2021-07-20T04:17:54.6095700Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6096534Z [cluster2]                                     },
2021-07-20T04:17:54.6097329Z [cluster2]                                     "clusterID": {
2021-07-20T04:17:54.6097883Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6098361Z [cluster2]                                     },
2021-07-20T04:17:54.6098861Z [cluster2]                                     "colorCodes": {
2021-07-20T04:17:54.6099400Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6099873Z [cluster2]                                     },
2021-07-20T04:17:54.6100485Z [cluster2]                                     "connectionHealthCheck": {
2021-07-20T04:17:54.6101142Z [cluster2]                                         "properties": {
2021-07-20T04:17:54.6101939Z [cluster2]                                             "enabled": {
2021-07-20T04:17:54.6102659Z [cluster2]                                                 "type": "boolean"
2021-07-20T04:17:54.6103198Z [cluster2]                                             },
2021-07-20T04:17:54.6103762Z [cluster2]                                             "intervalSeconds": {
2021-07-20T04:17:54.6104535Z [cluster2]                                                 "description": "The interval at which health check pings are sent.",
2021-07-20T04:17:54.6105267Z [cluster2]                                                 "format": "int64",
2021-07-20T04:17:54.6107944Z [cluster2]                                                 "type": "integer"
2021-07-20T04:17:54.6108473Z [cluster2]                                             },
2021-07-20T04:17:54.6109076Z [cluster2]                                             "maxPacketLossCount": {
2021-07-20T04:17:54.6110019Z [cluster2]                                                 "description": "The maximum number of packets lost at which the health checker will mark the connection as down.",
2021-07-20T04:17:54.6110871Z [cluster2]                                                 "format": "int64",
2021-07-20T04:17:54.6111402Z [cluster2]                                                 "type": "integer"
2021-07-20T04:17:54.6112085Z [cluster2]                                             }
2021-07-20T04:17:54.6112702Z [cluster2]                                         },
2021-07-20T04:17:54.6113221Z [cluster2]                                         "type": "object"
2021-07-20T04:17:54.6113734Z [cluster2]                                     },
2021-07-20T04:17:54.6114620Z [cluster2]                                     "coreDNSCustomConfig": {
2021-07-20T04:17:54.6115787Z [cluster2]                                         "properties": {
2021-07-20T04:17:54.6116602Z [cluster2]                                             "configMapName": {
2021-07-20T04:17:54.6117192Z [cluster2]                                                 "type": "string"
2021-07-20T04:17:54.6117681Z [cluster2]                                             },
2021-07-20T04:17:54.6118190Z [cluster2]                                             "namespace": {
2021-07-20T04:17:54.6119157Z [cluster2]                                                 "type": "string"
2021-07-20T04:17:54.6120142Z [cluster2]                                             }
2021-07-20T04:17:54.6120603Z [cluster2]                                         },
2021-07-20T04:17:54.6121083Z [cluster2]                                         "type": "object"
2021-07-20T04:17:54.6121827Z [cluster2]                                     },
2021-07-20T04:17:54.6122519Z [cluster2]                                     "customDomains": {
2021-07-20T04:17:54.6123252Z [cluster2]                                         "items": {
2021-07-20T04:17:54.6123924Z [cluster2]                                             "type": "string"
2021-07-20T04:17:54.6124715Z [cluster2]                                         },
2021-07-20T04:17:54.6125415Z [cluster2]                                         "type": "array",
2021-07-20T04:17:54.6127375Z [cluster2]                                         "x-kubernetes-list-type": "set"
2021-07-20T04:17:54.6128198Z [cluster2]                                     },
2021-07-20T04:17:54.6129001Z [cluster2]                                     "debug": {
2021-07-20T04:17:54.6129675Z [cluster2]                                         "type": "boolean"
2021-07-20T04:17:54.6130315Z [cluster2]                                     },
2021-07-20T04:17:54.6130816Z [cluster2]                                     "globalCIDR": {
2021-07-20T04:17:54.6131501Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6132155Z [cluster2]                                     },
2021-07-20T04:17:54.6132840Z [cluster2]                                     "imageOverrides": {
2021-07-20T04:17:54.6173790Z [cluster2]                                         "additionalProperties": {
2021-07-20T04:17:54.6174584Z [cluster2]                                             "type": "string"
2021-07-20T04:17:54.6175076Z [cluster2]                                         },
2021-07-20T04:17:54.6175582Z [cluster2]                                         "type": "object"
2021-07-20T04:17:54.6176076Z [cluster2]                                     },
2021-07-20T04:17:54.6176664Z [cluster2]                                     "loadBalancerEnabled": {
2021-07-20T04:17:54.6177502Z [cluster2]                                         "type": "boolean"
2021-07-20T04:17:54.6178009Z [cluster2]                                     },
2021-07-20T04:17:54.6178508Z [cluster2]                                     "namespace": {
2021-07-20T04:17:54.6179970Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6180521Z [cluster2]                                     },
2021-07-20T04:17:54.6181020Z [cluster2]                                     "natEnabled": {
2021-07-20T04:17:54.6181594Z [cluster2]                                         "type": "boolean"
2021-07-20T04:17:54.6182094Z [cluster2]                                     },
2021-07-20T04:17:54.6182607Z [cluster2]                                     "repository": {
2021-07-20T04:17:54.6183378Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6183850Z [cluster2]                                     },
2021-07-20T04:17:54.6184355Z [cluster2]                                     "serviceCIDR": {
2021-07-20T04:17:54.6184890Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6185559Z [cluster2]                                     },
2021-07-20T04:17:54.6186365Z [cluster2]                                     "serviceDiscoveryEnabled": {
2021-07-20T04:17:54.6187084Z [cluster2]                                         "type": "boolean"
2021-07-20T04:17:54.6187564Z [cluster2]                                     },
2021-07-20T04:17:54.6188042Z [cluster2]                                     "version": {
2021-07-20T04:17:54.6188571Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6189039Z [cluster2]                                     }
2021-07-20T04:17:54.6189567Z [cluster2]                                 },
2021-07-20T04:17:54.6190043Z [cluster2]                                 "required": [
2021-07-20T04:17:54.6190583Z [cluster2]                                     "broker",
2021-07-20T04:17:54.6191342Z [cluster2]                                     "brokerK8sApiServer",
2021-07-20T04:17:54.6192123Z [cluster2]                                     "brokerK8sApiServerToken",
2021-07-20T04:17:54.6192824Z [cluster2]                                     "brokerK8sCA",
2021-07-20T04:17:54.6193551Z [cluster2]                                     "brokerK8sRemoteNamespace",
2021-07-20T04:17:54.6194446Z [cluster2]                                     "ceIPSecDebug",
2021-07-20T04:17:54.6195007Z [cluster2]                                     "ceIPSecPSK",
2021-07-20T04:17:54.6195567Z [cluster2]                                     "clusterCIDR",
2021-07-20T04:17:54.6196155Z [cluster2]                                     "clusterID",
2021-07-20T04:17:54.6196668Z [cluster2]                                     "debug",
2021-07-20T04:17:54.6197291Z [cluster2]                                     "namespace",
2021-07-20T04:17:54.6197839Z [cluster2]                                     "natEnabled",
2021-07-20T04:17:54.6198381Z [cluster2]                                     "serviceCIDR"
2021-07-20T04:17:54.6198884Z [cluster2]                                 ],
2021-07-20T04:17:54.6199371Z [cluster2]                                 "type": "object"
2021-07-20T04:17:54.6199968Z [cluster2]                             },
2021-07-20T04:17:54.6200436Z [cluster2]                             "status": {
2021-07-20T04:17:54.6201282Z [cluster2]                                 "description": "SubmarinerStatus defines the observed state of Submariner",
2021-07-20T04:17:54.6202108Z [cluster2]                                 "properties": {
2021-07-20T04:17:54.6202832Z [cluster2]                                     "clusterCIDR": {
2021-07-20T04:17:54.6203561Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6204166Z [cluster2]                                     },
2021-07-20T04:17:54.6204649Z [cluster2]                                     "clusterID": {
2021-07-20T04:17:54.6205184Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6205655Z [cluster2]                                     },
2021-07-20T04:17:54.6206158Z [cluster2]                                     "colorCodes": {
2021-07-20T04:17:54.6206691Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6207174Z [cluster2]                                     },
2021-07-20T04:17:54.6207722Z [cluster2]                                     "deploymentInfo": {
2021-07-20T04:17:54.6208319Z [cluster2]                                         "properties": {
2021-07-20T04:17:54.6208908Z [cluster2]                                             "cloudProvider": {
2021-07-20T04:17:54.6216614Z [cluster2]                                                 "type": "string"
2021-07-20T04:17:54.6217552Z [cluster2]                                             },
2021-07-20T04:17:54.6218099Z [cluster2]                                             "kubernetesType": {
2021-07-20T04:17:54.6218697Z [cluster2]                                                 "type": "string"
2021-07-20T04:17:54.6219181Z [cluster2]                                             },
2021-07-20T04:17:54.6219814Z [cluster2]                                             "kubernetesTypeVersion": {
2021-07-20T04:17:54.6220628Z [cluster2]                                                 "type": "string"
2021-07-20T04:17:54.6221162Z [cluster2]                                             },
2021-07-20T04:17:54.6221865Z [cluster2]                                             "kubernetesVersion": {
2021-07-20T04:17:54.6222469Z [cluster2]                                                 "type": "string"
2021-07-20T04:17:54.6222970Z [cluster2]                                             }
2021-07-20T04:17:54.6223406Z [cluster2]                                         },
2021-07-20T04:17:54.6223912Z [cluster2]                                         "type": "object"
2021-07-20T04:17:54.6224388Z [cluster2]                                     },
2021-07-20T04:17:54.6225015Z [cluster2]                                     "gatewayDaemonSetStatus": {
2021-07-20T04:17:54.6225704Z [cluster2]                                         "properties": {
2021-07-20T04:17:54.6226402Z [cluster2]                                             "lastResourceVersion": {
2021-07-20T04:17:54.6227048Z [cluster2]                                                 "type": "string"
2021-07-20T04:17:54.6227536Z [cluster2]                                             },
2021-07-20T04:17:54.6228398Z [cluster2]                                             "mismatchedContainerImages": {
2021-07-20T04:17:54.6229088Z [cluster2]                                                 "type": "boolean"
2021-07-20T04:17:54.6256109Z [cluster2]                                             },
2021-07-20T04:17:54.6256962Z [cluster2]                                             "nonReadyContainerStates": {
2021-07-20T04:17:54.6257864Z [cluster2]                                                 "items": {
2021-07-20T04:17:54.6259066Z [cluster2]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-20T04:17:54.6260292Z [cluster2]                                                     "properties": {
2021-07-20T04:17:54.6260845Z [cluster2]                                                         "running": {
2021-07-20T04:17:54.6261707Z [cluster2]                                                             "description": "Details about a running container",
2021-07-20T04:17:54.6262673Z [cluster2]                                                             "properties": {
2021-07-20T04:17:54.6263283Z [cluster2]                                                                 "startedAt": {
2021-07-20T04:17:54.6264921Z [cluster2]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-20T04:17:54.6266054Z [cluster2]                                                                     "format": "date-time",
2021-07-20T04:17:54.6266644Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6267147Z [cluster2]                                                                 }
2021-07-20T04:17:54.6267615Z [cluster2]                                                             },
2021-07-20T04:17:54.6268108Z [cluster2]                                                             "type": "object"
2021-07-20T04:17:54.6268617Z [cluster2]                                                         },
2021-07-20T04:17:54.6269123Z [cluster2]                                                         "terminated": {
2021-07-20T04:17:54.6269823Z [cluster2]                                                             "description": "Details about a terminated container",
2021-07-20T04:17:54.6270537Z [cluster2]                                                             "properties": {
2021-07-20T04:17:54.6271100Z [cluster2]                                                                 "containerID": {
2021-07-20T04:17:54.6272344Z [cluster2]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-20T04:17:54.6273148Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6273646Z [cluster2]                                                                 },
2021-07-20T04:17:54.6274164Z [cluster2]                                                                 "exitCode": {
2021-07-20T04:17:54.6274907Z [cluster2]                                                                     "description": "Exit status from the last termination of the container",
2021-07-20T04:17:54.6275667Z [cluster2]                                                                     "format": "int32",
2021-07-20T04:17:54.6276324Z [cluster2]                                                                     "type": "integer"
2021-07-20T04:17:54.6276837Z [cluster2]                                                                 },
2021-07-20T04:17:54.6277364Z [cluster2]                                                                 "finishedAt": {
2021-07-20T04:17:54.6278099Z [cluster2]                                                                     "description": "Time at which the container last terminated",
2021-07-20T04:17:54.6279108Z [cluster2]                                                                     "format": "date-time",
2021-07-20T04:17:54.6279699Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6281878Z [cluster2]                                                                 },
2021-07-20T04:17:54.6282781Z [cluster2]                                                                 "message": {
2021-07-20T04:17:54.6283597Z [cluster2]                                                                     "description": "Message regarding the last termination of the container",
2021-07-20T04:17:54.6284439Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6286470Z [cluster2]                                                                 },
2021-07-20T04:17:54.6288851Z [cluster2]                                                                 "reason": {
2021-07-20T04:17:54.6289609Z [cluster2]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-20T04:17:54.6290597Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6291108Z [cluster2]                                                                 },
2021-07-20T04:17:54.6292213Z [cluster2]                                                                 "signal": {
2021-07-20T04:17:54.6293080Z [cluster2]                                                                     "description": "Signal from the last termination of the container",
2021-07-20T04:17:54.6293827Z [cluster2]                                                                     "format": "int32",
2021-07-20T04:17:54.6294382Z [cluster2]                                                                     "type": "integer"
2021-07-20T04:17:54.6295910Z [cluster2]                                                                 },
2021-07-20T04:17:54.6296848Z [cluster2]                                                                 "startedAt": {
2021-07-20T04:17:54.6297791Z [cluster2]                                                                     "description": "Time at which previous execution of the container started",
2021-07-20T04:17:54.6298946Z [cluster2]                                                                     "format": "date-time",
2021-07-20T04:17:54.6329072Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6329767Z [cluster2]                                                                 }
2021-07-20T04:17:54.6330781Z [cluster2]                                                             },
2021-07-20T04:17:54.6331390Z [cluster2]                                                             "required": [
2021-07-20T04:17:54.6331921Z [cluster2]                                                                 "exitCode"
2021-07-20T04:17:54.6332397Z [cluster2]                                                             ],
2021-07-20T04:17:54.6332892Z [cluster2]                                                             "type": "object"
2021-07-20T04:17:54.6333363Z [cluster2]                                                         },
2021-07-20T04:17:54.6333842Z [cluster2]                                                         "waiting": {
2021-07-20T04:17:54.6334683Z [cluster2]                                                             "description": "Details about a waiting container",
2021-07-20T04:17:54.6335365Z [cluster2]                                                             "properties": {
2021-07-20T04:17:54.6335923Z [cluster2]                                                                 "message": {
2021-07-20T04:17:54.6336849Z [cluster2]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-20T04:17:54.6337877Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6338875Z [cluster2]                                                                 },
2021-07-20T04:17:54.6339667Z [cluster2]                                                                 "reason": {
2021-07-20T04:17:54.6340799Z [cluster2]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-20T04:17:54.6341909Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6342794Z [cluster2]                                                                 }
2021-07-20T04:17:54.6343431Z [cluster2]                                                             },
2021-07-20T04:17:54.6343955Z [cluster2]                                                             "type": "object"
2021-07-20T04:17:54.6344482Z [cluster2]                                                         }
2021-07-20T04:17:54.6344933Z [cluster2]                                                     },
2021-07-20T04:17:54.6345446Z [cluster2]                                                     "type": "object"
2021-07-20T04:17:54.6345942Z [cluster2]                                                 },
2021-07-20T04:17:54.6346595Z [cluster2]                                                 "type": "array"
2021-07-20T04:17:54.6347065Z [cluster2]                                             },
2021-07-20T04:17:54.6347526Z [cluster2]                                             "status": {
2021-07-20T04:17:54.6348285Z [cluster2]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-20T04:17:54.6349099Z [cluster2]                                                 "properties": {
2021-07-20T04:17:54.6349684Z [cluster2]                                                     "collisionCount": {
2021-07-20T04:17:54.6350960Z [cluster2]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-20T04:17:54.6352395Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6352953Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6353451Z [cluster2]                                                     },
2021-07-20T04:17:54.6353949Z [cluster2]                                                     "conditions": {
2021-07-20T04:17:54.6355411Z [cluster2]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-20T04:17:54.6356281Z [cluster2]                                                         "items": {
2021-07-20T04:17:54.6357131Z [cluster2]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-20T04:17:54.6357995Z [cluster2]                                                             "properties": {
2021-07-20T04:17:54.6358649Z [cluster2]                                                                 "lastTransitionTime": {
2021-07-20T04:17:54.6359500Z [cluster2]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-20T04:17:54.6360941Z [cluster2]                                                                     "format": "date-time",
2021-07-20T04:17:54.6361582Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6362117Z [cluster2]                                                                 },
2021-07-20T04:17:54.6362654Z [cluster2]                                                                 "message": {
2021-07-20T04:17:54.6363504Z [cluster2]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-20T04:17:54.6364763Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6365272Z [cluster2]                                                                 },
2021-07-20T04:17:54.6365749Z [cluster2]                                                                 "reason": {
2021-07-20T04:17:54.6366943Z [cluster2]                                                                     "description": "The reason for the condition's last transition.",
2021-07-20T04:17:54.6367677Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6368191Z [cluster2]                                                                 },
2021-07-20T04:17:54.6368680Z [cluster2]                                                                 "status": {
2021-07-20T04:17:54.6369397Z [cluster2]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-20T04:17:54.6370160Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6370740Z [cluster2]                                                                 },
2021-07-20T04:17:54.6371232Z [cluster2]                                                                 "type": {
2021-07-20T04:17:54.6371895Z [cluster2]                                                                     "description": "Type of DaemonSet condition.",
2021-07-20T04:17:54.6372700Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6373392Z [cluster2]                                                                 }
2021-07-20T04:17:54.6373860Z [cluster2]                                                             },
2021-07-20T04:17:54.6374377Z [cluster2]                                                             "required": [
2021-07-20T04:17:54.6375061Z [cluster2]                                                                 "status",
2021-07-20T04:17:54.6375572Z [cluster2]                                                                 "type"
2021-07-20T04:17:54.6376050Z [cluster2]                                                             ],
2021-07-20T04:17:54.6376541Z [cluster2]                                                             "type": "object"
2021-07-20T04:17:54.6410502Z [cluster2]                                                         },
2021-07-20T04:17:54.6411349Z [cluster2]                                                         "type": "array"
2021-07-20T04:17:54.6411897Z [cluster2]                                                     },
2021-07-20T04:17:54.6412531Z [cluster2]                                                     "currentNumberScheduled": {
2021-07-20T04:17:54.6413946Z [cluster2]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.6415398Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6415986Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6416535Z [cluster2]                                                     },
2021-07-20T04:17:54.6417326Z [cluster2]                                                     "desiredNumberScheduled": {
2021-07-20T04:17:54.6418961Z [cluster2]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.6420353Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6421106Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6421736Z [cluster2]                                                     },
2021-07-20T04:17:54.6422477Z [cluster2]                                                     "numberAvailable": {
2021-07-20T04:17:54.6423706Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-20T04:17:54.6424893Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6425486Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6426350Z [cluster2]                                                     },
2021-07-20T04:17:54.6426964Z [cluster2]                                                     "numberMisscheduled": {
2021-07-20T04:17:54.6428337Z [cluster2]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.6429644Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6430389Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6430902Z [cluster2]                                                     },
2021-07-20T04:17:54.6431430Z [cluster2]                                                     "numberReady": {
2021-07-20T04:17:54.6432471Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-20T04:17:54.6433400Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6433978Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6434485Z [cluster2]                                                     },
2021-07-20T04:17:54.6435069Z [cluster2]                                                     "numberUnavailable": {
2021-07-20T04:17:54.6436338Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-20T04:17:54.6437476Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6438048Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6438553Z [cluster2]                                                     },
2021-07-20T04:17:54.6439152Z [cluster2]                                                     "observedGeneration": {
2021-07-20T04:17:54.6440393Z [cluster2]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-20T04:17:54.6441244Z [cluster2]                                                         "format": "int64",
2021-07-20T04:17:54.6441828Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6442514Z [cluster2]                                                     },
2021-07-20T04:17:54.6443168Z [cluster2]                                                     "updatedNumberScheduled": {
2021-07-20T04:17:54.6444068Z [cluster2]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-20T04:17:54.6444838Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6445389Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6446011Z [cluster2]                                                     }
2021-07-20T04:17:54.6446478Z [cluster2]                                                 },
2021-07-20T04:17:54.6446968Z [cluster2]                                                 "required": [
2021-07-20T04:17:54.6447796Z [cluster2]                                                     "currentNumberScheduled",
2021-07-20T04:17:54.6448576Z [cluster2]                                                     "desiredNumberScheduled",
2021-07-20T04:17:54.6449683Z [cluster2]                                                     "numberMisscheduled",
2021-07-20T04:17:54.6450346Z [cluster2]                                                     "numberReady"
2021-07-20T04:17:54.6451148Z [cluster2]                                                 ],
2021-07-20T04:17:54.6451830Z [cluster2]                                                 "type": "object"
2021-07-20T04:17:54.6452329Z [cluster2]                                             }
2021-07-20T04:17:54.6452988Z [cluster2]                                         },
2021-07-20T04:17:54.6453443Z [cluster2]                                         "required": [
2021-07-20T04:17:54.6454129Z [cluster2]                                             "mismatchedContainerImages"
2021-07-20T04:17:54.6454765Z [cluster2]                                         ],
2021-07-20T04:17:54.6455253Z [cluster2]                                         "type": "object"
2021-07-20T04:17:54.6455720Z [cluster2]                                     },
2021-07-20T04:17:54.6456190Z [cluster2]                                     "gateways": {
2021-07-20T04:17:54.6456684Z [cluster2]                                         "items": {
2021-07-20T04:17:54.6486623Z [cluster2]                                             "properties": {
2021-07-20T04:17:54.6487344Z [cluster2]                                                 "connections": {
2021-07-20T04:17:54.6488050Z [cluster2]                                                     "items": {
2021-07-20T04:17:54.6488647Z [cluster2]                                                         "properties": {
2021-07-20T04:17:54.6489235Z [cluster2]                                                             "endpoint": {
2021-07-20T04:17:54.6490020Z [cluster2]                                                                 "properties": {
2021-07-20T04:17:54.6490594Z [cluster2]                                                                     "backend": {
2021-07-20T04:17:54.6491404Z [cluster2]                                                                         "type": "string"
2021-07-20T04:17:54.6492064Z [cluster2]                                                                     },
2021-07-20T04:17:54.6492700Z [cluster2]                                                                     "backend_config": {
2021-07-20T04:17:54.6493601Z [cluster2]                                                                         "additionalProperties": {
2021-07-20T04:17:54.6494304Z [cluster2]                                                                             "type": "string"
2021-07-20T04:17:54.6494875Z [cluster2]                                                                         },
2021-07-20T04:17:54.6495447Z [cluster2]                                                                         "type": "object"
2021-07-20T04:17:54.6495984Z [cluster2]                                                                     },
2021-07-20T04:17:54.6496841Z [cluster2]                                                                     "cable_name": {
2021-07-20T04:17:54.6497831Z [cluster2]                                                                         "type": "string"
2021-07-20T04:17:54.6498708Z [cluster2]                                                                     },
2021-07-20T04:17:54.6499385Z [cluster2]                                                                     "cluster_id": {
2021-07-20T04:17:54.6499936Z [cluster2]                                                                         "type": "string"
2021-07-20T04:17:54.6500577Z [cluster2]                                                                     },
2021-07-20T04:17:54.6501125Z [cluster2]                                                                     "healthCheckIP": {
2021-07-20T04:17:54.6501727Z [cluster2]                                                                         "type": "string"
2021-07-20T04:17:54.6502228Z [cluster2]                                                                     },
2021-07-20T04:17:54.6502751Z [cluster2]                                                                     "hostname": {
2021-07-20T04:17:54.6503310Z [cluster2]                                                                         "type": "string"
2021-07-20T04:17:54.6503812Z [cluster2]                                                                     },
2021-07-20T04:17:54.6504327Z [cluster2]                                                                     "nat_enabled": {
2021-07-20T04:17:54.6505082Z [cluster2]                                                                         "type": "boolean"
2021-07-20T04:17:54.6505818Z [cluster2]                                                                     },
2021-07-20T04:17:54.6506370Z [cluster2]                                                                     "private_ip": {
2021-07-20T04:17:54.6506955Z [cluster2]                                                                         "type": "string"
2021-07-20T04:17:54.6507517Z [cluster2]                                                                     },
2021-07-20T04:17:54.6508046Z [cluster2]                                                                     "public_ip": {
2021-07-20T04:17:54.6508635Z [cluster2]                                                                         "type": "string"
2021-07-20T04:17:54.6509512Z [cluster2]                                                                     },
2021-07-20T04:17:54.6510022Z [cluster2]                                                                     "subnets": {
2021-07-20T04:17:54.6510554Z [cluster2]                                                                         "items": {
2021-07-20T04:17:54.6511085Z [cluster2]                                                                             "type": "string"
2021-07-20T04:17:54.6511603Z [cluster2]                                                                         },
2021-07-20T04:17:54.6512200Z [cluster2]                                                                         "type": "array"
2021-07-20T04:17:54.6512732Z [cluster2]                                                                     }
2021-07-20T04:17:54.6513188Z [cluster2]                                                                 },
2021-07-20T04:17:54.6513701Z [cluster2]                                                                 "required": [
2021-07-20T04:17:54.6514248Z [cluster2]                                                                     "backend",
2021-07-20T04:17:54.6514776Z [cluster2]                                                                     "cable_name",
2021-07-20T04:17:54.6515323Z [cluster2]                                                                     "cluster_id",
2021-07-20T04:17:54.6515854Z [cluster2]                                                                     "hostname",
2021-07-20T04:17:54.6516403Z [cluster2]                                                                     "nat_enabled",
2021-07-20T04:17:54.6517434Z [cluster2]                                                                     "private_ip",
2021-07-20T04:17:54.6518005Z [cluster2]                                                                     "public_ip",
2021-07-20T04:17:54.6518583Z [cluster2]                                                                     "subnets"
2021-07-20T04:17:54.6519109Z [cluster2]                                                                 ],
2021-07-20T04:17:54.6519662Z [cluster2]                                                                 "type": "object"
2021-07-20T04:17:54.6520283Z [cluster2]                                                             },
2021-07-20T04:17:54.6520839Z [cluster2]                                                             "latencyRTT": {
2021-07-20T04:17:54.6521882Z [cluster2]                                                                 "description": "LatencySpec describes the round trip time information for a packet between the gateway pods of two clusters.",
2021-07-20T04:17:54.6522919Z [cluster2]                                                                 "properties": {
2021-07-20T04:17:54.6523517Z [cluster2]                                                                     "average": {
2021-07-20T04:17:54.6524248Z [cluster2]                                                                         "type": "string"
2021-07-20T04:17:54.6524943Z [cluster2]                                                                     },
2021-07-20T04:17:54.6525434Z [cluster2]                                                                     "last": {
2021-07-20T04:17:54.6525957Z [cluster2]                                                                         "type": "string"
2021-07-20T04:17:54.6526836Z [cluster2]                                                                     },
2021-07-20T04:17:54.6527340Z [cluster2]                                                                     "max": {
2021-07-20T04:17:54.6527915Z [cluster2]                                                                         "type": "string"
2021-07-20T04:17:54.6528457Z [cluster2]                                                                     },
2021-07-20T04:17:54.6528977Z [cluster2]                                                                     "min": {
2021-07-20T04:17:54.6529551Z [cluster2]                                                                         "type": "string"
2021-07-20T04:17:54.6530090Z [cluster2]                                                                     },
2021-07-20T04:17:54.6530794Z [cluster2]                                                                     "stdDev": {
2021-07-20T04:17:54.6531366Z [cluster2]                                                                         "type": "string"
2021-07-20T04:17:54.6532075Z [cluster2]                                                                     }
2021-07-20T04:17:54.6532719Z [cluster2]                                                                 },
2021-07-20T04:17:54.6533385Z [cluster2]                                                                 "type": "object"
2021-07-20T04:17:54.6533918Z [cluster2]                                                             },
2021-07-20T04:17:54.6534394Z [cluster2]                                                             "status": {
2021-07-20T04:17:54.6534930Z [cluster2]                                                                 "type": "string"
2021-07-20T04:17:54.6535424Z [cluster2]                                                             },
2021-07-20T04:17:54.6535969Z [cluster2]                                                             "statusMessage": {
2021-07-20T04:17:54.6536552Z [cluster2]                                                                 "type": "string"
2021-07-20T04:17:54.6544244Z [cluster2]                                                             },
2021-07-20T04:17:54.6544905Z [cluster2]                                                             "usingIP": {
2021-07-20T04:17:54.6545689Z [cluster2]                                                                 "type": "string"
2021-07-20T04:17:54.6546221Z [cluster2]                                                             },
2021-07-20T04:17:54.6546821Z [cluster2]                                                             "usingNAT": {
2021-07-20T04:17:54.6547398Z [cluster2]                                                                 "type": "boolean"
2021-07-20T04:17:54.6547938Z [cluster2]                                                             }
2021-07-20T04:17:54.6548792Z [cluster2]                                                         },
2021-07-20T04:17:54.6549334Z [cluster2]                                                         "required": [
2021-07-20T04:17:54.6549900Z [cluster2]                                                             "endpoint",
2021-07-20T04:17:54.6550573Z [cluster2]                                                             "status",
2021-07-20T04:17:54.6551164Z [cluster2]                                                             "statusMessage"
2021-07-20T04:17:54.6551754Z [cluster2]                                                         ],
2021-07-20T04:17:54.6552300Z [cluster2]                                                         "type": "object"
2021-07-20T04:17:54.6552820Z [cluster2]                                                     },
2021-07-20T04:17:54.6553351Z [cluster2]                                                     "type": "array"
2021-07-20T04:17:54.6554073Z [cluster2]                                                 },
2021-07-20T04:17:54.6554581Z [cluster2]                                                 "haStatus": {
2021-07-20T04:17:54.6555113Z [cluster2]                                                     "type": "string"
2021-07-20T04:17:54.6555625Z [cluster2]                                                 },
2021-07-20T04:17:54.6556167Z [cluster2]                                                 "localEndpoint": {
2021-07-20T04:17:54.6556758Z [cluster2]                                                     "properties": {
2021-07-20T04:17:54.6557322Z [cluster2]                                                         "backend": {
2021-07-20T04:17:54.6557868Z [cluster2]                                                             "type": "string"
2021-07-20T04:17:54.6558388Z [cluster2]                                                         },
2021-07-20T04:17:54.6558905Z [cluster2]                                                         "backend_config": {
2021-07-20T04:17:54.6559595Z [cluster2]                                                             "additionalProperties": {
2021-07-20T04:17:54.6560468Z [cluster2]                                                                 "type": "string"
2021-07-20T04:17:54.6561202Z [cluster2]                                                             },
2021-07-20T04:17:54.6561902Z [cluster2]                                                             "type": "object"
2021-07-20T04:17:54.6562432Z [cluster2]                                                         },
2021-07-20T04:17:54.6563071Z [cluster2]                                                         "cable_name": {
2021-07-20T04:17:54.6563812Z [cluster2]                                                             "type": "string"
2021-07-20T04:17:54.6564337Z [cluster2]                                                         },
2021-07-20T04:17:54.6564848Z [cluster2]                                                         "cluster_id": {
2021-07-20T04:17:54.6565724Z [cluster2]                                                             "type": "string"
2021-07-20T04:17:54.6566253Z [cluster2]                                                         },
2021-07-20T04:17:54.6566797Z [cluster2]                                                         "healthCheckIP": {
2021-07-20T04:17:54.6567396Z [cluster2]                                                             "type": "string"
2021-07-20T04:17:54.6568009Z [cluster2]                                                         },
2021-07-20T04:17:54.6568531Z [cluster2]                                                         "hostname": {
2021-07-20T04:17:54.6569097Z [cluster2]                                                             "type": "string"
2021-07-20T04:17:54.6569606Z [cluster2]                                                         },
2021-07-20T04:17:54.6570130Z [cluster2]                                                         "nat_enabled": {
2021-07-20T04:17:54.6570690Z [cluster2]                                                             "type": "boolean"
2021-07-20T04:17:54.6571318Z [cluster2]                                                         },
2021-07-20T04:17:54.6571826Z [cluster2]                                                         "private_ip": {
2021-07-20T04:17:54.6572393Z [cluster2]                                                             "type": "string"
2021-07-20T04:17:54.6572913Z [cluster2]                                                         },
2021-07-20T04:17:54.6573612Z [cluster2]                                                         "public_ip": {
2021-07-20T04:17:54.6574192Z [cluster2]                                                             "type": "string"
2021-07-20T04:17:54.6574868Z [cluster2]                                                         },
2021-07-20T04:17:54.6575374Z [cluster2]                                                         "subnets": {
2021-07-20T04:17:54.6575891Z [cluster2]                                                             "items": {
2021-07-20T04:17:54.6576436Z [cluster2]                                                                 "type": "string"
2021-07-20T04:17:54.6576970Z [cluster2]                                                             },
2021-07-20T04:17:54.6581562Z [cluster2]                                                             "type": "array"
2021-07-20T04:17:54.6582306Z [cluster2]                                                         }
2021-07-20T04:17:54.6582784Z [cluster2]                                                     },
2021-07-20T04:17:54.6583322Z [cluster2]                                                     "required": [
2021-07-20T04:17:54.6583872Z [cluster2]                                                         "backend",
2021-07-20T04:17:54.6585993Z [cluster2]                                                         "cable_name",
2021-07-20T04:17:54.6586577Z [cluster2]                                                         "cluster_id",
2021-07-20T04:17:54.6587138Z [cluster2]                                                         "hostname",
2021-07-20T04:17:54.6588289Z [cluster2]                                                         "nat_enabled",
2021-07-20T04:17:54.6588946Z [cluster2]                                                         "private_ip",
2021-07-20T04:17:54.6590034Z [cluster2]                                                         "public_ip",
2021-07-20T04:17:54.6590678Z [cluster2]                                                         "subnets"
2021-07-20T04:17:54.6591362Z [cluster2]                                                     ],
2021-07-20T04:17:54.6591908Z [cluster2]                                                     "type": "object"
2021-07-20T04:17:54.6592408Z [cluster2]                                                 },
2021-07-20T04:17:54.6592952Z [cluster2]                                                 "statusFailure": {
2021-07-20T04:17:54.6593524Z [cluster2]                                                     "type": "string"
2021-07-20T04:17:54.6594039Z [cluster2]                                                 },
2021-07-20T04:17:54.6594523Z [cluster2]                                                 "version": {
2021-07-20T04:17:54.6595074Z [cluster2]                                                     "type": "string"
2021-07-20T04:17:54.6595584Z [cluster2]                                                 }
2021-07-20T04:17:54.6596136Z [cluster2]                                             },
2021-07-20T04:17:54.6596642Z [cluster2]                                             "required": [
2021-07-20T04:17:54.6597197Z [cluster2]                                                 "connections",
2021-07-20T04:17:54.6597756Z [cluster2]                                                 "haStatus",
2021-07-20T04:17:54.6598314Z [cluster2]                                                 "localEndpoint",
2021-07-20T04:17:54.6598924Z [cluster2]                                                 "statusFailure",
2021-07-20T04:17:54.6599476Z [cluster2]                                                 "version"
2021-07-20T04:17:54.6600115Z [cluster2]                                             ],
2021-07-20T04:17:54.6600622Z [cluster2]                                             "type": "object"
2021-07-20T04:17:54.6601106Z [cluster2]                                         },
2021-07-20T04:17:54.6601599Z [cluster2]                                         "type": "array"
2021-07-20T04:17:54.6602068Z [cluster2]                                     },
2021-07-20T04:17:54.6603007Z [cluster2]                                     "globalCIDR": {
2021-07-20T04:17:54.6603778Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6604461Z [cluster2]                                     },
2021-07-20T04:17:54.6605289Z [cluster2]                                     "globalnetDaemonSetStatus": {
2021-07-20T04:17:54.6606472Z [cluster2]                                         "properties": {
2021-07-20T04:17:54.6607691Z [cluster2]                                             "lastResourceVersion": {
2021-07-20T04:17:54.6609188Z [cluster2]                                                 "type": "string"
2021-07-20T04:17:54.6609701Z [cluster2]                                             },
2021-07-20T04:17:54.6610336Z [cluster2]                                             "mismatchedContainerImages": {
2021-07-20T04:17:54.6611355Z [cluster2]                                                 "type": "boolean"
2021-07-20T04:17:54.6611836Z [cluster2]                                             },
2021-07-20T04:17:54.6612473Z [cluster2]                                             "nonReadyContainerStates": {
2021-07-20T04:17:54.6613129Z [cluster2]                                                 "items": {
2021-07-20T04:17:54.6614309Z [cluster2]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-20T04:17:54.6617351Z [cluster2]                                                     "properties": {
2021-07-20T04:17:54.6618139Z [cluster2]                                                         "running": {
2021-07-20T04:17:54.6618857Z [cluster2]                                                             "description": "Details about a running container",
2021-07-20T04:17:54.6619915Z [cluster2]                                                             "properties": {
2021-07-20T04:17:54.6620966Z [cluster2]                                                                 "startedAt": {
2021-07-20T04:17:54.6622823Z [cluster2]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-20T04:17:54.6623837Z [cluster2]                                                                     "format": "date-time",
2021-07-20T04:17:54.6624455Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6624990Z [cluster2]                                                                 }
2021-07-20T04:17:54.6625473Z [cluster2]                                                             },
2021-07-20T04:17:54.6626154Z [cluster2]                                                             "type": "object"
2021-07-20T04:17:54.6626646Z [cluster2]                                                         },
2021-07-20T04:17:54.6627157Z [cluster2]                                                         "terminated": {
2021-07-20T04:17:54.6627863Z [cluster2]                                                             "description": "Details about a terminated container",
2021-07-20T04:17:54.6628581Z [cluster2]                                                             "properties": {
2021-07-20T04:17:54.6629159Z [cluster2]                                                                 "containerID": {
2021-07-20T04:17:54.6630585Z [cluster2]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-20T04:17:54.6631522Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6632026Z [cluster2]                                                                 },
2021-07-20T04:17:54.6632541Z [cluster2]                                                                 "exitCode": {
2021-07-20T04:17:54.6633309Z [cluster2]                                                                     "description": "Exit status from the last termination of the container",
2021-07-20T04:17:54.6634046Z [cluster2]                                                                     "format": "int32",
2021-07-20T04:17:54.6634605Z [cluster2]                                                                     "type": "integer"
2021-07-20T04:17:54.6635116Z [cluster2]                                                                 },
2021-07-20T04:17:54.6635641Z [cluster2]                                                                 "finishedAt": {
2021-07-20T04:17:54.6636388Z [cluster2]                                                                     "description": "Time at which the container last terminated",
2021-07-20T04:17:54.6637396Z [cluster2]                                                                     "format": "date-time",
2021-07-20T04:17:54.6637985Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6638494Z [cluster2]                                                                 },
2021-07-20T04:17:54.6639319Z [cluster2]                                                                 "message": {
2021-07-20T04:17:54.6640271Z [cluster2]                                                                     "description": "Message regarding the last termination of the container",
2021-07-20T04:17:54.6641289Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6641850Z [cluster2]                                                                 },
2021-07-20T04:17:54.6642365Z [cluster2]                                                                 "reason": {
2021-07-20T04:17:54.6643168Z [cluster2]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-20T04:17:54.6644263Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6644889Z [cluster2]                                                                 },
2021-07-20T04:17:54.6645401Z [cluster2]                                                                 "signal": {
2021-07-20T04:17:54.6646107Z [cluster2]                                                                     "description": "Signal from the last termination of the container",
2021-07-20T04:17:54.6646842Z [cluster2]                                                                     "format": "int32",
2021-07-20T04:17:54.6647396Z [cluster2]                                                                     "type": "integer"
2021-07-20T04:17:54.6647919Z [cluster2]                                                                 },
2021-07-20T04:17:54.6648438Z [cluster2]                                                                 "startedAt": {
2021-07-20T04:17:54.6649709Z [cluster2]                                                                     "description": "Time at which previous execution of the container started",
2021-07-20T04:17:54.6651113Z [cluster2]                                                                     "format": "date-time",
2021-07-20T04:17:54.6652061Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6652614Z [cluster2]                                                                 }
2021-07-20T04:17:54.6653267Z [cluster2]                                                             },
2021-07-20T04:17:54.6654024Z [cluster2]                                                             "required": [
2021-07-20T04:17:54.6654618Z [cluster2]                                                                 "exitCode"
2021-07-20T04:17:54.6655143Z [cluster2]                                                             ],
2021-07-20T04:17:54.6655686Z [cluster2]                                                             "type": "object"
2021-07-20T04:17:54.6656219Z [cluster2]                                                         },
2021-07-20T04:17:54.6656753Z [cluster2]                                                         "waiting": {
2021-07-20T04:17:54.6658158Z [cluster2]                                                             "description": "Details about a waiting container",
2021-07-20T04:17:54.6659014Z [cluster2]                                                             "properties": {
2021-07-20T04:17:54.6659612Z [cluster2]                                                                 "message": {
2021-07-20T04:17:54.6660416Z [cluster2]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-20T04:17:54.6661384Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6661904Z [cluster2]                                                                 },
2021-07-20T04:17:54.6662623Z [cluster2]                                                                 "reason": {
2021-07-20T04:17:54.6663790Z [cluster2]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-20T04:17:54.6664546Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6665357Z [cluster2]                                                                 }
2021-07-20T04:17:54.6665832Z [cluster2]                                                             },
2021-07-20T04:17:54.6666368Z [cluster2]                                                             "type": "object"
2021-07-20T04:17:54.6666898Z [cluster2]                                                         }
2021-07-20T04:17:54.6667351Z [cluster2]                                                     },
2021-07-20T04:17:54.6667871Z [cluster2]                                                     "type": "object"
2021-07-20T04:17:54.6668491Z [cluster2]                                                 },
2021-07-20T04:17:54.6669013Z [cluster2]                                                 "type": "array"
2021-07-20T04:17:54.6669619Z [cluster2]                                             },
2021-07-20T04:17:54.6670909Z [cluster2]                                             "status": {
2021-07-20T04:17:54.6672327Z [cluster2]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-20T04:17:54.6673384Z [cluster2]                                                 "properties": {
2021-07-20T04:17:54.6674342Z [cluster2]                                                     "collisionCount": {
2021-07-20T04:17:54.6676074Z [cluster2]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-20T04:17:54.6677368Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6678118Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6679669Z [cluster2]                                                     },
2021-07-20T04:17:54.6680387Z [cluster2]                                                     "conditions": {
2021-07-20T04:17:54.6683216Z [cluster2]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-20T04:17:54.6684309Z [cluster2]                                                         "items": {
2021-07-20T04:17:54.6686365Z [cluster2]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-20T04:17:54.6687990Z [cluster2]                                                             "properties": {
2021-07-20T04:17:54.6689069Z [cluster2]                                                                 "lastTransitionTime": {
2021-07-20T04:17:54.6689958Z [cluster2]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-20T04:17:54.6691170Z [cluster2]                                                                     "format": "date-time",
2021-07-20T04:17:54.6691774Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6692319Z [cluster2]                                                                 },
2021-07-20T04:17:54.6692841Z [cluster2]                                                                 "message": {
2021-07-20T04:17:54.6694001Z [cluster2]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-20T04:17:54.6694848Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6695371Z [cluster2]                                                                 },
2021-07-20T04:17:54.6695884Z [cluster2]                                                                 "reason": {
2021-07-20T04:17:54.6696917Z [cluster2]                                                                     "description": "The reason for the condition's last transition.",
2021-07-20T04:17:54.6697814Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6698356Z [cluster2]                                                                 },
2021-07-20T04:17:54.6698857Z [cluster2]                                                                 "status": {
2021-07-20T04:17:54.6699620Z [cluster2]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-20T04:17:54.6700874Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6701420Z [cluster2]                                                                 },
2021-07-20T04:17:54.6701929Z [cluster2]                                                                 "type": {
2021-07-20T04:17:54.6702604Z [cluster2]                                                                     "description": "Type of DaemonSet condition.",
2021-07-20T04:17:54.6703315Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6705111Z [cluster2]                                                                 }
2021-07-20T04:17:54.6705667Z [cluster2]                                                             },
2021-07-20T04:17:54.6706192Z [cluster2]                                                             "required": [
2021-07-20T04:17:54.6706727Z [cluster2]                                                                 "status",
2021-07-20T04:17:54.6707269Z [cluster2]                                                                 "type"
2021-07-20T04:17:54.6707753Z [cluster2]                                                             ],
2021-07-20T04:17:54.6708430Z [cluster2]                                                             "type": "object"
2021-07-20T04:17:54.6708921Z [cluster2]                                                         },
2021-07-20T04:17:54.6709727Z [cluster2]                                                         "type": "array"
2021-07-20T04:17:54.6710498Z [cluster2]                                                     },
2021-07-20T04:17:54.6711182Z [cluster2]                                                     "currentNumberScheduled": {
2021-07-20T04:17:54.6712555Z [cluster2]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.6713771Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6714331Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6714843Z [cluster2]                                                     },
2021-07-20T04:17:54.6715458Z [cluster2]                                                     "desiredNumberScheduled": {
2021-07-20T04:17:54.6716882Z [cluster2]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.6718146Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6718703Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6719244Z [cluster2]                                                     },
2021-07-20T04:17:54.6720136Z [cluster2]                                                     "numberAvailable": {
2021-07-20T04:17:54.6721379Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-20T04:17:54.6722695Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6723489Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6724171Z [cluster2]                                                     },
2021-07-20T04:17:54.6724748Z [cluster2]                                                     "numberMisscheduled": {
2021-07-20T04:17:54.6726183Z [cluster2]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.6727775Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6728330Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6728839Z [cluster2]                                                     },
2021-07-20T04:17:54.6729343Z [cluster2]                                                     "numberReady": {
2021-07-20T04:17:54.6730264Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-20T04:17:54.6731322Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6731892Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6732415Z [cluster2]                                                     },
2021-07-20T04:17:54.6732980Z [cluster2]                                                     "numberUnavailable": {
2021-07-20T04:17:54.6734329Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-20T04:17:54.6735485Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6736036Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6736545Z [cluster2]                                                     },
2021-07-20T04:17:54.6737205Z [cluster2]                                                     "observedGeneration": {
2021-07-20T04:17:54.6738612Z [cluster2]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-20T04:17:54.6739535Z [cluster2]                                                         "format": "int64",
2021-07-20T04:17:54.6740111Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6740622Z [cluster2]                                                     },
2021-07-20T04:17:54.6741270Z [cluster2]                                                     "updatedNumberScheduled": {
2021-07-20T04:17:54.6742192Z [cluster2]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-20T04:17:54.6742947Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6743521Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6744035Z [cluster2]                                                     }
2021-07-20T04:17:54.6744507Z [cluster2]                                                 },
2021-07-20T04:17:54.6745002Z [cluster2]                                                 "required": [
2021-07-20T04:17:54.6745688Z [cluster2]                                                     "currentNumberScheduled",
2021-07-20T04:17:54.6746510Z [cluster2]                                                     "desiredNumberScheduled",
2021-07-20T04:17:54.6747264Z [cluster2]                                                     "numberMisscheduled",
2021-07-20T04:17:54.6747922Z [cluster2]                                                     "numberReady"
2021-07-20T04:17:54.6748433Z [cluster2]                                                 ],
2021-07-20T04:17:54.6748949Z [cluster2]                                                 "type": "object"
2021-07-20T04:17:54.6749596Z [cluster2]                                             }
2021-07-20T04:17:54.6750150Z [cluster2]                                         },
2021-07-20T04:17:54.6750643Z [cluster2]                                         "required": [
2021-07-20T04:17:54.6751311Z [cluster2]                                             "mismatchedContainerImages"
2021-07-20T04:17:54.6751952Z [cluster2]                                         ],
2021-07-20T04:17:54.6752417Z [cluster2]                                         "type": "object"
2021-07-20T04:17:54.6752896Z [cluster2]                                     },
2021-07-20T04:17:54.6753641Z [cluster2]                                     "loadBalancerStatus": {
2021-07-20T04:17:54.6754281Z [cluster2]                                         "properties": {
2021-07-20T04:17:54.6754823Z [cluster2]                                             "status": {
2021-07-20T04:17:54.6756051Z [cluster2]                                                 "description": "LoadBalancerStatus represents the status of a load-balancer.",
2021-07-20T04:17:54.6757083Z [cluster2]                                                 "properties": {
2021-07-20T04:17:54.6757608Z [cluster2]                                                     "ingress": {
2021-07-20T04:17:54.6758982Z [cluster2]                                                         "description": "Ingress is a list containing ingress points for the load-balancer. Traffic intended for the service should be sent to these ingress points.",
2021-07-20T04:17:54.6760322Z [cluster2]                                                         "items": {
2021-07-20T04:17:54.6762163Z [cluster2]                                                             "description": "LoadBalancerIngress represents the status of a load-balancer ingress point: traffic intended for the service should be sent to an ingress point.",
2021-07-20T04:17:54.6763498Z [cluster2]                                                             "properties": {
2021-07-20T04:17:54.6764046Z [cluster2]                                                                 "hostname": {
2021-07-20T04:17:54.6765289Z [cluster2]                                                                     "description": "Hostname is set for load-balancer ingress points that are DNS based (typically AWS load-balancers)",
2021-07-20T04:17:54.6766223Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6766718Z [cluster2]                                                                 },
2021-07-20T04:17:54.6767198Z [cluster2]                                                                 "ip": {
2021-07-20T04:17:54.6769125Z [cluster2]                                                                     "description": "IP is set for load-balancer ingress points that are IP based (typically GCE or OpenStack load-balancers)",
2021-07-20T04:17:54.6770071Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6770567Z [cluster2]                                                                 }
2021-07-20T04:17:54.6771367Z [cluster2]                                                             },
2021-07-20T04:17:54.6772043Z [cluster2]                                                             "type": "object"
2021-07-20T04:17:54.6772533Z [cluster2]                                                         },
2021-07-20T04:17:54.6773030Z [cluster2]                                                         "type": "array"
2021-07-20T04:17:54.6773510Z [cluster2]                                                     }
2021-07-20T04:17:54.6773963Z [cluster2]                                                 },
2021-07-20T04:17:54.6774458Z [cluster2]                                                 "type": "object"
2021-07-20T04:17:54.6775074Z [cluster2]                                             }
2021-07-20T04:17:54.6775495Z [cluster2]                                         },
2021-07-20T04:17:54.6778106Z [cluster2]                                         "type": "object"
2021-07-20T04:17:54.6778995Z [cluster2]                                     },
2021-07-20T04:17:54.6779518Z [cluster2]                                     "natEnabled": {
2021-07-20T04:17:54.6780051Z [cluster2]                                         "type": "boolean"
2021-07-20T04:17:54.6780520Z [cluster2]                                     },
2021-07-20T04:17:54.6781032Z [cluster2]                                     "networkPlugin": {
2021-07-20T04:17:54.6781667Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6782136Z [cluster2]                                     },
2021-07-20T04:17:54.6782784Z [cluster2]                                     "routeAgentDaemonSetStatus": {
2021-07-20T04:17:54.6785424Z [cluster2]                                         "properties": {
2021-07-20T04:17:54.6786637Z [cluster2]                                             "lastResourceVersion": {
2021-07-20T04:17:54.6787328Z [cluster2]                                                 "type": "string"
2021-07-20T04:17:54.6787874Z [cluster2]                                             },
2021-07-20T04:17:54.6788859Z [cluster2]                                             "mismatchedContainerImages": {
2021-07-20T04:17:54.6789917Z [cluster2]                                                 "type": "boolean"
2021-07-20T04:17:54.6790399Z [cluster2]                                             },
2021-07-20T04:17:54.6790992Z [cluster2]                                             "nonReadyContainerStates": {
2021-07-20T04:17:54.6791619Z [cluster2]                                                 "items": {
2021-07-20T04:17:54.6792919Z [cluster2]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-20T04:17:54.6794104Z [cluster2]                                                     "properties": {
2021-07-20T04:17:54.6794633Z [cluster2]                                                         "running": {
2021-07-20T04:17:54.6795268Z [cluster2]                                                             "description": "Details about a running container",
2021-07-20T04:17:54.6796286Z [cluster2]                                                             "properties": {
2021-07-20T04:17:54.6797008Z [cluster2]                                                                 "startedAt": {
2021-07-20T04:17:54.6798115Z [cluster2]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-20T04:17:54.6799365Z [cluster2]                                                                     "format": "date-time",
2021-07-20T04:17:54.6800534Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6801540Z [cluster2]                                                                 }
2021-07-20T04:17:54.6802330Z [cluster2]                                                             },
2021-07-20T04:17:54.6803216Z [cluster2]                                                             "type": "object"
2021-07-20T04:17:54.6804060Z [cluster2]                                                         },
2021-07-20T04:17:54.6804920Z [cluster2]                                                         "terminated": {
2021-07-20T04:17:54.6806092Z [cluster2]                                                             "description": "Details about a terminated container",
2021-07-20T04:17:54.6806965Z [cluster2]                                                             "properties": {
2021-07-20T04:17:54.6807530Z [cluster2]                                                                 "containerID": {
2021-07-20T04:17:54.6808618Z [cluster2]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-20T04:17:54.6809525Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6810230Z [cluster2]                                                                 },
2021-07-20T04:17:54.6810913Z [cluster2]                                                                 "exitCode": {
2021-07-20T04:17:54.6811914Z [cluster2]                                                                     "description": "Exit status from the last termination of the container",
2021-07-20T04:17:54.6812824Z [cluster2]                                                                     "format": "int32",
2021-07-20T04:17:54.6813949Z [cluster2]                                                                     "type": "integer"
2021-07-20T04:17:54.6814864Z [cluster2]                                                                 },
2021-07-20T04:17:54.6815408Z [cluster2]                                                                 "finishedAt": {
2021-07-20T04:17:54.6816210Z [cluster2]                                                                     "description": "Time at which the container last terminated",
2021-07-20T04:17:54.6817675Z [cluster2]                                                                     "format": "date-time",
2021-07-20T04:17:54.6818441Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6819424Z [cluster2]                                                                 },
2021-07-20T04:17:54.6820855Z [cluster2]                                                                 "message": {
2021-07-20T04:17:54.6822135Z [cluster2]                                                                     "description": "Message regarding the last termination of the container",
2021-07-20T04:17:54.6823422Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6823935Z [cluster2]                                                                 },
2021-07-20T04:17:54.6824424Z [cluster2]                                                                 "reason": {
2021-07-20T04:17:54.6825554Z [cluster2]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-20T04:17:54.6826836Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6827389Z [cluster2]                                                                 },
2021-07-20T04:17:54.6828555Z [cluster2]                                                                 "signal": {
2021-07-20T04:17:54.6830184Z [cluster2]                                                                     "description": "Signal from the last termination of the container",
2021-07-20T04:17:54.6831130Z [cluster2]                                                                     "format": "int32",
2021-07-20T04:17:54.6831703Z [cluster2]                                                                     "type": "integer"
2021-07-20T04:17:54.6832219Z [cluster2]                                                                 },
2021-07-20T04:17:54.6832737Z [cluster2]                                                                 "startedAt": {
2021-07-20T04:17:54.6833687Z [cluster2]                                                                     "description": "Time at which previous execution of the container started",
2021-07-20T04:17:54.6834990Z [cluster2]                                                                     "format": "date-time",
2021-07-20T04:17:54.6835766Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6836302Z [cluster2]                                                                 }
2021-07-20T04:17:54.6837002Z [cluster2]                                                             },
2021-07-20T04:17:54.6837526Z [cluster2]                                                             "required": [
2021-07-20T04:17:54.6838250Z [cluster2]                                                                 "exitCode"
2021-07-20T04:17:54.6838945Z [cluster2]                                                             ],
2021-07-20T04:17:54.6839476Z [cluster2]                                                             "type": "object"
2021-07-20T04:17:54.6840058Z [cluster2]                                                         },
2021-07-20T04:17:54.6840569Z [cluster2]                                                         "waiting": {
2021-07-20T04:17:54.6841271Z [cluster2]                                                             "description": "Details about a waiting container",
2021-07-20T04:17:54.6841978Z [cluster2]                                                             "properties": {
2021-07-20T04:17:54.6842552Z [cluster2]                                                                 "message": {
2021-07-20T04:17:54.6843725Z [cluster2]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-20T04:17:54.6844507Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6845037Z [cluster2]                                                                 },
2021-07-20T04:17:54.6845535Z [cluster2]                                                                 "reason": {
2021-07-20T04:17:54.6846264Z [cluster2]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-20T04:17:54.6847089Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6847619Z [cluster2]                                                                 }
2021-07-20T04:17:54.6848103Z [cluster2]                                                             },
2021-07-20T04:17:54.6848611Z [cluster2]                                                             "type": "object"
2021-07-20T04:17:54.6849139Z [cluster2]                                                         }
2021-07-20T04:17:54.6849599Z [cluster2]                                                     },
2021-07-20T04:17:54.6850116Z [cluster2]                                                     "type": "object"
2021-07-20T04:17:54.6850616Z [cluster2]                                                 },
2021-07-20T04:17:54.6851115Z [cluster2]                                                 "type": "array"
2021-07-20T04:17:54.6851606Z [cluster2]                                             },
2021-07-20T04:17:54.6852087Z [cluster2]                                             "status": {
2021-07-20T04:17:54.6852897Z [cluster2]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-20T04:17:54.6853720Z [cluster2]                                                 "properties": {
2021-07-20T04:17:54.6854330Z [cluster2]                                                     "collisionCount": {
2021-07-20T04:17:54.6855655Z [cluster2]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-20T04:17:54.6856952Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6857665Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6858186Z [cluster2]                                                     },
2021-07-20T04:17:54.6858707Z [cluster2]                                                     "conditions": {
2021-07-20T04:17:54.6860675Z [cluster2]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-20T04:17:54.6861719Z [cluster2]                                                         "items": {
2021-07-20T04:17:54.6862623Z [cluster2]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-20T04:17:54.6863517Z [cluster2]                                                             "properties": {
2021-07-20T04:17:54.6864787Z [cluster2]                                                                 "lastTransitionTime": {
2021-07-20T04:17:54.6865891Z [cluster2]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-20T04:17:54.6867681Z [cluster2]                                                                     "format": "date-time",
2021-07-20T04:17:54.6868498Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6869357Z [cluster2]                                                                 },
2021-07-20T04:17:54.6870848Z [cluster2]                                                                 "message": {
2021-07-20T04:17:54.6872343Z [cluster2]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-20T04:17:54.6873990Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6874711Z [cluster2]                                                                 },
2021-07-20T04:17:54.6875594Z [cluster2]                                                                 "reason": {
2021-07-20T04:17:54.6878212Z [cluster2]                                                                     "description": "The reason for the condition's last transition.",
2021-07-20T04:17:54.6879268Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6880102Z [cluster2]                                                                 },
2021-07-20T04:17:54.6881016Z [cluster2]                                                                 "status": {
2021-07-20T04:17:54.6882592Z [cluster2]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-20T04:17:54.6883617Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6884401Z [cluster2]                                                                 },
2021-07-20T04:17:54.6885183Z [cluster2]                                                                 "type": {
2021-07-20T04:17:54.6886111Z [cluster2]                                                                     "description": "Type of DaemonSet condition.",
2021-07-20T04:17:54.6888334Z [cluster2]                                                                     "type": "string"
2021-07-20T04:17:54.6888915Z [cluster2]                                                                 }
2021-07-20T04:17:54.6889399Z [cluster2]                                                             },
2021-07-20T04:17:54.6889945Z [cluster2]                                                             "required": [
2021-07-20T04:17:54.6890498Z [cluster2]                                                                 "status",
2021-07-20T04:17:54.6891045Z [cluster2]                                                                 "type"
2021-07-20T04:17:54.6891554Z [cluster2]                                                             ],
2021-07-20T04:17:54.6892110Z [cluster2]                                                             "type": "object"
2021-07-20T04:17:54.6892655Z [cluster2]                                                         },
2021-07-20T04:17:54.6893319Z [cluster2]                                                         "type": "array"
2021-07-20T04:17:54.6894011Z [cluster2]                                                     },
2021-07-20T04:17:54.6895207Z [cluster2]                                                     "currentNumberScheduled": {
2021-07-20T04:17:54.6896701Z [cluster2]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.6898394Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6898992Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6899937Z [cluster2]                                                     },
2021-07-20T04:17:54.6900772Z [cluster2]                                                     "desiredNumberScheduled": {
2021-07-20T04:17:54.6902361Z [cluster2]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.6903760Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6904342Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6904892Z [cluster2]                                                     },
2021-07-20T04:17:54.6905466Z [cluster2]                                                     "numberAvailable": {
2021-07-20T04:17:54.6906986Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-20T04:17:54.6908119Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6908678Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6909202Z [cluster2]                                                     },
2021-07-20T04:17:54.6909780Z [cluster2]                                                     "numberMisscheduled": {
2021-07-20T04:17:54.6911453Z [cluster2]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-20T04:17:54.6912714Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6913272Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6913798Z [cluster2]                                                     },
2021-07-20T04:17:54.6914316Z [cluster2]                                                     "numberReady": {
2021-07-20T04:17:54.6915676Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-20T04:17:54.6916801Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6917355Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6932042Z [cluster2]                                                     },
2021-07-20T04:17:54.6932776Z [cluster2]                                                     "numberUnavailable": {
2021-07-20T04:17:54.6934209Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-20T04:17:54.6935373Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6936120Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6936692Z [cluster2]                                                     },
2021-07-20T04:17:54.6976193Z [cluster2]                                                     "observedGeneration": {
2021-07-20T04:17:54.6977294Z [cluster2]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-20T04:17:54.6978224Z [cluster2]                                                         "format": "int64",
2021-07-20T04:17:54.6978812Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6979343Z [cluster2]                                                     },
2021-07-20T04:17:54.6980082Z [cluster2]                                                     "updatedNumberScheduled": {
2021-07-20T04:17:54.6981015Z [cluster2]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-20T04:17:54.6981772Z [cluster2]                                                         "format": "int32",
2021-07-20T04:17:54.6982343Z [cluster2]                                                         "type": "integer"
2021-07-20T04:17:54.6982853Z [cluster2]                                                     }
2021-07-20T04:17:54.6983324Z [cluster2]                                                 },
2021-07-20T04:17:54.6984050Z [cluster2]                                                 "required": [
2021-07-20T04:17:54.6984720Z [cluster2]                                                     "currentNumberScheduled",
2021-07-20T04:17:54.6985541Z [cluster2]                                                     "desiredNumberScheduled",
2021-07-20T04:17:54.6986300Z [cluster2]                                                     "numberMisscheduled",
2021-07-20T04:17:54.6986967Z [cluster2]                                                     "numberReady"
2021-07-20T04:17:54.6987633Z [cluster2]                                                 ],
2021-07-20T04:17:54.6988875Z [cluster2]                                                 "type": "object"
2021-07-20T04:17:54.6989433Z [cluster2]                                             }
2021-07-20T04:17:54.6989890Z [cluster2]                                         },
2021-07-20T04:17:54.6990399Z [cluster2]                                         "required": [
2021-07-20T04:17:54.6991128Z [cluster2]                                             "mismatchedContainerImages"
2021-07-20T04:17:54.6991823Z [cluster2]                                         ],
2021-07-20T04:17:54.6992325Z [cluster2]                                         "type": "object"
2021-07-20T04:17:54.6993148Z [cluster2]                                     },
2021-07-20T04:17:54.6993646Z [cluster2]                                     "serviceCIDR": {
2021-07-20T04:17:54.6994177Z [cluster2]                                         "type": "string"
2021-07-20T04:17:54.6994816Z [cluster2]                                     }
2021-07-20T04:17:54.6995268Z [cluster2]                                 },
2021-07-20T04:17:54.6995818Z [cluster2]                                 "required": [
2021-07-20T04:17:54.6996367Z [cluster2]                                     "clusterID",
2021-07-20T04:17:54.6997055Z [cluster2]                                     "natEnabled"
2021-07-20T04:17:54.6997625Z [cluster2]                                 ],
2021-07-20T04:17:54.6998128Z [cluster2]                                 "type": "object"
2021-07-20T04:17:54.6998711Z [cluster2]                             }
2021-07-20T04:17:54.6999160Z [cluster2]                         },
2021-07-20T04:17:54.6999723Z [cluster2]                         "type": "object"
2021-07-20T04:17:54.7000225Z [cluster2]                     }
2021-07-20T04:17:54.7000892Z [cluster2]                 },
2021-07-20T04:17:54.7001523Z [cluster2]                 "served": true,
2021-07-20T04:17:54.7002143Z [cluster2]                 "storage": true,
2021-07-20T04:17:54.7002704Z [cluster2]                 "subresources": {
2021-07-20T04:17:54.7026130Z [cluster2]                     "status": {}
2021-07-20T04:17:54.7026817Z [cluster2]                 }
2021-07-20T04:17:54.7027232Z [cluster2]             }
2021-07-20T04:17:54.7027617Z [cluster2]         ]
2021-07-20T04:17:54.7028015Z [cluster2]     },
2021-07-20T04:17:54.7028418Z [cluster2]     "status": {
2021-07-20T04:17:54.7028939Z [cluster2]         "acceptedNames": {
2021-07-20T04:17:54.7029483Z [cluster2]             "kind": "Submariner",
2021-07-20T04:17:54.7030089Z [cluster2]             "listKind": "SubmarinerList",
2021-07-20T04:17:54.7030696Z [cluster2]             "plural": "submariners",
2021-07-20T04:17:54.7031291Z [cluster2]             "singular": "submariner"
2021-07-20T04:17:54.7031780Z [cluster2]         },
2021-07-20T04:17:54.7032216Z [cluster2]         "conditions": [
2021-07-20T04:17:54.7032671Z [cluster2]             {
2021-07-20T04:17:54.7033783Z [cluster2]                 "lastTransitionTime": "2021-07-20T04:13:12Z",
2021-07-20T04:17:56.7680393Z [cluster2]                 "message": "no conflicts found",
2021-07-20T04:17:56.7681041Z [cluster1]                 "reason": "NoConflicts",
2021-07-20T04:17:56.7681774Z [cluster1]                 "status": "True",
2021-07-20T04:17:56.7682329Z [cluster1]                 "type": "NamesAccepted"
2021-07-20T04:17:56.7682841Z [cluster1]             },
2021-07-20T04:17:56.7683690Z [cluster1]             {
2021-07-20T04:17:56.7684911Z [cluster1]                 "lastTransitionTime": "2021-07-20T04:13:01Z",
2021-07-20T04:17:56.7685608Z [cluster1]                 "message": "the initial names have been accepted",
2021-07-20T04:17:56.7686337Z [cluster1]                 "reason": "InitialNamesAccepted",
2021-07-20T04:17:56.7686920Z [cluster1]                 "status": "True",
2021-07-20T04:17:56.7687429Z [cluster1]                 "type": "Established"
2021-07-20T04:17:56.7687897Z [cluster1]             }
2021-07-20T04:17:56.7688260Z [cluster1]         ],
2021-07-20T04:17:56.7688718Z [cluster1]         "storedVersions": [
2021-07-20T04:17:56.7689190Z [cluster1]             "v1alpha1"
2021-07-20T04:17:56.7689599Z [cluster1]         ]
2021-07-20T04:17:56.7689952Z [cluster1]     }
2021-07-20T04:17:56.7690313Z [cluster1] }
2021-07-20T04:17:56.7691377Z [36m[submariner-operator]$ [cluster1] validate_equals .metadata.name submariners.submariner.io[0m
2021-07-20T04:17:56.7692966Z [36m[submariner-operator]$ [cluster1] validate_equals .metadata.name submariners.submariner.io[0m
2021-07-20T04:17:56.7696188Z [36m[submariner-operator]$ [cluster1] jq -r -M .metadata.name /tmp/submariners.submariner.io.cluster1.json[0m
2021-07-20T04:17:56.7698897Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced[0m
2021-07-20T04:17:56.7700063Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced[0m
2021-07-20T04:17:56.7701559Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.scope /tmp/submariners.submariner.io.cluster1.json[0m
2021-07-20T04:17:56.7703057Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io[0m
2021-07-20T04:17:56.7704224Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io[0m
2021-07-20T04:17:56.7705863Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.group /tmp/submariners.submariner.io.cluster1.json[0m
2021-07-20T04:17:56.7707406Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.names.kind Submariner[0m
2021-07-20T04:17:56.7709118Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.names.kind Submariner[0m
2021-07-20T04:17:56.7711259Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.names.kind /tmp/submariners.submariner.io.cluster1.json[0m
2021-07-20T04:17:56.7712861Z [36m[submariner-operator]$ [cluster1] verify_subm_operator[0m
2021-07-20T04:17:56.7713832Z [36m[submariner-operator]$ [cluster1] verify_subm_operator[0m
2021-07-20T04:17:56.7715436Z [36m[submariner-operator]$ [cluster1] kubectl get ns submariner-operator[0m
2021-07-20T04:17:56.7716744Z [36m[submariner-operator]$ [cluster1] kubectl get ns submariner-operator[0m
2021-07-20T04:17:56.7718085Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get ns submariner-operator[0m
2021-07-20T04:17:56.7718937Z [cluster1] NAME                  STATUS   AGE
2021-07-20T04:17:56.7719869Z [cluster1] submariner-operator   Active   4m54s
2021-07-20T04:17:56.7721274Z [36m[submariner-operator]$ [cluster1] kubectl get crds submariners.submariner.io[0m
2021-07-20T04:17:56.7722740Z [36m[submariner-operator]$ [cluster1] kubectl get crds submariners.submariner.io[0m
2021-07-20T04:17:56.7724689Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crds submariners.submariner.io[0m
2021-07-20T04:17:56.7725801Z [cluster1] NAME                        CREATED AT
2021-07-20T04:17:56.7726883Z [cluster1] submariners.submariner.io   2021-07-20T04:13:01Z
2021-07-20T04:17:56.7728141Z [36m[submariner-operator]$ [cluster1] kubectl api-resources[0m
2021-07-20T04:17:56.7729366Z [36m[submariner-operator]$ [cluster1] grep submariners[0m
2021-07-20T04:17:56.7730774Z [36m[submariner-operator]$ [cluster1] kubectl api-resources[0m
2021-07-20T04:17:56.7731949Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 api-resources[0m
2021-07-20T04:17:56.7733722Z [cluster1] submariners                                    submariner.io/v1alpha1                 true         Submariner
2021-07-20T04:17:56.7735494Z [36m[submariner-operator]$ [cluster1] kubectl get sa --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:56.7737325Z [36m[submariner-operator]$ [cluster1] kubectl get sa --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:56.7739143Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get sa --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:56.7740238Z [cluster1] NAME                  SECRETS   AGE
2021-07-20T04:17:56.7741034Z [cluster1] submariner-operator   1         4m55s
2021-07-20T04:17:56.7742347Z [36m[submariner-operator]$ [cluster1] kubectl get roles --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:56.7744025Z [36m[submariner-operator]$ [cluster1] kubectl get roles --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:56.7745829Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get roles --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:56.7746925Z [cluster1] NAME                  CREATED AT
2021-07-20T04:17:56.7747722Z [cluster1] submariner-operator   2021-07-20T04:13:01Z
2021-07-20T04:17:57.0793937Z [36m[submariner-operator]$ [cluster1] kubectl geto conflicts found",
2021-07-20T04:17:57.0794719Z [cluster2]                 "reason": "NoConflicts",
2021-07-20T04:17:57.0795278Z [cluster2]                 "status": "True",
2021-07-20T04:17:57.0796239Z [cluster2]                 "type": "NamesAccepted"
2021-07-20T04:17:57.0796779Z [cluster2]             },
2021-07-20T04:17:57.0797357Z [cluster2]             {
2021-07-20T04:17:57.0798188Z [cluster2]                 "lastTransitionTime": "2021-07-20T04:13:12Z",
2021-07-20T04:17:57.0798946Z [cluster2]                 "message": "the initial names have been accepted",
2021-07-20T04:17:57.0799707Z [cluster2]                 "reason": "InitialNamesAccepted",
2021-07-20T04:17:57.0800350Z [cluster2]                 "status": "True",
2021-07-20T04:17:57.0800994Z [cluster2]                 "type": "Established"
2021-07-20T04:17:57.0801496Z [cluster2]             }
2021-07-20T04:17:57.0801885Z [cluster2]         ],
2021-07-20T04:17:57.0802378Z [cluster2]         "storedVersions": [
2021-07-20T04:17:57.0802886Z [cluster2]             "v1alpha1"
2021-07-20T04:17:57.0803319Z [cluster2]         ]
2021-07-20T04:17:57.0803711Z [cluster2]     }
2021-07-20T04:17:57.0804083Z [cluster2] }
2021-07-20T04:17:57.0805451Z [36m[submariner-operator]$ [cluster2] validate_equals .metadata.name submariners.submariner.io[0m
2021-07-20T04:17:57.0807110Z [36m[submariner-operator]$ [cluster2] validate_equals .metadata.name submariners.submariner.io[0m
2021-07-20T04:17:57.0809278Z [36m[submariner-operator]$ [cluster2] jq -r -M .metadata.name /tmp/submariners.submariner.io.cluster2.json[0m
2021-07-20T04:17:57.0810878Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced[0m
2021-07-20T04:17:57.0812159Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced[0m
2021-07-20T04:17:57.0813767Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.scope /tmp/submariners.submariner.io.cluster2.json[0m
2021-07-20T04:17:57.0815350Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io[0m
2021-07-20T04:17:57.0816622Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io[0m
2021-07-20T04:17:57.0818856Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.group /tmp/submariners.submariner.io.cluster2.json[0m
2021-07-20T04:17:57.0820801Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.names.kind Submariner[0m
2021-07-20T04:17:57.0822116Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.names.kind Submariner[0m
2021-07-20T04:17:57.0823897Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.names.kind /tmp/submariners.submariner.io.cluster2.json[0m
2021-07-20T04:17:57.0825358Z [36m[submariner-operator]$ [cluster2] verify_subm_operator[0m
2021-07-20T04:17:57.0827409Z [36m[submariner-operator]$ [cluster2] verify_subm_operator[0m
2021-07-20T04:17:57.0828640Z [36m[submariner-operator]$ [cluster2] kubectl get ns submariner-operator[0m
2021-07-20T04:17:57.0829932Z [36m[submariner-operator]$ [cluster2] kubectl get ns submariner-operator[0m
2021-07-20T04:17:57.0831652Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get ns submariner-operator[0m
2021-07-20T04:17:57.0832533Z [cluster2] NAME                  STATUS   AGE
2021-07-20T04:17:57.0833314Z [cluster2] submariner-operator   Active   4m43s
2021-07-20T04:17:57.0834462Z [36m[submariner-operator]$ [cluster2] kubectl get crds submariners.submariner.io[0m
2021-07-20T04:17:57.0836025Z [36m[submariner-operator]$ [cluster2] kubectl get crds submariners.submariner.io[0m
2021-07-20T04:17:57.0837560Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crds submariners.submariner.io[0m
2021-07-20T04:17:57.0838672Z [cluster2] NAME                        CREATED AT
2021-07-20T04:17:57.0839761Z [cluster2] submariners.submariner.io   2021-07-20T04:13:12Z
2021-07-20T04:17:57.0840943Z [36m[submariner-operator]$ [cluster2] kubectl api-resources[0m
2021-07-20T04:17:57.0842059Z [36m[submariner-operator]$ [cluster2] kubectl api-resources[0m
2021-07-20T04:17:57.0843633Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 api-resources[0m
2021-07-20T04:17:57.0844746Z [36m[submariner-operator]$ [cluster2] grep submariners[0m
2021-07-20T04:17:57.0845606Z [cluster2] submariners                                    submariner.io/v1alpha1                 true         Submariner
2021-07-20T04:17:57.0847137Z [36m[submariner-operator]$ [cluster2] kubectl get sa --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:57.0848932Z [36m[submariner-operator]$ [cluster2] kubectl get sa --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:57.0851142Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get sa --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:57.0852251Z [cluster2] NAME                  SECRETS   AGE
2021-07-20T04:17:57.0853238Z [cluster2] submariner-operator   1         4m44s
2021-07-20T04:17:57.0854546Z [36m[submariner-operator]$ [cluster2] kubectl get roles --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:57.0856393Z [36m[submariner-operator]$ [cluster2] kubectl get roles --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:57.0858494Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get roles --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:57.0859584Z [cluster2] NAME                  CREATED AT
2021-07-20T04:17:57.0860401Z [cluster2] submariner-operator   2021-07-20T04:13:13Z
2021-07-20T04:17:58.6484751Z [36m[submariner-operator]$ [cluster2] kubectl get rolebindings --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:58.6487735Z [36m[submariner-operator]$ [cluster1] kubectl get rolebindings --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:58.6490361Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get rolebindings --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:58.6491515Z [cluster1] NAME                  ROLE                       AGE
2021-07-20T04:17:58.6493003Z [cluster1] submariner-operator   Role/submariner-operator   4m56s
2021-07-20T04:17:58.6494677Z [36m[submariner-operator]$ [cluster1] kubectl get deployments --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:58.6496397Z [36m[submariner-operator]$ [cluster1] kubectl get deployments --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:58.6498604Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get deployments --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:58.6500385Z [cluster1] NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
2021-07-20T04:17:58.6501609Z [cluster1] submariner-operator   1/1     1            1           4m50s
2021-07-20T04:17:58.6502598Z [36m[submariner-operator]$ [cluster1] verify_subm_op_pod[0m
2021-07-20T04:17:58.6503580Z [36m[submariner-operator]$ [cluster1] verify_subm_op_pod[0m
2021-07-20T04:17:58.6505164Z [36m[submariner-operator]$ [cluster1] subm_operator_pod_name=$(kubectl get pods --namespace=$subm_ns -l name=$operator_deployment_name -o=jsonpath='{.items..metadata.name}')[0m
2021-07-20T04:17:58.6507230Z [36m[submariner-operator]$ [cluster1] kubectl get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}[0m
2021-07-20T04:17:58.6509804Z [36m[submariner-operator]$ [cluster1] kubectl get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}[0m
2021-07-20T04:17:58.6511961Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}[0m
2021-07-20T04:17:58.6514153Z [36m[submariner-operator]$ [cluster1] kubectl get pod submariner-operator-5bfdb87494-bvr8z --namespace=submariner-operator -o json[0m
2021-07-20T04:17:58.6516260Z [36m[submariner-operator]$ [cluster1] kubectl get pod submariner-operator-5bfdb87494-bvr8z --namespace=submariner-operator -o json[0m
2021-07-20T04:17:58.6518713Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get pod submariner-operator-5bfdb87494-bvr8z --namespace=submariner-operator -o json[0m
2021-07-20T04:17:58.6520248Z [cluster1] {
2021-07-20T04:17:58.6520714Z [cluster1]     "apiVersion": "v1",
2021-07-20T04:17:58.6521230Z [cluster1]     "kind": "Pod",
2021-07-20T04:17:58.6521697Z [cluster1]     "metadata": {
2021-07-20T04:17:58.6522536Z [cluster1]         "creationTimestamp": "2021-07-20T04:16:44Z",
2021-07-20T04:17:58.6523979Z [cluster1]         "generateName": "submariner-operator-5bfdb87494-",
2021-07-20T04:17:58.6524758Z [cluster1]         "labels": {
2021-07-20T04:17:58.6525518Z [cluster1]             "name": "submariner-operator",
2021-07-20T04:17:58.6526581Z [cluster1]             "pod-template-hash": "5bfdb87494"
2021-07-20T04:17:58.6527141Z [cluster1]         },
2021-07-20T04:17:58.6527607Z [cluster1]         "managedFields": [
2021-07-20T04:17:58.6528450Z [cluster1]             {
2021-07-20T04:17:58.6529271Z [cluster1]                 "apiVersion": "v1",
2021-07-20T04:17:58.6530087Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-20T04:17:58.6530665Z [cluster1]                 "fieldsV1": {
2021-07-20T04:17:58.6531185Z [cluster1]                     "f:metadata": {
2021-07-20T04:17:58.6531738Z [cluster1]                         "f:generateName": {},
2021-07-20T04:17:58.6532635Z [cluster1]                         "f:labels": {
2021-07-20T04:17:58.6533564Z [cluster1]                             ".": {},
2021-07-20T04:17:58.6534003Z [cluster1]                             "f:name": {},
2021-07-20T04:17:58.6534820Z [cluster1]                             "f:pod-template-hash": {}
2021-07-20T04:17:58.6535338Z [cluster1]                         },
2021-07-20T04:17:58.6535860Z [cluster1]                         "f:ownerReferences": {
2021-07-20T04:17:58.6536367Z [cluster1]                             ".": {},
2021-07-20T04:17:58.6537363Z [cluster1]                             "k:{\"uid\":\"56805054-cd4e-419d-a3a8-e9b44852de36\"}": {
2021-07-20T04:17:58.6537967Z [cluster1]                                 ".": {},
2021-07-20T04:17:58.6538669Z [cluster1]                                 "f:apiVersion": {},
2021-07-20T04:17:58.6539311Z [cluster1]                                 "f:blockOwnerDeletion": {},
2021-07-20T04:17:58.6539927Z [cluster1]                                 "f:controller": {},
2021-07-20T04:17:58.6540615Z [cluster1]                                 "f:kind": {},
2021-07-20T04:17:58.6541495Z [cluster1]                                 "f:name": {},
2021-07-20T04:17:58.6542139Z [cluster1]                                 "f:uid": {}
2021-07-20T04:17:58.6542586Z [cluster1]                             }
2021-07-20T04:17:58.6544311Z [cluster1]                         }
2021-07-20T04:17:58.6546713Z [cluster1]                     },
2021-07-20T04:17:58.6547363Z [cluster1]                     "f:spec": {
2021-07-20T04:17:58.6548018Z [cluster1]                         "f:containers": {
2021-07-20T04:17:58.6550699Z [cluster1]                             "k:{\"name\":\"submariner-operator\"}": {
2021-07-20T04:17:58.6551309Z [cluster1]                                 ".": {},
2021-07-20T04:17:58.6551797Z [cluster1]                                 "f:command": {},
2021-07-20T04:17:58.6552279Z [cluster1]                                 "f:env": {
2021-07-20T04:17:58.6556891Z [cluster1]                                     ".": {},
2021-07-20T04:17:58.6557440Z [cluster1]                                     "k:{\"name\":\"OPERATOR_NAME\"}": {
2021-07-20T04:17:58.6557988Z [cluster1]                                         ".": {},
2021-07-20T04:17:58.6558491Z [cluster1]                                         "f:name": {},
2021-07-20T04:17:58.6559157Z [cluster1]                                         "f:value": {}
2021-07-20T04:17:58.6559650Z [cluster1]                                     },
2021-07-20T04:17:58.6560339Z [cluster1]                                     "k:{\"name\":\"POD_NAME\"}": {
2021-07-20T04:17:58.6560876Z [cluster1]                                         ".": {},
2021-07-20T04:17:58.6561367Z [cluster1]                                         "f:name": {},
2021-07-20T04:17:58.6561915Z [cluster1]                                         "f:valueFrom": {
2021-07-20T04:17:58.6562429Z [cluster1]                                             ".": {},
2021-07-20T04:17:58.6562957Z [cluster1]                                             "f:fieldRef": {
2021-07-20T04:17:58.6563485Z [cluster1]                                                 ".": {},
2021-07-20T04:17:58.6564022Z [cluster1]                                                 "f:apiVersion": {},
2021-07-20T04:17:58.6564627Z [cluster1]                                                 "f:fieldPath": {}
2021-07-20T04:17:58.6565298Z [cluster1]                                             }
2021-07-20T04:17:58.6565750Z [cluster1]                                         }
2021-07-20T04:17:58.6566186Z [cluster1]                                     },
2021-07-20T04:17:58.6566936Z [cluster1]                                     "k:{\"name\":\"WATCH_NAMESPACE\"}": {
2021-07-20T04:17:58.6607696Z [cluster1]                                         ".": {},
2021-07-20T04:17:58.6608380Z [cluster1]                                         "f:name": {},
2021-07-20T04:17:58.6608935Z [cluster1]                                         "f:valueFrom": {
2021-07-20T04:17:58.6609452Z [cluster1]                                             ".": {},
2021-07-20T04:17:58.6609983Z [cluster1]                                             "f:fieldRef": {
2021-07-20T04:17:58.6610675Z [cluster1]                                                 ".": {},
2021-07-20T04:17:58.6611544Z [cluster1]                                                 "f:apiVersion": {},
2021-07-20T04:17:58.6612129Z [cluster1]                                                 "f:fieldPath": {}
2021-07-20T04:17:58.6612630Z [cluster1]                                             }
2021-07-20T04:17:58.6613172Z [cluster1]                                         }
2021-07-20T04:17:58.6613658Z [cluster1]                                     }
2021-07-20T04:17:58.6614212Z [cluster1]                                 },
2021-07-20T04:17:58.6614671Z [cluster1]                                 "f:image": {},
2021-07-20T04:17:58.6615252Z [cluster1]                                 "f:imagePullPolicy": {},
2021-07-20T04:17:58.6615810Z [cluster1]                                 "f:name": {},
2021-07-20T04:17:58.6616325Z [cluster1]                                 "f:resources": {},
2021-07-20T04:17:58.6617174Z [cluster1]                                 "f:terminationMessagePath": {},
2021-07-20T04:17:58.6618206Z [cluster1]                                 "f:terminationMessagePolicy": {}
2021-07-20T04:17:58.6619052Z [cluster1]                             }
2021-07-20T04:17:58.6619480Z [cluster1]                         },
2021-07-20T04:17:58.6619983Z [cluster1]                         "f:dnsPolicy": {},
2021-07-20T04:17:58.6620614Z [cluster1]                         "f:enableServiceLinks": {},
2021-07-20T04:17:58.6621313Z [cluster1]                         "f:restartPolicy": {},
2021-07-20T04:17:58.6622071Z [cluster1]                         "f:schedulerName": {},
2021-07-20T04:17:58.6622861Z [cluster1]                         "f:securityContext": {},
2021-07-20T04:17:58.6623510Z [cluster1]                         "f:serviceAccount": {},
2021-07-20T04:17:58.6624318Z [cluster1]                         "f:serviceAccountName": {},
2021-07-20T04:17:58.6625384Z [cluster1]                         "f:terminationGracePeriodSeconds": {}
2021-07-20T04:17:58.6627549Z [cluster1]                     }
2021-07-20T04:17:58.6629536Z [cluster1]                 },
2021-07-20T04:17:58.6630814Z [cluster1]                 "manager": "kube-controller-manager",
2021-07-20T04:17:58.6631558Z [cluster1]                 "operation": "Update",
2021-07-20T04:17:58.6632330Z [cluster1]                 "time": "2021-07-20T04:16:44Z"
2021-07-20T04:17:58.6632805Z [cluster1]             },
2021-07-20T04:17:58.6633363Z [cluster1]             {
2021-07-20T04:17:58.6634027Z [cluster1]                 "apiVersion": "v1",
2021-07-20T04:17:58.6634632Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-20T04:17:58.6635342Z [cluster1]                 "fieldsV1": {
2021-07-20T04:17:58.6639778Z [cluster1]                     "f:status": {
2021-07-20T04:17:58.6640379Z [cluster1]                         "f:conditions": {
2021-07-20T04:17:58.6641028Z [cluster1]                             "k:{\"type\":\"ContainersReady\"}": {
2021-07-20T04:17:58.6641611Z [cluster1]                                 ".": {},
2021-07-20T04:17:58.6642179Z [cluster1]                                 "f:lastProbeTime": {},
2021-07-20T04:17:58.6642843Z [cluster1]                                 "f:lastTransitionTime": {},
2021-07-20T04:17:58.6643471Z [cluster1]                                 "f:status": {},
2021-07-20T04:17:58.6643987Z [cluster1]                                 "f:type": {}
2021-07-20T04:17:58.6644450Z [cluster1]                             },
2021-07-20T04:17:58.6645171Z [cluster1]                             "k:{\"type\":\"Initialized\"}": {
2021-07-20T04:17:58.6645729Z [cluster1]                                 ".": {},
2021-07-20T04:17:58.6646284Z [cluster1]                                 "f:lastProbeTime": {},
2021-07-20T04:17:58.6646951Z [cluster1]                                 "f:lastTransitionTime": {},
2021-07-20T04:17:58.6647743Z [cluster1]                                 "f:status": {},
2021-07-20T04:17:58.6648222Z [cluster1]                                 "f:type": {}
2021-07-20T04:17:58.6648690Z [cluster1]                             },
2021-07-20T04:17:58.6649185Z [cluster1]                             "k:{\"type\":\"Ready\"}": {
2021-07-20T04:17:58.6649661Z [cluster1]                                 ".": {},
2021-07-20T04:17:58.6650198Z [cluster1]                                 "f:lastProbeTime": {},
2021-07-20T04:17:58.6650841Z [cluster1]                                 "f:lastTransitionTime": {},
2021-07-20T04:17:58.6651451Z [cluster1]                                 "f:status": {},
2021-07-20T04:17:58.6652111Z [cluster1]                                 "f:type": {}
2021-07-20T04:17:58.6652587Z [cluster1]                             }
2021-07-20T04:17:58.6653013Z [cluster1]                         },
2021-07-20T04:17:58.6653583Z [cluster1]                         "f:containerStatuses": {},
2021-07-20T04:17:58.6654190Z [cluster1]                         "f:hostIP": {},
2021-07-20T04:17:58.6654689Z [cluster1]                         "f:phase": {},
2021-07-20T04:17:58.6655317Z [cluster1]                         "f:podIP": {},
2021-07-20T04:17:58.6655953Z [cluster1]                         "f:podIPs": {
2021-07-20T04:17:58.6656422Z [cluster1]                             ".": {},
2021-07-20T04:17:58.6656882Z [cluster1]                             "k:{\"ip\":\"10.1.224.4\"}": {
2021-07-20T04:17:58.6657593Z [cluster1]                                 ".": {},
2021-07-20T04:17:58.6658043Z [cluster1]                                 "f:ip": {}
2021-07-20T04:17:58.6658499Z [cluster1]                             }
2021-07-20T04:17:58.6658932Z [cluster1]                         },
2021-07-20T04:17:58.6659553Z [cluster1]                         "f:startTime": {}
2021-07-20T04:17:58.6660041Z [cluster1]                     }
2021-07-20T04:17:58.6660452Z [cluster1]                 },
2021-07-20T04:17:58.6660956Z [cluster1]                 "manager": "kubelet",
2021-07-20T04:17:58.6661520Z [cluster1]                 "operation": "Update",
2021-07-20T04:17:58.6662433Z [cluster1]                 "time": "2021-07-20T04:16:45Z"
2021-07-20T04:17:58.6662903Z [cluster1]             }
2021-07-20T04:17:58.6663319Z [cluster1]         ],
2021-07-20T04:17:58.6664343Z [cluster1]         "name": "submariner-operator-5bfdb87494-bvr8z",
2021-07-20T04:17:58.6665547Z [cluster1]         "namespace": "submariner-operator",
2021-07-20T04:17:58.6666248Z [cluster1]         "ownerReferences": [
2021-07-20T04:17:58.6666775Z [cluster1]             {
2021-07-20T04:17:58.6667449Z [cluster1]                 "apiVersion": "apps/v1",
2021-07-20T04:17:58.6668116Z [cluster1]                 "blockOwnerDeletion": true,
2021-07-20T04:17:58.6668776Z [cluster1]                 "controller": true,
2021-07-20T04:17:58.6669337Z [cluster1]                 "kind": "ReplicaSet",
2021-07-20T04:17:58.6670679Z [cluster1]                 "name": "submariner-operator-5bfdb87494",
2021-07-20T04:17:58.6671791Z [cluster1]                 "uid": "56805054-cd4e-419d-a3a8-e9b44852de36"
2021-07-20T04:17:58.6672366Z [cluster1]             }
2021-07-20T04:17:58.6672764Z [cluster1]         ],
2021-07-20T04:17:58.6673272Z [cluster1]         "resourceVersion": "2285",
2021-07-20T04:17:58.6675260Z [cluster1]         "uid": "b385b415-0816-433d-96bd-32fde9de6373"
2021-07-20T04:17:58.6676181Z [cluster1]     },
2021-07-20T04:17:58.6676625Z [cluster1]     "spec": {
2021-07-20T04:17:58.6677095Z [cluster1]         "containers": [
2021-07-20T04:17:58.6677569Z [cluster1]             {
2021-07-20T04:17:58.6678157Z [cluster1]                 "command": [
2021-07-20T04:17:58.6680773Z [cluster1]                     "submariner-operator",
2021-07-20T04:17:58.6681710Z [cluster1]                     "-alsologtostderr",
2021-07-20T04:17:58.6682438Z [cluster1]                     "-v=1"
2021-07-20T04:17:58.6682868Z [cluster1]                 ],
2021-07-20T04:17:58.6684156Z [cluster1]                 "env": [
2021-07-20T04:17:58.6684581Z [cluster1]                     {
2021-07-20T04:17:58.6685104Z [cluster1]                         "name": "WATCH_NAMESPACE",
2021-07-20T04:17:58.6686007Z [cluster1]                         "valueFrom": {
2021-07-20T04:17:58.6686739Z [cluster1]                             "fieldRef": {
2021-07-20T04:17:58.6687455Z [cluster1]                                 "apiVersion": "v1",
2021-07-20T04:17:58.6688116Z [cluster1]                                 "fieldPath": "metadata.namespace"
2021-07-20T04:17:58.6688727Z [cluster1]                             }
2021-07-20T04:17:58.6689136Z [cluster1]                         }
2021-07-20T04:17:58.6689606Z [cluster1]                     },
2021-07-20T04:17:58.6690600Z [cluster1]                     {
2021-07-20T04:17:58.6691137Z [cluster1]                         "name": "POD_NAME",
2021-07-20T04:17:58.6692719Z [cluster1]                         "valueFrom": {
2021-07-20T04:17:58.6693253Z [cluster1]                             "fieldRef": {
2021-07-20T04:17:58.6693800Z [cluster1]                                 "apiVersion": "v1",
2021-07-20T04:17:58.6694408Z [cluster1]                                 "fieldPath": "metadata.name"
2021-07-20T04:17:58.6695144Z [cluster1]                             }
2021-07-20T04:17:58.6695559Z [cluster1]                         }
2021-07-20T04:17:58.6695981Z [cluster1]                     },
2021-07-20T04:17:58.6696419Z [cluster1]                     {
2021-07-20T04:17:58.6696918Z [cluster1]                         "name": "OPERATOR_NAME",
2021-07-20T04:17:58.6738316Z [cluster1]                         "value": "submariner-operator"
2021-07-20T04:17:58.6739288Z [cluster1]                     }
2021-07-20T04:17:58.6739878Z [cluster1]                 ],
2021-07-20T04:17:58.6740940Z [cluster1]                 "image": "localhost:5000/submariner-operator:local",
2021-07-20T04:17:58.6741811Z [cluster1]                 "imagePullPolicy": "IfNotPresent",
2021-07-20T04:17:58.6742772Z [cluster1]                 "name": "submariner-operator",
2021-07-20T04:17:58.6743419Z [cluster1]                 "resources": {},
2021-07-20T04:17:58.6744405Z [cluster1]                 "terminationMessagePath": "/dev/termination-log",
2021-07-20T04:17:58.6745387Z [cluster1]                 "terminationMessagePolicy": "File",
2021-07-20T04:17:58.6746101Z [cluster1]                 "volumeMounts": [
2021-07-20T04:17:58.6746600Z [cluster1]                     {
2021-07-20T04:17:58.6747298Z [cluster1]                         "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
2021-07-20T04:17:58.6748819Z [cluster1]                         "name": "submariner-operator-token-7fkk7",
2021-07-20T04:17:58.6749601Z [cluster1]                         "readOnly": true
2021-07-20T04:17:58.6750043Z [cluster1]                     }
2021-07-20T04:17:58.6750438Z [cluster1]                 ]
2021-07-20T04:17:58.6751190Z [cluster1]             }
2021-07-20T04:17:58.6751604Z [cluster1]         ],
2021-07-20T04:17:58.6752128Z [cluster1]         "dnsPolicy": "ClusterFirst",
2021-07-20T04:17:58.6752827Z [cluster1]         "enableServiceLinks": true,
2021-07-20T04:17:58.6753735Z [cluster1]         "nodeName": "cluster1-worker",
2021-07-20T04:17:58.6754599Z [cluster1]         "preemptionPolicy": "PreemptLowerPriority",
2021-07-20T04:17:58.6755341Z [cluster1]         "priority": 0,
2021-07-20T04:17:58.6755895Z [cluster1]         "restartPolicy": "Always",
2021-07-20T04:17:58.6756988Z [cluster1]         "schedulerName": "default-scheduler",
2021-07-20T04:17:58.6757674Z [cluster1]         "securityContext": {},
2021-07-20T04:17:58.6758597Z [cluster1]         "serviceAccount": "submariner-operator",
2021-07-20T04:17:58.6759887Z [cluster1]         "serviceAccountName": "submariner-operator",
2021-07-20T04:17:58.6760880Z [cluster1]         "terminationGracePeriodSeconds": 30,
2021-07-20T04:17:58.6761787Z [cluster1]         "tolerations": [
2021-07-20T04:17:58.6762267Z [cluster1]             {
2021-07-20T04:17:58.6762759Z [cluster1]                 "effect": "NoExecute",
2021-07-20T04:17:58.6763712Z [cluster1]                 "key": "node.kubernetes.io/not-ready",
2021-07-20T04:17:58.6764406Z [cluster1]                 "operator": "Exists",
2021-07-20T04:17:58.6765342Z [cluster1]                 "tolerationSeconds": 300
2021-07-20T04:17:58.6766033Z [cluster1]             },
2021-07-20T04:17:58.6766418Z [cluster1]             {
2021-07-20T04:17:58.6767089Z [cluster1]                 "effect": "NoExecute",
2021-07-20T04:17:58.6767790Z [cluster1]                 "key": "node.kubernetes.io/unreachable",
2021-07-20T04:17:58.6768645Z [cluster1]                 "operator": "Exists",
2021-07-20T04:17:58.6769413Z [cluster1]                 "tolerationSeconds": 300
2021-07-20T04:17:58.6770113Z [cluster1]             }
2021-07-20T04:17:58.6770686Z [cluster1]         ],
2021-07-20T04:17:58.6771132Z [cluster1]         "volumes": [
2021-07-20T04:17:58.6771582Z [cluster1]             {
2021-07-20T04:17:58.6772557Z [cluster1]                 "name": "submariner-operator-token-7fkk7",
2021-07-20T04:17:58.6773348Z [cluster1]                 "secret": {
2021-07-20T04:17:58.6774042Z [cluster1]                     "defaultMode": 420,
2021-07-20T04:17:58.6775832Z [cluster1]                     "secretName": "submariner-operator-token-7fkk7"
2021-07-20T04:17:58.6776639Z [cluster1]                 }
2021-07-20T04:17:58.6777212Z [cluster1]             }
2021-07-20T04:17:58.6777622Z [cluster1]         ]
2021-07-20T04:17:58.6778030Z [cluster1]     },
2021-07-20T04:17:58.6778446Z [cluster1]     "status": {
2021-07-20T04:17:58.6778935Z [cluster1]         "conditions": [
2021-07-20T04:17:58.6779385Z [cluster1]             {
2021-07-20T04:17:58.6779922Z [cluster1]                 "lastProbeTime": null,
2021-07-20T04:17:58.6780934Z [cluster1]                 "lastTransitionTime": "2021-07-20T04:16:44Z",
2021-07-20T04:17:58.6781561Z [cluster1]                 "status": "True",
2021-07-20T04:17:58.6782123Z [cluster1]                 "type": "Initialized"
2021-07-20T04:17:58.6782612Z [cluster1]             },
2021-07-20T04:17:58.6783030Z [cluster1]             {
2021-07-20T04:17:58.6783539Z [cluster1]                 "lastProbeTime": null,
2021-07-20T04:17:58.6784463Z [cluster1]                 "lastTransitionTime": "2021-07-20T04:16:45Z",
2021-07-20T04:17:58.6785095Z [cluster1]                 "status": "True",
2021-07-20T04:17:58.6785615Z [cluster1]                 "type": "Ready"
2021-07-20T04:17:58.6786063Z [cluster1]             },
2021-07-20T04:17:58.6786477Z [cluster1]             {
2021-07-20T04:17:58.6787004Z [cluster1]                 "lastProbeTime": null,
2021-07-20T04:17:58.6787910Z [cluster1]                 "lastTransitionTime": "2021-07-20T04:16:45Z",
2021-07-20T04:17:58.6788706Z [cluster1]                 "status": "True",
2021-07-20T04:17:58.6789502Z [cluster1]                 "type": "ContainersReady"
2021-07-20T04:17:58.6790068Z [cluster1]             },
2021-07-20T04:17:58.6790468Z [cluster1]             {
2021-07-20T04:17:58.6790994Z [cluster1]                 "lastProbeTime": null,
2021-07-20T04:17:58.6791903Z [cluster1]                 "lastTransitionTime": "2021-07-20T04:16:44Z",
2021-07-20T04:17:58.6792683Z [cluster1]                 "status": "True",
2021-07-20T04:17:58.6793229Z [cluster1]                 "type": "PodScheduled"
2021-07-20T04:17:58.6793726Z [cluster1]             }
2021-07-20T04:17:58.6794124Z [cluster1]         ],
2021-07-20T04:17:58.6794624Z [cluster1]         "containerStatuses": [
2021-07-20T04:17:58.6795137Z [cluster1]             {
2021-07-20T04:17:58.6796656Z [cluster1]                 "containerID": "containerd://9d8e1969f937c8b40d6578416f15f030fb52e15178f7f026900c9c8899697cc9",
2021-07-20T04:17:58.6799002Z [cluster1]                 "image": "localhost:5000/submariner-operator:local",
2021-07-20T04:17:58.6800687Z [cluster1]                 "imageID": "localhost:5000/submariner-operator@sha256:4fc688208e3986e77644027577c7d3896e83b320cc64da4dcdbe1f94adcbda8b",
2021-07-20T04:17:58.6801834Z [cluster1]                 "lastState": {},
2021-07-20T04:17:58.6802687Z [cluster1]                 "name": "submariner-operator",
2021-07-20T04:17:58.6803304Z [cluster1]                 "ready": true,
2021-07-20T04:17:58.6803848Z [cluster1]                 "restartCount": 0,
2021-07-20T04:17:58.6804394Z [cluster1]                 "started": true,
2021-07-20T04:17:58.6804897Z [cluster1]                 "state": {
2021-07-20T04:17:58.6805367Z [cluster1]                     "running": {
2021-07-20T04:17:58.9006197Z [cluster1]                         "startedAt": "2 rolebindings --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:58.9008049Z [36m[submariner-operator]$ [cluster2] kubectl get rolebindings --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:58.9010076Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get rolebindings --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:58.9011237Z [cluster2] NAME                  ROLE                       AGE
2021-07-20T04:17:58.9012418Z [cluster2] submariner-operator   Role/submariner-operator   4m44s
2021-07-20T04:17:58.9014203Z [36m[submariner-operator]$ [cluster2] kubectl get deployments --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:58.9016227Z [36m[submariner-operator]$ [cluster2] kubectl get deployments --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:58.9018274Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get deployments --namespace=submariner-operator submariner-operator[0m
2021-07-20T04:17:58.9019662Z [cluster2] NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
2021-07-20T04:17:58.9020577Z [cluster2] submariner-operator   1/1     1            1           4m40s
2021-07-20T04:17:58.9021515Z [36m[submariner-operator]$ [cluster2] verify_subm_op_pod[0m
2021-07-20T04:17:58.9022473Z [36m[submariner-operator]$ [cluster2] verify_subm_op_pod[0m
2021-07-20T04:17:58.9023971Z [36m[submariner-operator]$ [cluster2] subm_operator_pod_name=$(kubectl get pods --namespace=$subm_ns -l name=$operator_deployment_name -o=jsonpath='{.items..metadata.name}')[0m
2021-07-20T04:17:58.9025964Z [36m[submariner-operator]$ [cluster2] kubectl get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}[0m
2021-07-20T04:17:58.9028011Z [36m[submariner-operator]$ [cluster2] kubectl get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}[0m
2021-07-20T04:17:58.9030282Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}[0m
2021-07-20T04:17:58.9032493Z [36m[submariner-operator]$ [cluster2] kubectl get pod submariner-operator-5bfdb87494-j25fv --namespace=submariner-operator -o json[0m
2021-07-20T04:17:58.9034877Z [36m[submariner-operator]$ [cluster2] kubectl get pod submariner-operator-5bfdb87494-j25fv --namespace=submariner-operator -o json[0m
2021-07-20T04:17:58.9037058Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pod submariner-operator-5bfdb87494-j25fv --namespace=submariner-operator -o json[0m
2021-07-20T04:17:58.9038243Z [cluster2] {
2021-07-20T04:17:58.9038661Z [cluster2]     "apiVersion": "v1",
2021-07-20T04:17:58.9039123Z [cluster2]     "kind": "Pod",
2021-07-20T04:17:58.9039559Z [cluster2]     "metadata": {
2021-07-20T04:17:58.9040308Z [cluster2]         "creationTimestamp": "2021-07-20T04:16:52Z",
2021-07-20T04:17:58.9041360Z [cluster2]         "generateName": "submariner-operator-5bfdb87494-",
2021-07-20T04:17:58.9042277Z [cluster2]         "labels": {
2021-07-20T04:17:58.9043724Z [cluster2]             "name": "submariner-operator",
2021-07-20T04:17:58.9044657Z [cluster2]             "pod-template-hash": "5bfdb87494"
2021-07-20T04:17:58.9045378Z [cluster2]         },
2021-07-20T04:17:58.9046048Z [cluster2]         "managedFields": [
2021-07-20T04:17:58.9046549Z [cluster2]             {
2021-07-20T04:17:58.9047317Z [cluster2]                 "apiVersion": "v1",
2021-07-20T04:17:58.9047857Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-20T04:17:58.9048373Z [cluster2]                 "fieldsV1": {
2021-07-20T04:17:58.9048835Z [cluster2]                     "f:metadata": {
2021-07-20T04:17:58.9049351Z [cluster2]                         "f:generateName": {},
2021-07-20T04:17:58.9049837Z [cluster2]                         "f:labels": {
2021-07-20T04:17:58.9050268Z [cluster2]                             ".": {},
2021-07-20T04:17:58.9050685Z [cluster2]                             "f:name": {},
2021-07-20T04:17:58.9051443Z [cluster2]                             "f:pod-template-hash": {}
2021-07-20T04:17:58.9051948Z [cluster2]                         },
2021-07-20T04:17:58.9052452Z [cluster2]                         "f:ownerReferences": {
2021-07-20T04:17:58.9052960Z [cluster2]                             ".": {},
2021-07-20T04:17:58.9053765Z [cluster2]                             "k:{\"uid\":\"a5d78fe2-4110-496f-8468-cb48089964ab\"}": {
2021-07-20T04:17:58.9054348Z [cluster2]                                 ".": {},
2021-07-20T04:17:58.9054809Z [cluster2]                                 "f:apiVersion": {},
2021-07-20T04:17:58.9055547Z [cluster2]                                 "f:blockOwnerDeletion": {},
2021-07-20T04:17:58.9056127Z [cluster2]                                 "f:controller": {},
2021-07-20T04:17:58.9056613Z [cluster2]                                 "f:kind": {},
2021-07-20T04:17:58.9057260Z [cluster2]                                 "f:name": {},
2021-07-20T04:17:58.9057714Z [cluster2]                                 "f:uid": {}
2021-07-20T04:17:58.9058139Z [cluster2]                             }
2021-07-20T04:17:58.9058524Z [cluster2]                         }
2021-07-20T04:17:58.9058920Z [cluster2]                     },
2021-07-20T04:17:58.9059322Z [cluster2]                     "f:spec": {
2021-07-20T04:17:58.9059796Z [cluster2]                         "f:containers": {
2021-07-20T04:17:58.9086861Z [cluster2]                             "k:{\"name\":\"submariner-operator\"}": {
2021-07-20T04:17:58.9087626Z [cluster2]                                 ".": {},
2021-07-20T04:17:58.9088089Z [cluster2]                                 "f:command": {},
2021-07-20T04:17:58.9088556Z [cluster2]                                 "f:env": {
2021-07-20T04:17:58.9088976Z [cluster2]                                     ".": {},
2021-07-20T04:17:58.9089484Z [cluster2]                                     "k:{\"name\":\"OPERATOR_NAME\"}": {
2021-07-20T04:17:58.9089976Z [cluster2]                                         ".": {},
2021-07-20T04:17:58.9090434Z [cluster2]                                         "f:name": {},
2021-07-20T04:17:58.9091045Z [cluster2]                                         "f:value": {}
2021-07-20T04:17:58.9091491Z [cluster2]                                     },
2021-07-20T04:17:58.9091971Z [cluster2]                                     "k:{\"name\":\"POD_NAME\"}": {
2021-07-20T04:17:58.9092527Z [cluster2]                                         ".": {},
2021-07-20T04:17:58.9092978Z [cluster2]                                         "f:name": {},
2021-07-20T04:17:58.9093463Z [cluster2]                                         "f:valueFrom": {
2021-07-20T04:17:58.9094554Z [cluster2]                                             ".": {},
2021-07-20T04:17:58.9095002Z [cluster2]                                             "f:fieldRef": {
2021-07-20T04:17:58.9095467Z [cluster2]                                                 ".": {},
2021-07-20T04:17:58.9095932Z [cluster2]                                                 "f:apiVersion": {},
2021-07-20T04:17:58.9096652Z [cluster2]                                                 "f:fieldPath": {}
2021-07-20T04:17:58.9097307Z [cluster2]                                             }
2021-07-20T04:17:58.9097713Z [cluster2]                                         }
2021-07-20T04:17:58.9098119Z [cluster2]                                     },
2021-07-20T04:17:58.9099833Z [cluster2]                                     "k:{\"name\":\"WATCH_NAMESPACE\"}": {
2021-07-20T04:17:58.9100458Z [cluster2]                                         ".": {},
2021-07-20T04:17:58.9100959Z [cluster2]                                         "f:name": {},
2021-07-20T04:17:58.9101511Z [cluster2]                                         "f:valueFrom": {
2021-07-20T04:17:58.9102042Z [cluster2]                                             ".": {},
2021-07-20T04:17:58.9102713Z [cluster2]                                             "f:fieldRef": {
2021-07-20T04:17:58.9103877Z [cluster2]                                                 ".": {},
2021-07-20T04:17:58.9104354Z [cluster2]                                                 "f:apiVersion": {},
2021-07-20T04:17:58.9105112Z [cluster2]                                                 "f:fieldPath": {}
2021-07-20T04:17:58.9105581Z [cluster2]                                             }
2021-07-20T04:17:58.9106008Z [cluster2]                                         }
2021-07-20T04:17:58.9106412Z [cluster2]                                     }
2021-07-20T04:17:58.9106827Z [cluster2]                                 },
2021-07-20T04:17:58.9107599Z [cluster2]                                 "f:image": {},
2021-07-20T04:17:58.9108108Z [cluster2]                                 "f:imagePullPolicy": {},
2021-07-20T04:17:58.9108819Z [cluster2]                                 "f:name": {},
2021-07-20T04:17:58.9109286Z [cluster2]                                 "f:resources": {},
2021-07-20T04:17:58.9109916Z [cluster2]                                 "f:terminationMessagePath": {},
2021-07-20T04:17:58.9111095Z [cluster2]                                 "f:terminationMessagePolicy": {}
2021-07-20T04:17:58.9111769Z [cluster2]                             }
2021-07-20T04:17:58.9112196Z [cluster2]                         },
2021-07-20T04:17:58.9112699Z [cluster2]                         "f:dnsPolicy": {},
2021-07-20T04:17:58.9113343Z [cluster2]                         "f:enableServiceLinks": {},
2021-07-20T04:17:58.9114158Z [cluster2]                         "f:restartPolicy": {},
2021-07-20T04:17:58.9114921Z [cluster2]                         "f:schedulerName": {},
2021-07-20T04:17:58.9116138Z [cluster2]                         "f:securityContext": {},
2021-07-20T04:17:58.9116897Z [cluster2]                         "f:serviceAccount": {},
2021-07-20T04:17:58.9117488Z [cluster2]                         "f:serviceAccountName": {},
2021-07-20T04:17:58.9118424Z [cluster2]                         "f:terminationGracePeriodSeconds": {}
2021-07-20T04:17:58.9119052Z [cluster2]                     }
2021-07-20T04:17:58.9119427Z [cluster2]                 },
2021-07-20T04:17:58.9120514Z [cluster2]                 "manager": "kube-controller-manager",
2021-07-20T04:17:58.9121165Z [cluster2]                 "operation": "Update",
2021-07-20T04:17:58.9122248Z [cluster2]                 "time": "2021-07-20T04:16:52Z"
2021-07-20T04:17:58.9122707Z [cluster2]             },
2021-07-20T04:17:58.9123469Z [cluster2]             {
2021-07-20T04:17:58.9123899Z [cluster2]                 "apiVersion": "v1",
2021-07-20T04:17:58.9124439Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-20T04:17:58.9124944Z [cluster2]                 "fieldsV1": {
2021-07-20T04:17:58.9125399Z [cluster2]                     "f:status": {
2021-07-20T04:17:58.9125859Z [cluster2]                         "f:conditions": {
2021-07-20T04:17:58.9126438Z [cluster2]                             "k:{\"type\":\"ContainersReady\"}": {
2021-07-20T04:17:58.9126974Z [cluster2]                                 ".": {},
2021-07-20T04:17:58.9127458Z [cluster2]                                 "f:lastProbeTime": {},
2021-07-20T04:17:58.9128209Z [cluster2]                                 "f:lastTransitionTime": {},
2021-07-20T04:17:58.9128780Z [cluster2]                                 "f:status": {},
2021-07-20T04:17:58.9129244Z [cluster2]                                 "f:type": {}
2021-07-20T04:17:58.9129659Z [cluster2]                             },
2021-07-20T04:17:58.9130158Z [cluster2]                             "k:{\"type\":\"Initialized\"}": {
2021-07-20T04:17:58.9130656Z [cluster2]                                 ".": {},
2021-07-20T04:17:58.9131144Z [cluster2]                                 "f:lastProbeTime": {},
2021-07-20T04:17:58.9131759Z [cluster2]                                 "f:lastTransitionTime": {},
2021-07-20T04:17:58.9132480Z [cluster2]                                 "f:status": {},
2021-07-20T04:17:58.9132941Z [cluster2]                                 "f:type": {}
2021-07-20T04:17:58.9133352Z [cluster2]                             },
2021-07-20T04:17:58.9133815Z [cluster2]                             "k:{\"type\":\"Ready\"}": {
2021-07-20T04:17:58.9134265Z [cluster2]                                 ".": {},
2021-07-20T04:17:58.9134764Z [cluster2]                                 "f:lastProbeTime": {},
2021-07-20T04:17:58.9135380Z [cluster2]                                 "f:lastTransitionTime": {},
2021-07-20T04:17:58.9135925Z [cluster2]                                 "f:status": {},
2021-07-20T04:17:58.9136543Z [cluster2]                                 "f:type": {}
2021-07-20T04:17:58.9137164Z [cluster2]                             }
2021-07-20T04:17:58.9137563Z [cluster2]                         },
2021-07-20T04:17:58.9138045Z [cluster2]                         "f:containerStatuses": {},
2021-07-20T04:17:58.9138574Z [cluster2]                         "f:hostIP": {},
2021-07-20T04:17:58.9139006Z [cluster2]                         "f:phase": {},
2021-07-20T04:17:58.9139450Z [cluster2]                         "f:podIP": {},
2021-07-20T04:17:58.9167492Z [cluster2]                         "f:podIPs": {
2021-07-20T04:17:58.9167940Z [cluster2]                             ".": {},
2021-07-20T04:17:58.9168557Z [cluster2]                             "k:{\"ip\":\"10.2.0.6\"}": {
2021-07-20T04:17:58.9168989Z [cluster2]                                 ".": {},
2021-07-20T04:17:58.9169422Z [cluster2]                                 "f:ip": {}
2021-07-20T04:17:58.9169978Z [cluster2]                             }
2021-07-20T04:17:58.9170365Z [cluster2]                         },
2021-07-20T04:17:58.9170793Z [cluster2]                         "f:startTime": {}
2021-07-20T04:17:58.9171324Z [cluster2]                     }
2021-07-20T04:17:58.9171685Z [cluster2]                 },
2021-07-20T04:17:58.9172123Z [cluster2]                 "manager": "kubelet",
2021-07-20T04:17:58.9172713Z [cluster2]                 "operation": "Update",
2021-07-20T04:17:58.9173511Z [cluster2]                 "time": "2021-07-20T04:16:54Z"
2021-07-20T04:17:58.9173929Z [cluster2]             }
2021-07-20T04:17:58.9174285Z [cluster2]         ],
2021-07-20T04:17:58.9175200Z [cluster2]         "name": "submariner-operator-5bfdb87494-j25fv",
2021-07-20T04:17:58.9176387Z [cluster2]         "namespace": "submariner-operator",
2021-07-20T04:17:58.9177344Z [cluster2]         "ownerReferences": [
2021-07-20T04:17:58.9177788Z [cluster2]             {
2021-07-20T04:17:58.9178399Z [cluster2]                 "apiVersion": "apps/v1",
2021-07-20T04:17:58.9178970Z [cluster2]                 "blockOwnerDeletion": true,
2021-07-20T04:17:58.9179737Z [cluster2]                 "controller": true,
2021-07-20T04:17:58.9180261Z [cluster2]                 "kind": "ReplicaSet",
2021-07-20T04:17:58.9181528Z [cluster2]                 "name": "submariner-operator-5bfdb87494",
2021-07-20T04:17:58.9182490Z [cluster2]                 "uid": "a5d78fe2-4110-496f-8468-cb48089964ab"
2021-07-20T04:17:58.9182979Z [cluster2]             }
2021-07-20T04:17:58.9183329Z [cluster2]         ],
2021-07-20T04:17:58.9183757Z [cluster2]         "resourceVersion": "2222",
2021-07-20T04:17:58.9184718Z [cluster2]         "uid": "be31fe80-0216-4364-af56-40744bf3b482"
2021-07-20T04:17:58.9185230Z [cluster2]     },
2021-07-20T04:17:58.9185596Z [cluster2]     "spec": {
2021-07-20T04:17:58.9185993Z [cluster2]         "containers": [
2021-07-20T04:17:58.9186389Z [cluster2]             {
2021-07-20T04:17:58.9186757Z [cluster2]                 "command": [
2021-07-20T04:17:58.9187493Z [cluster2]                     "submariner-operator",
2021-07-20T04:17:58.9188232Z [cluster2]                     "-alsologtostderr",
2021-07-20T04:17:58.9188842Z [cluster2]                     "-v=1"
2021-07-20T04:17:58.9189219Z [cluster2]                 ],
2021-07-20T04:17:58.9189769Z [cluster2]                 "env": [
2021-07-20T04:17:58.9190348Z [cluster2]                     {
2021-07-20T04:17:58.9191177Z [cluster2]                         "name": "WATCH_NAMESPACE",
2021-07-20T04:17:58.9191895Z [cluster2]                         "valueFrom": {
2021-07-20T04:17:58.9192813Z [cluster2]                             "fieldRef": {
2021-07-20T04:17:58.9193478Z [cluster2]                                 "apiVersion": "v1",
2021-07-20T04:17:58.9194250Z [cluster2]                                 "fieldPath": "metadata.namespace"
2021-07-20T04:17:58.9194766Z [cluster2]                             }
2021-07-20T04:17:58.9195148Z [cluster2]                         }
2021-07-20T04:17:58.9195499Z [cluster2]                     },
2021-07-20T04:17:58.9195867Z [cluster2]                     {
2021-07-20T04:17:58.9196409Z [cluster2]                         "name": "POD_NAME",
2021-07-20T04:17:58.9196867Z [cluster2]                         "valueFrom": {
2021-07-20T04:17:58.9197310Z [cluster2]                             "fieldRef": {
2021-07-20T04:17:58.9197785Z [cluster2]                                 "apiVersion": "v1",
2021-07-20T04:17:58.9198563Z [cluster2]                                 "fieldPath": "metadata.name"
2021-07-20T04:17:58.9199067Z [cluster2]                             }
2021-07-20T04:17:58.9200244Z [cluster2]                         }
2021-07-20T04:17:58.9200681Z [cluster2]                     },
2021-07-20T04:17:58.9201112Z [cluster2]                     {
2021-07-20T04:17:58.9201619Z [cluster2]                         "name": "OPERATOR_NAME",
2021-07-20T04:17:58.9202770Z [cluster2]                         "value": "submariner-operator"
2021-07-20T04:17:58.9203853Z [cluster2]                     }
2021-07-20T04:17:58.9204403Z [cluster2]                 ],
2021-07-20T04:17:58.9205181Z [cluster2]                 "image": "localhost:5000/submariner-operator:local",
2021-07-20T04:17:58.9205946Z [cluster2]                 "imagePullPolicy": "IfNotPresent",
2021-07-20T04:17:58.9206803Z [cluster2]                 "name": "submariner-operator",
2021-07-20T04:17:58.9207509Z [cluster2]                 "resources": {},
2021-07-20T04:17:58.9208410Z [cluster2]                 "terminationMessagePath": "/dev/termination-log",
2021-07-20T04:17:58.9209238Z [cluster2]                 "terminationMessagePolicy": "File",
2021-07-20T04:17:58.9209887Z [cluster2]                 "volumeMounts": [
2021-07-20T04:17:58.9210309Z [cluster2]                     {
2021-07-20T04:17:58.9210934Z [cluster2]                         "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
2021-07-20T04:17:58.9212041Z [cluster2]                         "name": "submariner-operator-token-kbk9m",
2021-07-20T04:17:58.9213336Z [cluster2]                         "readOnly": true
2021-07-20T04:17:58.9214045Z [cluster2]                     }
2021-07-20T04:17:58.9214768Z [cluster2]                 ]
2021-07-20T04:17:58.9215321Z [cluster2]             }
2021-07-20T04:17:58.9215669Z [cluster2]         ],
2021-07-20T04:17:58.9216139Z [cluster2]         "dnsPolicy": "ClusterFirst",
2021-07-20T04:17:58.9216727Z [cluster2]         "enableServiceLinks": true,
2021-07-20T04:17:58.9260191Z [cluster2]         "nodeName": "cluster2-worker2",
2021-07-20T04:17:58.9261006Z [cluster2]         "preemptionPolicy": "PreemptLowerPriority",
2021-07-20T04:17:58.9262168Z [cluster2]         "priority": 0,
2021-07-20T04:17:58.9262700Z [cluster2]         "restartPolicy": "Always",
2021-07-20T04:17:58.9263613Z [cluster2]         "schedulerName": "default-scheduler",
2021-07-20T04:17:58.9264272Z [cluster2]         "securityContext": {},
2021-07-20T04:17:58.9265121Z [cluster2]         "serviceAccount": "submariner-operator",
2021-07-20T04:17:58.9266332Z [cluster2]         "serviceAccountName": "submariner-operator",
2021-07-20T04:17:58.9267369Z [cluster2]         "terminationGracePeriodSeconds": 30,
2021-07-20T04:17:58.9268109Z [cluster2]         "tolerations": [
2021-07-20T04:17:58.9268542Z [cluster2]             {
2021-07-20T04:17:58.9269018Z [cluster2]                 "effect": "NoExecute",
2021-07-20T04:17:58.9270272Z [cluster2]                 "key": "node.kubernetes.io/not-ready",
2021-07-20T04:17:58.9270970Z [cluster2]                 "operator": "Exists",
2021-07-20T04:17:58.9271857Z [cluster2]                 "tolerationSeconds": 300
2021-07-20T04:17:58.9272402Z [cluster2]             },
2021-07-20T04:17:58.9272963Z [cluster2]             {
2021-07-20T04:17:58.9273422Z [cluster2]                 "effect": "NoExecute",
2021-07-20T04:17:58.9274092Z [cluster2]                 "key": "node.kubernetes.io/unreachable",
2021-07-20T04:17:58.9274729Z [cluster2]                 "operator": "Exists",
2021-07-20T04:17:58.9275308Z [cluster2]                 "tolerationSeconds": 300
2021-07-20T04:17:58.9275798Z [cluster2]             }
2021-07-20T04:17:58.9276183Z [cluster2]         ],
2021-07-20T04:17:58.9276729Z [cluster2]         "volumes": [
2021-07-20T04:17:58.9277151Z [cluster2]             {
2021-07-20T04:17:58.9278089Z [cluster2]                 "name": "submariner-operator-token-kbk9m",
2021-07-20T04:17:58.9278827Z [cluster2]                 "secret": {
2021-07-20T04:17:58.9279322Z [cluster2]                     "defaultMode": 420,
2021-07-20T04:17:58.9280797Z [cluster2]                     "secretName": "submariner-operator-token-kbk9m"
2021-07-20T04:17:58.9281604Z [cluster2]                 }
2021-07-20T04:17:58.9282170Z [cluster2]             }
2021-07-20T04:17:58.9282566Z [cluster2]         ]
2021-07-20T04:17:58.9283100Z [cluster2]     },
2021-07-20T04:17:58.9283503Z [cluster2]     "status": {
2021-07-20T04:17:58.9283944Z [cluster2]         "conditions": [
2021-07-20T04:17:58.9284380Z [cluster2]             {
2021-07-20T04:17:58.9284856Z [cluster2]                 "lastProbeTime": null,
2021-07-20T04:17:58.9285731Z [cluster2]                 "lastTransitionTime": "2021-07-20T04:16:52Z",
2021-07-20T04:17:58.9286326Z [cluster2]                 "status": "True",
2021-07-20T04:17:58.9286850Z [cluster2]                 "type": "Initialized"
2021-07-20T04:17:58.9287324Z [cluster2]             },
2021-07-20T04:17:58.9287701Z [cluster2]             {
2021-07-20T04:17:58.9288192Z [cluster2]                 "lastProbeTime": null,
2021-07-20T04:17:58.9289043Z [cluster2]                 "lastTransitionTime": "2021-07-20T04:16:54Z",
2021-07-20T04:17:58.9289829Z [cluster2]                 "status": "True",
2021-07-20T04:17:58.9290469Z [cluster2]                 "type": "Ready"
2021-07-20T04:17:58.9290923Z [cluster2]             },
2021-07-20T04:17:58.9291310Z [cluster2]             {
2021-07-20T04:17:58.9291819Z [cluster2]                 "lastProbeTime": null,
2021-07-20T04:17:58.9292913Z [cluster2]                 "lastTransitionTime": "2021-07-20T04:16:54Z",
2021-07-20T04:17:58.9293556Z [cluster2]                 "status": "True",
2021-07-20T04:17:58.9294317Z [cluster2]                 "type": "ContainersReady"
2021-07-20T04:17:58.9295224Z [cluster2]             },
2021-07-20T04:17:58.9295630Z [cluster2]             {
2021-07-20T04:17:58.9296125Z [cluster2]                 "lastProbeTime": null,
2021-07-20T04:17:58.9297256Z [cluster2]                 "lastTransitionTime": "2021-07-20T04:16:52Z",
2021-07-20T04:17:58.9297995Z [cluster2]                 "status": "True",
2021-07-20T04:17:58.9298553Z [cluster2]                 "type": "PodScheduled"
2021-07-20T04:17:58.9299128Z [cluster2]             }
2021-07-20T04:17:58.9299669Z [cluster2]         ],
2021-07-20T04:17:58.9301821Z [cluster2]         "containerStatuses": [
2021-07-20T04:17:58.9302425Z [cluster2]             {
2021-07-20T04:17:58.9304338Z [cluster2]                 "containerID": "containerd://25c55084eec2fb43059e8d5d6a4e8a43f6da0df433e2d6b073c88e1ac57fc874",
2021-07-20T04:17:58.9306805Z [cluster2]                 "image": "localhost:5000/submariner-operator:local",
2021-07-20T04:17:58.9308463Z [cluster2]                 "imageID": "localhost:5000/submariner-operator@sha256:4fc688208e3986e77644027577c7d3896e83b320cc64da4dcdbe1f94adcbda8b",
2021-07-20T04:17:58.9309604Z [cluster2]                 "lastState": {},
2021-07-20T04:17:58.9310592Z [cluster2]                 "name": "submariner-operator",
2021-07-20T04:17:58.9311219Z [cluster2]                 "ready": true,
2021-07-20T04:17:58.9311763Z [cluster2]                 "restartCount": 0,
2021-07-20T04:17:58.9312318Z [cluster2]                 "started": true,
2021-07-20T04:17:58.9312811Z [cluster2]                 "state": {
2021-07-20T04:17:58.9313305Z [cluster2]                     "running": {
2021-07-20T04:17:59.5867992Z [cluster2]                         "startedAt": "20021-07-20T04:16:45Z"
2021-07-20T04:17:59.5868680Z [cluster1]                     }
2021-07-20T04:17:59.5869107Z [cluster1]                 }
2021-07-20T04:17:59.5869532Z [cluster1]             }
2021-07-20T04:17:59.5869933Z [cluster1]         ],
2021-07-20T04:17:59.5870398Z [cluster1]         "hostIP": "172.18.0.4",
2021-07-20T04:17:59.5871191Z [cluster1]         "phase": "Running",
2021-07-20T04:17:59.5871693Z [cluster1]         "podIP": "10.1.224.4",
2021-07-20T04:17:59.5872180Z [cluster1]         "podIPs": [
2021-07-20T04:17:59.5872613Z [cluster1]             {
2021-07-20T04:17:59.5873057Z [cluster1]                 "ip": "10.1.224.4"
2021-07-20T04:17:59.5873484Z [cluster1]             }
2021-07-20T04:17:59.5873898Z [cluster1]         ],
2021-07-20T04:17:59.5874394Z [cluster1]         "qosClass": "BestEffort",
2021-07-20T04:17:59.5875281Z [cluster1]         "startTime": "2021-07-20T04:16:44Z"
2021-07-20T04:17:59.5875755Z [cluster1]     }
2021-07-20T04:17:59.5876156Z [cluster1] }
2021-07-20T04:17:59.5877771Z [36m[submariner-operator]$ [cluster1] kubectl get pod submariner-operator-5bfdb87494-bvr8z --namespace=submariner-operator -o jsonpath={.status.phase}[0m
2021-07-20T04:17:59.5879469Z [36m[submariner-operator]$ [cluster1] grep Running[0m
2021-07-20T04:17:59.5881270Z [36m[submariner-operator]$ [cluster1] kubectl get pod submariner-operator-5bfdb87494-bvr8z --namespace=submariner-operator -o jsonpath={.status.phase}[0m
2021-07-20T04:17:59.5883876Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get pod submariner-operator-5bfdb87494-bvr8z --namespace=submariner-operator -o jsonpath={.status.phase}[0m
2021-07-20T04:17:59.5885320Z [cluster1] Running
2021-07-20T04:17:59.5886862Z [36m[submariner-operator]$ [cluster1] kubectl logs submariner-operator-5bfdb87494-bvr8z --namespace=submariner-operator[0m
2021-07-20T04:17:59.5889072Z [36m[submariner-operator]$ [cluster1] kubectl logs submariner-operator-5bfdb87494-bvr8z --namespace=submariner-operator[0m
2021-07-20T04:17:59.5891593Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 logs submariner-operator-5bfdb87494-bvr8z --namespace=submariner-operator[0m
2021-07-20T04:17:59.5893149Z [cluster1] I0720 04:16:45.475658       1 main.go:77] cmd "msg"="Go Version: go1.16.5"
2021-07-20T04:17:59.5893945Z [cluster1] I0720 04:16:45.475720       1 main.go:78] cmd "msg"="Go OS/Arch: linux/amd64"
2021-07-20T04:17:59.5895023Z [cluster1] I0720 04:16:45.475729       1 main.go:79] cmd "msg"="Version of operator-sdk: v0.19.0"
2021-07-20T04:17:59.5896377Z [cluster1] I0720 04:16:45.475736       1 main.go:80] cmd "msg"="Submariner operator version: devel-0105e4d"
2021-07-20T04:17:59.5897540Z [cluster1] I0720 04:16:45.476199       1 leader.go:89] leader "msg"="Trying to become the leader."
2021-07-20T04:17:59.5899283Z [cluster1] I0720 04:16:46.235713       1 leader.go:126] leader "msg"="Found existing lock"  "LockOwner"="submariner-operator-586cb56578-4hd7q"
2021-07-20T04:17:59.5900504Z [cluster1] I0720 04:16:46.302957       1 leader.go:192] leader "msg"="Not the leader. Waiting."
2021-07-20T04:17:59.5901329Z [cluster1] I0720 04:16:47.443615       1 leader.go:192] leader "msg"="Not the leader. Waiting."
2021-07-20T04:17:59.5902126Z [cluster1] I0720 04:16:49.827310       1 leader.go:149] leader "msg"="Became the leader."
2021-07-20T04:17:59.5903507Z [cluster1] I0720 04:16:50.431570       1 listener.go:44] controller-runtime/metrics "msg"="metrics server is starting to listen"  "addr"="0.0.0.0:8383"
2021-07-20T04:17:59.5904547Z [cluster1] I0720 04:16:50.431972       1 main.go:129] cmd "msg"="Registering Components."
2021-07-20T04:17:59.5905353Z [cluster1] I0720 04:16:50.432145       1 main.go:137] cmd "msg"="Creating the Lighthouse CRDs"
2021-07-20T04:17:59.5906446Z [cluster1] I0720 04:16:54.431690       1 main.go:182] cmd "msg"="Could not create ServiceMonitor object"  "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.5908334Z [cluster1] I0720 04:16:54.431744       1 main.go:186] cmd "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects"  "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.5941105Z [cluster1] I0720 04:16:54.431790       1 main.go:203] cmd "msg"="Starting the Cmd."
2021-07-20T04:17:59.5942741Z [cluster1] I0720 04:16:54.432125       1 internal.go:406] controller-runtime/manager "msg"="starting metrics server"  "path"="/metrics"
2021-07-20T04:17:59.5946321Z [cluster1] I0720 04:16:54.432430       1 controller.go:158] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting EventSource"  "source"={"Type":{"status":{"deploymentInfo":{}},"metadata":{"creationTimestamp":null},"spec":{"brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","clusterID":"","namespace":"","debug":false}}}
2021-07-20T04:17:59.5985718Z [cluster1] I0720 04:16:54.432634       1 controller.go:158] controller-runtime/manager/controller/servicediscovery "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" "source"={"Type":{"status":{"deploymentInfo":{}},"metadata":{"creationTimestamp":null},"spec":{"brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","clusterID":"","namespace":"","debug":false}}}
2021-07-20T04:17:59.5989589Z [cluster1] I0720 04:16:54.432643       1 controller.go:158] controller-runtime/manager/controller/broker "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Broker" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{},"status":{}}}
2021-07-20T04:17:59.5995688Z [cluster1] I0720 04:16:54.432998       1 controller.go:158] controller-runtime/manager/controller/submariner "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"broker":"","brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","ceIPSecPSK":"","clusterCIDR":"","clusterID":"","serviceCIDR":"","namespace":"","ceIPSecDebug":false,"debug":false,"natEnabled":false},"status":{"natEnabled":false,"clusterID":"","gatewayDaemonSetStatus":{"mismatchedContainerImages":false},"routeAgentDaemonSetStatus":{"mismatchedContainerImages":false},"globalnetDaemonSetStatus":{"mismatchedContainerImages":false},"loadBalancerStatus":{},"deploymentInfo":{}}}}
2021-07-20T04:17:59.6004744Z [cluster1] I0720 04:16:54.433086       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"broker":"","brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","ceIPSecPSK":"","clusterCIDR":"","clusterID":"","serviceCIDR":"","namespace":"","ceIPSecDebug":false,"debug":false,"natEnabled":false},"status":{"natEnabled":false,"clusterID":"","gatewayDaemonSetStatus":{"mismatchedContainerImages":false},"routeAgentDaemonSetStatus":{"mismatchedContainerImages":false},"globalnetDaemonSetStatus":{"mismatchedContainerImages":false},"loadBalancerStatus":{},"deploymentInfo":{}}}}
2021-07-20T04:17:59.6010258Z [cluster1] I0720 04:16:54.533038       1 controller.go:165] controller-runtime/manager/controller/servicediscovery "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery"
2021-07-20T04:17:59.6012774Z [cluster1] I0720 04:16:54.533104       1 controller.go:192] controller-runtime/manager/controller/servicediscovery "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" "worker count"=1
2021-07-20T04:17:59.6015397Z [cluster1] I0720 04:16:54.533348       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6017974Z [cluster1] I0720 04:16:54.551466       1 controller.go:165] controller-runtime/manager/controller/broker "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:17:59.6065213Z [cluster1] I0720 04:16:54.551506       1 controller.go:192] controller-runtime/manager/controller/broker "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="Broker" "worker count"=1
2021-07-20T04:17:59.6069421Z [cluster1] I0720 04:16:54.565122       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"updateStrategy":{}},"status":{"currentNumberScheduled":0,"numberMisscheduled":0,"desiredNumberScheduled":0,"numberReady":0}}}
2021-07-20T04:17:59.6073636Z [cluster1] I0720 04:16:54.570748       1 controller.go:158] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"strategy":{}},"status":{}}}
2021-07-20T04:17:59.6077494Z [cluster1] I0720 04:16:54.570775       1 controller.go:158] controller-runtime/manager/controller/submariner "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"strategy":{}},"status":{}}}
2021-07-20T04:17:59.6080270Z [cluster1] I0720 04:16:54.671389       1 controller.go:165] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting Controller"
2021-07-20T04:17:59.6082342Z [cluster1] I0720 04:16:54.671490       1 controller.go:192] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting workers"  "worker count"=1
2021-07-20T04:17:59.6085063Z [cluster1] I0720 04:16:54.671599       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6088706Z [cluster1] I0720 04:16:54.671812       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"status":{"version":"","haStatus":"","localEndpoint":{"cluster_id":"","cable_name":"","hostname":"","subnets":null,"private_ip":"","public_ip":"","nat_enabled":false,"backend":""},"statusFailure":"","connections":null}}}
2021-07-20T04:17:59.6092443Z [cluster1] I0720 04:16:54.672374       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6095096Z [cluster1] I0720 04:16:54.684712       1 controller.go:165] controller-runtime/manager/controller/submariner "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="Submariner"
2021-07-20T04:17:59.6097625Z [cluster1] I0720 04:16:54.772440       1 controller.go:165] controller-runtime/manager/controller/submariner-controller "msg"="Starting Controller"
2021-07-20T04:17:59.6100155Z [cluster1] I0720 04:16:54.772548       1 controller.go:192] controller-runtime/manager/controller/submariner-controller "msg"="Starting workers"  "worker count"=1
2021-07-20T04:17:59.6102345Z [cluster1] I0720 04:16:54.772662       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6104617Z [cluster1] I0720 04:16:54.786343       1 controller.go:192] controller-runtime/manager/controller/submariner "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "worker count"=1
2021-07-20T04:17:59.6107181Z [cluster1] I0720 04:16:54.786498       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6110113Z [cluster1] E0720 04:16:54.823761       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:17:59.6113808Z [cluster1] E0720 04:16:55.254779       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:17:59.6117520Z [cluster1] E0720 04:16:55.549124       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:17:59.6121027Z [cluster1] E0720 04:16:55.787403       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:17:59.6124364Z [cluster1] E0720 04:16:56.011420       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:17:59.6127904Z [cluster1] E0720 04:16:56.199726       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:17:59.6130842Z [cluster1] I0720 04:16:56.420160       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6133828Z [cluster1] I0720 04:16:56.534852       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6136665Z [cluster1] I0720 04:16:56.583595       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6146779Z [cluster1] E0720 04:16:56.623610       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:17:59.6150265Z [cluster1] I0720 04:16:56.972804       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6152727Z [cluster1] I0720 04:16:57.119936       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6155567Z [cluster1] I0720 04:16:57.178525       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6158428Z [cluster1] I0720 04:16:57.315886       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6161798Z [cluster1] E0720 04:16:57.353558       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:17:59.6165081Z [cluster1] I0720 04:16:57.458537       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6168112Z [cluster1] I0720 04:16:57.458588       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6172542Z [cluster1] E0720 04:16:58.105614       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:17:59.6176010Z [cluster1] E0720 04:16:59.444270       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:17:59.6190898Z [cluster1] E0720 04:17:02.092539       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:17:59.6193842Z [cluster1] I0720 04:17:03.539495       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6196073Z [cluster1] I0720 04:17:04.229392       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6198351Z [cluster1] I0720 04:17:04.270336       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6201520Z [cluster1] I0720 04:17:04.270421       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6203804Z [cluster1] I0720 04:17:05.144855       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6206299Z [cluster1] I0720 04:17:05.145196       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6207803Z [cluster1] I0720 04:17:05.228041       1 submariner_networkdiscovery.go:42] controller_submariner "msg"="Cluster network discovered"
2021-07-20T04:17:59.6209548Z [cluster1] I0720 04:17:05.228410       1 network.go:56] controller_submariner "msg"="Discovered K8s network details"  "clusterCIDRs"=["10.1.0.0/16"] "plugin"="weave-net" "serviceCIDRs"=["100.1.0.0/16"]
2021-07-20T04:17:59.6211235Z [cluster1] I0720 04:17:05.228700       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6212767Z [cluster1] I0720 04:17:05.228853       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6215437Z [cluster1] I0720 04:17:05.245508       1 helpers.go:83] controller_submariner "msg"="Created a new DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6219682Z [cluster1] E0720 04:17:05.245879       1 controller.go:267] controller-runtime/manager/controller/submariner "msg"="Reconciler error" "error"="error creating or updating DaemonSet submariner-operator/submariner-gateway: DaemonSet.apps \"submariner-gateway\" not found" "name"="submariner" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Submariner"
2021-07-20T04:17:59.6223261Z [cluster1] I0720 04:17:05.251888       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6227108Z [cluster1] I0720 04:17:05.252231       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6228541Z [cluster1] I0720 04:17:05.252398       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6230160Z [cluster1] I0720 04:17:05.278819       1 submariner_networkdiscovery.go:42] controller_submariner "msg"="Cluster network discovered"
2021-07-20T04:17:59.6231976Z [cluster1] I0720 04:17:05.279196       1 network.go:56] controller_submariner "msg"="Discovered K8s network details"  "clusterCIDRs"=["10.1.0.0/16"] "plugin"="weave-net" "serviceCIDRs"=["100.1.0.0/16"]
2021-07-20T04:17:59.6233564Z [cluster1] I0720 04:17:05.279379       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6234943Z [cluster1] I0720 04:17:05.279517       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6237937Z [cluster1] I0720 04:17:05.291402       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6247043Z [cluster1] I0720 04:17:05.337728       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6251861Z [cluster1] I0720 04:17:05.412756       1 helpers.go:231] controller_submariner "msg"="Created a new Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6259170Z [cluster1] E0720 04:17:05.578811       1 controller.go:267] controller-runtime/manager/controller/submariner "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-gateway-metrics: services \"submariner-gateway-metrics\" already exists" "name"="submariner" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Submariner"
2021-07-20T04:17:59.6262699Z [cluster1] I0720 04:17:05.588955       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6264251Z [cluster1] I0720 04:17:05.589269       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6265531Z [cluster1] I0720 04:17:05.589475       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6296725Z [cluster1] I0720 04:17:05.610142       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6300813Z [cluster1] I0720 04:17:05.632748       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6304071Z [cluster1] I0720 04:17:06.015613       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6307485Z [cluster1] I0720 04:17:06.033183       1 helpers.go:83] controller_submariner "msg"="Created a new DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6310460Z [cluster1] I0720 04:17:06.049027       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6313326Z [cluster1] I0720 04:17:06.051118       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6316383Z [cluster1] I0720 04:17:06.057967       1 servicediscovery_resources.go:66] controller_submariner "msg"="Created Service Discovery CR" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Name"="service-discovery" "Namespace"="submariner-operator"
2021-07-20T04:17:59.6319623Z [cluster1] I0720 04:17:06.095494       1 helpers.go:129] controller_servicediscovery "msg"="Created a new Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.6327705Z [cluster1] E0720 04:17:06.134350       1 controller.go:267] controller-runtime/manager/controller/servicediscovery "msg"="Reconciler error" "error"="error creating or updating Deployment submariner-operator/submariner-lighthouse-agent: deployments.apps \"submariner-lighthouse-agent\" already exists" "name"="service-discovery" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery"
2021-07-20T04:17:59.6339024Z [cluster1] I0720 04:17:06.141015       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6342506Z [cluster1] I0720 04:17:06.158588       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.6346575Z [cluster1] I0720 04:17:06.232431       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6349572Z [cluster1] I0720 04:17:06.235474       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6351433Z [cluster1] I0720 04:17:06.235513       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6352715Z [cluster1] I0720 04:17:06.235523       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6355018Z [cluster1] I0720 04:17:06.260924       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6361982Z [cluster1] I0720 04:17:06.366585       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6365521Z [cluster1] I0720 04:17:06.429332       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6370430Z [cluster1] E0720 04:17:06.434024       1 controller.go:267] controller-runtime/manager/controller/servicediscovery "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-lighthouse-agent-metrics: services \"submariner-lighthouse-agent-metrics\" already exists" "name"="service-discovery" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery"
2021-07-20T04:17:59.6377692Z [cluster1] I0720 04:17:06.445077       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6381412Z [cluster1] E0720 04:17:06.542287       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6384591Z [cluster1] I0720 04:17:06.542336       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6387864Z [cluster1] I0720 04:17:06.542375       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6389253Z [cluster1] I0720 04:17:06.542387       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6392249Z [cluster1] I0720 04:17:06.695676       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.6395942Z [cluster1] I0720 04:17:06.696526       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6399058Z [cluster1] I0720 04:17:06.740554       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6403392Z [cluster1] I0720 04:17:06.760278       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6407265Z [cluster1] I0720 04:17:06.849533       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6427069Z [cluster1] I0720 04:17:06.851068       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6431062Z [cluster1] I0720 04:17:07.004040       1 helpers.go:175] controller_servicediscovery "msg"="Created a new ConfigMap" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "ConfigMap.Name"="submariner-lighthouse-coredns" "ConfigMap.Namespace"="submariner-operator"
2021-07-20T04:17:59.6434674Z [cluster1] I0720 04:17:07.046766       1 helpers.go:129] controller_servicediscovery "msg"="Created a new Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.6438441Z [cluster1] I0720 04:17:07.129945       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6441807Z [cluster1] I0720 04:17:07.342132       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6445064Z [cluster1] I0720 04:17:07.481331       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6448312Z [cluster1] I0720 04:17:07.500957       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6452137Z [cluster1] I0720 04:17:07.513573       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6456268Z [cluster1] I0720 04:17:07.590474       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6461713Z [cluster1] E0720 04:17:07.590552       1 controller.go:267] controller-runtime/manager/controller/servicediscovery-controller "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-lighthouse-coredns-metrics: Service \"submariner-lighthouse-coredns-metrics\" not found" "name"="service-discovery" "namespace"="submariner-operator"
2021-07-20T04:17:59.6466740Z [cluster1] I0720 04:17:07.590598       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6489599Z [cluster1] I0720 04:17:07.737067       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6493451Z [cluster1] I0720 04:17:07.873020       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.6497800Z [cluster1] I0720 04:17:07.873579       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.6501585Z [cluster1] E0720 04:17:07.960453       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6504688Z [cluster1] I0720 04:17:07.960884       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6506326Z [cluster1] I0720 04:17:07.968986       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6507896Z [cluster1] I0720 04:17:07.969027       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6511232Z [cluster1] I0720 04:17:07.960828       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6512896Z [cluster1] I0720 04:17:07.973422       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6514273Z [cluster1] I0720 04:17:07.973436       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6516840Z [cluster1] I0720 04:17:08.059634       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6520022Z [cluster1] I0720 04:17:08.062189       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6523716Z [cluster1] I0720 04:17:08.081644       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6527905Z [cluster1] I0720 04:17:08.082007       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6531767Z [cluster1] I0720 04:17:08.201412       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6535034Z [cluster1] I0720 04:17:08.201489       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6538528Z [cluster1] E0720 04:17:08.619442       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:17:59.6542377Z [cluster1] I0720 04:17:08.703513       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6545779Z [cluster1] I0720 04:17:08.707661       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6549793Z [cluster1] I0720 04:17:08.754064       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6554353Z [cluster1] I0720 04:17:08.754108       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.1.210.120\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n\n\n\n\n"
2021-07-20T04:17:59.6558606Z [cluster1] I0720 04:17:08.824955       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.6562211Z [cluster1] I0720 04:17:08.825896       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6565592Z [cluster1] I0720 04:17:08.877200       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6568704Z [cluster1] I0720 04:17:08.939987       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6572040Z [cluster1] I0720 04:17:08.962812       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6577689Z [cluster1] I0720 04:17:09.026973       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6581385Z [cluster1] I0720 04:17:09.630626       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6584551Z [cluster1] I0720 04:17:09.636246       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6590895Z [cluster1] I0720 04:17:09.636295       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.1.210.120\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n\n\n\n\n\n"
2021-07-20T04:17:59.6595212Z [cluster1] I0720 04:17:09.651793       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6598593Z [cluster1] I0720 04:17:09.683338       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.6603015Z [cluster1] I0720 04:17:09.766776       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6606849Z [cluster1] I0720 04:17:10.398867       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6614322Z [cluster1] I0720 04:17:10.428205       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.6618573Z [cluster1] I0720 04:17:10.458769       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6622880Z [cluster1] I0720 04:17:11.061674       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6626609Z [cluster1] I0720 04:17:11.233507       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6631024Z [cluster1] I0720 04:17:11.233577       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.1.210.120\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n\n\n\n\n\n\n"
2021-07-20T04:17:59.6635522Z [cluster1] I0720 04:17:13.832967       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6639986Z [cluster1] I0720 04:17:13.943411       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.6644661Z [cluster1] I0720 04:17:14.184706       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6648799Z [cluster1] I0720 04:17:14.811670       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6652395Z [cluster1] I0720 04:17:14.874652       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.6656596Z [cluster1] I0720 04:17:14.952240       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6660884Z [cluster1] I0720 04:17:15.557781       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6664191Z [cluster1] I0720 04:17:15.609744       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6669730Z [cluster1] I0720 04:17:15.609799       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.1.210.120\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n\n\n\n\n\n\n\n"
2021-07-20T04:17:59.6673589Z [cluster1] I0720 04:17:18.578323       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6675997Z [cluster1] I0720 04:17:18.578396       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6677393Z [cluster1] I0720 04:17:18.578638       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6680888Z [cluster1] I0720 04:17:18.703172       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6684485Z [cluster1] I0720 04:17:18.755973       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6688142Z [cluster1] I0720 04:17:19.433125       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6691108Z [cluster1] I0720 04:17:19.509355       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6693625Z [cluster1] I0720 04:17:19.638667       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6695717Z [cluster1] I0720 04:17:19.638722       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6697299Z [cluster1] I0720 04:17:19.638733       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6700195Z [cluster1] I0720 04:17:19.781296       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6702881Z [cluster1] I0720 04:17:19.782672       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6706009Z [cluster1] I0720 04:17:19.782705       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6707389Z [cluster1] I0720 04:17:19.782716       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6709945Z [cluster1] I0720 04:17:19.889414       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6714008Z [cluster1] I0720 04:17:20.055695       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6717711Z [cluster1] I0720 04:17:20.058561       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6721448Z [cluster1] E0720 04:17:20.063492       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:17:59.6724719Z [cluster1] I0720 04:17:20.682361       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6727840Z [cluster1] I0720 04:17:20.682419       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6731016Z [cluster1] I0720 04:17:20.834675       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6734515Z [cluster1] I0720 04:17:20.843787       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6737326Z [cluster1] I0720 04:17:21.794194       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6739101Z [cluster1] I0720 04:17:21.794260       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6740480Z [cluster1] I0720 04:17:21.794272       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6742733Z [cluster1] I0720 04:17:21.854473       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6745806Z [cluster1] I0720 04:17:21.981230       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6749146Z [cluster1] I0720 04:17:22.001448       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.6753063Z [cluster1] I0720 04:17:22.208915       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6757009Z [cluster1] I0720 04:17:22.210377       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6761796Z [cluster1] I0720 04:17:22.832494       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6765160Z [cluster1] I0720 04:17:22.920347       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6768745Z [cluster1] I0720 04:17:22.981671       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6772282Z [cluster1] I0720 04:17:22.998644       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.6778581Z [cluster1] I0720 04:17:23.242488       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6783649Z [cluster1] I0720 04:17:23.264548       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6785333Z [cluster1] I0720 04:17:23.264590       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6786725Z [cluster1] I0720 04:17:23.264601       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6790835Z [cluster1] I0720 04:17:23.270559       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6792472Z [cluster1] I0720 04:17:23.270589       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6793941Z [cluster1] I0720 04:17:23.270600       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6796799Z [cluster1] I0720 04:17:23.331630       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6801613Z [cluster1] I0720 04:17:23.530384       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6804996Z [cluster1] I0720 04:17:23.607323       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6808898Z [cluster1] I0720 04:17:23.730554       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6812502Z [cluster1] I0720 04:17:23.928003       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6815625Z [cluster1] I0720 04:17:23.948699       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6822001Z [cluster1] I0720 04:17:23.953901       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.1.210.120\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n\n\n\n\n\n\n\n\n"
2021-07-20T04:17:59.6825851Z [cluster1] I0720 04:17:24.233930       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6829577Z [cluster1] I0720 04:17:24.294949       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6832289Z [cluster1] I0720 04:17:24.334727       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6834061Z [cluster1] I0720 04:17:24.334813       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6835395Z [cluster1] I0720 04:17:24.334824       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6837753Z [cluster1] I0720 04:17:24.354245       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6841040Z [cluster1] I0720 04:17:24.383631       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6844398Z [cluster1] I0720 04:17:24.383676       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6847498Z [cluster1] I0720 04:17:24.446140       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6850367Z [cluster1] I0720 04:17:24.813439       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6853898Z [cluster1] I0720 04:17:24.842276       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.6857924Z [cluster1] I0720 04:17:24.890087       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6861441Z [cluster1] I0720 04:17:25.052691       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6865037Z [cluster1] I0720 04:17:25.085941       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6869265Z [cluster1] I0720 04:17:25.502834       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6874938Z [cluster1] I0720 04:17:25.522640       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.6883121Z [cluster1] I0720 04:17:25.546814       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6887764Z [cluster1] I0720 04:17:26.150243       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6891295Z [cluster1] I0720 04:17:26.238414       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6895964Z [cluster1] I0720 04:17:26.238659       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.1.210.120\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n\n\n\n\n\n\n\n\n\n"
2021-07-20T04:17:59.6899757Z [cluster1] I0720 04:17:26.509330       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6901422Z [cluster1] I0720 04:17:26.509385       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6902945Z [cluster1] I0720 04:17:26.509396       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6905677Z [cluster1] I0720 04:17:26.534088       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6909516Z [cluster1] I0720 04:17:26.593907       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6912828Z [cluster1] I0720 04:17:27.196950       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6916037Z [cluster1] I0720 04:17:27.212048       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6918751Z [cluster1] I0720 04:17:27.226408       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6920387Z [cluster1] I0720 04:17:27.227984       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6921760Z [cluster1] I0720 04:17:27.230651       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6924766Z [cluster1] I0720 04:17:27.253098       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6926402Z [cluster1] I0720 04:17:27.253214       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6928325Z [cluster1] I0720 04:17:27.253267       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6931106Z [cluster1] I0720 04:17:27.261180       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6934329Z [cluster1] I0720 04:17:27.289695       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6937696Z [cluster1] I0720 04:17:27.290472       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6942001Z [cluster1] I0720 04:17:27.301928       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6945820Z [cluster1] I0720 04:17:27.891600       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6949103Z [cluster1] I0720 04:17:27.903082       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6952466Z [cluster1] I0720 04:17:27.917859       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6955178Z [cluster1] I0720 04:17:27.966772       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6958453Z [cluster1] I0720 04:17:27.968572       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6960580Z [cluster1] I0720 04:17:27.970134       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6962250Z [cluster1] I0720 04:17:27.971202       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6965422Z [cluster1] I0720 04:17:28.030778       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6973998Z [cluster1] E0720 04:17:28.031579       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6979664Z [cluster1] I0720 04:17:28.031653       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.6981348Z [cluster1] I0720 04:17:28.031716       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.6982742Z [cluster1] I0720 04:17:28.031729       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.6985695Z [cluster1] I0720 04:17:28.040140       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6988915Z [cluster1] I0720 04:17:28.106075       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.6992539Z [cluster1] I0720 04:17:28.114832       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.6996467Z [cluster1] I0720 04:17:28.682146       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.6999840Z [cluster1] I0720 04:17:28.811833       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.7003365Z [cluster1] I0720 04:17:28.918150       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.7006653Z [cluster1] I0720 04:17:28.990061       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.7009365Z [cluster1] I0720 04:17:32.646083       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.7011027Z [cluster1] I0720 04:17:32.646160       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.7012401Z [cluster1] I0720 04:17:32.646171       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.7014878Z [cluster1] I0720 04:17:32.656911       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.7019046Z [cluster1] I0720 04:17:32.662406       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.7023795Z [cluster1] I0720 04:17:33.264802       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.7027126Z [cluster1] I0720 04:17:33.271403       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.7029971Z [cluster1] I0720 04:17:33.281158       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.7031657Z [cluster1] I0720 04:17:33.281231       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.7033019Z [cluster1] I0720 04:17:33.281261       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.7035013Z [cluster1] I0720 04:17:33.281415       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.7037353Z [cluster1] I0720 04:17:33.281524       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.7038710Z [cluster1] I0720 04:17:33.281593       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.7041179Z [cluster1] I0720 04:17:33.291253       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.7044594Z [cluster1] I0720 04:17:33.291797       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.7047745Z [cluster1] I0720 04:17:33.295905       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.7051118Z [cluster1] I0720 04:17:33.297625       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.7054320Z [cluster1] I0720 04:17:33.897944       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.7057610Z [cluster1] I0720 04:17:33.900355       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.7060805Z [cluster1] I0720 04:17:33.907918       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.7064211Z [cluster1] I0720 04:17:33.908730       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.7066914Z [cluster1] I0720 04:17:37.682015       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.7068545Z [cluster1] I0720 04:17:37.682092       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.7069924Z [cluster1] I0720 04:17:37.682134       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.7072374Z [cluster1] I0720 04:17:37.702905       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.7075653Z [cluster1] I0720 04:17:37.709347       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.7079545Z [cluster1] I0720 04:17:38.312028       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.7083691Z [cluster1] I0720 04:17:38.319072       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.7086502Z [cluster1] I0720 04:17:38.360082       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.7088152Z [cluster1] I0720 04:17:38.360279       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.7089685Z [cluster1] I0720 04:17:38.360427       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.7091711Z [cluster1] I0720 04:17:38.366787       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.7093357Z [cluster1] I0720 04:17:38.366828       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.7094723Z [cluster1] I0720 04:17:38.366838       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-20T04:17:59.7097428Z [cluster1] I0720 04:17:38.394593       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.7100624Z [cluster1] I0720 04:17:38.414645       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.7103842Z [cluster1] I0720 04:17:38.422895       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.7107812Z [cluster1] I0720 04:17:38.452272       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.7111041Z [cluster1] I0720 04:17:39.024961       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.7114218Z [cluster1] I0720 04:17:39.036188       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.7117398Z [cluster1] I0720 04:17:39.065173       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.8392840Z [cluster1] I0720 04:17:39.073129       1 helpers.go:85] controller_submariner "21-07-20T04:16:53Z"
2021-07-20T04:17:59.8393865Z [cluster2]                     }
2021-07-20T04:17:59.8394283Z [cluster2]                 }
2021-07-20T04:17:59.8394680Z [cluster2]             }
2021-07-20T04:17:59.8395081Z [cluster2]         ],
2021-07-20T04:17:59.8395508Z [cluster2]         "hostIP": "172.18.0.6",
2021-07-20T04:17:59.8396012Z [cluster2]         "phase": "Running",
2021-07-20T04:17:59.8396503Z [cluster2]         "podIP": "10.2.0.6",
2021-07-20T04:17:59.8396957Z [cluster2]         "podIPs": [
2021-07-20T04:17:59.8397389Z [cluster2]             {
2021-07-20T04:17:59.8397813Z [cluster2]                 "ip": "10.2.0.6"
2021-07-20T04:17:59.8398240Z [cluster2]             }
2021-07-20T04:17:59.8398624Z [cluster2]         ],
2021-07-20T04:17:59.8399117Z [cluster2]         "qosClass": "BestEffort",
2021-07-20T04:17:59.8399920Z [cluster2]         "startTime": "2021-07-20T04:16:52Z"
2021-07-20T04:17:59.8400577Z [cluster2]     }
2021-07-20T04:17:59.8400958Z [cluster2] }
2021-07-20T04:17:59.8402768Z [36m[submariner-operator]$ [cluster2] kubectl get pod submariner-operator-5bfdb87494-j25fv --namespace=submariner-operator -o jsonpath={.status.phase}[0m
2021-07-20T04:17:59.8404690Z [36m[submariner-operator]$ [cluster2] grep Running[0m
2021-07-20T04:17:59.8406580Z [36m[submariner-operator]$ [cluster2] kubectl get pod submariner-operator-5bfdb87494-j25fv --namespace=submariner-operator -o jsonpath={.status.phase}[0m
2021-07-20T04:17:59.8409405Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pod submariner-operator-5bfdb87494-j25fv --namespace=submariner-operator -o jsonpath={.status.phase}[0m
2021-07-20T04:17:59.8410830Z [cluster2] Running
2021-07-20T04:17:59.8412328Z [36m[submariner-operator]$ [cluster2] kubectl logs submariner-operator-5bfdb87494-j25fv --namespace=submariner-operator[0m
2021-07-20T04:17:59.8414515Z [36m[submariner-operator]$ [cluster2] kubectl logs submariner-operator-5bfdb87494-j25fv --namespace=submariner-operator[0m
2021-07-20T04:17:59.8417263Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 logs submariner-operator-5bfdb87494-j25fv --namespace=submariner-operator[0m
2021-07-20T04:17:59.8418709Z [cluster2] I0720 04:16:54.357758       1 main.go:77] cmd "msg"="Go Version: go1.16.5"
2021-07-20T04:17:59.8419561Z [cluster2] I0720 04:16:54.357827       1 main.go:78] cmd "msg"="Go OS/Arch: linux/amd64"
2021-07-20T04:17:59.8420578Z [cluster2] I0720 04:16:54.357835       1 main.go:79] cmd "msg"="Version of operator-sdk: v0.19.0"
2021-07-20T04:17:59.8421832Z [cluster2] I0720 04:16:54.357842       1 main.go:80] cmd "msg"="Submariner operator version: devel-0105e4d"
2021-07-20T04:17:59.8423182Z [cluster2] I0720 04:16:54.358009       1 leader.go:89] leader "msg"="Trying to become the leader."
2021-07-20T04:17:59.8424756Z [cluster2] I0720 04:16:55.021325       1 leader.go:126] leader "msg"="Found existing lock"  "LockOwner"="submariner-operator-586cb56578-wpllt"
2021-07-20T04:17:59.8425956Z [cluster2] I0720 04:16:55.189936       1 leader.go:192] leader "msg"="Not the leader. Waiting."
2021-07-20T04:17:59.8426788Z [cluster2] I0720 04:16:56.376524       1 leader.go:192] leader "msg"="Not the leader. Waiting."
2021-07-20T04:17:59.8427608Z [cluster2] I0720 04:16:58.771432       1 leader.go:192] leader "msg"="Not the leader. Waiting."
2021-07-20T04:17:59.8428398Z [cluster2] I0720 04:17:03.307199       1 leader.go:149] leader "msg"="Became the leader."
2021-07-20T04:17:59.8429743Z [cluster2] I0720 04:17:03.911246       1 listener.go:44] controller-runtime/metrics "msg"="metrics server is starting to listen"  "addr"="0.0.0.0:8383"
2021-07-20T04:17:59.8430775Z [cluster2] I0720 04:17:03.911668       1 main.go:129] cmd "msg"="Registering Components."
2021-07-20T04:17:59.8431598Z [cluster2] I0720 04:17:03.911878       1 main.go:137] cmd "msg"="Creating the Lighthouse CRDs"
2021-07-20T04:17:59.8432683Z [cluster2] I0720 04:17:08.018104       1 main.go:182] cmd "msg"="Could not create ServiceMonitor object"  "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.8434718Z [cluster2] I0720 04:17:08.018210       1 main.go:186] cmd "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects"  "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.8435952Z [cluster2] I0720 04:17:08.018317       1 main.go:203] cmd "msg"="Starting the Cmd."
2021-07-20T04:17:59.8437232Z [cluster2] I0720 04:17:08.018678       1 internal.go:406] controller-runtime/manager "msg"="starting metrics server"  "path"="/metrics"
2021-07-20T04:17:59.8471260Z [cluster2] I0720 04:17:08.019362       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"broker":"","brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","ceIPSecPSK":"","clusterCIDR":"","clusterID":"","serviceCIDR":"","namespace":"","ceIPSecDebug":false,"debug":false,"natEnabled":false},"status":{"natEnabled":false,"clusterID":"","gatewayDaemonSetStatus":{"mismatchedContainerImages":false},"routeAgentDaemonSetStatus":{"mismatchedContainerImages":false},"globalnetDaemonSetStatus":{"mismatchedContainerImages":false},"loadBalancerStatus":{},"deploymentInfo":{}}}}
2021-07-20T04:17:59.8480069Z [cluster2] I0720 04:17:08.019642       1 controller.go:158] controller-runtime/manager/controller/submariner "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"broker":"","brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","ceIPSecPSK":"","clusterCIDR":"","clusterID":"","serviceCIDR":"","namespace":"","ceIPSecDebug":false,"debug":false,"natEnabled":false},"status":{"natEnabled":false,"clusterID":"","gatewayDaemonSetStatus":{"mismatchedContainerImages":false},"routeAgentDaemonSetStatus":{"mismatchedContainerImages":false},"globalnetDaemonSetStatus":{"mismatchedContainerImages":false},"loadBalancerStatus":{},"deploymentInfo":{}}}}
2021-07-20T04:17:59.8486745Z [cluster2] I0720 04:17:08.019864       1 controller.go:158] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting EventSource"  "source"={"Type":{"status":{"deploymentInfo":{}},"metadata":{"creationTimestamp":null},"spec":{"brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","clusterID":"","namespace":"","debug":false}}}
2021-07-20T04:17:59.8492197Z [cluster2] I0720 04:17:08.020078       1 controller.go:158] controller-runtime/manager/controller/servicediscovery "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" "source"={"Type":{"status":{"deploymentInfo":{}},"metadata":{"creationTimestamp":null},"spec":{"brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","clusterID":"","namespace":"","debug":false}}}
2021-07-20T04:17:59.8496305Z [cluster2] I0720 04:17:08.020275       1 controller.go:158] controller-runtime/manager/controller/broker "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Broker" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{},"status":{}}}
2021-07-20T04:17:59.8499782Z [cluster2] I0720 04:17:08.120458       1 controller.go:165] controller-runtime/manager/controller/servicediscovery "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery"
2021-07-20T04:17:59.8503268Z [cluster2] I0720 04:17:08.120507       1 controller.go:192] controller-runtime/manager/controller/servicediscovery "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" "worker count"=1
2021-07-20T04:17:59.8507120Z [cluster2] I0720 04:17:08.120564       1 controller.go:165] controller-runtime/manager/controller/broker "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:17:59.8510893Z [cluster2] I0720 04:17:08.120585       1 controller.go:192] controller-runtime/manager/controller/broker "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="Broker" "worker count"=1
2021-07-20T04:17:59.8514314Z [cluster2] I0720 04:17:08.121913       1 controller.go:158] controller-runtime/manager/controller/submariner "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"strategy":{}},"status":{}}}
2021-07-20T04:17:59.8526702Z [cluster2] I0720 04:17:08.122060       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"updateStrategy":{}},"status":{"currentNumberScheduled":0,"numberMisscheduled":0,"desiredNumberScheduled":0,"numberReady":0}}}
2021-07-20T04:17:59.8530919Z [cluster2] I0720 04:17:08.122097       1 controller.go:158] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"strategy":{}},"status":{}}}
2021-07-20T04:17:59.8588986Z [cluster2] I0720 04:17:08.125458       1 controller.go:165] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting Controller"
2021-07-20T04:17:59.8592374Z [cluster2] I0720 04:17:08.222454       1 controller.go:165] controller-runtime/manager/controller/submariner "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="Submariner"
2021-07-20T04:17:59.8594784Z [cluster2] I0720 04:17:08.222650       1 controller.go:192] controller-runtime/manager/controller/submariner "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "worker count"=1
2021-07-20T04:17:59.8596969Z [cluster2] I0720 04:17:08.222877       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.8600229Z [cluster2] I0720 04:17:08.226918       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"status":{"version":"","haStatus":"","localEndpoint":{"cluster_id":"","cable_name":"","hostname":"","subnets":null,"private_ip":"","public_ip":"","nat_enabled":false,"backend":""},"statusFailure":"","connections":null}}}
2021-07-20T04:17:59.8603825Z [cluster2] I0720 04:17:08.227006       1 controller.go:192] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting workers"  "worker count"=1
2021-07-20T04:17:59.8605789Z [cluster2] I0720 04:17:08.261476       1 submariner_networkdiscovery.go:42] controller_submariner "msg"="Cluster network discovered"
2021-07-20T04:17:59.8607600Z [cluster2] I0720 04:17:08.261664       1 network.go:56] controller_submariner "msg"="Discovered K8s network details"  "clusterCIDRs"=["10.2.0.0/16"] "plugin"="weave-net" "serviceCIDRs"=["100.2.0.0/16"]
2021-07-20T04:17:59.8608993Z [cluster2] I0720 04:17:08.261845       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.8610326Z [cluster2] I0720 04:17:08.261965       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.8612883Z [cluster2] I0720 04:17:08.318380       1 helpers.go:83] controller_submariner "msg"="Created a new DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.8615799Z [cluster2] I0720 04:17:08.327954       1 controller.go:165] controller-runtime/manager/controller/submariner-controller "msg"="Starting Controller"
2021-07-20T04:17:59.8617964Z [cluster2] I0720 04:17:08.328018       1 controller.go:192] controller-runtime/manager/controller/submariner-controller "msg"="Starting workers"  "worker count"=1
2021-07-20T04:17:59.8620231Z [cluster2] I0720 04:17:08.328133       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.8621899Z [cluster2] I0720 04:17:08.328191       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.8623255Z [cluster2] I0720 04:17:08.328208       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.8625902Z [cluster2] I0720 04:17:08.379074       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.8629314Z [cluster2] I0720 04:17:08.512084       1 helpers.go:231] controller_submariner "msg"="Created a new Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.8633117Z [cluster2] E0720 04:17:08.743896       1 controller.go:267] controller-runtime/manager/controller/submariner "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-gateway-metrics: services \"submariner-gateway-metrics\" already exists" "name"="submariner" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Submariner"
2021-07-20T04:17:59.8680436Z [cluster2] I0720 04:17:08.746436       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.8682158Z [cluster2] I0720 04:17:08.746656       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.8683546Z [cluster2] I0720 04:17:08.746785       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.8687076Z [cluster2] I0720 04:17:08.892652       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.8690382Z [cluster2] I0720 04:17:08.930239       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.8695050Z [cluster2] I0720 04:17:09.125548       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.8698554Z [cluster2] I0720 04:17:09.148201       1 helpers.go:83] controller_submariner "msg"="Created a new DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.8701621Z [cluster2] I0720 04:17:09.215066       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.8704353Z [cluster2] I0720 04:17:09.215362       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.8707380Z [cluster2] I0720 04:17:09.230287       1 servicediscovery_resources.go:66] controller_submariner "msg"="Created Service Discovery CR" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Name"="service-discovery" "Namespace"="submariner-operator"
2021-07-20T04:17:59.8710643Z [cluster2] I0720 04:17:09.245236       1 helpers.go:129] controller_servicediscovery "msg"="Created a new Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.8715161Z [cluster2] E0720 04:17:09.346235       1 controller.go:267] controller-runtime/manager/controller/servicediscovery-controller "msg"="Reconciler error" "error"="error creating or updating Deployment submariner-operator/submariner-lighthouse-agent: deployments.apps \"submariner-lighthouse-agent\" already exists" "name"="service-discovery" "namespace"="submariner-operator"
2021-07-20T04:17:59.8719374Z [cluster2] I0720 04:17:09.352247       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.8722523Z [cluster2] I0720 04:17:09.583305       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.8726207Z [cluster2] I0720 04:17:09.819366       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.8729015Z [cluster2] I0720 04:17:09.826925       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.8730952Z [cluster2] I0720 04:17:09.826975       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.8732267Z [cluster2] I0720 04:17:09.826986       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.8734856Z [cluster2] I0720 04:17:10.111960       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.8738365Z [cluster2] I0720 04:17:10.148120       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.8741850Z [cluster2] I0720 04:17:10.148988       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.8745432Z [cluster2] I0720 04:17:10.507371       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.8749794Z [cluster2] E0720 04:17:10.764740       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.8752880Z [cluster2] I0720 04:17:10.764806       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.8754474Z [cluster2] I0720 04:17:10.764866       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.8860666Z [cluster2] I0720 04:17:10.764876       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.8864411Z [cluster2] I0720 04:17:10.827190       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.8868375Z [cluster2] I0720 04:17:10.827547       1 helpers.go:175] controller_servicediscovery "msg"="Created a new ConfigMap" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "ConfigMap.Name"="submariner-lighthouse-coredns" "ConfigMap.Namespace"="submariner-operator"
2021-07-20T04:17:59.8871807Z [cluster2] I0720 04:17:11.484702       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.8875218Z [cluster2] I0720 04:17:11.487817       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.8878921Z [cluster2] I0720 04:17:11.491805       1 helpers.go:129] controller_servicediscovery "msg"="Created a new Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.8882514Z [cluster2] I0720 04:17:11.509522       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.8886163Z [cluster2] I0720 04:17:11.779749       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.8889671Z [cluster2] I0720 04:17:11.780164       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.8893074Z [cluster2] I0720 04:17:12.267667       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.8897940Z [cluster2] I0720 04:17:12.268676       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.8901393Z [cluster2] I0720 04:17:12.882694       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.8904811Z [cluster2] I0720 04:17:13.022787       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.8907877Z [cluster2] I0720 04:17:13.346339       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.8910559Z [cluster2] I0720 04:17:13.824636       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.8912247Z [cluster2] I0720 04:17:13.824685       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.8913536Z [cluster2] I0720 04:17:13.824698       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.8915900Z [cluster2] I0720 04:17:13.863120       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.8920418Z [cluster2] E0720 04:17:13.982854       1 controller.go:267] controller-runtime/manager/controller/servicediscovery-controller "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-lighthouse-coredns-metrics: services \"submariner-lighthouse-coredns-metrics\" already exists" "name"="service-discovery" "namespace"="submariner-operator"
2021-07-20T04:17:59.8924403Z [cluster2] I0720 04:17:13.982904       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.8971203Z [cluster2] E0720 04:17:14.054264       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.8974717Z [cluster2] I0720 04:17:14.054319       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.8976447Z [cluster2] I0720 04:17:14.054361       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.8978087Z [cluster2] I0720 04:17:14.054382       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.8980537Z [cluster2] I0720 04:17:14.055839       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.8983577Z [cluster2] I0720 04:17:14.073553       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.8987369Z [cluster2] I0720 04:17:14.133288       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.8991153Z [cluster2] I0720 04:17:14.134293       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.8997899Z [cluster2] I0720 04:17:14.134364       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.2.152.70\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n\n\n\n"
2021-07-20T04:17:59.9002182Z [cluster2] I0720 04:17:14.145620       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9005480Z [cluster2] I0720 04:17:14.258270       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9009174Z [cluster2] I0720 04:17:14.939176       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9012651Z [cluster2] I0720 04:17:14.968863       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9016300Z [cluster2] I0720 04:17:15.066545       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9020148Z [cluster2] I0720 04:17:15.202267       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9023565Z [cluster2] I0720 04:17:15.577306       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9027027Z [cluster2] I0720 04:17:15.595860       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.9046153Z [cluster2] I0720 04:17:15.640330       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9055717Z [cluster2] I0720 04:17:15.775651       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9100871Z [cluster2] I0720 04:17:15.885732       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9104452Z [cluster2] I0720 04:17:16.247679       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9108027Z [cluster2] I0720 04:17:16.262545       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9113275Z [cluster2] I0720 04:17:16.262603       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.2.152.70\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n\n\n\n\n"
2021-07-20T04:17:59.9117153Z [cluster2] I0720 04:17:16.324755       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9132765Z [cluster2] I0720 04:17:16.367256       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.9136835Z [cluster2] I0720 04:17:16.400384       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9140923Z [cluster2] I0720 04:17:17.013272       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9144648Z [cluster2] I0720 04:17:17.078915       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.9148652Z [cluster2] I0720 04:17:17.103991       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9152217Z [cluster2] I0720 04:17:17.813962       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9155732Z [cluster2] I0720 04:17:17.846014       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9160297Z [cluster2] I0720 04:17:17.846077       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.2.152.70\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n\n\n\n\n\n"
2021-07-20T04:17:59.9163813Z [cluster2] I0720 04:17:19.395104       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9165712Z [cluster2] I0720 04:17:19.395156       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9167111Z [cluster2] I0720 04:17:19.395180       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9169705Z [cluster2] I0720 04:17:19.515700       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9178658Z [cluster2] I0720 04:17:19.554605       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9182131Z [cluster2] I0720 04:17:20.157518       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9187351Z [cluster2] I0720 04:17:20.233399       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9192358Z [cluster2] I0720 04:17:20.254272       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9195046Z [cluster2] I0720 04:17:20.254397       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9198142Z [cluster2] I0720 04:17:20.254454       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9201385Z [cluster2] I0720 04:17:20.265476       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9203364Z [cluster2] I0720 04:17:20.265746       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9204883Z [cluster2] I0720 04:17:20.265828       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9207762Z [cluster2] I0720 04:17:20.416216       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9211150Z [cluster2] I0720 04:17:20.416778       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9214592Z [cluster2] I0720 04:17:20.522268       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9218991Z [cluster2] I0720 04:17:20.522300       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9223548Z [cluster2] I0720 04:17:21.170714       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9242644Z [cluster2] I0720 04:17:21.170843       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9249985Z [cluster2] I0720 04:17:21.254318       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9253426Z [cluster2] I0720 04:17:21.322993       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9256464Z [cluster2] I0720 04:17:21.766131       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9258257Z [cluster2] I0720 04:17:21.766497       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9259868Z [cluster2] I0720 04:17:21.766659       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9262394Z [cluster2] I0720 04:17:22.087380       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9265610Z [cluster2] I0720 04:17:22.104789       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9268875Z [cluster2] I0720 04:17:22.756706       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9274749Z [cluster2] I0720 04:17:22.864769       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9299947Z [cluster2] I0720 04:17:22.905366       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9301820Z [cluster2] I0720 04:17:22.905417       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9303360Z [cluster2] I0720 04:17:22.905429       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9305336Z [cluster2] I0720 04:17:22.980847       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9306941Z [cluster2] I0720 04:17:22.980895       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9308644Z [cluster2] I0720 04:17:22.980907       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9311326Z [cluster2] I0720 04:17:23.175280       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9315053Z [cluster2] I0720 04:17:23.178605       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9318558Z [cluster2] I0720 04:17:23.269947       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9321832Z [cluster2] I0720 04:17:23.303519       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9325212Z [cluster2] I0720 04:17:23.944272       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9328558Z [cluster2] I0720 04:17:23.966037       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9332052Z [cluster2] I0720 04:17:24.123928       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9335302Z [cluster2] I0720 04:17:24.137858       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9360250Z [cluster2] E0720 04:17:24.243645       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9363353Z [cluster2] I0720 04:17:24.243869       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9365024Z [cluster2] I0720 04:17:24.243957       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9366394Z [cluster2] I0720 04:17:24.243970       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9369367Z [cluster2] I0720 04:17:24.323881       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9372492Z [cluster2] I0720 04:17:24.373972       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9375767Z [cluster2] I0720 04:17:25.373342       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9379600Z [cluster2] I0720 04:17:25.650289       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9382478Z [cluster2] I0720 04:17:26.024177       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9385639Z [cluster2] I0720 04:17:26.083525       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.9390107Z [cluster2] I0720 04:17:26.120669       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9394042Z [cluster2] I0720 04:17:26.729273       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9397578Z [cluster2] I0720 04:17:26.751970       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.9402549Z [cluster2] I0720 04:17:26.764117       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9460020Z [cluster2] I0720 04:17:26.927533       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9461660Z [cluster2] I0720 04:17:26.927859       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9462973Z [cluster2] I0720 04:17:26.927944       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9465399Z [cluster2] I0720 04:17:26.951669       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9469087Z [cluster2] I0720 04:17:26.961593       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9472898Z [cluster2] I0720 04:17:27.371787       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9476636Z [cluster2] I0720 04:17:27.377922       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9481244Z [cluster2] I0720 04:17:27.378076       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.2.152.70\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n\n\n\n\n\n\n"
2021-07-20T04:17:59.9485373Z [cluster2] I0720 04:17:27.569356       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9488573Z [cluster2] I0720 04:17:27.583622       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9492043Z [cluster2] I0720 04:17:27.624454       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9493840Z [cluster2] I0720 04:17:27.624528       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9495352Z [cluster2] I0720 04:17:27.624540       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9497973Z [cluster2] I0720 04:17:27.625107       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9500256Z [cluster2] I0720 04:17:27.625133       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9501640Z [cluster2] I0720 04:17:27.625236       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9504242Z [cluster2] I0720 04:17:27.688260       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9507638Z [cluster2] I0720 04:17:27.710987       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9510953Z [cluster2] I0720 04:17:27.780860       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9514392Z [cluster2] I0720 04:17:27.858019       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9517747Z [cluster2] I0720 04:17:28.413413       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9520970Z [cluster2] I0720 04:17:28.424409       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9523850Z [cluster2] I0720 04:17:28.450063       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9525516Z [cluster2] I0720 04:17:28.450281       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9527177Z [cluster2] I0720 04:17:28.450305       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9529497Z [cluster2] I0720 04:17:28.460224       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9532800Z [cluster2] I0720 04:17:28.474972       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9536357Z [cluster2] I0720 04:17:28.475917       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9540217Z [cluster2] E0720 04:17:28.499633       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9543126Z [cluster2] I0720 04:17:28.499862       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9544733Z [cluster2] I0720 04:17:28.500022       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9546261Z [cluster2] I0720 04:17:28.500106       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9548804Z [cluster2] I0720 04:17:28.518268       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9552118Z [cluster2] I0720 04:17:28.519797       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9555109Z [cluster2] I0720 04:17:28.576038       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9558772Z [cluster2] I0720 04:17:29.122964       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9562125Z [cluster2] I0720 04:17:29.166377       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9565209Z [cluster2] I0720 04:17:29.210721       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9568439Z [cluster2] I0720 04:17:29.238171       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9571436Z [cluster2] I0720 04:17:29.488414       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9572957Z [cluster2] I0720 04:17:29.488491       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9574236Z [cluster2] I0720 04:17:29.488503       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9576659Z [cluster2] I0720 04:17:29.539652       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9661136Z [cluster2] I0720 04:17:29.555535       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9664492Z [cluster2] I0720 04:17:30.158319       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9667457Z [cluster2] I0720 04:17:30.167811       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9670373Z [cluster2] I0720 04:17:30.192130       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9672041Z [cluster2] I0720 04:17:30.192230       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9673702Z [cluster2] I0720 04:17:30.192252       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9675530Z [cluster2] I0720 04:17:30.194183       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9677353Z [cluster2] I0720 04:17:30.194228       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9679095Z [cluster2] I0720 04:17:30.194241       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9681922Z [cluster2] I0720 04:17:30.222249       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9685538Z [cluster2] I0720 04:17:30.236016       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9688759Z [cluster2] I0720 04:17:30.249084       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9692328Z [cluster2] I0720 04:17:30.304218       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9695434Z [cluster2] I0720 04:17:30.651128       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9698749Z [cluster2] I0720 04:17:30.664824       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.9703405Z [cluster2] I0720 04:17:30.676088       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9707168Z [cluster2] I0720 04:17:30.838112       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9710343Z [cluster2] I0720 04:17:30.854844       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9713326Z [cluster2] I0720 04:17:30.907390       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9717062Z [cluster2] I0720 04:17:30.916070       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9720434Z [cluster2] I0720 04:17:31.278868       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9723875Z [cluster2] I0720 04:17:31.286948       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.9728313Z [cluster2] I0720 04:17:31.292648       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9732122Z [cluster2] I0720 04:17:31.894199       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9735161Z [cluster2] I0720 04:17:31.896821       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9771806Z [cluster2] I0720 04:17:31.896962       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.2.152.70\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n\n\n\n\n\n\n\n"
2021-07-20T04:17:59.9776127Z [cluster2] I0720 04:17:31.900778       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9780534Z [cluster2] I0720 04:17:31.905563       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.9784416Z [cluster2] I0720 04:17:31.913626       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9787542Z [cluster2] I0720 04:17:31.953619       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9789528Z [cluster2] I0720 04:17:31.953949       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9791015Z [cluster2] I0720 04:17:31.954051       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9793843Z [cluster2] I0720 04:17:31.965012       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9797544Z [cluster2] I0720 04:17:31.969001       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9800749Z [cluster2] I0720 04:17:32.515936       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9805605Z [cluster2] I0720 04:17:32.527681       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-20T04:17:59.9809703Z [cluster2] I0720 04:17:32.531576       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9813257Z [cluster2] I0720 04:17:32.570728       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9816808Z [cluster2] I0720 04:17:32.577061       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9820085Z [cluster2] I0720 04:17:32.593047       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9821895Z [cluster2] I0720 04:17:32.593869       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9823444Z [cluster2] I0720 04:17:32.593928       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9826350Z [cluster2] I0720 04:17:32.595326       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9834835Z [cluster2] I0720 04:17:32.595376       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9836173Z [cluster2] I0720 04:17:32.595388       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9839154Z [cluster2] I0720 04:17:32.602641       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9842480Z [cluster2] I0720 04:17:32.605063       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9845853Z [cluster2] I0720 04:17:32.608352       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9849209Z [cluster2] I0720 04:17:32.622527       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9852750Z [cluster2] I0720 04:17:33.133504       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9855889Z [cluster2] I0720 04:17:33.135604       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9860862Z [cluster2] I0720 04:17:33.135654       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.2.152.70\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n\n\n\n\n\n\n\n\n"
2021-07-20T04:17:59.9865150Z [cluster2] I0720 04:17:33.210304       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9868228Z [cluster2] I0720 04:17:33.216936       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9871493Z [cluster2] I0720 04:17:33.225688       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9874819Z [cluster2] I0720 04:17:33.230826       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9877607Z [cluster2] I0720 04:17:36.974955       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9879191Z [cluster2] I0720 04:17:36.975024       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9880874Z [cluster2] I0720 04:17:36.975035       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9883556Z [cluster2] I0720 04:17:36.984273       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9888116Z [cluster2] I0720 04:17:36.987824       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:17:59.9891234Z [cluster2] I0720 04:17:37.592469       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:17:59.9949772Z [cluster2] I0720 04:17:37.599234       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9953083Z [cluster2] I0720 04:17:37.612841       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9958511Z [cluster2] I0720 04:17:37.612893       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9960560Z [cluster2] I0720 04:17:37.612906       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9962703Z [cluster2] I0720 04:17:37.613900       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:17:59.9964852Z [cluster2] I0720 04:17:37.613945       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:17:59.9966197Z [cluster2] I0720 04:17:37.613963       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:17:59.9968661Z [cluster2] I0720 04:17:37.627248       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9972170Z [cluster2] I0720 04:17:37.627681       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:17:59.9975540Z [cluster2] I0720 04:17:37.633616       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:18:00.0019717Z [cluster2] I0720 04:17:37.633839       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:18:00.0023223Z [cluster2] I0720 04:17:38.237327       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:18:00.0026364Z [cluster2] I0720 04:17:38.238374       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:18:00.0029983Z [cluster2] I0720 04:17:38.248785       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0033444Z [cluster2] I0720 04:17:38.249631       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0035952Z [cluster2] I0720 04:17:42.002551       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:18:00.0037491Z [cluster2] I0720 04:17:42.002612       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:18:00.0038749Z [cluster2] I0720 04:17:42.002623       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:18:00.0041342Z [cluster2] I0720 04:17:42.028844       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0044836Z [cluster2] I0720 04:17:42.040011       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:18:00.0047927Z [cluster2] I0720 04:17:42.642499       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:18:00.0051065Z [cluster2] I0720 04:17:42.649095       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0054029Z [cluster2] I0720 04:17:42.661277       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:18:00.0055965Z [cluster2] I0720 04:17:42.661316       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:18:00.0058110Z [cluster2] I0720 04:17:42.661319       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:18:00.0059653Z [cluster2] I0720 04:17:42.661327       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:18:00.0061252Z [cluster2] I0720 04:17:42.661378       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:18:00.0062826Z [cluster2] I0720 04:17:42.661389       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:18:00.0065366Z [cluster2] I0720 04:17:42.669179       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0068575Z [cluster2] I0720 04:17:42.673327       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:18:00.0071891Z [cluster2] I0720 04:17:42.678581       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0075387Z [cluster2] I0720 04:17:42.690263       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:18:00.0078602Z [cluster2] I0720 04:17:43.275912       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:18:00.0081761Z [cluster2] I0720 04:17:43.288530       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0085066Z [cluster2] I0720 04:17:43.299719       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:18:00.0088285Z [cluster2] I0720 04:17:43.322489       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0090975Z [cluster2] I0720 04:17:47.056261       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:18:00.0092637Z [cluster2] I0720 04:17:47.056337       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:18:00.0094020Z [cluster2] I0720 04:17:47.056353       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:18:00.0096800Z [cluster2] I0720 04:17:47.080159       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0140783Z [cluster2] I0720 04:17:47.094204       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:18:00.0144316Z [cluster2] I0720 04:17:47.696182       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:18:00.0147518Z [cluster2] I0720 04:17:47.703034       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0150369Z [cluster2] I0720 04:17:47.751878       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:18:00.0152163Z [cluster2] I0720 04:17:47.751934       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:18:00.0153535Z [cluster2] I0720 04:17:47.751945       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:18:00.0155512Z [cluster2] I0720 04:17:47.752494       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:18:00.0157283Z [cluster2] I0720 04:17:47.752549       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:18:00.0158590Z [cluster2] I0720 04:17:47.752562       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:18:00.0161999Z [cluster2] I0720 04:17:47.795210       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0165234Z [cluster2] I0720 04:17:47.804130       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0168693Z [cluster2] I0720 04:17:47.815464       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:18:00.0171889Z [cluster2] I0720 04:17:47.855790       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:18:00.0175198Z [cluster2] I0720 04:17:48.422974       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:18:00.0178524Z [cluster2] I0720 04:17:48.463367       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0181673Z [cluster2] I0720 04:17:48.468359       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:18:00.0184863Z [cluster2] I0720 04:17:48.547255       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0187907Z [cluster2] I0720 04:17:52.113410       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:18:00.0189552Z [cluster2] I0720 04:17:52.113499       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:18:00.0191173Z [cluster2] I0720 04:17:52.113511       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:18:00.0193726Z [cluster2] I0720 04:17:52.131418       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0197274Z [cluster2] I0720 04:17:52.137174       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:18:00.0200490Z [cluster2] I0720 04:17:52.741109       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:18:00.0203804Z [cluster2] I0720 04:17:52.755742       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0206862Z [cluster2] I0720 04:17:52.776482       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:18:00.0212351Z [cluster2] I0720 04:17:52.776740       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:18:00.0214078Z [cluster2] I0720 04:17:52.776842       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:18:00.0216119Z [cluster2] I0720 04:17:52.780248       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:18:00.0218309Z [cluster2] I0720 04:17:52.780583       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:18:00.0219695Z [cluster2] I0720 04:17:52.780606       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:18:00.0222210Z [cluster2] I0720 04:17:52.804342       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0225516Z [cluster2] I0720 04:17:52.808430       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0228628Z [cluster2] I0720 04:17:52.812812       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:18:00.0232208Z [cluster2] I0720 04:17:52.832270       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:18:00.0235726Z [cluster2] I0720 04:17:53.417325       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:18:00.0238893Z [cluster2] I0720 04:17:53.425688       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0242387Z [cluster2] I0720 04:17:53.441280       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:18:00.0245550Z [cluster2] I0720 04:17:53.447611       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0248319Z [cluster2] I0720 04:17:57.155885       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:18:00.0249975Z [cluster2] I0720 04:17:57.155963       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:18:00.0251338Z [cluster2] I0720 04:17:57.155975       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:18:00.0253823Z [cluster2] I0720 04:17:57.172467       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0257286Z [cluster2] I0720 04:17:57.179983       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:18:00.0260564Z [cluster2] I0720 04:17:57.782670       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:18:00.0264065Z [cluster2] I0720 04:17:57.798801       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.0266771Z [cluster2] I0720 04:17:57.825103       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:18:00.0268539Z [cluster2] I0720 04:17:57.825155       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:18:00.0269874Z [cluster2] I0720 04:17:57.825168       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:18:00.0271772Z [cluster2] I0720 04:17:57.826417       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator"
2021-07-20T04:18:00.0273475Z [cluster2] I0720 04:17:57.826446       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-20T04:18:00.0274977Z [cluster2] I0720 04:17:57.826456       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-20T04:18:00.0277480Z [cluster2] I0720 04:17:57.838655       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.7753046Z [cluster2] I0720 04:17:57.845883       1 helpers.go:233] controller_submariner "msg"msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:00.7756968Z [cluster1] E0720 04:17:40.645843       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker"
2021-07-20T04:18:00.7759894Z [36m[submariner-operator]$ [cluster1] verify_subm_cr[0m
2021-07-20T04:18:00.7760924Z [36m[submariner-operator]$ [cluster1] verify_subm_cr[0m
2021-07-20T04:18:00.7763107Z [36m[submariner-operator]$ [cluster1] kubectl get submariner --namespace=submariner-operator[0m
2021-07-20T04:18:00.7764599Z [36m[submariner-operator]$ [cluster1] grep submariner[0m
2021-07-20T04:18:00.7766441Z [36m[submariner-operator]$ [cluster1] kubectl get submariner --namespace=submariner-operator[0m
2021-07-20T04:18:00.7768159Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get submariner --namespace=submariner-operator[0m
2021-07-20T04:18:00.7769564Z [cluster1] submariner   55s
2021-07-20T04:18:00.7771157Z [36m[submariner-operator]$ [cluster1] json_file=/tmp/submariner.cluster1.json[0m
2021-07-20T04:18:00.7773030Z [36m[submariner-operator]$ [cluster1] kubectl get submariner submariner --namespace=submariner-operator -o json[0m
2021-07-20T04:18:00.7774686Z [36m[submariner-operator]$ [cluster1] kubectl get submariner submariner --namespace=submariner-operator -o json[0m
2021-07-20T04:18:00.7776742Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get submariner submariner --namespace=submariner-operator -o json[0m
2021-07-20T04:18:00.7778938Z [36m[submariner-operator]$ [cluster1] tee /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:00.7779911Z [cluster1] {
2021-07-20T04:18:00.7780500Z [cluster1]     "apiVersion": "submariner.io/v1alpha1",
2021-07-20T04:18:00.7781211Z [cluster1]     "kind": "Submariner",
2021-07-20T04:18:00.7781792Z [cluster1]     "metadata": {
2021-07-20T04:18:00.7782658Z [cluster1]         "creationTimestamp": "2021-07-20T04:17:05Z",
2021-07-20T04:18:00.7784206Z [cluster1]         "generation": 1,
2021-07-20T04:18:00.7784800Z [cluster1]         "managedFields": [
2021-07-20T04:18:00.7785381Z [cluster1]             {
2021-07-20T04:18:00.7786006Z [cluster1]                 "apiVersion": "submariner.io/v1alpha1",
2021-07-20T04:18:00.7786783Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-20T04:18:00.7787424Z [cluster1]                 "fieldsV1": {
2021-07-20T04:18:00.7788016Z [cluster1]                     "f:spec": {
2021-07-20T04:18:00.7788515Z [cluster1]                         ".": {},
2021-07-20T04:18:00.7789078Z [cluster1]                         "f:broker": {},
2021-07-20T04:18:00.7790019Z [cluster1]                         "f:brokerK8sApiServer": {},
2021-07-20T04:18:00.7790885Z [cluster1]                         "f:brokerK8sApiServerToken": {},
2021-07-20T04:18:00.7791685Z [cluster1]                         "f:brokerK8sCA": {},
2021-07-20T04:18:00.7792652Z [cluster1]                         "f:brokerK8sRemoteNamespace": {},
2021-07-20T04:18:00.7793463Z [cluster1]                         "f:cableDriver": {},
2021-07-20T04:18:00.7794103Z [cluster1]                         "f:ceIPSecDebug": {},
2021-07-20T04:18:00.7794870Z [cluster1]                         "f:ceIPSecIKEPort": {},
2021-07-20T04:18:00.7795561Z [cluster1]                         "f:ceIPSecNATTPort": {},
2021-07-20T04:18:00.7796267Z [cluster1]                         "f:ceIPSecPSK": {},
2021-07-20T04:18:00.7797083Z [cluster1]                         "f:clusterCIDR": {},
2021-07-20T04:18:00.7797668Z [cluster1]                         "f:clusterID": {},
2021-07-20T04:18:00.7798305Z [cluster1]                         "f:colorCodes": {},
2021-07-20T04:18:00.7799030Z [cluster1]                         "f:connectionHealthCheck": {
2021-07-20T04:18:00.7799705Z [cluster1]                             ".": {},
2021-07-20T04:18:00.7800213Z [cluster1]                             "f:enabled": {},
2021-07-20T04:18:00.7800891Z [cluster1]                             "f:intervalSeconds": {},
2021-07-20T04:18:00.7801579Z [cluster1]                             "f:maxPacketLossCount": {}
2021-07-20T04:18:00.7802239Z [cluster1]                         },
2021-07-20T04:18:00.7802771Z [cluster1]                         "f:debug": {},
2021-07-20T04:18:00.7803303Z [cluster1]                         "f:namespace": {},
2021-07-20T04:18:00.7803930Z [cluster1]                         "f:natEnabled": {},
2021-07-20T04:18:00.7804503Z [cluster1]                         "f:repository": {},
2021-07-20T04:18:00.7805169Z [cluster1]                         "f:serviceCIDR": {},
2021-07-20T04:18:00.7805888Z [cluster1]                         "f:serviceDiscoveryEnabled": {},
2021-07-20T04:18:00.7806635Z [cluster1]                         "f:version": {}
2021-07-20T04:18:00.7823475Z [cluster1]                     }
2021-07-20T04:18:00.7823951Z [cluster1]                 },
2021-07-20T04:18:00.7824397Z [cluster1]                 "manager": "subctl",
2021-07-20T04:18:00.7824948Z [cluster1]                 "operation": "Update",
2021-07-20T04:18:00.7825796Z [cluster1]                 "time": "2021-07-20T04:17:05Z"
2021-07-20T04:18:00.7826245Z [cluster1]             },
2021-07-20T04:18:00.7826624Z [cluster1]             {
2021-07-20T04:18:00.7827190Z [cluster1]                 "apiVersion": "submariner.io/v1alpha1",
2021-07-20T04:18:00.7827832Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-20T04:18:00.7828533Z [cluster1]                 "fieldsV1": {
2021-07-20T04:18:00.7829036Z [cluster1]                     "f:status": {
2021-07-20T04:18:00.7829524Z [cluster1]                         ".": {},
2021-07-20T04:18:00.7830014Z [cluster1]                         "f:clusterCIDR": {},
2021-07-20T04:18:00.7830525Z [cluster1]                         "f:clusterID": {},
2021-07-20T04:18:00.7831046Z [cluster1]                         "f:colorCodes": {},
2021-07-20T04:18:00.7831590Z [cluster1]                         "f:deploymentInfo": {},
2021-07-20T04:18:00.7832278Z [cluster1]                         "f:gatewayDaemonSetStatus": {
2021-07-20T04:18:00.7832860Z [cluster1]                             ".": {},
2021-07-20T04:18:00.7854878Z [cluster1]                             "f:lastResourceVersion": {},
2021-07-20T04:18:00.7855864Z [cluster1]                             "f:mismatchedContainerImages": {},
2021-07-20T04:18:00.7856905Z [cluster1]                             "f:nonReadyContainerStates": {},
2021-07-20T04:18:00.7858025Z [cluster1]                             "f:status": {
2021-07-20T04:18:00.7858669Z [cluster1]                                 ".": {},
2021-07-20T04:18:00.7859483Z [cluster1]                                 "f:currentNumberScheduled": {},
2021-07-20T04:18:00.7861048Z [cluster1]                                 "f:desiredNumberScheduled": {},
2021-07-20T04:18:00.7861909Z [cluster1]                                 "f:numberAvailable": {},
2021-07-20T04:18:00.7862524Z [cluster1]                                 "f:numberMisscheduled": {},
2021-07-20T04:18:00.7863287Z [cluster1]                                 "f:numberReady": {},
2021-07-20T04:18:00.7863889Z [cluster1]                                 "f:observedGeneration": {},
2021-07-20T04:18:00.7864946Z [cluster1]                                 "f:updatedNumberScheduled": {}
2021-07-20T04:18:00.7865522Z [cluster1]                             }
2021-07-20T04:18:00.7865906Z [cluster1]                         },
2021-07-20T04:18:00.7866356Z [cluster1]                         "f:gateways": {},
2021-07-20T04:18:00.7866982Z [cluster1]                         "f:globalnetDaemonSetStatus": {
2021-07-20T04:18:00.7867583Z [cluster1]                             ".": {},
2021-07-20T04:18:00.7868356Z [cluster1]                             "f:mismatchedContainerImages": {}
2021-07-20T04:18:00.7868978Z [cluster1]                         },
2021-07-20T04:18:00.7869516Z [cluster1]                         "f:loadBalancerStatus": {},
2021-07-20T04:18:00.7870097Z [cluster1]                         "f:natEnabled": {},
2021-07-20T04:18:00.7870656Z [cluster1]                         "f:networkPlugin": {},
2021-07-20T04:18:00.7871356Z [cluster1]                         "f:routeAgentDaemonSetStatus": {
2021-07-20T04:18:00.7871990Z [cluster1]                             ".": {},
2021-07-20T04:18:00.7872537Z [cluster1]                             "f:lastResourceVersion": {},
2021-07-20T04:18:00.7873309Z [cluster1]                             "f:mismatchedContainerImages": {},
2021-07-20T04:18:00.7874114Z [cluster1]                             "f:nonReadyContainerStates": {},
2021-07-20T04:18:00.7874750Z [cluster1]                             "f:status": {
2021-07-20T04:18:00.7875203Z [cluster1]                                 ".": {},
2021-07-20T04:18:00.7875793Z [cluster1]                                 "f:currentNumberScheduled": {},
2021-07-20T04:18:00.7876653Z [cluster1]                                 "f:desiredNumberScheduled": {},
2021-07-20T04:18:00.7877336Z [cluster1]                                 "f:numberAvailable": {},
2021-07-20T04:18:00.7877991Z [cluster1]                                 "f:numberMisscheduled": {},
2021-07-20T04:18:00.7878590Z [cluster1]                                 "f:numberReady": {},
2021-07-20T04:18:00.7879206Z [cluster1]                                 "f:observedGeneration": {},
2021-07-20T04:18:00.7880085Z [cluster1]                                 "f:updatedNumberScheduled": {}
2021-07-20T04:18:00.7880859Z [cluster1]                             }
2021-07-20T04:18:00.7881564Z [cluster1]                         },
2021-07-20T04:18:00.7882064Z [cluster1]                         "f:serviceCIDR": {}
2021-07-20T04:18:00.7882549Z [cluster1]                     }
2021-07-20T04:18:00.7883184Z [cluster1]                 },
2021-07-20T04:18:00.7886250Z [cluster1]                 "manager": "submariner-operator",
2021-07-20T04:18:00.7886945Z [cluster1]                 "operation": "Update",
2021-07-20T04:18:00.7887882Z [cluster1]                 "time": "2021-07-20T04:17:27Z"
2021-07-20T04:18:00.7888497Z [cluster1]             }
2021-07-20T04:18:00.7888892Z [cluster1]         ],
2021-07-20T04:18:00.7889346Z [cluster1]         "name": "submariner",
2021-07-20T04:18:00.7890164Z [cluster1]         "namespace": "submariner-operator",
2021-07-20T04:18:00.7890840Z [cluster1]         "resourceVersion": "2711",
2021-07-20T04:18:00.7891696Z [cluster1]         "uid": "8e6f527a-b483-4b9e-8238-4c8b8112ee57"
2021-07-20T04:18:00.7892257Z [cluster1]     },
2021-07-20T04:18:00.7892649Z [cluster1]     "spec": {
2021-07-20T04:18:00.7893089Z [cluster1]         "broker": "k8s",
2021-07-20T04:18:00.7893640Z [cluster1]         "brokerK8sApiServer": "172.18.0.8:6443",
2021-07-20T04:18:00.8100638Z [cluster1]         "brokerK8sApiServerToken": "***",
2021-07-20T04:18:00.8690023Z [cluster1]         "brokerK8sCA": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3lNREEwTURjd05Gb1hEVE14TURjeE9EQTBNRGN3TkZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTDMvCm5MZDVnKzRsa2hja3FVZC9ZdHJ2VzZrN3FuM0NrWXlRSkEzL3hvVU1DY1BNUElzR0RySlNhK0tDdzVDVkY4ZE8KWWNQUStpU3ozOXBaNFc0M1dsUjhNVDZkUGx4VzhOVGJDWmFzK3RtKzBFOHE0ZlFpbEdIQWZNenUrYTJQdjRkUgpUTm9BZE9nRU9WcXZ6NExjNmhaUnhnUkFzeGExRHZieUVySXNleDNiU1RuREROb0svSkhQSnFNVkJ3bFNFZTVCCkErelFNV0IrOWJmeHloWmhBZGpMZ1pacm5VT3ZkREp2V0Mzay9TaEZSOWplMWZJRGVQaXE1YncrYWJ4c1lHRXMKaHEyZHFHa1B0bkFkN0laZTV1cnJLdVhicURNei9tOENDSWYyd2ovd05adnMxTkRWaEFrL1pndUxOSUt0Y0loLwp4cnk3YllEZW9rVUtCQ2hlS1pVQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZESjRwZkJmQ0s2Y2hCdmlINWtKQ2RBUmlEVWFNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTlRHZFgyMXZuYWNFY2J6MGlEMGxvRjlYbTcxVmxZRGJGR1h0VzF1a25nZ3R6c3BJOApOL0RGVWFsOU1RZmtmT05zUDVMaWRVSDlDRzRsc0Z4ekloWWxoOHUwMzNiNFVRbTJkN09lTUIwTGc3dnFkT01yCmxLQWZPYzhHa1ViemRQUWh2YTd2SDlkN3dzRWdtVFlzTGVqTEdMaG5sbFFTN0tIbVdLNTN2Qml6amtJamVyeWkKbE4rVEhTelIxbTVwME1QMk8xMmRxVXE5UHhUZHd4VzFkNVc4ZWZDT3VGM0wxdlVseVAyZXVWVXo4dWxGNGFlZApVOHVOVmhMK3NZZ3hVcE9KRVU0Z1hTMEpXYk9FSVpXaFRBUHpoTXRSb04rNjZCK2s0S3VxR2xkaHJZYlpOT0U1CkV2Z3J3a0Z0RUJ6WmRYNWNNMnZ4NE96Zzh4c1NEMHV5U3pRSAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==",
2021-07-20T04:18:00.9202648Z [cluster1]         "brokerK8sRemoteNamespace": "submariner-k8s-broker",
2021-07-20T04:18:00.9203849Z [cluster1]         "cableDriver": "libreswan",
2021-07-20T04:18:00.9204426Z [cluster1]         "ceIPSecDebug": false,
2021-07-20T04:18:00.9204989Z [cluster1]         "ceIPSecIKEPort": 500,
2021-07-20T04:18:00.9205587Z [cluster1]         "ceIPSecNATTPort": 4500,
2021-07-20T04:18:00.9206954Z [cluster1]         "ceIPSecPSK": "8cestBDIYj1jV6cQ5aPRjh5ToQhcnUNDJ87qeMu+K0C+PFNpF4VJX+CFj9D0C8yx",
2021-07-20T04:18:00.9208288Z [cluster1]         "clusterCIDR": "",
2021-07-20T04:18:00.9209134Z [cluster1]         "clusterID": "cluster1",
2021-07-20T04:18:00.9209606Z [cluster1]         "colorCodes": "blue",
2021-07-20T04:18:00.9210168Z [cluster1]         "connectionHealthCheck": {
2021-07-20T04:18:00.9211087Z [cluster1]             "enabled": true,
2021-07-20T04:18:00.9211609Z [cluster1]             "intervalSeconds": 1,
2021-07-20T04:18:00.9212366Z [cluster1]             "maxPacketLossCount": 5
2021-07-20T04:18:00.9212876Z [cluster1]         },
2021-07-20T04:18:00.9213290Z [cluster1]         "debug": false,
2021-07-20T04:18:00.9214443Z [cluster1]         "namespace": "submariner-operator",
2021-07-20T04:18:00.9215062Z [cluster1]         "natEnabled": false,
2021-07-20T04:18:00.9215893Z [cluster1]         "repository": "localhost:5000",
2021-07-20T04:18:00.9216439Z [cluster1]         "serviceCIDR": "",
2021-07-20T04:18:00.9217169Z [cluster1]         "serviceDiscoveryEnabled": true,
2021-07-20T04:18:00.9217785Z [cluster1]         "version": "local"
2021-07-20T04:18:00.9218175Z [cluster1]     },
2021-07-20T04:18:00.9218557Z [cluster1]     "status": {
2021-07-20T04:18:00.9218996Z [cluster1]         "clusterCIDR": "10.1.0.0/16",
2021-07-20T04:18:00.9219500Z [cluster1]         "clusterID": "cluster1",
2021-07-20T04:18:00.9219994Z [cluster1]         "colorCodes": "blue",
2021-07-20T04:18:00.9220602Z [cluster1]         "deploymentInfo": {},
2021-07-20T04:18:00.9221197Z [cluster1]         "gatewayDaemonSetStatus": {
2021-07-20T04:18:00.9222043Z [cluster1]             "lastResourceVersion": "2665",
2021-07-20T04:18:00.9222784Z [cluster1]             "mismatchedContainerImages": false,
2021-07-20T04:18:00.9223700Z [cluster1]             "nonReadyContainerStates": [],
2021-07-20T04:18:00.9224266Z [cluster1]             "status": {
2021-07-20T04:18:00.9224807Z [cluster1]                 "currentNumberScheduled": 1,
2021-07-20T04:18:00.9225505Z [cluster1]                 "desiredNumberScheduled": 1,
2021-07-20T04:18:00.9226114Z [cluster1]                 "numberAvailable": 1,
2021-07-20T04:18:00.9226691Z [cluster1]                 "numberMisscheduled": 0,
2021-07-20T04:18:00.9227223Z [cluster1]                 "numberReady": 1,
2021-07-20T04:18:00.9227772Z [cluster1]                 "observedGeneration": 1,
2021-07-20T04:18:00.9228563Z [cluster1]                 "updatedNumberScheduled": 1
2021-07-20T04:18:00.9229079Z [cluster1]             }
2021-07-20T04:18:00.9229444Z [cluster1]         },
2021-07-20T04:18:00.9229826Z [cluster1]         "gateways": [
2021-07-20T04:18:00.9230220Z [cluster1]             {
2021-07-20T04:18:00.9230630Z [cluster1]                 "connections": [
2021-07-20T04:18:00.9231055Z [cluster1]                     {
2021-07-20T04:18:00.9231466Z [cluster1]                         "endpoint": {
2021-07-20T04:18:00.9231968Z [cluster1]                             "backend": "libreswan",
2021-07-20T04:18:00.9232468Z [cluster1]                             "backend_config": {
2021-07-20T04:18:00.9233669Z [cluster1]                                 "natt-discovery-port": "4490",
2021-07-20T04:18:00.9235151Z [cluster1]                                 "preferred-server": "false"
2021-07-20T04:18:00.9235694Z [cluster1]                             },
2021-07-20T04:18:00.9236879Z [cluster1]                             "cable_name": "submariner-cable-cluster2-172-18-0-5",
2021-07-20T04:18:00.9238229Z [cluster1]                             "cluster_id": "cluster2",
2021-07-20T04:18:00.9238769Z [cluster1]                             "healthCheckIP": "10.2.128.0",
2021-07-20T04:18:00.9239562Z [cluster1]                             "hostname": "cluster2-worker",
2021-07-20T04:18:00.9240324Z [cluster1]                             "nat_enabled": false,
2021-07-20T04:18:00.9241025Z [cluster1]                             "private_ip": "172.18.0.5",
2021-07-20T04:18:00.9241516Z [cluster1]                             "public_ip": "52.137.72.137",
2021-07-20T04:18:00.9241999Z [cluster1]                             "subnets": [
2021-07-20T04:18:00.9242436Z [cluster1]                                 "100.2.0.0/16",
2021-07-20T04:18:00.9242864Z [cluster1]                                 "10.2.0.0/16"
2021-07-20T04:18:00.9243428Z [cluster1]                             ]
2021-07-20T04:18:00.9243800Z [cluster1]                         },
2021-07-20T04:18:00.9244243Z [cluster1]                         "latencyRTT": {
2021-07-20T04:18:00.9244904Z [cluster1]                             "average": "0s",
2021-07-20T04:18:00.9245866Z [cluster1]                             "last": "0s",
2021-07-20T04:18:00.9246451Z [cluster1]                             "max": "0s",
2021-07-20T04:18:00.9246886Z [cluster1]                             "min": "0s",
2021-07-20T04:18:00.9247321Z [cluster1]                             "stdDev": "0s"
2021-07-20T04:18:00.9248356Z [cluster1]                         },
2021-07-20T04:18:00.9248979Z [cluster1]                         "status": "error",
2021-07-20T04:18:00.9249843Z [cluster1]                         "statusMessage": "Failed to successfully ping the remote endpoint IP \"10.2.128.0\"",
2021-07-20T04:18:00.9250705Z [cluster1]                         "usingIP": "172.18.0.5"
2021-07-20T04:18:00.9251126Z [cluster1]                     }
2021-07-20T04:18:00.9251509Z [cluster1]                 ],
2021-07-20T04:18:00.9251949Z [cluster1]                 "haStatus": "active",
2021-07-20T04:18:00.9252466Z [cluster1]                 "localEndpoint": {
2021-07-20T04:18:00.9253191Z [cluster1]                     "backend": "libreswan",
2021-07-20T04:18:00.9253723Z [cluster1]                     "backend_config": {
2021-07-20T04:18:00.9254550Z [cluster1]                         "natt-discovery-port": "4490",
2021-07-20T04:18:00.9255566Z [cluster1]                         "preferred-server": "false"
2021-07-20T04:18:00.9256093Z [cluster1]                     },
2021-07-20T04:18:00.9257181Z [cluster1]                     "cable_name": "submariner-cable-cluster1-172-18-0-4",
2021-07-20T04:18:00.9258153Z [cluster1]                     "cluster_id": "cluster1",
2021-07-20T04:18:00.9258693Z [cluster1]                     "healthCheckIP": "10.1.224.0",
2021-07-20T04:18:00.9259519Z [cluster1]                     "hostname": "cluster1-worker",
2021-07-20T04:18:00.9260087Z [cluster1]                     "nat_enabled": false,
2021-07-20T04:18:00.9260780Z [cluster1]                     "private_ip": "172.18.0.4",
2021-07-20T04:18:00.9261259Z [cluster1]                     "public_ip": "52.137.72.137",
2021-07-20T04:18:00.9261728Z [cluster1]                     "subnets": [
2021-07-20T04:18:00.9262163Z [cluster1]                         "100.1.0.0/16",
2021-07-20T04:18:00.9272091Z [cluster1]                         "10.1.0.0/16"
2021-07-20T04:18:00.9272561Z [cluster1]                     ]
2021-07-20T04:18:00.9272959Z [cluster1]                 },
2021-07-20T04:18:00.9273442Z [cluster1]                 "statusFailure": "",
2021-07-20T04:18:00.9274668Z [cluster1]                 "version": "devel-1441a80"
2021-07-20T04:18:00.9275161Z [cluster1]             }
2021-07-20T04:18:00.9275533Z [cluster1]         ],
2021-07-20T04:18:00.9276123Z [cluster1]         "globalnetDaemonSetStatus": {
2021-07-20T04:18:00.9283757Z [cluster1]             "mismatchedContainerImages": false
2021-07-20T04:18:00.9284396Z [cluster1]         },
2021-07-20T04:18:00.9284925Z [cluster1]         "loadBalancerStatus": {},
2021-07-20T04:18:00.9285496Z [cluster1]         "natEnabled": false,
2021-07-20T04:18:00.9286404Z [cluster1]         "networkPlugin": "weave-net",
2021-07-20T04:18:00.9287127Z [cluster1]         "routeAgentDaemonSetStatus": {
2021-07-20T04:18:00.9287873Z [cluster1]             "lastResourceVersion": "2635",
2021-07-20T04:18:00.9297574Z [cluster1]             "mismatchedContainerImages": false,
2021-07-20T04:18:00.9298483Z [cluster1]             "nonReadyContainerStates": [],
2021-07-20T04:18:00.9299081Z [cluster1]             "status": {
2021-07-20T04:18:00.9299680Z [cluster1]                 "currentNumberScheduled": 3,
2021-07-20T04:18:00.9301636Z [cluster1]                 "desiredNumberScheduled": 3,
2021-07-20T04:18:00.9302349Z [cluster1]                 "numberAvailable": 3,
2021-07-20T04:18:00.9303019Z [cluster1]                 "numberMisscheduled": 0,
2021-07-20T04:18:00.9303630Z [cluster1]                 "numberReady": 3,
2021-07-20T04:18:00.9304844Z [cluster1]                 "observedGeneration": 1,
2021-07-20T04:18:00.9305731Z [cluster1]                 "updatedNumberScheduled": 3
2021-07-20T04:18:00.9306378Z [cluster1]             }
2021-07-20T04:18:00.9306754Z [cluster1]         },
2021-07-20T04:18:00.9307220Z [cluster1]         "serviceCIDR": "100.1.0.0/16"
2021-07-20T04:18:00.9307664Z [cluster1]     }
2021-07-20T04:18:00.9308037Z [cluster1] }
2021-07-20T04:18:00.9309398Z [36m[submariner-operator]$ [cluster1] validate_equals .metadata.namespace submariner-operator[0m
2021-07-20T04:18:00.9310971Z [36m[submariner-operator]$ [cluster1] validate_equals .metadata.namespace submariner-operator[0m
2021-07-20T04:18:00.9312502Z [36m[submariner-operator]$ [cluster1] jq -r -M .metadata.namespace /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:00.9313914Z [36m[submariner-operator]$ [cluster1] validate_equals .apiVersion submariner.io/v1alpha1[0m
2021-07-20T04:18:00.9315219Z [36m[submariner-operator]$ [cluster1] validate_equals .apiVersion submariner.io/v1alpha1[0m
2021-07-20T04:18:01.0740451Z [="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:18:01.0744314Z [cluster2] I0720 04:17:57.879530       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:01.0748533Z [cluster2] I0720 04:17:57.892541       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-20T04:18:01.0751725Z [cluster2] I0720 04:17:58.451419       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:18:01.0755596Z [cluster2] I0720 04:17:58.462515       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:01.0759253Z [cluster2] I0720 04:17:58.494923       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-20T04:18:01.0765316Z [cluster2] I0720 04:17:58.501175       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-20T04:18:01.0767418Z [36m[submariner-operator]$ [cluster2] verify_subm_cr[0m
2021-07-20T04:18:01.0768433Z [36m[submariner-operator]$ [cluster2] verify_subm_cr[0m
2021-07-20T04:18:01.0769956Z [36m[submariner-operator]$ [cluster2] kubectl get submariner --namespace=submariner-operator[0m
2021-07-20T04:18:01.0771141Z [36m[submariner-operator]$ [cluster2] grep submariner[0m
2021-07-20T04:18:01.0772571Z [36m[submariner-operator]$ [cluster2] kubectl get submariner --namespace=submariner-operator[0m
2021-07-20T04:18:01.0774232Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get submariner --namespace=submariner-operator[0m
2021-07-20T04:18:01.0775401Z [cluster2] submariner   59s
2021-07-20T04:18:01.0776450Z [36m[submariner-operator]$ [cluster2] json_file=/tmp/submariner.cluster2.json[0m
2021-07-20T04:18:01.0778199Z [36m[submariner-operator]$ [cluster2] kubectl get submariner submariner --namespace=submariner-operator -o json[0m
2021-07-20T04:18:01.0779780Z [36m[submariner-operator]$ [cluster2] tee /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:01.0781299Z [36m[submariner-operator]$ [cluster2] kubectl get submariner submariner --namespace=submariner-operator -o json[0m
2021-07-20T04:18:01.0783037Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get submariner submariner --namespace=submariner-operator -o json[0m
2021-07-20T04:18:01.0784227Z [cluster2] {
2021-07-20T04:18:01.0784919Z [cluster2]     "apiVersion": "submariner.io/v1alpha1",
2021-07-20T04:18:01.0785557Z [cluster2]     "kind": "Submariner",
2021-07-20T04:18:01.0786045Z [cluster2]     "metadata": {
2021-07-20T04:18:01.0786840Z [cluster2]         "creationTimestamp": "2021-07-20T04:17:01Z",
2021-07-20T04:18:01.0787424Z [cluster2]         "generation": 1,
2021-07-20T04:18:01.0787930Z [cluster2]         "managedFields": [
2021-07-20T04:18:01.0788396Z [cluster2]             {
2021-07-20T04:18:01.0788935Z [cluster2]                 "apiVersion": "submariner.io/v1alpha1",
2021-07-20T04:18:01.0789594Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-20T04:18:01.0790108Z [cluster2]                 "fieldsV1": {
2021-07-20T04:18:01.0790570Z [cluster2]                     "f:spec": {
2021-07-20T04:18:01.0790991Z [cluster2]                         ".": {},
2021-07-20T04:18:01.0791439Z [cluster2]                         "f:broker": {},
2021-07-20T04:18:01.0792015Z [cluster2]                         "f:brokerK8sApiServer": {},
2021-07-20T04:18:01.0792734Z [cluster2]                         "f:brokerK8sApiServerToken": {},
2021-07-20T04:18:01.0793399Z [cluster2]                         "f:brokerK8sCA": {},
2021-07-20T04:18:01.0794254Z [cluster2]                         "f:brokerK8sRemoteNamespace": {},
2021-07-20T04:18:01.0794946Z [cluster2]                         "f:cableDriver": {},
2021-07-20T04:18:01.0795489Z [cluster2]                         "f:ceIPSecDebug": {},
2021-07-20T04:18:01.0796082Z [cluster2]                         "f:ceIPSecIKEPort": {},
2021-07-20T04:18:01.0796831Z [cluster2]                         "f:ceIPSecNATTPort": {},
2021-07-20T04:18:01.0797420Z [cluster2]                         "f:ceIPSecPSK": {},
2021-07-20T04:18:01.0797966Z [cluster2]                         "f:clusterCIDR": {},
2021-07-20T04:18:01.0798477Z [cluster2]                         "f:clusterID": {},
2021-07-20T04:18:01.0799024Z [cluster2]                         "f:colorCodes": {},
2021-07-20T04:18:01.0799638Z [cluster2]                         "f:connectionHealthCheck": {
2021-07-20T04:18:01.0800412Z [cluster2]                             ".": {},
2021-07-20T04:18:01.0801046Z [cluster2]                             "f:enabled": {},
2021-07-20T04:18:01.0801839Z [cluster2]                             "f:intervalSeconds": {},
2021-07-20T04:18:01.0802472Z [cluster2]                             "f:maxPacketLossCount": {}
2021-07-20T04:18:01.0803018Z [cluster2]                         },
2021-07-20T04:18:01.0803613Z [cluster2]                         "f:debug": {},
2021-07-20T04:18:01.0804081Z [cluster2]                         "f:namespace": {},
2021-07-20T04:18:01.0804584Z [cluster2]                         "f:natEnabled": {},
2021-07-20T04:18:01.0805078Z [cluster2]                         "f:repository": {},
2021-07-20T04:18:01.0805596Z [cluster2]                         "f:serviceCIDR": {},
2021-07-20T04:18:01.0806221Z [cluster2]                         "f:serviceDiscoveryEnabled": {},
2021-07-20T04:18:01.0806833Z [cluster2]                         "f:version": {}
2021-07-20T04:18:01.0807248Z [cluster2]                     }
2021-07-20T04:18:01.0807634Z [cluster2]                 },
2021-07-20T04:18:01.0808059Z [cluster2]                 "manager": "subctl",
2021-07-20T04:18:01.0808569Z [cluster2]                 "operation": "Update",
2021-07-20T04:18:01.0809324Z [cluster2]                 "time": "2021-07-20T04:17:01Z"
2021-07-20T04:18:01.0809736Z [cluster2]             },
2021-07-20T04:18:01.0810147Z [cluster2]             {
2021-07-20T04:18:01.0810667Z [cluster2]                 "apiVersion": "submariner.io/v1alpha1",
2021-07-20T04:18:01.0811302Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-20T04:18:01.0811800Z [cluster2]                 "fieldsV1": {
2021-07-20T04:18:01.0812253Z [cluster2]                     "f:status": {
2021-07-20T04:18:01.0812664Z [cluster2]                         ".": {},
2021-07-20T04:18:01.0813132Z [cluster2]                         "f:clusterCIDR": {},
2021-07-20T04:18:01.0813641Z [cluster2]                         "f:clusterID": {},
2021-07-20T04:18:01.0814657Z [cluster2]                         "f:colorCodes": {},
2021-07-20T04:18:01.0815236Z [cluster2]                         "f:deploymentInfo": {},
2021-07-20T04:18:01.0815879Z [cluster2]                         "f:gatewayDaemonSetStatus": {
2021-07-20T04:18:01.0816460Z [cluster2]                             ".": {},
2021-07-20T04:18:01.0857811Z [cluster2]                             "f:lastResourceVersion": {},
2021-07-20T04:18:01.0858642Z [cluster2]                             "f:mismatchedContainerImages": {},
2021-07-20T04:18:01.0859499Z [cluster2]                             "f:nonReadyContainerStates": {},
2021-07-20T04:18:01.0860162Z [cluster2]                             "f:status": {
2021-07-20T04:18:01.0860634Z [cluster2]                                 ".": {},
2021-07-20T04:18:01.0861246Z [cluster2]                                 "f:currentNumberScheduled": {},
2021-07-20T04:18:01.0862237Z [cluster2]                                 "f:desiredNumberScheduled": {},
2021-07-20T04:18:01.0862952Z [cluster2]                                 "f:numberAvailable": {},
2021-07-20T04:18:01.0863628Z [cluster2]                                 "f:numberMisscheduled": {},
2021-07-20T04:18:01.0864252Z [cluster2]                                 "f:numberReady": {},
2021-07-20T04:18:01.0864896Z [cluster2]                                 "f:observedGeneration": {},
2021-07-20T04:18:01.0865644Z [cluster2]                                 "f:updatedNumberScheduled": {}
2021-07-20T04:18:01.0866264Z [cluster2]                             }
2021-07-20T04:18:01.0866899Z [cluster2]                         },
2021-07-20T04:18:01.0867360Z [cluster2]                         "f:gateways": {},
2021-07-20T04:18:01.0868048Z [cluster2]                         "f:globalnetDaemonSetStatus": {
2021-07-20T04:18:01.0868682Z [cluster2]                             ".": {},
2021-07-20T04:18:01.0869343Z [cluster2]                             "f:mismatchedContainerImages": {}
2021-07-20T04:18:01.0869972Z [cluster2]                         },
2021-07-20T04:18:01.0870535Z [cluster2]                         "f:loadBalancerStatus": {},
2021-07-20T04:18:01.0871136Z [cluster2]                         "f:natEnabled": {},
2021-07-20T04:18:01.0871704Z [cluster2]                         "f:networkPlugin": {},
2021-07-20T04:18:01.0872440Z [cluster2]                         "f:routeAgentDaemonSetStatus": {
2021-07-20T04:18:01.0873084Z [cluster2]                             ".": {},
2021-07-20T04:18:01.0873661Z [cluster2]                             "f:lastResourceVersion": {},
2021-07-20T04:18:01.0874699Z [cluster2]                             "f:mismatchedContainerImages": {},
2021-07-20T04:18:01.0875582Z [cluster2]                             "f:nonReadyContainerStates": {},
2021-07-20T04:18:01.0876248Z [cluster2]                             "f:status": {
2021-07-20T04:18:01.0876800Z [cluster2]                                 ".": {},
2021-07-20T04:18:01.0877441Z [cluster2]                                 "f:currentNumberScheduled": {},
2021-07-20T04:18:01.0878435Z [cluster2]                                 "f:desiredNumberScheduled": {},
2021-07-20T04:18:01.0879277Z [cluster2]                                 "f:numberAvailable": {},
2021-07-20T04:18:01.0880103Z [cluster2]                                 "f:numberMisscheduled": {},
2021-07-20T04:18:01.0880768Z [cluster2]                                 "f:numberReady": {},
2021-07-20T04:18:01.0881415Z [cluster2]                                 "f:observedGeneration": {},
2021-07-20T04:18:01.0882338Z [cluster2]                                 "f:updatedNumberScheduled": {}
2021-07-20T04:18:01.0882939Z [cluster2]                             }
2021-07-20T04:18:01.0883367Z [cluster2]                         },
2021-07-20T04:18:01.0883862Z [cluster2]                         "f:serviceCIDR": {}
2021-07-20T04:18:01.0884332Z [cluster2]                     }
2021-07-20T04:18:01.0884742Z [cluster2]                 },
2021-07-20T04:18:01.0885678Z [cluster2]                 "manager": "submariner-operator",
2021-07-20T04:18:01.0886483Z [cluster2]                 "operation": "Update",
2021-07-20T04:18:01.0887268Z [cluster2]                 "time": "2021-07-20T04:17:30Z"
2021-07-20T04:18:01.0888019Z [cluster2]             }
2021-07-20T04:18:01.0888415Z [cluster2]         ],
2021-07-20T04:18:01.0889037Z [cluster2]         "name": "submariner",
2021-07-20T04:18:01.0890664Z [cluster2]         "namespace": "submariner-operator",
2021-07-20T04:18:01.0891835Z [cluster2]         "resourceVersion": "2659",
2021-07-20T04:18:01.0892737Z [cluster2]         "uid": "96422515-ce64-46b3-8e98-aeb48188a435"
2021-07-20T04:18:01.0893488Z [cluster2]     },
2021-07-20T04:18:01.0893914Z [cluster2]     "spec": {
2021-07-20T04:18:01.0894368Z [cluster2]         "broker": "k8s",
2021-07-20T04:18:01.0894982Z [cluster2]         "brokerK8sApiServer": "172.18.0.8:6443",
2021-07-20T04:18:01.1049938Z [cluster2]         "brokerK8sApiServerToken": "***",
2021-07-20T04:18:01.1589305Z [cluster2]         "brokerK8sCA": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3lNREEwTURjd05Gb1hEVE14TURjeE9EQTBNRGN3TkZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTDMvCm5MZDVnKzRsa2hja3FVZC9ZdHJ2VzZrN3FuM0NrWXlRSkEzL3hvVU1DY1BNUElzR0RySlNhK0tDdzVDVkY4ZE8KWWNQUStpU3ozOXBaNFc0M1dsUjhNVDZkUGx4VzhOVGJDWmFzK3RtKzBFOHE0ZlFpbEdIQWZNenUrYTJQdjRkUgpUTm9BZE9nRU9WcXZ6NExjNmhaUnhnUkFzeGExRHZieUVySXNleDNiU1RuREROb0svSkhQSnFNVkJ3bFNFZTVCCkErelFNV0IrOWJmeHloWmhBZGpMZ1pacm5VT3ZkREp2V0Mzay9TaEZSOWplMWZJRGVQaXE1YncrYWJ4c1lHRXMKaHEyZHFHa1B0bkFkN0laZTV1cnJLdVhicURNei9tOENDSWYyd2ovd05adnMxTkRWaEFrL1pndUxOSUt0Y0loLwp4cnk3YllEZW9rVUtCQ2hlS1pVQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZESjRwZkJmQ0s2Y2hCdmlINWtKQ2RBUmlEVWFNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTlRHZFgyMXZuYWNFY2J6MGlEMGxvRjlYbTcxVmxZRGJGR1h0VzF1a25nZ3R6c3BJOApOL0RGVWFsOU1RZmtmT05zUDVMaWRVSDlDRzRsc0Z4ekloWWxoOHUwMzNiNFVRbTJkN09lTUIwTGc3dnFkT01yCmxLQWZPYzhHa1ViemRQUWh2YTd2SDlkN3dzRWdtVFlzTGVqTEdMaG5sbFFTN0tIbVdLNTN2Qml6amtJamVyeWkKbE4rVEhTelIxbTVwME1QMk8xMmRxVXE5UHhUZHd4VzFkNVc4ZWZDT3VGM0wxdlVseVAyZXVWVXo4dWxGNGFlZApVOHVOVmhMK3NZZ3hVcE9KRVU0Z1hTMEpXYk9FSVpXaFRBUHpoTXRSb04rNjZCK2s0S3VxR2xkaHJZYlpOT0U1CkV2Z3J3a0Z0RUJ6WmRYNWNNMnZ4NE96Zzh4c1NEMHV5U3pRSAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==",
2021-07-20T04:18:01.2145528Z [cluster2]         "brokerK8sRemoteNamespace": "submariner-k8s-broker",
2021-07-20T04:18:01.2146627Z [cluster2]         "cableDriver": "libreswan",
2021-07-20T04:18:01.2147298Z [cluster2]         "ceIPSecDebug": false,
2021-07-20T04:18:01.2153592Z [cluster2]         "ceIPSecIKEPort": 500,
2021-07-20T04:18:01.2154217Z [cluster2]         "ceIPSecNATTPort": 4500,
2021-07-20T04:18:01.2155699Z [cluster2]         "ceIPSecPSK": "8cestBDIYj1jV6cQ5aPRjh5ToQhcnUNDJ87qeMu+K0C+PFNpF4VJX+CFj9D0C8yx",
2021-07-20T04:18:01.2157102Z [cluster2]         "clusterCIDR": "",
2021-07-20T04:18:01.2157641Z [cluster2]         "clusterID": "cluster2",
2021-07-20T04:18:01.2158183Z [cluster2]         "colorCodes": "blue",
2021-07-20T04:18:01.2158806Z [cluster2]         "connectionHealthCheck": {
2021-07-20T04:18:01.2159410Z [cluster2]             "enabled": true,
2021-07-20T04:18:01.2159939Z [cluster2]             "intervalSeconds": 1,
2021-07-20T04:18:01.2161041Z [cluster2]             "maxPacketLossCount": 5
2021-07-20T04:18:01.2161573Z [cluster2]         },
2021-07-20T04:18:01.2162030Z [cluster2]         "debug": false,
2021-07-20T04:18:01.2163023Z [cluster2]         "namespace": "submariner-operator",
2021-07-20T04:18:01.2164043Z [cluster2]         "natEnabled": false,
2021-07-20T04:18:01.2164592Z [cluster2]         "repository": "localhost:5000",
2021-07-20T04:18:01.2165138Z [cluster2]         "serviceCIDR": "",
2021-07-20T04:18:01.2165776Z [cluster2]         "serviceDiscoveryEnabled": true,
2021-07-20T04:18:01.2166416Z [cluster2]         "version": "local"
2021-07-20T04:18:01.2167038Z [cluster2]     },
2021-07-20T04:18:01.2167773Z [cluster2]     "status": {
2021-07-20T04:18:01.2168454Z [cluster2]         "clusterCIDR": "10.2.0.0/16",
2021-07-20T04:18:01.2168976Z [cluster2]         "clusterID": "cluster2",
2021-07-20T04:18:01.2169520Z [cluster2]         "colorCodes": "blue",
2021-07-20T04:18:01.2170048Z [cluster2]         "deploymentInfo": {},
2021-07-20T04:18:01.2170694Z [cluster2]         "gatewayDaemonSetStatus": {
2021-07-20T04:18:01.2171382Z [cluster2]             "lastResourceVersion": "2465",
2021-07-20T04:18:01.2172147Z [cluster2]             "mismatchedContainerImages": false,
2021-07-20T04:18:01.2172945Z [cluster2]             "nonReadyContainerStates": [],
2021-07-20T04:18:01.2173546Z [cluster2]             "status": {
2021-07-20T04:18:01.2174500Z [cluster2]                 "currentNumberScheduled": 1,
2021-07-20T04:18:01.2175234Z [cluster2]                 "desiredNumberScheduled": 1,
2021-07-20T04:18:01.2175904Z [cluster2]                 "numberAvailable": 1,
2021-07-20T04:18:01.2176504Z [cluster2]                 "numberMisscheduled": 0,
2021-07-20T04:18:01.2177212Z [cluster2]                 "numberReady": 1,
2021-07-20T04:18:01.2177796Z [cluster2]                 "observedGeneration": 1,
2021-07-20T04:18:01.2178503Z [cluster2]                 "updatedNumberScheduled": 1
2021-07-20T04:18:01.2179058Z [cluster2]             }
2021-07-20T04:18:01.2179992Z [cluster2]         },
2021-07-20T04:18:01.2180660Z [cluster2]         "gateways": [
2021-07-20T04:18:01.2181091Z [cluster2]             {
2021-07-20T04:18:01.2181525Z [cluster2]                 "connections": [
2021-07-20T04:18:01.2182146Z [cluster2]                     {
2021-07-20T04:18:01.2182600Z [cluster2]                         "endpoint": {
2021-07-20T04:18:01.2183122Z [cluster2]                             "backend": "libreswan",
2021-07-20T04:18:01.2183674Z [cluster2]                             "backend_config": {
2021-07-20T04:18:01.2184846Z [cluster2]                                 "natt-discovery-port": "4490",
2021-07-20T04:18:01.2185831Z [cluster2]                                 "preferred-server": "false"
2021-07-20T04:18:01.2186371Z [cluster2]                             },
2021-07-20T04:18:01.2187546Z [cluster2]                             "cable_name": "submariner-cable-cluster1-172-18-0-4",
2021-07-20T04:18:01.2188597Z [cluster2]                             "cluster_id": "cluster1",
2021-07-20T04:18:01.2189538Z [cluster2]                             "healthCheckIP": "10.1.224.0",
2021-07-20T04:18:01.2190539Z [cluster2]                             "hostname": "cluster1-worker",
2021-07-20T04:18:01.2191475Z [cluster2]                             "nat_enabled": false,
2021-07-20T04:18:01.2191994Z [cluster2]                             "private_ip": "172.18.0.4",
2021-07-20T04:18:01.2192664Z [cluster2]                             "public_ip": "52.137.72.137",
2021-07-20T04:18:01.2193343Z [cluster2]                             "subnets": [
2021-07-20T04:18:01.2193996Z [cluster2]                                 "100.1.0.0/16",
2021-07-20T04:18:01.2194467Z [cluster2]                                 "10.1.0.0/16"
2021-07-20T04:18:01.2194919Z [cluster2]                             ]
2021-07-20T04:18:01.2195800Z [cluster2]                         },
2021-07-20T04:18:01.2196248Z [cluster2]                         "latencyRTT": {
2021-07-20T04:18:01.2196965Z [cluster2]                             "average": "253.848µs",
2021-07-20T04:18:01.2197825Z [cluster2]                             "last": "216.799µs",
2021-07-20T04:18:01.2198468Z [cluster2]                             "max": "593.596µs",
2021-07-20T04:18:01.2199137Z [cluster2]                             "min": "153.6µs",
2021-07-20T04:18:01.2200171Z [cluster2]                             "stdDev": "90.352µs"
2021-07-20T04:18:01.2200656Z [cluster2]                         },
2021-07-20T04:18:01.2201326Z [cluster2]                         "status": "connected",
2021-07-20T04:18:01.2202032Z [cluster2]                         "statusMessage": "",
2021-07-20T04:18:01.2202562Z [cluster2]                         "usingIP": "172.18.0.4"
2021-07-20T04:18:01.2203120Z [cluster2]                     }
2021-07-20T04:18:01.2203537Z [cluster2]                 ],
2021-07-20T04:18:01.2203988Z [cluster2]                 "haStatus": "active",
2021-07-20T04:18:01.2204957Z [cluster2]                 "localEndpoint": {
2021-07-20T04:18:01.2205535Z [cluster2]                     "backend": "libreswan",
2021-07-20T04:18:01.2206108Z [cluster2]                     "backend_config": {
2021-07-20T04:18:01.2206987Z [cluster2]                         "natt-discovery-port": "4490",
2021-07-20T04:18:01.2207937Z [cluster2]                         "preferred-server": "false"
2021-07-20T04:18:01.2208660Z [cluster2]                     },
2021-07-20T04:18:01.2209691Z [cluster2]                     "cable_name": "submariner-cable-cluster2-172-18-0-5",
2021-07-20T04:18:01.2210923Z [cluster2]                     "cluster_id": "cluster2",
2021-07-20T04:18:01.2211498Z [cluster2]                     "healthCheckIP": "10.2.128.0",
2021-07-20T04:18:01.2212577Z [cluster2]                     "hostname": "cluster2-worker",
2021-07-20T04:18:01.2213353Z [cluster2]                     "nat_enabled": false,
2021-07-20T04:18:01.2213899Z [cluster2]                     "private_ip": "172.18.0.5",
2021-07-20T04:18:01.2214408Z [cluster2]                     "public_ip": "52.137.72.137",
2021-07-20T04:18:01.2214912Z [cluster2]                     "subnets": [
2021-07-20T04:18:01.2215373Z [cluster2]                         "100.2.0.0/16",
2021-07-20T04:18:01.2215798Z [cluster2]                         "10.2.0.0/16"
2021-07-20T04:18:01.2216538Z [cluster2]                     ]
2021-07-20T04:18:01.2217000Z [cluster2]                 },
2021-07-20T04:18:01.2217780Z [cluster2]                 "statusFailure": "",
2021-07-20T04:18:01.2218779Z [cluster2]                 "version": "devel-1441a80"
2021-07-20T04:18:01.2219283Z [cluster2]             }
2021-07-20T04:18:01.2219667Z [cluster2]         ],
2021-07-20T04:18:01.2220276Z [cluster2]         "globalnetDaemonSetStatus": {
2021-07-20T04:18:01.2221467Z [cluster2]             "mismatchedContainerImages": false
2021-07-20T04:18:01.2222120Z [cluster2]         },
2021-07-20T04:18:01.2222649Z [cluster2]         "loadBalancerStatus": {},
2021-07-20T04:18:01.2243974Z [cluster2]         "natEnab36m[submariner-operator]$ [cluster1] jq -r -M .apiVersion /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:01.2245691Z [36m[submariner-operator]$ [cluster1] validate_equals .kind Submariner[0m
2021-07-20T04:18:01.2246990Z [36m[submariner-operator]$ [cluster1] validate_equals .kind Submariner[0m
2021-07-20T04:18:01.2248703Z [36m[submariner-operator]$ [cluster1] jq -r -M .kind /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:01.2250344Z [36m[submariner-operator]$ [cluster1] validate_equals .metadata.name submariner[0m
2021-07-20T04:18:01.2251768Z [36m[submariner-operator]$ [cluster1] validate_equals .metadata.name submariner[0m
2021-07-20T04:18:01.2253453Z [36m[submariner-operator]$ [cluster1] jq -r -M .metadata.name /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:01.2255053Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sApiServer 172.18.0.8:6443[0m
2021-07-20T04:18:01.2256591Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sApiServer 172.18.0.8:6443[0m
2021-07-20T04:18:01.2258727Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.brokerK8sApiServer /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:01.2400881Z [36m[submariner-operator]$ [cluster1] validate_not_equals .spec.brokerK8sApiServerToken ***
2021-07-20T04:18:01.2574911Z [36m[submariner-operator]$ [cluster1] validate_not_equals .spec.brokerK8sApiServerToken ***
2021-07-20T04:18:01.2579039Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.brokerK8sApiServerToken /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:01.4731357Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sCA LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3lNREEwTURjd05Gb1hEVE14TURjeE9EQTBNRGN3TkZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTDMvCm5MZDVnKzRsa2hja3FVZC9ZdHJ2VzZrN3FuM0NrWXlRSkEzL3hvVU1DY1BNUElzR0RySlNhK0tDdzVDVkY4ZE8KWWNQUStpU3ozOXBaNFc0M1dsUjhNVDZkUGx4VzhOVGJDWmFzK3RtKzBFOHE0ZlFpbEdIQWZNenUrYTJQdjRkUgpUTm9BZE9nRU9WcXZ6NExjNmhaUnhnUkFzeGExRHZieUVySXNleDNiU1RuREROb0svSkhQSnFNVkJ3bFNFZTVCCkErelFNV0IrOWJmeHloWmhBZGpMZ1pacm5VT3ZkREp2V0Mzay9TaEZSOWplMWZJRGVQaXE1YncrYWJ4c1lHRXMKaHEyZHFHa1B0bkFkN0laZTV1cnJLdVhicURNei9tOENDSWYyd2ovd05adnMxTkRWaEFrL1pndUxOSUt0Y0loLwp4cnk3YllEZW9rVUtCQ2hlS1pVQ0F3RUFBYU5DTUVBd0Rled": false,
2021-07-20T04:18:01.4916572Z [cluster2]         "networkPlugin": "weave-net",
2021-07-20T04:18:01.4917505Z [cluster2]         "routeAgentDaemonSetStatus": {
2021-07-20T04:18:01.4919050Z [cluster2]             "lastResourceVersion": "2525",
2021-07-20T04:18:01.4919871Z [cluster2]             "mismatchedContainerImages": false,
2021-07-20T04:18:01.4924863Z [cluster2]             "nonReadyContainerStates": [],
2021-07-20T04:18:01.4925527Z [cluster2]             "status": {
2021-07-20T04:18:01.4926115Z [cluster2]                 "currentNumberScheduled": 3,
2021-07-20T04:18:01.4926863Z [cluster2]                 "desiredNumberScheduled": 3,
2021-07-20T04:18:01.4927997Z [cluster2]                 "numberAvailable": 3,
2021-07-20T04:18:01.4929337Z [cluster2]                 "numberMisscheduled": 0,
2021-07-20T04:18:01.4929952Z [cluster2]                 "numberReady": 3,
2021-07-20T04:18:01.4930895Z [cluster2]                 "observedGeneration": 1,
2021-07-20T04:18:01.4931634Z [cluster2]                 "updatedNumberScheduled": 3
2021-07-20T04:18:01.4932390Z [cluster2]             }
2021-07-20T04:18:01.4933188Z [cluster2]         },
2021-07-20T04:18:01.4933680Z [cluster2]         "serviceCIDR": "100.2.0.0/16"
2021-07-20T04:18:01.4934130Z [cluster2]     }
2021-07-20T04:18:01.4934488Z [cluster2] }
2021-07-20T04:18:01.4936033Z [36m[submariner-operator]$ [cluster2] validate_equals .metadata.namespace submariner-operator[0m
2021-07-20T04:18:01.4937811Z [36m[submariner-operator]$ [cluster2] validate_equals .metadata.namespace submariner-operator[0m
2021-07-20T04:18:01.4940164Z [36m[submariner-operator]$ [cluster2] jq -r -M .metadata.namespace /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:01.4941598Z [36m[submariner-operator]$ [cluster2] validate_equals .apiVersion submariner.io/v1alpha1[0m
2021-07-20T04:18:01.4942909Z [36m[submariner-operator]$ [cluster2] validate_equals .apiVersion submariner.io/v1alpha1[0m
2021-07-20T04:18:01.4945237Z [36m[submariner-operator]$ [cluster2] jq -r -M .apiVersion /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:01.4946649Z [36m[submariner-operator]$ [cluster2] validate_equals .kind Submariner[0m
2021-07-20T04:18:01.4948206Z [36m[submariner-operator]$ [cluster2] validate_equals .kind Submariner[0m
2021-07-20T04:18:01.4950153Z [36m[submariner-operator]$ [cluster2] jq -r -M .kind /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:01.4951622Z [36m[submariner-operator]$ [cluster2] validate_equals .metadata.name submariner[0m
2021-07-20T04:18:01.4952927Z [36m[submariner-operator]$ [cluster2] validate_equals .metadata.name submariner[0m
2021-07-20T04:18:01.4954941Z [36m[submariner-operator]$ [cluster2] jq -r -M .metadata.name /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:01.4958521Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sApiServer 172.18.0.8:6443[0m
2021-07-20T04:18:01.4960146Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sApiServer 172.18.0.8:6443[0m
2021-07-20T04:18:01.4962146Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.brokerK8sApiServer /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:01.5178460Z [36m[submariner-operator]$ [cluster2] validate_not_equals .spec.brokerK8sApiServerToken ***
2021-07-20T04:18:01.6800264Z [36m[submariner-operator]$ [cluster2] validate_not_equals .spec.brokerK8sApiServerToken ***
2021-07-20T04:18:01.7438576Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sCA LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3lNREEwTURjd05Gb1hEVE14TURjeE9EQTBNRGN3TkZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTDMvCm5MZDVnKzRsa2hja3FVZC9ZdHJ2VzZrN3FuM0NrWXlRSkEzL3hvVU1DY1BNUElzR0RySlNhK0tDdzVDVkY4ZE8KWWNQUStpU3ozOXBaNFc0M1dsUjhNVDZkUGx4VzhOVGJDWmFzK3RtKzBFOHE0ZlFpbEdIQWZNenUrYTJQdjRkUgpUTm9BZE9nRU9WcXZ6NExjNmhaUnhnUkFzeGExRHZieUVySXNleDNiU1RuREROb0svSkhQSnFNVkJ3bFNFZTVCCkErelFNV0IrOWJmeHloWmhBZGpMZ1pacm5VT3ZkREp2V0Mzay9TaEZSOWplMWZJRGVQaXE1YncrYWJ4c1lHRXMKaHEyZHFHa1B0bkFkN0laZTV1cnJLdVhicURNei9tOENDSWYyd2ovd05adnMxTkRWaEFrL1pndUxOSUt0Y0loLwp4cnk3YllEZW9rVUtCQ2hlS1pVQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZESjRwZkJmQ0s2Y2hCdmlINWtKQ2RBUmlEVWFNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTlRHZFgyMXZuYWNFY2J6MGlEMGxvRjlYbTcxVmxZRGJGR1h0VzF1a25nZ3R6c3BJOApOL0RGVWFsOU1RZmtmT05zUDVMaWRVSDlDRzRsc0Z4ekloWWxoOHUwMzNiNFVRbTJkN09lTUIwTGc3dnFkT01yCmxLQWZPYzhHa1ViemRQUWh2YTd2SDlkN3dzRWdtVFlzTGVqTEdMaG5sbFFTN0tIbVdLNTN2Qml6amtJamVyeWkKbE4rVEhTelIxbTVwME1QMk8xMmRxVXE5UHhUZHd4VzFkNVc4ZWZDT3VGM0wxdlVseVAyZXVWVXo4dWxGNGFlZApVOHVOVmhMK3NZZ3hVcE9KRVU0Z1hTMEpXYk9FSVpXaFRBUHpoTXRSb04rNjZCK2s0S3VxR2xkaHJZYlpOT0U1CkV2Z3J3a0Z0RUJ6WmRYNWNNMnZ4NE96Zzh4c1NEMHV5U3pRSAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==[0m
2021-07-20T04:18:01.8189484Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.brokerK8sCA /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:01.8191841Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sRemoteNamespace submariner-k8s-broker[0m
2021-07-20T04:18:01.8194620Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sRemoteNamespace submariner-k8s-broker[0m
2021-07-20T04:18:01.8196810Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.brokerK8sRemoteNamespace /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:01.8198474Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecDebug false[0m
2021-07-20T04:18:01.8199833Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecDebug false[0m
2021-07-20T04:18:01.8207525Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.ceIPSecDebug /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:01.8209192Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecIKEPort 500[0m
2021-07-20T04:18:01.8210504Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecIKEPort 500[0m
2021-07-20T04:18:01.8212200Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.ceIPSecIKEPort /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:01.8213745Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecNATTPort 4500[0m
2021-07-20T04:18:01.8215250Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecNATTPort 4500[0m
2021-07-20T04:18:01.8216767Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.ceIPSecNATTPort /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:01.8222116Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.repository localhost:5000[0m
2021-07-20T04:18:01.8225392Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.repository localhost:5000[0m
2021-07-20T04:18:01.8227356Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.repository /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:01.8228682Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.version local[0m
2021-07-20T04:18:01.8229845Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.version local[0m
2021-07-20T04:18:01.8231498Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.version /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:01.8232807Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.broker k8s[0m
2021-07-20T04:18:01.8233939Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.broker k8s[0m
2021-07-20T04:18:01.8244564Z jHreGAwCYauk0cmQHkGGCMQ-qT6i5QQ1JZDDyG65vdKuGuy74g2WNvNC-hAKOAQ43iRMscyr9gg9nCXpGgy7fs64tLOaf5NXokSAx18pu6RW0Jk8_YIaFGqyMhx9KTwymZA8uVxTFx54fxvdw312g6PvyVpRPJgUSxbdfoFQXEKJb64F_uM0uifekNEdjDEQCY3OA9CbZzphH50hv517SfWnmCSltXmWB3wpHedWKuGQGYhmJjVgd60NFN8JSqwTwFW7JgmERhNznwiDAVZOWX4IoEVGctXhCDmSRHK0DKCD8Ms4TZDF3JyCsAVVzW5oQQYRA_nd0A[0m
2021-07-20T04:18:01.8257459Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.brokerK8sApiServerToken /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:01.8848605Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sCA LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3lNREEwTURjd05Gb1hEVE14TURjeE9EQTBNRGN3TkZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTDMvCm5MZDVnKzRsa2hja3FVZC9ZdHJ2VzZrN3FuM0NrWXlRSkEzL3hvVU1DY1BNUElzR0RySlNhK0tDdzVDVkY4ZE8KWWNQUStpU3ozOXBaNFc0M1dsUjhNVDZkUGx4VzhOVGJDWmFzK3RtKzBFOHE0ZlFpbEdIQWZNenUrYTJQdjRkUgpUTm9BZE9nRU9WcXZ6NExjNmhaUnhnUkFzeGExRHZieUVySXNleDNiU1RuREROb0svSkhQSnFNVkJ3bFNFZTVCCkErelFNV0IrOWJmeHloWmhBZGpMZ1pacm5VT3ZkREp2V0Mzay9TaEZSOWplMWZJRGVQaXE1YncrYWJ4c1lHRXMKaHEyZHFHa1B0bkFkN0laZTV1cnJLdVhicURNei9tOENDSWYyd2ovd05adnMxTkRWaEFrL1pndUxOSUt0Y0loLwp4cnk3YllEZW9rVUtCQ2hlS1pVQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZESjRwZkJmQ0s2Y2hCdmlINWtKQ2RBUmlEVWFNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTlRHZFgyMXZuYWNFY2J6MGlEMGxvRjlYbTcxVmxZRGJGR1h0VzF1a25nZ3R6c3BJOApOL0RGVWFsOU1RZmtmT05zUDVMaWRVSDlDRzRsc0Z4ekloWWxoOHUwMzNiNFVRbTJkN09lTUIwTGc3dnFkT01yCmxLQWZPYzhHa1ViemRQUWh2YTd2SDlkN3dzRWdtVFlzTGVqTEdMaG5sbFFTN0tIbVdLNTN2Qml6amtJamVyeWkKbE4rVEhTelIxbTVwME1QMk8xMmRxVXE5UHhUZHd4VzFkNVc4ZWZDT3VGM0wxdlVseVAyZXVWVXo4dWxGNGFlZApVOHVOVmhMK3NZZ3hVcE9KRVU0Z1hTMEpXYk9FSVpXaFRBUHpoTXRSb04rNjZCK2s0S3VxR2xkaHJZYlpOT0U1CkV2Z3J3a0Z0RUJ6WmRYNWNNMnZ4NE96Zzh4c1NEMHV5U3pRSAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==[0m
2021-07-20T04:18:01.9961585Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sCA LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3lNREEwTURjd05Gb1hEVE14TURjeE9EQTBNRGN3TkZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTDMvCm5MZDVnKzRsa2hja3FVZC9ZdHJ2VzZrN3FuM0NrWXlRSkEzL3hvVU1DY1BNUElzR0RySlNhK0tDdzVDVkY4ZE8KWWNQUStpU3ozOXBaNFc0M1dsUjhNVDZkUGx4VzhOVGJDWmFzK3RtKzBFOHE0ZlFpbEdIQWZNenUrYTJQdjRkUgpUTm9BZE9nRU9WcXZ6NExjNmhaUnhnUkFzeGExRHZieUVySXNleDNiU1RuREROb0svSkhQSnFNVkJ3bFNFZTVCCkErelFNV0IrOWJmeHloWmhBZGpMZ1pacm5VT3ZkREp2V0Mzay9TaEZSOWplMWZJRGVQaXE1YncrYWJ4c1lHRXMKaHEyZHFHa1B0bkFkN0laZTV1cnJLdVhicURNei9tOENDSWYyd2ovd05adnMxTkRWaEFrL1pndUxOSUt0Y0loLwp4cnk3YllEZW9rVUtCQ2hlS1pVQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZESjRwZkJmQ0s2Y2hCdmlINWtKQ2RBUmlEVWFNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTlRHZFgyMXZuYWNFY2J6MGlEMGxvRjlYbTcxVmxZRGJGR1h0VzF1a25nZ3R6c3BJOApOL0RGVWFsOU1RZmtmT05zUDVMaWRVSDlDRzRsc0Z4ekloWWxoOHUwMzNiNFVRbTJkN09lTUIwTGc3dnFkT01yCmxLQWZPYzhHa1ViemRQUWh2YTd2SDlkN3dzRWdtVFlzTGVqTEdMaG5sbFFTN0tIbVdLNTN2Qml6amtJamVyeWkKbE4rVEhTelIxbTVwME1QMk8xMmRxVXE5UHhUZHd4VzFkNVc4ZWZDT3VGM0wxdlVseVAyZXVWVXo4dWxGNGFlZApVOHVOVmhMK3NZZ3hVcE9KRVU0Z1hTMEpXYk9FSVpXaFRBUHpoTXRSb04rNjZCK2s0S3VxR2xkaHJZYlpOT0U1CkV2Z3J3a0Z0RUJ6WmRYNWNNMnZ4NE96Zzh4c1NEMHV5U3pRSAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==[0m
2021-07-20T04:18:02.0510480Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.brokerK8sCA /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:02.0512607Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sRemoteNamespace submariner-k8s-broker[0m
2021-07-20T04:18:02.0515545Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sRemoteNamespace submariner-k8s-broker[0m
2021-07-20T04:18:02.0517806Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.brokerK8sRemoteNamespace /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:02.0521039Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecDebug false[0m
2021-07-20T04:18:02.0522599Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecDebug false[0m
2021-07-20T04:18:03.3068270Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.ceIPSecDebug /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:03.3069860Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecIKEPort 500[0m
2021-07-20T04:18:03.3071124Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecIKEPort 500[0m
2021-07-20T04:18:03.3072774Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.ceIPSecIKEPort /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:03.3074588Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecNATTPort 4500[0m
2021-07-20T04:18:03.3076333Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecNATTPort 4500[0m
2021-07-20T04:18:03.3077861Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.ceIPSecNATTPort /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:03.3079244Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.repository localhost:5000[0m
2021-07-20T04:18:03.3080680Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.repository localhost:5000[0m
2021-07-20T04:18:03.3082250Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.repository /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:03.3084143Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.version local[0m
2021-07-20T04:18:03.3085449Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.version local[0m
2021-07-20T04:18:03.3087048Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.version /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:03.3088310Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.broker k8s[0m
2021-07-20T04:18:03.3089402Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.broker k8s[0m
2021-07-20T04:18:03.3090669Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.broker /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:03.3092078Z [36m[submariner-operator]$ [cluster2] jq -r .spec.clusterID /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:03.3093011Z [cluster2] Generated cluster id: cluster2
2021-07-20T04:18:03.3094024Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.colorCodes blue[0m
2021-07-20T04:18:03.3095385Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.colorCodes blue[0m
2021-07-20T04:18:03.3096888Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.colorCodes /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:03.3098566Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.debug false[0m
2021-07-20T04:18:03.3099649Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.debug false[0m
2021-07-20T04:18:03.3100852Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.debug /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:03.3102216Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.namespace submariner-operator[0m
2021-07-20T04:18:03.3103764Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.namespace submariner-operator[0m
2021-07-20T04:18:03.3105598Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.namespace /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:03.3107340Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.natEnabled false[0m
2021-07-20T04:18:03.3108567Z [36m[submariner-operator]$ [cluster2] validate_equals .spec.natEnabled false[0m
2021-07-20T04:18:03.3110418Z [36m[submariner-operator]$ [cluster2] jq -r -M .spec.natEnabled /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:03.3111708Z [36m[submariner-operator]$ [cluster2] verify_subm_cr_status_with_retries[0m
2021-07-20T04:18:03.3112774Z [36m[submariner-operator]$ [cluster2] verify_subm_cr_status_with_retries[0m
2021-07-20T04:18:03.3113858Z [36m[submariner-operator]$ [cluster2] with_retries 5 verify_subm_cr_status_[0m
2021-07-20T04:18:03.3114933Z [36m[submariner-operator]$ [cluster2] with_retries 5 verify_subm_cr_status_[0m
2021-07-20T04:18:03.3116092Z [36m[submariner-operator]$ [cluster2] iteration=1[0m
2021-07-20T04:18:03.3117523Z [36m[submariner-operator]$ [cluster2] verify_subm_cr_status_[0m
2021-07-20T04:18:03.3118643Z [36m[submariner-operator]$ [cluster2] verify_subm_cr_status_[0m
2021-07-20T04:18:03.3119591Z [36m[submariner-operator]$ [cluster2] verify_subm_cr_status[0m
2021-07-20T04:18:03.3121097Z [36m[submariner-operator]$ [cluster2] verify_subm_cr_status[0m
2021-07-20T04:18:03.3122336Z [36m[submariner-operator]$ [cluster2] json_file=/tmp/submariner.cluster2.json[0m
2021-07-20T04:18:03.3123295Z [cluster2] [36m[0m
2021-07-20T04:18:03.3124426Z [36m[submariner-operator]$ [cluster2] wait 10008[0m
2021-07-20T04:18:03.3125080Z [cluster2] [36m[0m
2021-07-20T04:18:03.3126473Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get submariner submariner --namespace=submariner-operator -o json[0m
2021-07-20T04:18:03.3127966Z [36m[submariner-operator]$ [cluster2] validate_equals .status.serviceCIDR 100.2.0.0/16[0m
2021-07-20T04:18:03.3129227Z [36m[submariner-operator]$ [cluster2] validate_equals .status.serviceCIDR 100.2.0.0/16[0m
2021-07-20T04:18:03.3130628Z [36m[submariner-operator]$ [cluster2] jq -r -M .status.serviceCIDR /tmp/submariner.cluster2.json[0m
2021-07-20T04:18:03.3132000Z [36m[submariner-operator]$ [cluster2] validate_equals .status.clusterCIDR 10.2.0.0/16[0m
2021-07-20T04:18:03.3133235Z [36m[submariner-operator]$ [cluster2] validate_equals .status.clusterCIDR 10.2.0.0/16[0m
2021-07-20T04:18:03.8159671Z [36m[submariner-operator]$ [cluster2] jq -r -M .status.clusterCIDR /tmp/submariner[36m[submariner-operator]$ [cluster1] jq -r -M .spec.broker /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:03.8162096Z [36m[submariner-operator]$ [cluster1] jq -r .spec.clusterID /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:03.8163045Z [cluster1] Generated cluster id: cluster1
2021-07-20T04:18:03.8164077Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.colorCodes blue[0m
2021-07-20T04:18:03.8165304Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.colorCodes blue[0m
2021-07-20T04:18:03.8166634Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.colorCodes /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:03.8168006Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.debug false[0m
2021-07-20T04:18:03.8169326Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.debug false[0m
2021-07-20T04:18:03.8170737Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.debug /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:03.8172158Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.namespace submariner-operator[0m
2021-07-20T04:18:03.8173548Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.namespace submariner-operator[0m
2021-07-20T04:18:03.8174984Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.namespace /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:03.8176515Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.natEnabled false[0m
2021-07-20T04:18:03.8178729Z [36m[submariner-operator]$ [cluster1] validate_equals .spec.natEnabled false[0m
2021-07-20T04:18:03.8180497Z [36m[submariner-operator]$ [cluster1] jq -r -M .spec.natEnabled /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:03.8181989Z [36m[submariner-operator]$ [cluster1] verify_subm_cr_status_with_retries[0m
2021-07-20T04:18:03.8183110Z [36m[submariner-operator]$ [cluster1] verify_subm_cr_status_with_retries[0m
2021-07-20T04:18:03.8184235Z [36m[submariner-operator]$ [cluster1] with_retries 5 verify_subm_cr_status_[0m
2021-07-20T04:18:03.8185363Z [36m[submariner-operator]$ [cluster1] with_retries 5 verify_subm_cr_status_[0m
2021-07-20T04:18:03.8186434Z [36m[submariner-operator]$ [cluster1] iteration=1[0m
2021-07-20T04:18:03.8187383Z [36m[submariner-operator]$ [cluster1] wait 9979[0m
2021-07-20T04:18:03.8188346Z [36m[submariner-operator]$ [cluster1] verify_subm_cr_status_[0m
2021-07-20T04:18:03.8189594Z [36m[submariner-operator]$ [cluster1] verify_subm_cr_status_[0m
2021-07-20T04:18:03.8190694Z [36m[submariner-operator]$ [cluster1] verify_subm_cr_status[0m
2021-07-20T04:18:03.8191687Z [36m[submariner-operator]$ [cluster1] verify_subm_cr_status[0m
2021-07-20T04:18:03.8192882Z [36m[submariner-operator]$ [cluster1] json_file=/tmp/submariner.cluster1.json[0m
2021-07-20T04:18:03.8193797Z [cluster1] [36m[0m
2021-07-20T04:18:03.8194361Z [cluster1] [36m[0m
2021-07-20T04:18:03.8195749Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get submariner submariner --namespace=submariner-operator -o json[0m
2021-07-20T04:18:03.8197341Z [36m[submariner-operator]$ [cluster1] validate_equals .status.serviceCIDR 100.1.0.0/16[0m
2021-07-20T04:18:03.8198633Z [36m[submariner-operator]$ [cluster1] validate_equals .status.serviceCIDR 100.1.0.0/16[0m
2021-07-20T04:18:03.8202820Z [36m[submariner-operator]$ [cluster1] jq -r -M .status.serviceCIDR /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:03.8204706Z [36m[submariner-operator]$ [cluster1] validate_equals .status.clusterCIDR 10.1.0.0/16[0m
2021-07-20T04:18:03.8206012Z [36m[submariner-operator]$ [cluster1] validate_equals .status.clusterCIDR 10.1.0.0/16[0m
2021-07-20T04:18:03.8207600Z [36m[submariner-operator]$ [cluster1] jq -r -M .status.clusterCIDR /tmp/submariner.cluster1.json[0m
2021-07-20T04:18:03.8208834Z [36m[submariner-operator]$ [cluster1] return 0[0m
2021-07-20T04:18:03.8209745Z [36m[submariner-operator]$ [cluster1] return 0[0m
2021-07-20T04:18:03.8211319Z [36m[submariner-operator]$ [cluster1] verify_subm_gateway_deployment[0m
2021-07-20T04:18:03.8212403Z [36m[submariner-operator]$ [cluster1] verify_subm_gateway_deployment[0m
2021-07-20T04:18:03.8213697Z [36m[submariner-operator]$ [cluster1] verify_daemonset submariner-gateway submariner-gateway[0m
2021-07-20T04:18:03.8215150Z [36m[submariner-operator]$ [cluster1] verify_daemonset submariner-gateway submariner-gateway[0m
2021-07-20T04:18:03.8216488Z [36m[submariner-operator]$ [cluster1] daemonset_app=submariner-gateway[0m
2021-07-20T04:18:03.8222299Z [36m[submariner-operator]$ [cluster1] daemonset_name=submariner-gateway[0m
2021-07-20T04:18:03.8223480Z [36m[submariner-operator]$ [cluster1] with_retries 60 daemonset_created[0m
2021-07-20T04:18:03.8224587Z [36m[submariner-operator]$ [cluster1] with_retries 60 daemonset_created[0m
2021-07-20T04:18:03.8225572Z [36m[submariner-operator]$ [cluster1] iteration=1[0m
2021-07-20T04:18:03.8226474Z [36m[submariner-operator]$ [cluster1] wait 10055[0m
2021-07-20T04:18:03.8227583Z [36m[submariner-operator]$ [cluster1] daemonset_created[0m
2021-07-20T04:18:03.8228544Z [36m[submariner-operator]$ [cluster1] daemonset_created[0m
2021-07-20T04:18:03.8229254Z [cluster1] [36m[0m
2021-07-20T04:18:03.8230111Z [36m[submariner-operator]$ [cluster1] return 0[0m
2021-07-20T04:18:03.8231136Z [36m[submariner-operator]$ [cluster1] with_retries 120 daemonset_deployed[0m
2021-07-20T04:18:03.8232219Z [36m[submariner-operator]$ [cluster1] with_retries 120 daemonset_deployed[0m
2021-07-20T04:18:03.8233202Z [36m[submariner-operator]$ [cluster1] iteration=1[0m
2021-07-20T04:18:05.5609814Z [36m.cluster2.json[0m
2021-07-20T04:18:05.5610717Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:18:05.5611619Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:18:05.5612723Z [36m[submariner-operator]$ [cluster2] verify_subm_gateway_deployment[0m
2021-07-20T04:18:05.5613817Z [36m[submariner-operator]$ [cluster2] verify_subm_gateway_deployment[0m
2021-07-20T04:18:05.5615141Z [36m[submariner-operator]$ [cluster2] verify_daemonset submariner-gateway submariner-gateway[0m
2021-07-20T04:18:05.5616613Z [36m[submariner-operator]$ [cluster2] verify_daemonset submariner-gateway submariner-gateway[0m
2021-07-20T04:18:05.5618441Z [36m[submariner-operator]$ [cluster2] daemonset_app=submariner-gateway[0m
2021-07-20T04:18:05.5619622Z [36m[submariner-operator]$ [cluster2] daemonset_name=submariner-gateway[0m
2021-07-20T04:18:05.5620949Z [36m[submariner-operator]$ [cluster2] with_retries 60 daemonset_created[0m
2021-07-20T04:18:05.5622075Z [36m[submariner-operator]$ [cluster2] with_retries 60 daemonset_created[0m
2021-07-20T04:18:05.5623051Z [36m[submariner-operator]$ [cluster2] iteration=1[0m
2021-07-20T04:18:05.5623909Z [36m[submariner-operator]$ [cluster2] wait 10075[0m
2021-07-20T04:18:05.5624813Z [36m[submariner-operator]$ [cluster2] daemonset_created[0m
2021-07-20T04:18:05.5625761Z [36m[submariner-operator]$ [cluster2] daemonset_created[0m
2021-07-20T04:18:05.5626496Z [cluster2] [36m[0m
2021-07-20T04:18:05.5627185Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:18:05.5628177Z [36m[submariner-operator]$ [cluster2] with_retries 120 daemonset_deployed[0m
2021-07-20T04:18:05.5629268Z [36m[submariner-operator]$ [cluster2] with_retries 120 daemonset_deployed[0m
2021-07-20T04:18:05.5630233Z [36m[submariner-operator]$ [cluster2] iteration=1[0m
2021-07-20T04:18:05.5631109Z [36m[submariner-operator]$ [cluster2] wait 10112[0m
2021-07-20T04:18:05.5632006Z [36m[submariner-operator]$ [cluster2] daemonset_deployed[0m
2021-07-20T04:18:05.5632982Z [36m[submariner-operator]$ [cluster2] daemonset_deployed[0m
2021-07-20T04:18:05.5634431Z [36m[submariner-operator]$ [cluster2] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}[0m
2021-07-20T04:18:05.5636329Z [36m[submariner-operator]$ [cluster2] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}[0m
2021-07-20T04:18:05.5639059Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}[0m
2021-07-20T04:18:05.5641724Z [36m[submariner-operator]$ [cluster2] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}[0m
2021-07-20T04:18:05.5643746Z [36m[submariner-operator]$ [cluster2] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}[0m
2021-07-20T04:18:05.5645971Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}[0m
2021-07-20T04:18:05.5647423Z [36m[submariner-operator]$ [cluster2] return 0[0m
2021-07-20T04:18:05.5648348Z [36m[submariner-operator]$ [cluster2] verify_subm_gateway_pod[0m
2021-07-20T04:18:05.5649313Z [36m[submariner-operator]$ [cluster2] verify_subm_gateway_pod[0m
2021-07-20T04:18:05.5650778Z [36m[submariner-operator]$ [cluster2] kubectl wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator[0m
2021-07-20T04:18:05.5652609Z [36m[submariner-operator]$ [cluster2] kubectl wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator[0m
2021-07-20T04:18:05.5654569Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator[0m
2021-07-20T04:18:05.5656071Z [cluster2] pod/submariner-gateway-fgb8w condition met
2021-07-20T04:18:05.5657787Z [36m[submariner-operator]$ [cluster2] subm_gateway_pod_name=$(kubectl get pods --namespace=$subm_ns -l app=$gateway_deployment_name -o=jsonpath='{.items..metadata.name}')[0m
2021-07-20T04:18:05.5659886Z [36m[submariner-operator]$ [cluster2] kubectl get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}[0m
2021-07-20T04:18:05.5661780Z [36m[submariner-operator]$ [cluster2] kubectl get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}[0m
2021-07-20T04:18:05.5663743Z [36m[submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}[0m
2021-07-20T04:18:05.5665567Z [36m[submariner-operator]$ [cluster2] json_file=/tmp/submariner-gateway-fgb8w.cluster2.json[0m
2021-07-20T04:18:05.5667861Z [36m[submariner-operator]$ [cluster2] kubectl get pod submariner-gateway-fgb8w --namespace=submariner-operator -o json[0m
2021-07-20T04:18:05.5670072Z [36m[submariner-operator]$ [cluster2] kubectl get pod submariner-gateway-fgb8w --namespace=submariner-operator -o json[0m
2021-07-20T04:18:05.8741350Z [36m[submariner-operator]$ [cluster2] command kubectl --contex[submariner-operator]$ [cluster1] daemonset_deployed[0m
2021-07-20T04:18:05.8743125Z [36m[submariner-operator]$ [cluster1] wait 10097[0m
2021-07-20T04:18:05.8744177Z [36m[submariner-operator]$ [cluster1] daemonset_deployed[0m
2021-07-20T04:18:05.8745700Z [36m[submariner-operator]$ [cluster1] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}[0m
2021-07-20T04:18:05.8747790Z [36m[submariner-operator]$ [cluster1] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}[0m
2021-07-20T04:18:05.8750378Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}[0m
2021-07-20T04:18:05.8752598Z [36m[submariner-operator]$ [cluster1] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}[0m
2021-07-20T04:18:05.8754212Z [36m[submariner-operator]$ [cluster1] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}[0m
2021-07-20T04:18:05.8756177Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}[0m
2021-07-20T04:18:05.8757783Z [36m[submariner-operator]$ [cluster1] return 0[0m
2021-07-20T04:18:05.8758765Z [36m[submariner-operator]$ [cluster1] verify_subm_gateway_pod[0m
2021-07-20T04:18:05.8759960Z [36m[submariner-operator]$ [cluster1] verify_subm_gateway_pod[0m
2021-07-20T04:18:05.8762176Z [36m[submariner-operator]$ [cluster1] kubectl wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator[0m
2021-07-20T04:18:05.8764573Z [36m[submariner-operator]$ [cluster1] kubectl wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator[0m
2021-07-20T04:18:05.8766611Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator[0m
2021-07-20T04:18:05.8768158Z [cluster1] pod/submariner-gateway-bxqsk condition met
2021-07-20T04:18:05.8769719Z [36m[submariner-operator]$ [cluster1] subm_gateway_pod_name=$(kubectl get pods --namespace=$subm_ns -l app=$gateway_deployment_name -o=jsonpath='{.items..metadata.name}')[0m
2021-07-20T04:18:05.8772555Z [36m[submariner-operator]$ [cluster1] kubectl get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}[0m
2021-07-20T04:18:05.8775028Z [36m[submariner-operator]$ [cluster1] kubectl get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}[0m
2021-07-20T04:18:05.8777346Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}[0m
2021-07-20T04:18:05.8779413Z [36m[submariner-operator]$ [cluster1] json_file=/tmp/submariner-gateway-bxqsk.cluster1.json[0m
2021-07-20T04:18:05.8781254Z [36m[submariner-operator]$ [cluster1] kubectl get pod submariner-gateway-bxqsk --namespace=submariner-operator -o json[0m
2021-07-20T04:18:05.8783040Z [36m[submariner-operator]$ [cluster1] tee /tmp/submariner-gateway-bxqsk.cluster1.json[0m
2021-07-20T04:18:05.8784859Z [36m[submariner-operator]$ [cluster1] kubectl get pod submariner-gateway-bxqsk --namespace=submariner-operator -o json[0m
2021-07-20T04:18:05.8787680Z [36m[submariner-operator]$ [cluster1] command kubectl --context=cluster1 get pod submariner-gateway-bxqsk --namespace=submariner-operator -o json[0m
2021-07-20T04:18:05.8789084Z [cluster1] {
2021-07-20T04:18:05.8789689Z [cluster1]     "apiVersion": "v1",
2021-07-20T04:18:05.8790173Z [cluster1]     "kind": "Pod",
2021-07-20T04:18:05.8790625Z [cluster1]     "metadata": {
2021-07-20T04:18:05.8791426Z [cluster1]         "creationTimestamp": "2021-07-20T04:17:05Z",
2021-07-20T04:18:05.8792347Z [cluster1]         "generateName": "submariner-gateway-",
2021-07-20T04:18:05.8792951Z [cluster1]         "labels": {
2021-07-20T04:18:05.8793875Z [cluster1]             "app": "submariner-gateway",
2021-07-20T04:18:05.8794855Z [cluster1]             "controller-revision-hash": "76bcf8cd46",
2021-07-20T04:18:05.8796157Z [cluster1]             "gateway.submariner.io/node": "cluster1-worker",
2021-07-20T04:18:05.8797224Z [cluster1]             "gateway.submariner.io/status": "active",
2021-07-20T04:18:05.8798202Z [cluster1]             "pod-template-generation": "1"
2021-07-20T04:18:05.8798793Z [cluster1]         },
2021-07-20T04:18:05.8799259Z [cluster1]         "managedFields": [
2021-07-20T04:18:05.8799748Z [cluster1]             {
2021-07-20T04:18:05.8800209Z [cluster1]                 "apiVersion": "v1",
2021-07-20T04:18:05.8800787Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-20T04:18:05.8801318Z [cluster1]                 "fieldsV1": {
2021-07-20T04:18:05.8801815Z [cluster1]                     "f:metadata": {
2021-07-20T04:18:05.8802353Z [cluster1]                         "f:generateName": {},
2021-07-20T04:18:05.8802892Z [cluster1]                         "f:labels": {
2021-07-20T04:18:05.8803499Z [cluster1]                             ".": {},
2021-07-20T04:18:05.8803942Z [cluster1]                             "f:app": {},
2021-07-20T04:18:05.8804852Z [cluster1]                             "f:controller-revision-hash": {},
2021-07-20T04:18:05.8805870Z [cluster1]                             "f:pod-template-generation": {}
2021-07-20T04:18:05.8806487Z [cluster1]                         },
2021-07-20T04:18:05.8807186Z [cluster1]                         "f:ownerReferences": {
2021-07-20T04:18:05.8807751Z [cluster1]                             ".": {},
2021-07-20T04:18:05.8808753Z [cluster1]                             "k:{\"uid\":\"f1145985-519e-4c64-9688-84c5152ebbe5\"}": {
2021-07-20T04:18:05.8809342Z [cluster1]                                 ".": {},
2021-07-20T04:18:05.8810826Z [cluster1]                                 "f:apiVersion": {},
2021-07-20T04:18:05.8811513Z [cluster1]                                 "f:blockOwnerDeletion": {},
2021-07-20T04:18:05.8812170Z [cluster1]                                 "f:controller": {},
2021-07-20T04:18:05.8812700Z [cluster1]                                 "f:kind": {},
2021-07-20T04:18:05.8813190Z [cluster1]                                 "f:name": {},
2021-07-20T04:18:05.8813832Z [cluster1]                                 "f:uid": {}
2021-07-20T04:18:05.8814279Z [cluster1]                             }
2021-07-20T04:18:05.8814687Z [cluster1]                         }
2021-07-20T04:18:05.8815111Z [cluster1]                     },
2021-07-20T04:18:05.8815742Z [cluster1]                     "f:spec": {
2021-07-20T04:18:05.8845306Z [cluster1]                         "f:affinity": {
2021-07-20T04:18:05.8845920Z [cluster1]                             ".": {},
2021-07-20T04:18:05.8846400Z [cluster1]                             "f:nodeAffinity": {
2021-07-20T04:18:05.8846898Z [cluster1]                                 ".": {},
2021-07-20T04:18:05.8847916Z [cluster1]                                 "f:requiredDuringSchedulingIgnoredDuringExecution": {
2021-07-20T04:18:05.8849098Z [cluster1]                                     ".": {},
2021-07-20T04:18:05.8849645Z [cluster1]                                     "f:nodeSelectorTerms": {}
2021-07-20T04:18:05.8850378Z [cluster1]                                 }
2021-07-20T04:18:05.8851068Z [cluster1]                             },
2021-07-20T04:18:05.8851614Z [cluster1]                             "f:podAntiAffinity": {
2021-07-20T04:18:05.8852176Z [cluster1]                                 ".": {},
2021-07-20T04:18:05.8853613Z [cluster1]                                 "f:requiredDuringSchedulingIgnoredDuringExecution": {}
2021-07-20T04:18:05.8854868Z [cluster1]                             }
2021-07-20T04:18:05.8855265Z [cluster1]                         },
2021-07-20T04:18:05.8855926Z [cluster1]                         "f:containers": {
2021-07-20T04:18:05.8856942Z [cluster1]                             "k:{\"name\":\"submariner-gateway\"}": {
2021-07-20T04:18:05.8857700Z [cluster1]                                 ".": {},
2021-07-20T04:18:05.8858196Z [cluster1]                                 "f:command": {},
2021-07-20T04:18:05.8858680Z [cluster1]                                 "f:env": {
2021-07-20T04:18:05.8859148Z [cluster1]                                     ".": {},
2021-07-20T04:18:05.8859701Z [cluster1]                                     "k:{\"name\":\"BROKER_K8S_APISERVER\"}": {
2021-07-20T04:18:05.8860264Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8860738Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8861244Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8861706Z [cluster1]                                     },
2021-07-20T04:18:05.8862309Z [cluster1]                                     "k:{\"name\":\"BROKER_K8S_APISERVERTOKEN\"}": {
2021-07-20T04:18:05.8862918Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8863430Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8864083Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8864541Z [cluster1]                                     },
2021-07-20T04:18:05.8865062Z [cluster1]                                     "k:{\"name\":\"BROKER_K8S_CA\"}": {
2021-07-20T04:18:05.8865574Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8866054Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8866693Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8867149Z [cluster1]                                     },
2021-07-20T04:18:05.8867761Z [cluster1]                                     "k:{\"name\":\"BROKER_K8S_REMOTENAMESPACE\"}": {
2021-07-20T04:18:05.8868361Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8868841Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8869327Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8869803Z [cluster1]                                     },
2021-07-20T04:18:05.8870315Z [cluster1]                                     "k:{\"name\":\"CE_IPSEC_DEBUG\"}": {
2021-07-20T04:18:05.8870857Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8871438Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8871928Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8872401Z [cluster1]                                     },
2021-07-20T04:18:05.8872959Z [cluster1]                                     "k:{\"name\":\"CE_IPSEC_FORCEENCAPS\"}": {
2021-07-20T04:18:05.8873536Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8874002Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8874501Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8874958Z [cluster1]                                     },
2021-07-20T04:18:05.8875510Z [cluster1]                                     "k:{\"name\":\"CE_IPSEC_IKEPORT\"}": {
2021-07-20T04:18:05.8876059Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8876523Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8877019Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8877474Z [cluster1]                                     },
2021-07-20T04:18:05.8878124Z [cluster1]                                     "k:{\"name\":\"CE_IPSEC_NATTPORT\"}": {
2021-07-20T04:18:05.8878677Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8879157Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8879658Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8880115Z [cluster1]                                     },
2021-07-20T04:18:05.8880719Z [cluster1]                                     "k:{\"name\":\"CE_IPSEC_PREFERREDSERVER\"}": {
2021-07-20T04:18:05.8881318Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8881800Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8882285Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8882754Z [cluster1]                                     },
2021-07-20T04:18:05.8883269Z [cluster1]                                     "k:{\"name\":\"CE_IPSEC_PSK\"}": {
2021-07-20T04:18:05.8883796Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8884280Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8884960Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8885446Z [cluster1]                                     },
2021-07-20T04:18:05.8885956Z [cluster1]                                     "k:{\"name\":\"NODE_NAME\"}": {
2021-07-20T04:18:05.8886598Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8887384Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8887888Z [cluster1]                                         "f:valueFrom": {
2021-07-20T04:18:05.8888382Z [cluster1]                                             ".": {},
2021-07-20T04:18:05.8888855Z [cluster1]                                             "f:fieldRef": {
2021-07-20T04:18:05.8889357Z [cluster1]                                                 ".": {},
2021-07-20T04:18:05.8889849Z [cluster1]                                                 "f:apiVersion": {},
2021-07-20T04:18:05.8890407Z [cluster1]                                                 "f:fieldPath": {}
2021-07-20T04:18:05.8890889Z [cluster1]                                             }
2021-07-20T04:18:05.8891329Z [cluster1]                                         }
2021-07-20T04:18:05.8891744Z [cluster1]                                     },
2021-07-20T04:18:05.8892242Z [cluster1]                                     "k:{\"name\":\"POD_NAME\"}": {
2021-07-20T04:18:05.8892737Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8893185Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8893698Z [cluster1]                                         "f:valueFrom": {
2021-07-20T04:18:05.8894177Z [cluster1]                                             ".": {},
2021-07-20T04:18:05.8894860Z [cluster1]                                             "f:fieldRef": {
2021-07-20T04:18:05.8895542Z [cluster1]                                                 ".": {},
2021-07-20T04:18:05.8896091Z [cluster1]                                                 "f:apiVersion": {},
2021-07-20T04:18:05.8896832Z [cluster1]                                                 "f:fieldPath": {}
2021-07-20T04:18:05.8897640Z [cluster1]                                             }
2021-07-20T04:18:05.8898084Z [cluster1]                                         }
2021-07-20T04:18:05.8898509Z [cluster1]                                     },
2021-07-20T04:18:05.8899043Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_BROKER\"}": {
2021-07-20T04:18:05.8899576Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8900034Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8900501Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8901051Z [cluster1]                                     },
2021-07-20T04:18:05.8901641Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_CABLEDRIVER\"}": {
2021-07-20T04:18:05.8902196Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8902659Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8903126Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8903584Z [cluster1]                                     },
2021-07-20T04:18:05.8904142Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_CLUSTERCIDR\"}": {
2021-07-20T04:18:05.8904714Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8905175Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8905642Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8906117Z [cluster1]                                     },
2021-07-20T04:18:05.8906657Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_CLUSTERID\"}": {
2021-07-20T04:18:05.8907211Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8907660Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8908135Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8908572Z [cluster1]                                     },
2021-07-20T04:18:05.8909131Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_COLORCODES\"}": {
2021-07-20T04:18:05.8909799Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8910245Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8910731Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8911778Z [cluster1]                                     },
2021-07-20T04:18:05.8912370Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_DEBUG\"}": {
2021-07-20T04:18:05.8912936Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8913431Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8913928Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8914420Z [cluster1]                                     },
2021-07-20T04:18:05.8915202Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_GLOBALCIDR\"}": {
2021-07-20T04:18:05.8915758Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8916227Z [cluster1]                                         "f:name": {}
2021-07-20T04:18:05.8916736Z [cluster1]                                     },
2021-07-20T04:18:05.8917363Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_HEALTHCHECKENABLED\"}": {
2021-07-20T04:18:05.8919965Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8922902Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8923406Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8923849Z [cluster1]                                     },
2021-07-20T04:18:05.8924493Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_HEALTHCHECKINTERVAL\"}": {
2021-07-20T04:18:05.8925354Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8925801Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8926258Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8926692Z [cluster1]                                     },
2021-07-20T04:18:05.8928088Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_HEALTHCHECKMAXPACKETLOSSCOUNT\"}": {
2021-07-20T04:18:05.8931614Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8932243Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8932724Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8933167Z [cluster1]                                     },
2021-07-20T04:18:05.8933694Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_NAMESPACE\"}": {
2021-07-20T04:18:05.8934241Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8934677Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8935152Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8935599Z [cluster1]                                     },
2021-07-20T04:18:05.8936172Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_NATENABLED\"}": {
2021-07-20T04:18:05.8936722Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8937558Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8938062Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8938493Z [cluster1]                                     },
2021-07-20T04:18:05.8974023Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_SERVICECIDR\"}": {
2021-07-20T04:18:05.8974595Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8975150Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8975629Z [cluster1]                                         "f:value": {}
2021-07-20T04:18:05.8976293Z [cluster1]                                     }
2021-07-20T04:18:05.8976708Z [cluster1]                                 },
2021-07-20T04:18:05.8977290Z [cluster1]                                 "f:image": {},
2021-07-20T04:18:05.8977847Z [cluster1]                                 "f:imagePullPolicy": {},
2021-07-20T04:18:05.8978370Z [cluster1]                                 "f:name": {},
2021-07-20T04:18:05.8978917Z [cluster1]                                 "f:ports": {
2021-07-20T04:18:05.8979374Z [cluster1]                                     ".": {},
2021-07-20T04:18:05.8979932Z [cluster1]                                     "k:{\"containerPort\":4490,\"protocol\":\"UDP\"}": {
2021-07-20T04:18:05.8980506Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8981001Z [cluster1]                                         "f:containerPort": {},
2021-07-20T04:18:05.8981546Z [cluster1]                                         "f:hostPort": {},
2021-07-20T04:18:05.8982029Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8982519Z [cluster1]                                         "f:protocol": {}
2021-07-20T04:18:05.8982963Z [cluster1]                                     },
2021-07-20T04:18:05.8983528Z [cluster1]                                     "k:{\"containerPort\":4500,\"protocol\":\"UDP\"}": {
2021-07-20T04:18:05.8984094Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8984846Z [cluster1]                                         "f:containerPort": {},
2021-07-20T04:18:05.8985561Z [cluster1]                                         "f:hostPort": {},
2021-07-20T04:18:05.8986030Z [cluster1]                                         "f:name": {},
2021-07-20T04:18:05.8986505Z [cluster1]                                         "f:protocol": {}
2021-07-20T04:18:05.8986949Z [cluster1]                                     }
2021-07-20T04:18:05.8987477Z [cluster1]                                 },
2021-07-20T04:18:05.8988119Z [cluster1]                                 "f:resources": {},
2021-07-20T04:18:05.8988642Z [cluster1]                                 "f:securityContext": {
2021-07-20T04:18:05.8989139Z [cluster1]                                     ".": {},
2021-07-20T04:18:05.8989720Z [cluster1]                                     "f:allowPrivilegeEscalation": {},
2021-07-20T04:18:05.8990375Z [cluster1]                                     "f:capabilities": {
2021-07-20T04:18:05.8990943Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8991393Z [cluster1]                                         "f:add": {},
2021-07-20T04:18:05.8991822Z [cluster1]                                         "f:drop": {}
2021-07-20T04:18:05.8992243Z [cluster1]                                     },
2021-07-20T04:18:05.8992708Z [cluster1]                                     "f:privileged": {},
2021-07-20T04:18:05.8993419Z [cluster1]                                     "f:readOnlyRootFilesystem": {},
2021-07-20T04:18:05.8994068Z [cluster1]                                     "f:runAsNonRoot": {}
2021-07-20T04:18:05.8994520Z [cluster1]                                 },
2021-07-20T04:18:05.8995086Z [cluster1]                                 "f:terminationMessagePath": {},
2021-07-20T04:18:05.8996040Z [cluster1]                                 "f:terminationMessagePolicy": {},
2021-07-20T04:18:05.8996966Z [cluster1]                                 "f:volumeMounts": {
2021-07-20T04:18:05.8997427Z [cluster1]                                     ".": {},
2021-07-20T04:18:05.8997937Z [cluster1]                                     "k:{\"mountPath\":\"/etc/ipsec.d\"}": {
2021-07-20T04:18:05.8998523Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.8998970Z [cluster1]                                         "f:mountPath": {},
2021-07-20T04:18:05.8999448Z [cluster1]                                         "f:name": {}
2021-07-20T04:18:05.9000036Z [cluster1]                                     },
2021-07-20T04:18:05.9000984Z [cluster1]                                     "k:{\"mountPath\":\"/var/lib/ipsec/nss\"}": {
2021-07-20T04:18:05.9001540Z [cluster1]                                         ".": {},
2021-07-20T04:18:05.9002051Z [cluster1]                                         "f:mountPath": {},
2021-07-20T04:18:05.9002580Z [cluster1]                                         "f:name": {}
2021-07-20T04:18:05.9003033Z [cluster1]                                     }
2021-07-20T04:18:05.9003797Z [cluster1]                                 }
2021-07-20T04:18:05.9004180Z [cluster1]                             }
2021-07-20T04:18:05.9004571Z [cluster1]                         },
2021-07-20T04:18:05.9004995Z [cluster1]                         "f:dnsPolicy": {},
2021-07-20T04:18:05.9005557Z [cluster1]                         "f:enableServiceLinks": {},
2021-07-20T04:18:05.9006119Z [cluster1]                         "f:hostNetwork": {},
2021-07-20T04:18:05.9006762Z [cluster1]                         "f:nodeSelector": {
2021-07-20T04:18:05.9007424Z [cluster1]                             ".": {},
2021-07-20T04:18:05.9008106Z [cluster1]                             "f:submariner.io/gateway": {}
2021-07-20T04:18:05.9008592Z [cluster1]                         },
2021-07-20T04:18:05.9009042Z [cluster1]                         "f:restartPolicy": {},
2021-07-20T04:18:05.9009583Z [cluster1]                         "f:schedulerName": {},
2021-07-20T04:18:05.9010127Z [cluster1]                         "f:securityContext": {},
2021-07-20T04:18:05.9010692Z [cluster1]                         "f:serviceAccount": {},
2021-07-20T04:18:05.9011272Z [cluster1]                         "f:serviceAccountName": {},
2021-07-20T04:18:05.9013259Z [cluster1]                         "f:terminationGracePeriodSeconds": {},
2021-07-20T04:18:05.9014238Z [cluster1]                         "f:tolerations": {},
2021-07-20T04:18:05.9014781Z [cluster1]                         "f:volumes": {
2021-07-20T04:18:05.9015271Z [cluster1]                             ".": {},
2021-07-20T04:18:05.9016317Z [cluster1]                             "k:{\"name\":\"ipsecd\"}": {
2021-07-20T04:18:05.9016785Z [cluster1]                                 ".": {},
2021-07-20T04:18:05.9017341Z [cluster1]                                 "f:emptyDir": {},
2021-07-20T04:18:05.9017819Z [cluster1]                                 "f:name": {}
2021-07-20T04:18:05.9018235Z [cluster1]                             },
2021-07-20T04:18:05.9018857Z [cluster1]                             "k:{\"name\":\"ipsecnss\"}": {
2021-07-20T04:18:05.9019367Z [cluster1]                                 ".": {},
2021-07-20T04:18:05.9019810Z [cluster1]                                 "f:emptyDir": {},
2021-07-20T04:18:05.9020288Z [cluster1]                                 "f:name": {}
2021-07-20T04:18:05.9020698Z [cluster1]                             }
2021-07-20T04:18:05.9021098Z [cluster1]                         }
2021-07-20T04:18:05.9021476Z [cluster1]                     }
2021-07-20T04:18:05.9021866Z [cluster1]                 },
2021-07-20T04:18:05.9022839Z [cluster1]                 "manager": "kube-controller-manager",
2021-07-20T04:18:05.9023509Z [cluster1]                 "operation": "Update",
2021-07-20T04:18:05.9024203Z [