benoit74

import subprocess
import re
import json


def run_pyright():
    # Run Pyright and capture the output
    result = subprocess.run(["pyright", "--outputjson"], capture_output=True, text=True)
    return result.stdout

benoit74

import glob
from warcio import ArchiveIterator

for warc in glob.glob("output/.tmph919m5n3/collections/crawl-*/archive/*.warc.gz"):
    with open(warc, "rb") as fh:
        for record in ArchiveIterator(fh):
            if record.rec_type == "request":
                print(record.rec_headers.get_header('WARC-Target-URI'))

print("DONE")

benoit74

One might need to get a better grasp at what Browsertrix is performing in terms of trafic and where it is being blocked.

You might use the following docker compose stack to run:

• a mitmweb web proxy to intercept all Browsertrix traffic and dump them on file
• a screencasting Browsertrix crawler to get a grasp on crawler behavior

If you open http://localhost:9037, you will see Browsertrix screencasting of the browser.

mitmdump are placed in the output/mitmdump folder, with one file per minute.

benoit74

import json
import subprocess

nodes_str = subprocess.check_output(
    ["kubectl", "get", "nodes",  "-o", "json"]
)
nodes = json.loads(nodes_str)

node_data = {}
for node in nodes.get("items"):

benoit74

{  
    "[python]": {
        "editor.defaultFormatter": "ms-python.black-formatter",
        "editor.formatOnSave": true,
        "editor.codeActionsOnSave": {
            "source.organizeImports": true
        },
      },
    "isort.args": [
        "--profile",

benoit74

my-super-secret-machine:/tmp/testbbe$ TF_LOG=debug terraform plan -out plan.out
2023-04-25T14:14:10.817Z [INFO]  Terraform version: 1.3.7
2023-04-25T14:14:10.817Z [DEBUG] using github.com/hashicorp/go-tfe v1.9.0
2023-04-25T14:14:10.817Z [DEBUG] using github.com/hashicorp/hcl/v2 v2.15.0
2023-04-25T14:14:10.817Z [DEBUG] using github.com/hashicorp/terraform-config-inspect v0.0.0-20210209133302-4fd17a0faac2
2023-04-25T14:14:10.817Z [DEBUG] using github.com/hashicorp/terraform-svchost v0.0.0-20200729002733-f050f53b9734
2023-04-25T14:14:10.817Z [DEBUG] using github.com/zclconf/go-cty v1.12.1
2023-04-25T14:14:10.817Z [INFO]  Go runtime version: go1.19.4
2023-04-25T14:14:10.817Z [INFO]  CLI args: []string{"terraform", "plan", "-out", "plan.out"}
2023-04-25T14:14:10.817Z [DEBUG] Attempting to open CLI config file: /home/atlantis/.terraformrc

benoit74

/*
 * Copyright 2002-2015 the original author or authors.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software

benoit74

license: gpl-3.0
height: 600

benoit74

license: gpl-3.0
height: 600

benoit74

#PS C:\Windows\system32> 
#Import-Module $env:appdata\boxstarter\Boxstarter.Chocolatey\Boxstarter.Chocolatey.psd1
#Invoke-ChocolateyBoxstarter https://gist.githubusercontent.com/fschafer/f04aef74c5d9e19cf8ae/raw/a06b630d2c4843b283b32db38f95d5c662de3e26/BoxStarter-DEV
#START http://boxstarter.org/package/nr/url?https://gist.githubusercontent.com/benoit74/7d98ed4b958804cac4d8/raw/3195b11b6e533a0df33b950d706ec15e30f38805/BoxStarter-DEV

# Boxstarter options
$Boxstarter.RebootOk=$true # Allow reboots?
$Boxstarter.NoPassword=$false # Is this a machine with no login password?
$Boxstarter.AutoLogin=$true # Save my password securely and auto-login after a reboot