Skip to content

Instantly share code, notes, and snippets.

View victorkane's full-sized avatar

Victor Kane victorkane

View GitHub Profile
@victorkane
victorkane / crawler.sh
Created March 2, 2021 13:05 — forked from xymox12/crawler.sh
Convert a csv of URLs and Page Titles created using WGET to Freemind XML and UL list
#!/bin/bash
#
# Crawls a domain
# Retreives all visible URLs and their page titles
# Saves to CSV
# $1 = URL (no http(s))
# $2 = csv title
# MODIFY - wget include directories, domain, and --reject-regex TODO - make a variable
# Text color variables
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>JS Bin</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.11.0/moment.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/react/0.14.3/react.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/react/0.14.3/react-dom.js">
</script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>JS Bin</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/redux/3.0.4/redux.js"></script>
<script src="https://fb.me/react-0.14.0.js"></script>
<script src="https://fb.me/react-dom-0.14.0.js"></script>
</head>
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>JS Bin</title>
</head>
<body>
<script id="jsbin-javascript">
'use strict';
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>JS Bin</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/redux/3.0.4/redux.js"></script>
</head>
<body>
<script id="jsbin-javascript">
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>JS Bin</title>
<script src="https://wzrd.in/standalone/expect@latest"></script>
</head>
<body>
<script id="jsbin-javascript">
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>JS Bin</title>
<script src="https://fb.me/react-0.14.0.js"></script>
<script src="https://fb.me/react-dom-0.14.0.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/redux/3.0.3/redux.js"></script>
</head>
<body>