Skip to content

Instantly share code, notes, and snippets.

View norandom's full-sized avatar
💭
/dev/norandom

Marius Ciepluch norandom

💭
/dev/norandom
View GitHub Profile
@norandom
norandom / upload_markdown_to_ragflow.sh
Created July 8, 2025 10:56
Upload Markdown files to RAGflow for parsing (Collection)
#!/bin/bash
# Check for jq and install if not found
if ! command -v jq &> /dev/null
then
echo "jq could not be found, attempting to install."
if [ -f /etc/os-release ]; then
. /etc/os-release
if [ "$ID" = "ubuntu" ] || [ "$ID" = "debian" ]; then
sudo apt-get update && sudo apt-get install -y jq
#!/bin/bash
# Script to combine all markdown files in each folder into one file per folder
# Usage: ./combine_markdown.sh [base_directory]
BASE_DIR="${1:-crawled_docs}"
OUTPUT_DIR="combined_docs"
if [ ! -d "$BASE_DIR" ]; then
echo "Error: Directory $BASE_DIR does not exist"
@norandom
norandom / crawl_documentation_zscaler.py
Created July 8, 2025 10:51
Crawl4AI Markdown scraper for documentation dataset generation (Zscaler)
import asyncio
import os
from urllib.parse import urlparse
from pathlib import Path
from crawl4ai import AsyncWebCrawler, CrawlerRunConfig
from crawl4ai.deep_crawling import BestFirstCrawlingStrategy
from crawl4ai.deep_crawling.scorers import KeywordRelevanceScorer
from crawl4ai.markdown_generation_strategy import DefaultMarkdownGenerator
from crawl4ai.deep_crawling.filters import FilterChain
@norandom
norandom / upload_multiple_batches_of_pdfs_to_ragflow.sh
Last active July 8, 2025 11:01
Upload multiple batches of PDFs to RAGflow via Bash
#!/bin/bash
# Check for jq and install if not found
if ! command -v jq &> /dev/null
then
echo "jq could not be found, attempting to install."
if [ -f /etc/os-release ]; then
. /etc/os-release
if [ "$ID" = "ubuntu" ] || [ "$ID" = "debian" ]; then
sudo apt-get update && sudo apt-get install -y jq
@norandom
norandom / upload_single_batch_of_pdfs_to_ragflow.sh
Last active July 8, 2025 11:02
Upload a batch of PDFs to RAGflow via Bash
#!/bin/bash
# Check for jq and install if not found
if ! command -v jq &> /dev/null
then
echo "jq could not be found, attempting to install."
if [ -f /etc/os-release ]; then
. /etc/os-release
if [ "$ID" = "ubuntu" ] || [ "$ID" = "debian" ]; then
sudo apt-get update && sudo apt-get install -y jq
########################
# Windows 10 Prefetch Parser
# Created by 505Forensics (http://www.505forensics.com)
# Modified by Marius Ciepluch (2024, Python 3)
#
# Usage: Utilize this script to parse either a single or set of Windows 10 prefetch files
#
# Dependencies: This script requires the installation of libscca (https://github.com/libyal/libscca), and was only tested in a Linux environment
#
# Output: Script will output in CSV to stdout by default.
@norandom
norandom / windows_registry_date_convert.py
Created February 25, 2024 12:10
Converts Windows Registry dates to a human readable format
"""
Copyright (c) 2024 Marius Ciepluch
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
@norandom
norandom / random.cpp
Created February 16, 2024 18:59
Quick benchmark - Randomness via CPU
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
// Function to convert a byte into its hexadecimal string representation
void byteToHex(unsigned char byte, char hexStr[3]) {
const char hexDigits[] = "0123456789abcdef";
hexStr[0] = hexDigits[byte >> 4]; // Extract the high nibble (4 bits) and find its hex character
hexStr[1] = hexDigits[byte & 0x0F]; // Extract the low nibble and find its hex character
hexStr[2] = '\0'; // Null-terminate the string
@norandom
norandom / setup_net_arch.sh
Last active February 16, 2024 09:59
Persist DHCP provisioned IP config (Arch, systemd-networkd)
#!/bin/bash
# Interface name, e.g., enp1s0
IFACE="enp1s0"
# Ensure systemd-networkd is enabled
systemctl enable systemd-networkd
systemctl start systemd-networkd
systemctl start systemd-resolved
@norandom
norandom / setup_net.sh
Last active February 15, 2024 17:40
Persist DHCP provisioned IP config (Debian, Ubuntu, netplan)
#!/bin/bash
# Interface name
IFACE="enp1s0"
# Get current IP, netmask, and gateway
IP=$(ip addr show $IFACE | grep 'inet ' | awk '{print $2}')
GATEWAY=$(ip route | grep default | awk '{print $3}')
DNS=$(awk '/^nameserver/ {print $2}' /run/systemd/resolve/resolv.conf | tr '\n' ' ')