catid

How to quantize 70B model so it will fit on 2x4090 GPUs:

I tried EXL2, AutoAWQ, and SqueezeLLM and they all failed for different reasons (issues opened).

HQQ worked:

I rented a 4x GPU 1TB RAM ($19/hr) instance on runpod with 1024GB container and 1024GB workspace disk space.
I think you only need 2x GPU with 80GB VRAM and 512GB+ system RAM so probably overpaid.

Note you need to fill in the form to get access to the 70B Meta weights.

thesamesam

FAQ on the xz-utils backdoor (CVE-2024-3094)

This is a living document. Everything in this document is made in good faith of being accurate, but like I just said; we don't yet know everything about what's going on.

Background

On March 29th, 2024, a backdoor was discovered in xz-utils, a suite of software that

yunginnanet

package main

// ▄─▄  ▄ ▄ ▄  ──▄  ▄─▄  ▄─▄
// ▓─   ▓ ▓ ▓  ▓ ▓  ▀─▄  ▀─▄
// ▀    ▀─▀─▀  ──▀  ▀─▀  ▀─▀
//      f w d  -->  5 5
//
// simple rfc1928 proxy server
//
//

dunderhay

#!/bin/bash
set -e

GO_VERSION="1.22.3"
GO_URL="https://go.dev/dl/go${GO_VERSION}.linux-amd64.tar.gz"
EXPECTED_CHECKSUM="8920ea521bad8f6b7bc377b4824982e011c19af27df88a815e3586ea895f1b36"

# Log output of script
exec > >(tee -i /home/ubuntu/install.log)
exec 2>&1

0xdevalias

Deobfuscating / Unminifying Obfuscated Web App / JavaScript Code

Table of Contents

• PoC
• Tools
  • Unsorted
• wakaru

krzemienski

#!/bin/bash
# ==================================================================
# This script will compile and install a static ffmpeg build with
#   support for NVENC in Ubuntu. Developed in Ubuntu 22.04 LTS,
#   with NVIDIA Drivers v510.73 and CUDA v11.6
# It assumes NVIDA drivers are installed and that you have a
#   CUDA-compatible GPU. You can check installed drivers with:
#       $ apt list *nvidia-driver-* | grep installed
#       $ nvidia-smi
# ==================================================================

ustayready

import openai
import boto3
import json
import time
from typing import Dict, List

openai.api_key = '### SET YOUR OPENAPI API KEY HERE ###'
session = boto3.session.Session()
client = session.client('iam')

lucaspar

#!/bin/bash
# =========================================================================
# Source: https://gist.github.com/lucaspar/27f5e108b80524b315be10b2a9049817
# =========================================================================
# This script will compile and install a static FFmpeg build with
#   support for NVENC in Ubuntu. Developed in Ubuntu 23.10,
#   with NVIDIA Drivers v535.129.03 and CUDA v12.2 with a GPU
#   with CUDA capability 8.6 (RTX 3080) (see ccap below).
# It assumes NVIDA drivers are installed and that you have a
#   CUDA-compatible GPU. You can check installed drivers with:

zrruziev

What is NUMA (Non-Uniformed Memory Access)

Non-Uniform Memory Access (NUMA) is one of the computer memory design methods used in multiprocessor systems, and the time to access the memory varies depending on the relative position between the memory and the processor. In the NUMA architecture, when a processor accesses its local memory, it is faster than when it accesses the remote memory. Remote memory refers to memory that is connected to another processor, and local memory refers to memory that is connected to its own processor. In other words, it is a technology to increase memory access efficiency while using multiple processors on one motherboard. When a specific processor runs out of memory, it monopolizes the bus by itself, so other processors have to play. , and designate 'access only here', and call it a NUMA node.

1. Check Nodes

lspci | grep -i nvidia
  
01:00.0 VGA compatible controller: NVIDIA Corporation TU106 [GeForce RTX 2060 12GB] (rev a1)

ibireme

// =============================================================================
// XNU kperf/kpc demo
// Available for 64-bit Intel/Apple Silicon, macOS/iOS, with root privileges
//
//
// Demo 1 (profile a function in current thread):
// 1. Open directory '/usr/share/kpep/', find your CPU PMC database.
//    M1 (Pro/Max/Ultra): /usr/share/kpep/a14.plist
//    M2 (Pro/Max):       /usr/share/kpep/a15.plist
//    M3:                 /usr/share/kpep/as1.plist