Skip to content

Instantly share code, notes, and snippets.

@mutaguchi
mutaguchi / 00_LLM_translator.md
Last active May 28, 2024 23:30
LLM_translator.py

llama.cpp serverを使って、入力文字列を翻訳するスクリプト。

使用方法

python LLM_Translator.py 
[-h] 
[-i 入力ファイルパス] [-o 出力ファイルパス] [-p プロンプトファイルパス] [-d 辞書ファイルパス] 
[-u llama.cpp serverのURL] [--quiet] [--no-outfile] [--include-input] [--min-context-count 最小保持ターン数]
function Filter-Duplication
{
[CmdletBinding()]
param (
[Parameter(ValueFromPipeline)]
[object[]]
$InputObject
)
begin
{
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
models = {
"stablelm": "stabilityai/japanese-stablelm-base-gamma-7b",
"chatntq": "NTQAI/chatntq-ja-7b-v1.0",
"mistral": "mistralai/Mistral-7B-v0.1",
"starling": "Nexusflow/Starling-LM-7B-beta",
"antler": "Elizezen/Antler-7B",
}

How to avoid PSAMSIMethodInvocationLogging. In the previously introduced method, a cmdlet is dynamically generated to execute a method by reflection. In this method, the method is executed by reflection by defining a type for the method invocation and having the Hashtable cast to that type. In the example, with Windows Defender real-time protection enabled, a process that would take up to 35 seconds to complete would take about 1 second using this method.

from transformers import GPTJForCausalLM, AlbertTokenizer
import torch
model = 'AIBunCho/japanese-novel-gpt-j-6b'
tokenizer = AlbertTokenizer.from_pretrained(model, keep_accents=True, remove_space=False)
model = GPTJForCausalLM.from_pretrained(
model,
load_in_4bit = True,
torch_dtype = torch.bfloat16,
device_map = 'auto')

Mitigating the Impact of PSAMSIMethodInvocationLogging on PowerShell Performance: An Exploration of the Invoke-Method Cmdlet

Method execution in PowerShell 7.3 and later has been slowed down due to PSAMSIMethodInvocationLogging. This feature executes the AMSI's Logging method by specifying the method information and arguments to be executed just before method execution. Originally, this was an experimental feature, but it has been promoted to an official feature.

The problem of slow operation has been partially solved, but still, with Windows Defender's real-time protection enabled, especially when the argument size is large, the execution of methods within a loop is extremely slow. PowerShell/PowerShell#19431

(Measure-Command{$a='x';$b='a'*1000;foreach($i in (1..1000000)){$y=$a.Contains($b)}}).TotalSeconds
@mutaguchi
mutaguchi / Test_All_Any.ps1
Created May 30, 2023 09:59
Test-All, Test-Any
function Test-All
{
[CmdletBinding()]
param(
[scriptblock]
$Predicate,
[Parameter(ValueFromPipeline)]
[PSObject]
$InputObject
)
# based on StableLM chat
# https://huggingface.co/spaces/stabilityai/stablelm-tuned-alpha-chat
import gradio as gr
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, StoppingCriteria, StoppingCriteriaList, TextIteratorStreamer
import time
import numpy as np
from torch.nn import functional as F
import os
@mutaguchi
mutaguchi / 00README.md
Last active September 30, 2023 13:38
rinna_chat
@mutaguchi
mutaguchi / 00music_poll_aggregate_using_PGT.ps1
Last active April 8, 2023 05:14
PGTで表記ゆれを修正する。それ以外の集計作業は普通にPowerShellで行う。
$env:OPENAI_API_KEY = "your api key"
Import-Module PowerGenerativeToolkit
$answers = echo 夜に駆ける ハルジオン あの夢を追って ツバメ たぶん 怪物 ミスタ 群青色 三原色 ラブレタ 優しい彗星 セブンティーン ハルカ もう少しだけ アイドル 海の真ん中に ハルジオン アドベンチャー 大正浪漫 好き 夜に走る ハルジオン あの夢を踏んで ツバメ 多分 怪物さん ミスター 群青 3原色 ラブレター 優しいすい星 セブンティーン 春風 もう少しの間 アイドル 海のまにまに ハルジオン 冒険 大正ロマン 好きだ
$titles = echo 夜に駆ける あの夢をなぞって ハルジオン たぶん 群青 ハルカ 怪物 優しい彗星 もう少しだけ 三原色 ラブレター 大正浪漫 ツバメ ミスター 好きだ 海のまにまに アドベンチャー セブンティーン アイドル
$list = $answers | foreach {
if ($_ -in $titles) { [pscustomobject]@{"raw" = $_; "title" = $_; "isCorrect" = $true } }
else { [pscustomobject]@{"raw" = $_; "title" = ""; "isCorrect" = $false } }
}