cnmoro

import tiktoken

gpt_encoding = tiktoken.encoding_for_model("gpt-3.5-turbo-16k")

def chunk_text(full_text, tokens_per_chunk=300, chunk_overlap=20):
    chunks = []
    current_chunk = []
    current_chunk_length = 0
    tokens = gpt_encoding.encode(full_text)
    for i, token in enumerate(tokens):

cnmoro

zRam is a virtual memory compression using block devices named /dev/zram using a fast compression algorithm (LZ4) that compress the least recently used (LRU) or inactive space in the memory allows the GNU/Linux kernel to free up more memory with less performance hit.

zRam is greatly increased the available amount of memory by compressing memory without swap disks/partition. It is recommended for the user to use zRam instead of not use/disable the swap to prevent out of memory (OOM) killer. Create a zRam block devices Load the zRam modules to the kernel using modprobe:

sudo modprobe zram

Set the zRam extremely fast compression algorithm using lz4:

cnmoro

import uvicorn, asyncio
import motor.motor_asyncio
from fastapi import FastAPI
import pandas as pd

app = FastAPI()
client = motor.motor_asyncio.AsyncIOMotorClient()
db = client['MYDB']

def process_data(data):

cnmoro

import category_encoders as ce

categoric_features = ['a', 'b', 'c']

woe_encoder = ce.WOEEncoder(cols=categoric_features)
woe_encoder.fit(X[categorics], y)

def get_woe_value(feature):
    for map in woe_encoder.ordinal_encoder.mapping:
        if map['col'] == feature:

cnmoro

Reference: https://help.ubuntu.com/community/Oracle%20Instant%20Client

Tested on: Ubuntu 18.04, 20.04

1. Decide which version of the Oracle client to install

   • Each version of the client supports certain versions of the database; for example, Oracle client 12.1 supports Oracle database 10.2 - 19c
   • See here for more information: http://support.oracle.com/epmos/faces/DocumentDisplay?id=207303.1

2. Download the Oracle client packages

   • Go here and download the desired RPM package(s): https://www.oracle.com/database/technologies/instant-client/downloads.html

cnmoro

# CLUSTERING

# Davies Bouldin Index -> Menor Melhor para escolha do K

# Descrição das estatísticas das features
df.groupby("cluster").describe()

centroids = kmeans.cluster_centers_
max = centroids[0]

cnmoro

import shap
import numpy as np
import pandas as pd

categoric_features = tuple(['FEATURE1', 'FEATURE2', 'ETC'])

def avaliar_importancias_features(modelo_treinado, X):
    explainer = shap.Explainer(modelo_treinado)
    shap_values = explainer.shap_values(X)

cnmoro

SERVER-SIDE

$ sudo apt install wireguard
$ sudo -i
$ cd /etc/wireguard/
$ umask 077; wg genkey | tee privatekey | wg pubkey > publickey
$ cat privatekey
( Save the key )
$ cat publickey
( Save the key )

cnmoro

sudo apt install nginx (ubuntu)
sudo yum install nginx (rhel)

#apache2
#httpd
sudo apt install apache2-utils (ubuntu)
sudo yum install httpd-tools (rhel)

sudo htpasswd -c /etc/nginx/.htpasswd USUARIO

cnmoro

$ pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org pip setuptools