redcpp

import openai
import pinecone
from sentence_transformers import SentenceTransformer

class GPTConversationManager:
    def __init__(self, api_key, pinecone_api_key, index_name):
        self.api_key = api_key
        openai.api_key = self.api_key
        self.conversation_history = []
        self.pinecone_api_key = pinecone_api_key

akhan619

Exploring Tokenizers from Hugging Face

Hugging Face (HF) has made NLP (Natural Language Processing) a breeze. In this post, we are going to take a look at tokenization using a hands on approach with the help of the Tokenizers library. We are going to load a real world dataset containing 10-K filings of public firms and see how to train a tokenizer from scratch based on the BERT tokenization scheme. In the process we will understand tokenization in detail and some gotchas to keep an eye out for.

Background on NLP (Optional)

If you already have an understanding of the NLP pipeline, you can safely skip this section.

For any NLP task, one of the first steps is pre-processing the data so that it can be fed into our NLP models. For those new to NLP, the general pipeline for any NLP task (text classification, question answering, etc.) is as follows:

stettix

Things I believe

This is a collection of the things I believe about software development. I have worked for years building backend and data processing systems, so read the below within that context.

Agree? Disagree? Feel free to let me know at @JanStette. See also my blog at www.janvsmachine.net.

Fundamentals

Keep it simple, stupid. You ain't gonna need it.

tarbaig

Backend Architectures

Twitter

ror, scala, jetty, erlang, thrift, mongrel, comet server, my-sql, memchached, varnish, kestrel(mq), starling, gizzard, cassandra, hadoop, vertica, munin, nagios, awstats

• Scaling Twitter: Making Twitter 10000 Percent Faster
• Scaling Twitter
• Blaine Cook on Scaling Twitter - YouTube

ipmb

Default Django Logging Tree

app.py

#!/usr/bin/env python
import os

import django
import logging_tree

kevin-smets

Requirements

Minikube requires that VT-x/AMD-v virtualization is enabled in BIOS. To check that this is enabled on OSX / macOS run:

sysctl -a | grep machdep.cpu.features | grep VMX

If there's output, you're good!

Prerequisites

bhtucker

"""
Upsert gist

Requires at least postgres 9.5 and sqlalchemy 1.1

Initial state:

[]
Initial upsert:

ivanleoncz

#/usr/bin/python3
""" Demonstration of logging feature for a Flask App. """

from logging.handlers import RotatingFileHandler
from flask import Flask, request, jsonify
from time import strftime

__author__ = "@ivanleoncz"

import logging

conormm

R to python data wrangling snippets

The dplyr package in R makes data wrangling significantly easier. The beauty of dplyr is that, by design, the options available are limited. Specifically, a set of key verbs form the core of the package. Using these verbs you can solve a wide range of data problems effectively in a shorter timeframe. Whilse transitioning to Python I have greatly missed the ease with which I can think through and solve problems using dplyr in R. The purpose of this document is to demonstrate how to execute the key dplyr verbs when manipulating data using Python (with the pandas package).

dplyr is organised around six key verbs:

vasanthk

System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

Basic Steps

1. Clarify and agree on the scope of the system

• User cases (description of sequences of events that, taken together, lead to a system doing something useful)
  • Who is going to use it?
  • How are they going to use it?