dburner

ChatGPT Resources

Context

ChatGPT appeared like an explosion on all my social media timelines in early December 2022. While I keep up with machine learning as an industry, I wasn't focused so much on this particular corner, and all the screenshots seemed like they came out of nowhere. What was this model? How did the chat prompting work? What was the context of OpenAI doing this work and collecting my prompts for training data?

I decided to do a quick investigation. Here's all the information I've found so far. I'm aggregating and synthesizing it as I go, so it's currently changing pretty frequently.

Model Architecture

dburner

public static class EvilStringHelper
{
    private static readonly Action<string, int, char> _setChar;
    private static readonly Action<string, int> _setLength;

    static EvilStringHelper()
    {
        if (Environment.Version.Major < 4)
        {
            MethodInfo setCharMethod = typeof(string).GetMethod(

dburner

# Procedure is for Ubuntu 14.04 LTS.

# Using these guides:
# http://datacenteroverlords.com/2012/03/01/creating-your-own-ssl-certificate-authority/
# https://turboflash.wordpress.com/2009/06/23/curl-adding-installing-trusting-new-self-signed-certificate/
# https://jamielinux.com/articles/2013/08/act-as-your-own-certificate-authority/

# Generate the root (GIVE IT A PASSWORD IF YOU'RE NOT AUTOMATING SIGNING!):
openssl genrsa -aes256 -out ca.key 2048
openssl req -new -x509 -days 7300 -key ca.key -sha256 -extensions v3_ca -out ca.crt

dburner

Ikea Bekant standing desk with Gerton table top

dburner

Client-side SSL

For excessively paranoid client authentication.

Using self-signed certificate.

Create a Certificate Authority root (which represents this server)

Organization & Common Name: Some human identifier for this server CA.

openssl genrsa -des3 -out ca.key 4096
openssl req -new -x509 -days 365 -key ca.key -out ca.crt

dburner

class AttentionLSTM(LSTM):
    """LSTM with attention mechanism

    This is an LSTM incorporating an attention mechanism into its hidden states.
    Currently, the context vector calculated from the attended vector is fed
    into the model's internal states, closely following the model by Xu et al.
    (2016, Sec. 3.1.2), using a soft attention model following
    Bahdanau et al. (2014).

    The layer expects two inputs instead of the usual one:

dburner

using System;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations.Schema;
using System.Data.Entity;
using System.Linq;
using System.Linq.Expressions;
using System.Reflection;
using System.Text;

namespace EFExtensions

dburner

"""
A keras attention layer that wraps RNN layers.

Based on tensorflows [attention_decoder](https://github.com/tensorflow/tensorflow/blob/c8a45a8e236776bed1d14fd71f3b6755bd63cc58/tensorflow/python/ops/seq2seq.py#L506) 
and [Grammar as a Foreign Language](https://arxiv.org/abs/1412.7449).

date: 20161101
author: wassname
url: https://gist.github.com/wassname/5292f95000e409e239b9dc973295327a
"""

dburner

# vocabulary = OrderedDict()
input_length = None
vocabulary_size = max(vocabulary.values()) + 1
weights_w2v = list(map(Word2Vec.__getitem__, vocabulary.keys()))
embedding_size len(weights_w2v[0])
nb_classes = 5

# CNN hyperparms
nb_filter = 64
filter_length = 5

dburner

namespace Analogy
{
    /// <summary>
    /// This example shows that a library that needs access to target .NET Standard 1.3
    /// can only access APIs available in that .NET Standard. Even though similar the APIs exist on .NET 
    /// Framework 4.5, it implements a version of .NET Standard that isn't compatible with the library.
    /// </summary>INetCoreApp10
    class Example1
    {
        public void Net45Application(INetFramework45 platform)