vinhnx

Which GGUF is right for me? (Opinionated)

Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggerganov/llama.cpp#5962

In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.

llama.cpp feature matrix

See the wiki upstream: https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix

vinhnx

# install DSPy: pip install dspy
import dspy

# Ollam is now compatible with OpenAI APIs
# 
# To get this to work you must include `model_type='chat'` in the `dspy.OpenAI` call. 
# If you do not include this you will get an error. 
# 
# I have also found that `stop='\n\n'` is required to get the model to stop generating text after the ansewr is complete. 
# At least with mistral.

vinhnx

# vim:fileencoding=utf-8:foldmethod=marker

# Include theme
include ./theme.conf

# ===== Config =====

font_family Menlo
font_size 15.0

vinhnx

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

vinhnx

import UIKit

extension UIViewController {
    func nxv_addChildViewController(_ childVC: UIViewController, containerView: UIView) {
        addChild(childVC)
        containerView.addSubview(childVC.view)
        containerView.translatesAutoresizingMaskIntoConstraints = false
        childVC.view.translatesAutoresizingMaskIntoConstraints = false
        

vinhnx

Code is clean if it can be understood easily – by everyone on the team. Clean code can be read and enhanced by a developer other than its original author. With understandability comes readability, changeability, extensibility and maintainability.

---

General rules

1. Follow standard conventions.
2. Keep it simple stupid. Simpler is always better. Reduce complexity as much as possible.
3. Boy scout rule. Leave the campground cleaner than you found it.
4. Always find root cause. Always look for the root cause of a problem.

Design rules

vinhnx

class MyCell: UITableViewCell {
    let titleLabel = UILabel()
    let subtitleLabel = UILabel()
    
    override init(style: UITableViewCell.CellStyle, reuseIdentifier: String?) {
        super.init(style: style, reuseIdentifier: reuseIdentifier)
    }
    
    required init?(coder: NSCoder) { nil }
    

vinhnx

import Foundation

public protocol NetworkLoggable {
  func log<T: CustomStringConvertible>(
    label: String,
    value: T?,
    level: NetworkLogger,
    function: StaticString,
    line: UInt,
    file: String

vinhnx

Architecture

• Doordash mobile architecture
• Evernote rewrite mobile app
• Scaling up an iOS app using modules

Offline Sync and Cache

• Dropbox photo offline sync
• Airbnb message sync
• Netflix download
• Trello offline works
• Expedia API cache

vinhnx

{"packages":[{"url":"https:\/\/github.com\/vinhnx\/DictionaryNestedSubscript.git","versions":[{"license":{"url":"https:\/\/github.com\/vinhnx\/DictionaryNestedSubscript\/blob\/master\/LICENSE","name":"MIT"},"defaultToolsVersion":"5.1.0","verifiedCompatibility":[{"swiftVersion":"5.1","platform":{"name":"ios"}},{"swiftVersion":"5.2","platform":{"name":"ios"}},{"swiftVersion":"5.3","platform":{"name":"ios"}},{"swiftVersion":"5.4","platform":{"name":"ios"}},{"swiftVersion":"5.5","platform":{"name":"ios"}},{"swiftVersion":"5.1","platform":{"name":"linux"}},{"swiftVersion":"5.2","platform":{"name":"linux"}},{"swiftVersion":"5.3","platform":{"name":"linux"}},{"swiftVersion":"5.4","platform":{"name":"linux"}},{"swiftVersion":"5.5","platform":{"name":"linux"}},{"swiftVersion":"5.1","platform":{"name":"macos"}},{"swiftVersion":"5.2","platform":{"name":"macos"}},{"swiftVersion":"5.3","platform":{"name":"macos"}},{"swiftVersion":"5.4","platform":{"name":"macos"}},{"swiftVersion":"5.5","platform":{"name":"macos"}},{"swift