Skip to content

Instantly share code, notes, and snippets.

View sighingnow's full-sized avatar
💭
typing...

Tao He sighingnow

💭
typing...
View GitHub Profile
@sighingnow
sighingnow / gpt-model.py
Created January 14, 2024 06:00
llm-arithmetic: flops, memory footprint, memory access I/O, and more for both training and inference (prefill and generation)
class GPTModel:
def __init__(
self,
name: str = None,
vocab_size: int = 51200,
sequence_length: int = 2048,
attention_heads: int = 32,
hidden_size: int = 2304,
layers: int = 24,
micro_batch_size: int = 1,
@sighingnow
sighingnow / kvcache.ipynb
Created December 26, 2023 02:39
LLM kv cache analysis
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@sighingnow
sighingnow / pipeline-model-parallel-visualization.ipynb
Last active June 14, 2024 19:58
Visualizing various different pipeline model parallel scheduling algorithms: GPipe, Pipedream(1F1B), Pipedream-2BW(async, no-flushes), and eager-1F1B
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@sighingnow
sighingnow / display-kconfig.sh
Created March 25, 2023 02:06
Display build configurations about current Linux kernel
cat /proc/config.gz | gunzip
@sighingnow
sighingnow / CRTP-(de)incomplete-(de)ambiguous.cpp
Created March 7, 2023 17:04
Fixes the incomplete type and ambiguous dependent name problem for CRTP: https://godbolt.org/z/3WGe1q5K5
template <typename U>
struct A_trait {
using T = int;
};
template <typename U, template <typename ...> class Trait = A_trait>
struct A {
using T = typename Trait<U>::T;
};
@sighingnow
sighingnow / PrintVariables.cmake
Last active June 5, 2022 04:03
Print all accessible variables in current context in CMake
function(print_cmake_variables)
get_cmake_property(_variableNames VARIABLES)
list (SORT _variableNames)
foreach (_variableName ${_variableNames})
if (ARGV0)
unset(MATCHED)
string(REGEX MATCH ${ARGV0} MATCHED ${_variableName})
if (NOT MATCHED)
continue()
endif()
@sighingnow
sighingnow / PrintTargetProperties.cmake
Last active November 13, 2022 11:40
Print properties of a target in CMake
# Get all propreties that cmake supports
if(NOT CMAKE_PROPERTY_LIST)
execute_process(COMMAND cmake --help-property-list OUTPUT_VARIABLE CMAKE_PROPERTY_LIST)
# Convert command output into a CMake list
string(REGEX REPLACE ";" "\\\\;" CMAKE_PROPERTY_LIST "${CMAKE_PROPERTY_LIST}")
string(REGEX REPLACE "\n" ";" CMAKE_PROPERTY_LIST "${CMAKE_PROPERTY_LIST}")
endif()
list(APPEND CMAKE_PROPERTY_LIST "IMPORTED_LOCATION")
@sighingnow
sighingnow / FindBrewOpenSSL.cmake
Created October 9, 2021 11:24
CMake: find brew-installed openssl on Mac
macro(find_openssl_libraries)
if (APPLE)
# If we're on OS X check for Homebrew's copy of OpenSSL instead of Apple's
if (NOT OpenSSL_DIR)
find_program(HOMEBREW brew)
if (HOMEBREW STREQUAL "HOMEBREW-NOTFOUND")
message(WARNING "Homebrew not found: not using Homebrew's OpenSSL")
if (NOT OPENSSL_ROOT_DIR)
message(WARNING "Use -DOPENSSL_ROOT_DIR for non-Apple OpenSSL")
endif()

Why COW was deemed ungood for std::string.

COW, short for copy on write, is a way to implement mutable strings so that creating strings and logically copying strings, is reduced to almost nothing; conceptually they become free operations like no-ops.

Basic idea: to share a data buffer among string instances, and only make a copy for a specific instance (the copy on write) when that instance's data is modified. The general cost of this is only an extra indirection for accessing the value of a string, so a COW implementation is highly desirable. And so the original C++ standard, C++98, and its correction C++03, had special support for COW implementations, and e.g. the g++ compiler's std::string implementations used COW.

So why was that support dropped in C++11?

In particular, would the same reason or reasons apply to a reference counted immutable string value class?