sighingnow
/ gpt-model.py
Created
January 14, 2024 06:00
llm-arithmetic: flops, memory footprint, memory access I/O, and more for both training and inference (prefill and generation)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| class GPTModel: | |
| def __init__( | |
| self, | |
| name: str = None, | |
| vocab_size: int = 51200, | |
| sequence_length: int = 2048, | |
| attention_heads: int = 32, | |
| hidden_size: int = 2304, | |
| layers: int = 24, | |
| micro_batch_size: int = 1, |
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
sighingnow
/ pipeline-model-parallel-visualization.ipynb
Last active
December 25, 2023 02:02
Visualizing various different pipeline model parallel scheduling algorithms: GPipe, Pipedream(1F1B), Pipedream-2BW(async, no-flushes), and eager-1F1B
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
sighingnow
/ display-kconfig.sh
Created
March 25, 2023 02:06
Display build configurations about current Linux kernel
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cat /proc/config.gz | gunzip |
sighingnow
/ CRTP-(de)incomplete-(de)ambiguous.cpp
Created
March 7, 2023 17:04
Fixes the incomplete type and ambiguous dependent name problem for CRTP: https://godbolt.org/z/3WGe1q5K5
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| template <typename U> | |
| struct A_trait { | |
| using T = int; | |
| }; | |
| template <typename U, template <typename ...> class Trait = A_trait> | |
| struct A { | |
| using T = typename Trait<U>::T; | |
| }; |
sighingnow
/ PrintVariables.cmake
Last active
June 5, 2022 04:03
Print all accessible variables in current context in CMake
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| function(print_cmake_variables) | |
| get_cmake_property(_variableNames VARIABLES) | |
| list (SORT _variableNames) | |
| foreach (_variableName ${_variableNames}) | |
| if (ARGV0) | |
| unset(MATCHED) | |
| string(REGEX MATCH ${ARGV0} MATCHED ${_variableName}) | |
| if (NOT MATCHED) | |
| continue() | |
| endif() |
sighingnow
/ PrintTargetProperties.cmake
Last active
November 13, 2022 11:40
Print properties of a target in CMake
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Get all propreties that cmake supports | |
| if(NOT CMAKE_PROPERTY_LIST) | |
| execute_process(COMMAND cmake --help-property-list OUTPUT_VARIABLE CMAKE_PROPERTY_LIST) | |
| # Convert command output into a CMake list | |
| string(REGEX REPLACE ";" "\\\\;" CMAKE_PROPERTY_LIST "${CMAKE_PROPERTY_LIST}") | |
| string(REGEX REPLACE "\n" ";" CMAKE_PROPERTY_LIST "${CMAKE_PROPERTY_LIST}") | |
| endif() | |
| list(APPEND CMAKE_PROPERTY_LIST "IMPORTED_LOCATION") |
sighingnow
/ FindBrewOpenSSL.cmake
Created
October 9, 2021 11:24
CMake: find brew-installed openssl on Mac
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| macro(find_openssl_libraries) | |
| if (APPLE) | |
| # If we're on OS X check for Homebrew's copy of OpenSSL instead of Apple's | |
| if (NOT OpenSSL_DIR) | |
| find_program(HOMEBREW brew) | |
| if (HOMEBREW STREQUAL "HOMEBREW-NOTFOUND") | |
| message(WARNING "Homebrew not found: not using Homebrew's OpenSSL") | |
| if (NOT OPENSSL_ROOT_DIR) | |
| message(WARNING "Use -DOPENSSL_ROOT_DIR for non-Apple OpenSSL") | |
| endif() |
sighingnow
/ strings 1 - Why COW is ungood for std string\strings 1 - Why COW is ungood for std string .md
Created
September 26, 2021 09:48
— forked from alf-p-steinbach/strings 1 - Why COW is ungood for std string\strings 1 - Why COW is ungood for std string .md
Why COW was deemed ungood for std::string
COW, short for copy on write, is a way to implement mutable strings so that creating strings and logically copying strings, is reduced to almost nothing; conceptually they become free operations like no-ops.
Basic idea: to share a data buffer among string instances, and only make a copy for a specific instance (the copy on write) when that instance's data is modified. The general cost of this is only an extra indirection for accessing the value of a string, so a COW implementation is highly desirable. And so the original C++ standard, C++98, and its correction C++03, had special support for COW implementations, and e.g. the g++ compiler's std::string implementations used COW.
So why was that support dropped in C++11?
In particular, would the same reason or reasons apply to a reference counted immutable string value class?
NewerOlder