Jacob Kahn jacobkahn
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| python src/transformers/models/chameleon/convert_chameleon_weights_to_hf.py --input_dir ~/chameleon/data/ --output_dir ~/chameleon/hf_converted/30B --model_size 30B | |
| Fetching all parameters from the checkpoint at ~/chameleon/data/models/30b. | |
| Loading the checkpoint in a Chameleon model... | |
| **************************************************************************************************** | |
| Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 15/15 [00:30<00:00, 2.02s/it] | |
| Setting `pad_token_id` to `eos_token_id`:2 for open-end generation. | |
| Traceback (most recent call last): | |
| File "/storage/home/jacobkahn/chameleon/transformers/src/transformers/models/chameleon/convert_chameleon_weights_to_hf.py", line 417, in <module> | |
| main() | |
| File "/storage/home/jacobkahn/chameleon/transformers/src/transformers/models/chameleon/convert_chameleon_weights_to_hf.py", line 408, in main |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Fetching all parameters from the checkpoint at /home/jacobkahn/chameleon/data/models/30b. | |
| Loading the checkpoint in a Chameleon model... | |
| Loading checkpoint shards: 100%|███████████████████████████████████████████| 15/15 [00:29<00:00, 1.97s/it] | |
| Setting `pad_token_id` to `eos_token_id`:2 for open-end generation. | |
| Traceback (most recent call last): | |
| File "/storage/home/jacobkahn/chameleon/transformers/src/transformers/models/chameleon/convert_chameleon_weights_to_hf.py", line 415, in <module> | |
| main() | |
| File "/storage/home/jacobkahn/chameleon/transformers/src/transformers/models/chameleon/convert_chameleon_weights_to_hf.py", line 406, in main | |
| write_model( | |
| File "/storage/home/jacobkahn/chameleon/transformers/src/transformers/models/chameleon/convert_chameleon_weights_to_hf.py", line 357, in write_model |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| diff --git a/bindings/python/CMakeLists.txt b/bindings/python/CMakeLists.txt | |
| index 08b4c486..7f32e9b8 100644 | |
| --- a/bindings/python/CMakeLists.txt | |
| +++ b/bindings/python/CMakeLists.txt | |
| @@ -21,7 +21,7 @@ function (add_pybind11_extension ext_name) | |
| target_link_libraries( | |
| ${ext_name} | |
| PUBLIC | |
| - fl_lib_audio | |
| + # fl_lib_audio |
jacobkahn
/ cast.new.jit.kernel.cl
Created
October 28, 2020 01:12
ArrayFire JIT kernel Generated from AF_JIT_KERNEL_TRACE=stderr ./test/cast_opencl --gtest_filter="*Test_JIT_DuplicateCastNoop"
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| __kernel void | |
| KER9041217350160086296( | |
| __global float *in0, dim_t iInfo0_offset, | |
| __global float *out0, | |
| KParam oInfo, uint groups_0, uint groups_1, uint num_odims) | |
| { | |
| uint groupId = get_group_id(1) * get_num_groups(0) + get_group_id(0); | |
| uint threadId = get_local_id(0); |
jacobkahn
/ cast.jit.kernel.master.cu
Last active
October 28, 2020 01:11
ArrayFire JIT kernel Generated from AF_JIT_KERNEL_TRACE=stderr ./test/cast_cuda --gtest_filter="*Test_JIT_DuplicateCastNoop" [old master]
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| typedef unsigned int uint; | |
| typedef long long dim_t; | |
| /******************************************************* | |
| * Copyright (c) 2014, ArrayFire | |
| * All rights reserved. | |
| * | |
| * This file is distributed under 3-clause BSD license. | |
| * The complete license agreement can be obtained at: | |
| * http://arrayfire.com/licenses/BSD-3-Clause | |
| ********************************************************/ |
jacobkahn
/ cast.new.jit.kernel.cu
Last active
October 28, 2020 01:11
ArrayFire JIT kernel Generated from AF_JIT_KERNEL_TRACE=stderr ./test/cast_cuda --gtest_filter="*Test_JIT_DuplicateCastNoop"
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| typedef unsigned int uint; | |
| typedef long long dim_t; | |
| /******************************************************* | |
| * Copyright (c) 2014, ArrayFire | |
| * All rights reserved. | |
| * | |
| * This file is distributed under 3-clause BSD license. | |
| * The complete license agreement can be obtained at: | |
| * http://arrayfire.com/licenses/BSD-3-Clause | |
| ********************************************************/ |
jacobkahn
/ librispeech.train.manual.excluded.titles.lst
Created
April 7, 2020 02:03
List of LibriSpeech train books found in Gutenberg with fuzzy matching procedure (word-level Levenshtein, score = 0.3, distancesim = 0.75)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 100 the story of a patriot|100 the story of a patriot|0 | |
| 32 caliber|32 caliber|0 | |
| a birding on a bronco|a birding on a bronco|0 | |
| acres of diamonds|acres of diamonds|0 | |
| buccaneers and pirates of our coasts|buccaneers and pirates of our coasts|0 | |
| byways around san francisco bay|byways around san francisco bay|0 | |
| caleb williams or things as they are|caleb williams or things as they are|0 | |
| candide|candide|0 | |
| candide|candide|0 | |
| canyons of the colorado|canyons of the colorado|0 |
jacobkahn
/ gutenberg_minus_librilight.manual.excluded.titles.lst
Last active
April 6, 2020 20:49
List of LibriLight books found in Gutenberg with fuzzy matching procedure (word-level Levenshtein, score = 0.3, distancesim = 0.75)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 100 the story of a patriot|100 the story of a patriot|0 | |
| 1914 and other poems|1914 and other poems|0 | |
| 32 caliber|32 caliber|0 | |
| 35 sonnets|35 sonnets|0 | |
| 813|813|0 | |
| aaron's rod|aaron's rod|0 | |
| abaft the funnel|abaft the funnel|0 | |
| abandoned|abandoned|0 | |
| abbotsford and newstead abbey|abbotsford and newstead abbey|0 | |
| abe and mawruss being further adventures of potash and perlmutter|abe and mawruss being further adventures of potash and perlmutter|0 |
jacobkahn
/ arrayfire.2801.changes.diff
Last active
March 23, 2020 14:14
Updated changes for ArrayFire PR #2801
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| diff --git a/src/backend/common/DefaultMemoryManager.cpp b/src/backend/common/DefaultMemoryManager.cpp | |
| index f3921a6b..0b1e44ca 100644 | |
| --- a/src/backend/common/DefaultMemoryManager.cpp | |
| +++ b/src/backend/common/DefaultMemoryManager.cpp | |
| @@ -162,7 +162,8 @@ void* DefaultMemoryManager::alloc(bool user_lock, const unsigned ndims, | |
| if (!this->debug_mode) { | |
| // FIXME: Add better checks for garbage collection | |
| // Perhaps look at total memory available as a metric | |
| - if (getMemoryPressure() > getMemoryPressureThreshold()) { | |
| + if (current.lock_bytes >= current.max_bytes || |
jacobkahn
/ test_many.sh
Last active
February 29, 2020 05:51
Run some stuff a bunch of times and see if it prints FAIL anywhere
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/sh | |
| # Usage: | |
| # bash ./test_many.sh [test_name] [num_runs=100] [parallelism=32] | |
| # | |
| # Example: | |
| # bash ./test_many.sh 2B | |
| test_name=$1 | |
| num_runs="${2:-100}" |
NewerOlder