Tyler Lisowski relyt0925
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| {"system": "You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.", "user": "Identify any Personal Identifying Information in the following input:.\nHere's an example of an input with Personal Identifying Information (PII) - {\"email\": \"[johndoe@gmail.com](mailto:johndoe@gmail.com)\", \"phone\\_number\": \"+1 555-123-4567\", \"age\": 35}", "assistant": "The Personal Identifying Information in the given input are the email and phone number - [johndoe@gmail.com](mailto:johndoe@gmail.com) and +1 555-123-4567."} | |
| {"system": "You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.", "user": "Mask all Personal Identifying Information in the input with ***.\nHere's an example of an input with Personal |
relyt0925
/ mt_bench_branch.log
Created
July 7, 2024 02:54
mt_bench_branch.log (ilab model evaluate --benchmark mt_bench_branch --model /instructlab/models/tuned-0701-1954/samples_4992 --judge-model /instructlab/models/mistralai/Mixtral-8x7B-Instruct-v0.1 --taxonomy-path /instructlab/taxonomy --output-dir /instructlab/mtbench --base-model /instructlab/models/ibm/granite-7b-base --branch main --base-bran…
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| INFO 2024-07-05 19:34:55,630 utils.py:145: _init_num_threads Note: detected 80 virtual cores but NumExpr set to maximum of 64, check "NUMEXPR_MAX_THREADS" environment variable. | |
| INFO 2024-07-05 19:34:55,630 utils.py:148: _init_num_threads Note: NumExpr detected 80 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 16. | |
| INFO 2024-07-05 19:34:55,630 utils.py:161: _init_num_threads NumExpr defaulting to 16 threads. | |
| INFO 2024-07-05 19:34:55,789 config.py:58: <module> PyTorch version 2.3.1 available. | |
| Generating questions and reference answers from qna files for branch main... | |
| INFO 2024-07-05 19:35:02,464 vllm.py:148: run_vllm vLLM starting up on pid 212 at http://127.0.0.1:48895/v1 | |
| 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 148/148 [00:00<00:00, 169.53it/s] | |
| generated 416 questions | |
| 100%|█████████████████████████████████████████████████████████████ |
relyt0925
/ mt_bench.log
Created
July 7, 2024 02:52
ilab mt_bench eval log (ilab model evaluate --benchmark mt_bench --model /instructlab/models/tuned-0701-1954/samples_4992 --judge-model /instructlab/models/mistralai/Mixtral-8x7B-Instruct-v0.1 --taxonomy-path /instructlab/taxonomy/ --output-dir /instructlab/mtbench)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| INFO 2024-07-05 18:53:02,883 utils.py:145: _init_num_threads Note: detected 80 virtual cores but NumExpr set to maximum of 64, check "NUMEXPR_MAX_THREADS" environment variable. | |
| INFO 2024-07-05 18:53:02,883 utils.py:148: _init_num_threads Note: NumExpr detected 80 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 16. | |
| INFO 2024-07-05 18:53:02,883 utils.py:161: _init_num_threads NumExpr defaulting to 16 threads. | |
| INFO 2024-07-05 18:53:03,050 config.py:58: <module> PyTorch version 2.3.1 available. | |
| Generating answers... | |
| INFO 2024-07-05 18:53:12,971 vllm.py:148: run_vllm vLLM starting up on pid 212 at http://127.0.0.1:58173/v1 | |
| 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 80/80 [01:02<00:00, 1.27it/s] | |
| Evaluating answers... | |
| INFO 2024-07-05 18:56:08,297 vllm.py:148: run_vllm vLLM starting up on pid 255 at http://127.0.0.1:48517/v1 | |
| 100%|██████████ |
relyt0925
/ mmlubranch.log
Created
July 7, 2024 02:50
mmlu_branch execution log (ilab model evaluate --model models/tuned-0701-1954/samples_4992 --base-model models/ibm/granite-7b-base --benchmark mmlu_branch --sdg-path generated/lm_eval/tasks)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| INFO 2024-07-06 18:00:23,647 utils.py:145: _init_num_threads Note: detected 80 virtual cores but NumExpr set to maximum of 64, check "NUMEXPR_MAX_THREADS" environment variable. | |
| INFO 2024-07-06 18:00:23,647 utils.py:148: _init_num_threads Note: NumExpr detected 80 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 16. | |
| INFO 2024-07-06 18:00:23,647 utils.py:161: _init_num_threads NumExpr defaulting to 16 threads. | |
| INFO 2024-07-06 18:00:23,802 config.py:58: <module> PyTorch version 2.3.1 available. | |
| INFO 2024-07-06 18:00:35,580 evaluator.py:152: simple_evaluate Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | |
| INFO 2024-07-06 18:00:35,580 evaluator.py:189: simple_evaluate Initializing hf model, with arguments: {'pretrained': 'models/tuned-0701-1954/samples_4992', 'dtype': 'bfloat16'} | |
| /usr/local/lib64/python3.11/site-packages/torch/cuda/__init__.py:619: UserWarning: Can't initialize NVML | |
| warnings.warn("Can't initialize NVML") | |
| INFO 2024-07-06 18:00:35,598 hug |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| INFO 2024-07-06 21:21:06,958 utils.py:145: _init_num_threads Note: detected 80 virtual cores but NumExpr set to maximum of 64, check "NUMEXPR_MAX_THREADS" environment variable. | |
| INFO 2024-07-06 21:21:06,958 utils.py:148: _init_num_threads Note: NumExpr detected 80 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 16. | |
| INFO 2024-07-06 21:21:06,958 utils.py:161: _init_num_threads NumExpr defaulting to 16 threads. | |
| INFO 2024-07-06 21:21:07,119 config.py:58: <module> PyTorch version 2.3.1 available. | |
| INFO 2024-07-06 21:21:13,649 evaluator.py:152: simple_evaluate Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | |
| INFO 2024-07-06 21:21:13,650 evaluator.py:189: simple_evaluate Initializing hf model, with arguments: {'pretrained': 'models/tuned-0701-1954/samples_4992', 'dtype': 'bfloat16'} | |
| /usr/local/lib64/python3.11/site-packages/torch/cuda/__init__.py:619: UserWarning: Can't initialize NVML | |
| warnings.warn("Can't initialize NVML") | |
| INFO 2024-07-06 21:21:13,666 hug |
relyt0925
/ sdggeneratefullrun.log
Last active
July 8, 2024 01:42
log file from full run of SDG generate across public taxonomy/skills (look for ilab generate to see the individual SDG generate calls)
This file has been truncated, but you can view the full file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| knowledge/technical_manual/ibm_redbooks/qna.yaml | |
| ilab generate --model-family mixtral --model /instructlab/models/mistralai/Mixtral-8x7B-Instruct-v0.1 --taxonomy-path /instructlab/taxonomytar/taxonomy/knowledge/technical_manual/ibm_redbooks/qna.yaml --output-dir /instructlab/ibmgenerate/taxonomytar/taxonomy/knowledge/technical_manual/ibm_redbooks --num-instructions 100 --endpoint-url http://0.0.0.0:8080/v1 --model /instructlab/models/mistralai/Mixtral-8x7B-Instruct-v0.1 | |
| time="2024-06-26T20:39:05Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" | |
| You are using an aliased command, this will be deprecated in a future release. Please consider using `ilab data generate` instead | |
| Generating synthetic data using '/instructlab/models/mistralai/Mixtral-8x7B-Instruct-v0.1' model, taxonomy:'/instructlab/taxonomytar/taxonomy/knowledge/technical_manual/ibm_redbooks/qna.yaml' against http://0.0.0.0:8080/v1 server | |
| Traceback (most recent call last): | |
| File "/usr/loca |
This file has been truncated, but you can view the full file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| nohup: ignoring input | |
| Converting /instructlab/generated/train_combinedknowlegeskills.jsonl | |
| Converting /instructlab/generated/test_combinedknowlegeskills.jsonl | |
| [16:39:00] INFO !!!!!!!! tokenizer has add_bos_token or utils.py:192 | |
| add_eos_token | |
| INFO eos: 32000, pad: 32001, system: 32002, user: utils.py:192 | |
| 32003, assistant: 32004 | |
| Generating train split: 32962 examples [00:00, 295454.57 examples/s] | |
| removing pretraining samples system msg | |
| Map (num_proc=72): 100%|██████████| 32962/32962 [00:00<00:00, 71731.50 examples/s] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| {"system": "You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.", "user": "Explain the formula of finding the area of a circle?", "assistant": "The formula for finding the area of a circle is πr², where π is a mathematical constant approximately equal to 3.14159 and r represents the radius of the circle. It is calculated as the product of π and the radius of the circle square. The radius of a circle is any line segment that passes through the center and terminates at the circumference of the circle. Let me know if you have any other questions or need assistance with something else."} | |
| {"system": "You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.", "user": "How to find the area of a circle |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| {"question_id": 81, "answer_id": "fTrzjmxaRqHjuA4uThwGG6", "model_id": "test_model", "choices": [{"index": 0, "turns": ["\nAloha, fellow wanderers! I've just returned from a tropical paradise that has left me with memories to last a lifetime - the enchanting islands of Hawaii. This archipelago in the Pacific Ocean is not only known for its stunning beaches and lush landscapes but also its rich culture and history.\n\nMy journey began on the island of Oahu, home to the bustling city of Honolulu. Here, I immersed myself in native Hawaiian culture at the Polynesian Cultural Center. The center showcases various Polynesian islands through interactive exhibits, traditional music, dance, and craft demonstrations. I participated in a hula lesson, learned about the art of lei-making, and even tried my hand at throwing a spear! \n\nNo trip to Oahu would be complete without visiting the historic Pearl Harbor. The USS Arizona Memorial was a humbling experience, providing an insight into the events leading up to America's |
relyt0925
/ judge_model_single.jsonl
Last active
July 7, 2024 02:22
mt_bench judge_model_single.jsonl example
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| {"question_id": 150, "model": "test_model", "judge": ["judge_model", "single-v1-multi-turn"], "user_prompt": "<|The Start of Assistant A's Conversation with User|>\n\n### User:\nHow have the Alps and Rhine River influenced settlement and agriculture in Western Europe? List three impacts.\n\n### Assistant A:\n\n1. Formation of distinct cultural regions: The Alps and the Rhine River have contributed to the formation of distinct cultural regions in Western Europe. The Alps, with their steep slopes and harsh climate, have traditionally been home to pastoral communities, while the fertile plains along the Rhine River have been ideal for agriculture. This has led to the development of unique cultural practices, languages, and cuisines in different regions.\n\n2. Trade and transportation: The Rhine River has been a crucial trade route since ancient times, connecting the Mediterranean region with the North Sea. Its navigability has facilitated the exchange of goods, ideas, and people between various parts of Western |
NewerOlder