gistfile1.txt

Charles Frye      19:09
How do i make sure the template and tokenization strategy I use in my axolotl training is exactly matched in my production server? Assuming I use something like vllm, sglang, trt-llm+trtion.
This question has been answered live

rensdimmendaal      19:05
can you show how DPO works in axolotl?
This question has been answered live

Sanyam Bhutani      19:07
Dumb Q: My understanding is Conversational Datasets are most effective for Fine Tuning? Any tips on setting these up or can we get good enough perf with just QA Pairs?
This question has been answered live

Dylan      19:12
Can we get some concrete examples for training dataset of finetuning for 

1) Completion model
2) Instruction model
3) Multiturn chat model

Thank you!
This question has been answered live

Sean Bergman      19:05
If I want a fine-tuned llama3 that works smoothly with a library like llama-index, which is tightly integrated with openAI template, which prompt template name should be used in axolotl? In this case, I could also also feed it the jsonl I use to train gpt-3.5 without any modifications.
This question has been answered live

Anonymous attendee      19:31
Hamel has a blog post on almost everything. Its great.

Charles Frye      19:46
<3

Dylan      19:32
can we just get a document with the different types of common use finetuning dataset template for

1) completion
2) instruction 
3) chat

that would help a lot!
This question has been answered live

Ankur Singh      19:35
Any comments on Axolotl vs Unsloth? The performance as well as memory gains with Unsloth are quite significant.

Ben Eyal      18:59
I was wondering about Template-free prompt construction, I really didn't understand how it works. The config only needs an output, and the example shows something about masking tokens? Where does the input-output come into play? I got so lost there :-)
This question has been answered live

Kranthi Kiran GV      19:02
Can you talk about the future directions of axolotl? What areas do you need contributors help in?
This question has been answered live

Artem Dinaburg      19:02
Is there some way/tool to estimate VRAM usage given an axolotl config?
This question has been answered live

Venetis Pallikaras      19:02
What is the difference between the chat_template and datasets:type parameters? To me it seems they are both achieving the same thing
This question has been answered live

Wayde Gillliam      19:06
What are your best practices for formulating and verifying prompt templates in the config and also where to look in the code to see where the inputs and outputs are being put together for debugging.
This question has been answered live

Anand Narayan      19:10
while training, can you pause. run an inference to see vibes, and they restart the training from where you paused?
This question has been answered live

Wayde Gillliam      19:20
I would assume for function calling finetunes you need to be able to specify a custom format … is that correct?  

Any examples?

Artem Dinaburg      19:00
I asked something similar in the discord, but is there a way to do a training run that is an effective no-op to validate that saving/loading the base model doesn’t have any unexpected side effects?
This question has been answered live

Sridhar Iyer      19:01
Does Axolotl support training of multimodal LLM - can i finetune PaliGemma or Phi-3-Vision? (any sample dataset - yaml file) thanks.
This question has been answered live

Mircea Ene      19:05
How would you structure your dataset for a chatbot application if you wanted to use DPO?
What I understood from yesterday’s conference there’s no conversational (sharegpt) that also handles DPO (preferred vs rejected).
So in order to use DPO on a chatbot for example would structuring the dataset like:

User: Input Text

LLM: Question 1?
User: Answer 1

LLM: Question 2?
User: Answer 2

Prefered: Prefered_Question
Rejected: Rejected_Question

*We want our chatbot to ask the questions.
This question has been answered live

Wayde Gillliam      19:13
What is the best way to train a quick and dirty fine-tune and then verify your model works at inference time as expected?

kian      19:20
If you have a conversational dataset, that don't have a system role, and the model does use system role, is there any base system role you recommend.

And for the other way round, when the model doesn't have a system role.
Do you do something of the sort.
User: {System Prompt}
assitant: Understood.
--- remaining conversation ---

Jose Pozuelo      19:28
Is there any configuration or Data set shape that allows you to “add” context/documentation to a model. I know the previouse lectures have explicitly said not to focus on this, but I think it’s a very common usecase. 

Do you ‘have’ to convert it to question answer pairs, or is there a data shape where you can add the documentation as if it was part of the base training, without overfitting to it?

Anonymous attendee      19:59
When would you fine tune on the cloud versus fine tuning on a local PC with 2x 4090 GPUs, assuming cost is not an issue? What kind of models would you try to fine tune using 2x 4090s ?

Chris Levy      19:08
To follow up on Wayde Gillliam’s question, Not just looking at the processed dataset. But being able to see what is used for an actual train step. GIven the batch size, sample packing etc.

Dylan      19:09
Do we have any examples with Axolotl to fine tune vision models?  For example Llava models?

Sanyam Bhutani      19:09
Which bits of axolotl do you think are super early right now?

keith      19:12
Is it possible/practical to start with a template-free approach to learn a particular tone and vocabulary, and then follow that up with conventional chat tuning?

I am thinking like training a chatbot on the works of an author, to pick up that author’s tone, but then also be able to answer questions and engage in chat.

Cian P      19:26
I thought if finetuning a LLAMA model we had to use LLAMA template, the same structure that the base model was trained on? Or we can use any template?

Selva      19:36
I want to master finetuning. 

Can you suggest a challenge I can embark on?   
- finetune a small benchmark and compete in a benchmark ranking (please suggest)
- competitions

Charles Frye      20:01
Try fine-tuning just on completion first. You can take literally _any_ text dataset and run this. Try to get the loss on a held-out dataset as low as possible.

Ideally, it's a text dataset that is continually updated, e.g. social media feeds, so you can compare hold-out loss to loss on data generated _after_ training.

vincentwarmerdam      19:44
What’s the most suprising thing you’ve learned while working on Axocotl that you did not expect to learn … but did?

Artem Dinaburg      19:01
Also, what is the difference between the “pretrain” data type and the “completion” data type?

Nehil      19:03
Where can I find great fine-tine examples like Honeycomb example? Are there hugging face datasets that people commonly use?

Artem Dinaburg      19:03
When should we fill out the “special_tokens” or “tokens” portions of the config?

Mark      19:03
is there a way I can specificy in the yaml so I can remember to login to my huggingface account when models are restricted and I need remember to login?

Nehil      19:05
@wing what are some of the challenges you see people facing when using axolotl? I feel Hamel and Dan made it so easy for us but I am have a feeling its not the same as a whole for everyone.

Anonymous attendee      19:14
are these DPO examples manually created?

cjameyson      19:24
Are there any important learnings or tricks to know when your task is purely a 'schema extraction' from semi-structured documents (like utility bills, invoices)?

Sean Bergman      19:28
When using qlora what is the best optimizer to use if needing to reduce vram usage? The example templates for llama3 uses a paged_adamw_32bit optimizer. I changed it to adam_bnb_8bit to try to reduce vram, but could use some guidance on that.

Anonymous attendee      19:38
is that best practice to create your own prompt template vs reuse someone else’s? responding to hamel’s comment

Cian P      19:41
Any advice on raising PRs on GitHub in relation to learning.

Charles Frye      19:46
I'd advise looking for open issues that are tagged as "good first issue" to see what work is needed. Docs PRs are also lower stakes.

And remember that the folks on the other side are almost always volunteers.

Anonymous attendee      19:52
Is there a way to add to vocab during finetuning a base model. For eg, I want it to recognize Acrononyms in Health care domain like medical codes. Appreciate any pointers.

Avi      19:02
Can RLHF is still a common fine tuning technique?
This question has been answered live

Sridhar Iyer      19:02
before we started, just wanted to ask if you have any sample files for PaliGemma / PHi-3 finetuning. thanks
This question has been answered live

laith      19:06
can you comment on custom datasets inside dharma for in-train evals? i saw a callback class or two maybe for mlflow but not sure how hackable the callback is.thanks!

Anthony      19:07
Are there plans for axolotl to support other model_type beside CausalLM?

Dylan      19:10
For people finetuning on their own GPU cards on a window system; do you use powershell? When finetuning, do you turn off all other applications to give maximum vram to the GPU?

rensdimmendaal      19:19
does it matter which dataset template you use for which base model?

Ataliba Miguel      19:40
I already fine-tuned some sort of spam dataset however it was on my own dataset, it produced a model size of 1.4Gb as it was fine-tuned on gpt2-medium. The model is now uploaded on my huggingface and I am to have it run on simple streamlit app. Any goby example on that?

Raqib Hayder      19:48
Do you finetune one task per model? If not, how does one group tasks?

EkShunya      19:49
is there a way to know who is the speaker in zoom as people pitch in and out ?

a. mishra      19:52
If the dataset does not fit any of the dataset format, then is the way to use axolotl is to tokenize it first and then forward it in the yaml, as per: https://openaccess-ai-collective.github.io/axolotl/docs/dataset-formats/tokenized.html

Ashita Achuthan      19:56
running list of zoom questions for this Axolotl workshop - 

https://docs.google.com/document/d/1944izw_gwWq9EuaZcNN5lQwiVOKaIYXbYIKi6e9Efsw/edit?usp=sharing

Dylan      20:02
For 2x 4090, you can try finetuning up to 7b models

a. mishra      18:59
How to set/decide data type for data sets on huggingface
This question has been answered live

Rubén      19:01
Can you talk about the limitations of the RL / DPO configurations ?
This question has been answered live

Sahar Nesaei      19:02
if the number does not excced, maybe it is possible.

Avi      19:04
Can LLM be used  for use cases that primarily use transaction information?

Sahar Nesaei      19:04
what is wrong with this command: oot@1b9c02068f36:~/axolotl# accelerate launch -m axolotl.cli.inference examples/tiny-llama/simple-lora.yml     --lora_model_dir="./simple-lora-out" --gradio

Nehil      19:09
@wing are you an avid runner? Love Nike Run Club ;)

Sebastian      19:16
What about the dataset for SFT and DPO/KTO/... . Can I use the chosen for the SFT and then do DPO/KTO with the SAME chosen but also not chose?

Jeffery Lovely      19:17
Any specific changes with IPO  vs DPO?

Raqib Hayder      19:34
I have the same sentiment as @Hamel. I do it myself. All this shit makes it difficult because debugging other people’s stuff is harder than debugging my own stuff.

Anonymous attendee      19:48
how do you finetune on openai models?

Jose Pozuelo      19:48
What are the best resources on dataset shapes for different usecases? It would be great to have a cheatsheet with an example and a config that could be used for that usecase

Anonymous attendee      19:51
I exported https://gist.github.com/strickvl/e1591b83e3b290fb176e780e7ce7d383

Dylan      19:51
We need screenshot and pass it to gpt4-o output in json

Anonymous attendee      19:51
Is there a way to use axolotl for a classification dataset?