Row 34594

Row ID: 34594 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 34594 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

Hi everyone. This is my first Reddit post, so please pardon if I am using some wrong format.

I want to learn about Fine Tuning LLMs such as LLama-2, Gemma models, Llama-3. I have read plenty of medium articles and have been exposed to plenty of new topics such as LoRA, QLoRA, PEFT, etc. I have tried my best to understand them. But while implementing the code by following along the medium articles, I sometimes come across numerous parameters that are initialized without explanation which is overwhelming sometimes. I have also been exposed to Unsloth. I want to know where can I begin from. Should I begin with understanding how to fine-tune small models such as BERT, T5, etc using PyTorch and Transformers library? If you would like to provide some roadmap it would be really helpful. Thank you for your time.

I am attaching an example of the code I struggled to understand. How does one know which parameters are supposed to be used and which not?

model_name = "NousResearch/Llama-2-7b-chat-hf" dataset_name = "mlabonne/guanaco-llama2-1k" new_model = "Llama-2-7b-chat-finetune" # QLoRA parameters lora_r = 64 lora_alpha = 16 lora_dropout = 0.1 #bitsandbytes parameters #activate 4-bit precision base model loading use_4bit = True #compute dtype for 4-bit base models bnb_4bit_compute_dtype = "float16" #quantization type fp4 or nf4 bnb_4bit_quant_type = "nf4" #activate nested quantization for 4 bit base models use_nested_quant = False #training parameters #output dir output_dir = "./results" #no. of training epochs num_train_epochs = 1 fp16 = False bf16 = False per_device_train_batch_size = 4 per_device_eval_batch_size = 4 gradient_accumulation_steps = 1 gradient_checkpointing = True max_grad_norm = 0.3 learning_rate = 2e-4 weight_decay = 0.001 optim = "paged_adamw_32bit" lr_scheduler_type = "cosine" max_steps = -1 warmup_ratio = 0.03 group_by_length = True save_steps = 0 logging_steps = 25 # SFT Parameters # Maximum sequence length to use max_seq_length = None # Pack multiple short examples in the same input sequence to increase efficiency packing = False # Load the entire model on the GPU 0 device_map = {"": 0}

# Load dataset (you can process it here) dataset = load_dataset(dataset_name, split="train") # Load tokenizer and model with QLoRA configuration compute_dtype = getattr(torch, bnb_4bit_compute_dtype) bnb_config = BitsAndBytesConfig(     load_in_4bit=use_4bit,     bnb_4bit_quant_type=bnb_4bit_quant_type,     bnb_4bit_compute_dtype=compute_dtype,     bnb_4bit_use_double_quant=use_nested_quant, ) # Check GPU compatibility with bfloat16 if compute_dtype == torch.float16 and use_4bit:     major, _ = torch.cuda.get_device_capability()     if major >= 8:         print("=" * 80)         print("Your GPU supports bfloat16: accelerate training with bf16=True")         print("=" * 80) # Load base model model = AutoModelForCausalLM.from_pretrained(     model_name,     quantization_config=bnb_config,     device_map=device_map ) model.config.use_cache = False model.config.pretraining_tp = 1 # Load LLaMA tokenizer tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True) tokenizer.pad_token = tokenizer.eos_token tokenizer.padding_side = "right" # Fix weird overflow issue with fp16 training # Load LoRA configuration peft_config = LoraConfig(     lora_alpha=lora_alpha,     lora_dropout=lora_dropout,     r=lora_r,     bias="none",     task_type="CAUSAL_LM", ) # Set training parameters training_arguments = TrainingArguments(     output_dir=output_dir,     num_train_epochs=num_train_epochs,     per_device_train_batch_size=per_device_train_batch_size,     gradient_accumulation_steps=gradient_accumulation_steps,     optim=optim,     save_steps=save_steps,     logging_steps=logging_steps,     learning_rate=learning_rate,     weight_decay=weight_decay,     fp16=fp16,     bf16=bf16,     max_grad_norm=max_grad_norm,     max_steps=max_steps,     warmup_ratio=warmup_ratio,     group_by_length=group_by_length,     lr_scheduler_type=lr_scheduler_type,     report_to="tensorboard" ) # Set supervised fine-tuning parameters trainer = SFTTrainer(     model=model,     train_dataset=dataset,     peft_config=peft_config,     dataset_text_field="text",     max_seq_length=max_seq_length,     tokenizer=tokenizer,     args=training_arguments,     packing=packing, ) # Train model trainer.train()

FieldValue
text Hi everyone. This is my first Reddit post, so please pardon if I am using some wrong format. I want to learn about Fine Tuning LLMs such as LLama-2, Gemma models, Llama-3. I have read plenty of medium articles and have been exposed to plenty of new topics such as LoRA, QLoRA, PEFT, etc. I have tried my best to understand them. But while implementing the code by following along the medium articles, I sometimes come across numerous parameters that are initialized without explanation which is ove…
label r/deeplearning
dataType post
communityName r/deeplearning
datetime 2024-05-21
username_encoded Z0FBQUFBQm5Lak1JenNNb2lyUWs5RGM4aDFBOUhXVEx4MHFJQlZRR2VYYTY3emZvdXc2WkgxdHFkTC1rczZRVnFjNGN1TXdZZmJqQVh6aWlfSlBpUUcwLVhLeU5neFBVem03UmJBNjdTX21oMHZMN1ZpVW40bGM9
url_encoded Z0FBQUFBQm5Lak9YVl9ISWdLaEVieS1kN3Bpa3dJU2JiLVhXNF9CUGdZZU5WVFVGS0RWMnl1R0xfTUtWQTFNZG1ObTl1eTZFai1tZ2NmY1l6R09meGQtVTJicm5LQXRJaVF6bUFmOXJmWVo0RDJPZkJybVM0T3Rnb3ZwYzJ0eV9LeTZYVXBnZDJfYTVZdWgxbUU0czd4MHc0TDNXQU5fNDc0WmlkOEt5UURXdkVybE5HN01fWTRSdVdZVW9zX2JqQ3JmUVJYU21OYjg5

Raw Record

{
  "text": "Hi everyone. This is my first Reddit post, so please pardon if I am using some wrong format. \n\nI want to learn about Fine Tuning LLMs such as LLama-2, Gemma models, Llama-3. I have read plenty of medium articles and have been exposed to plenty of new topics such as LoRA, QLoRA, PEFT, etc. I have tried my best to understand them. But while implementing the code by following along the medium articles, I sometimes come across numerous parameters that are initialized without explanation which is overwhelming sometimes. I have also been exposed to Unsloth. I want to know where can I begin from. Should I begin with understanding how to fine-tune small models such as BERT, T5, etc using PyTorch and Transformers library? If you would like to provide some roadmap it would be really helpful. Thank you for your time.\n\nI am attaching an example of the code I struggled to understand. How does one know which parameters are supposed to be used and which not?\n\n    model_name = \"NousResearch/Llama-2-7b-chat-hf\"\n    \n    dataset_name = \"mlabonne/guanaco-llama2-1k\"\n    \n    new_model = \"Llama-2-7b-chat-finetune\"\n    \n    # QLoRA parameters\n    \n    lora_r = 64\n    \n    lora_alpha = 16\n    \n    lora_dropout = 0.1\n    \n    #bitsandbytes parameters\n    \n    #activate 4-bit precision base model loading\n    use_4bit = True\n    \n    #compute dtype for 4-bit base models\n    bnb_4bit_compute_dtype = \"float16\"\n    \n    #quantization type fp4 or nf4\n    bnb_4bit_quant_type = \"nf4\"\n    \n    #activate nested quantization for 4 bit base models\n    use_nested_quant = False\n    \n    #training parameters\n    \n    #output dir\n    output_dir = \"./results\"\n    \n    #no. of training epochs\n    num_train_epochs = 1\n    \n    fp16 = False\n    bf16 = False\n    \n    per_device_train_batch_size = 4\n    \n    per_device_eval_batch_size = 4\n    \n    gradient_accumulation_steps = 1\n    \n    gradient_checkpointing = True\n    \n    max_grad_norm = 0.3\n    \n    learning_rate = 2e-4\n    \n    weight_decay = 0.001\n    \n    optim = \"paged_adamw_32bit\"\n    \n    lr_scheduler_type = \"cosine\"\n    \n    max_steps = -1\n    warmup_ratio = 0.03\n    \n    group_by_length = True\n    \n    save_steps = 0\n    \n    logging_steps = 25\n    \n    \n    # SFT Parameters\n    # Maximum sequence length to use\n    max_seq_length = None\n    \n    # Pack multiple short examples in the same input sequence to increase efficiency\n    packing = False\n    \n    # Load the entire model on the GPU 0\n    device_map = {\"\": 0}\n\n    # Load dataset (you can process it here)\n    dataset = load_dataset(dataset_name, split=\"train\")\n    \n    # Load tokenizer and model with QLoRA configuration\n    compute_dtype = getattr(torch, bnb_4bit_compute_dtype)\n    \n    bnb_config = BitsAndBytesConfig(\n        load_in_4bit=use_4bit,\n        bnb_4bit_quant_type=bnb_4bit_quant_type,\n        bnb_4bit_compute_dtype=compute_dtype,\n        bnb_4bit_use_double_quant=use_nested_quant,\n    )\n    \n    # Check GPU compatibility with bfloat16\n    if compute_dtype == torch.float16 and use_4bit:\n        major, _ = torch.cuda.get_device_capability()\n        if major >= 8:\n            print(\"=\" * 80)\n            print(\"Your GPU supports bfloat16: accelerate training with bf16=True\")\n            print(\"=\" * 80)\n    \n    # Load base model\n    model = AutoModelForCausalLM.from_pretrained(\n        model_name,\n        quantization_config=bnb_config,\n        device_map=device_map\n    )\n    model.config.use_cache = False\n    model.config.pretraining_tp = 1\n    \n    # Load LLaMA tokenizer\n    tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)\n    tokenizer.pad_token = tokenizer.eos_token\n    tokenizer.padding_side = \"right\" # Fix weird overflow issue with fp16 training\n    \n    # Load LoRA configuration\n    peft_config = LoraConfig(\n        lora_alpha=lora_alpha,\n        lora_dropout=lora_dropout,\n        r=lora_r,\n        bias=\"none\",\n        task_type=\"CAUSAL_LM\",\n    )\n    \n    # Set training parameters\n    training_arguments = TrainingArguments(\n        output_dir=output_dir,\n        num_train_epochs=num_train_epochs,\n        per_device_train_batch_size=per_device_train_batch_size,\n        gradient_accumulation_steps=gradient_accumulation_steps,\n        optim=optim,\n        save_steps=save_steps,\n        logging_steps=logging_steps,\n        learning_rate=learning_rate,\n        weight_decay=weight_decay,\n        fp16=fp16,\n        bf16=bf16,\n        max_grad_norm=max_grad_norm,\n        max_steps=max_steps,\n        warmup_ratio=warmup_ratio,\n        group_by_length=group_by_length,\n        lr_scheduler_type=lr_scheduler_type,\n        report_to=\"tensorboard\"\n    )\n    \n    # Set supervised fine-tuning parameters\n    trainer = SFTTrainer(\n        model=model,\n        train_dataset=dataset,\n        peft_config=peft_config,\n        dataset_text_field=\"text\",\n        max_seq_length=max_seq_length,\n        tokenizer=tokenizer,\n        args=training_arguments,\n        packing=packing,\n    )\n    \n    # Train model\n    trainer.train()\n    ",
  "label": "r/deeplearning",
  "dataType": "post",
  "communityName": "r/deeplearning",
  "datetime": "2024-05-21",
  "username_encoded": "Z0FBQUFBQm5Lak1JenNNb2lyUWs5RGM4aDFBOUhXVEx4MHFJQlZRR2VYYTY3emZvdXc2WkgxdHFkTC1rczZRVnFjNGN1TXdZZmJqQVh6aWlfSlBpUUcwLVhLeU5neFBVem03UmJBNjdTX21oMHZMN1ZpVW40bGM9",
  "url_encoded": "Z0FBQUFBQm5Lak9YVl9ISWdLaEVieS1kN3Bpa3dJU2JiLVhXNF9CUGdZZU5WVFVGS0RWMnl1R0xfTUtWQTFNZG1ObTl1eTZFai1tZ2NmY1l6R09meGQtVTJicm5LQXRJaVF6bUFmOXJmWVo0RDJPZkJybVM0T3Rnb3ZwYzJ0eV9LeTZYVXBnZDJfYTVZdWgxbUU0czd4MHc0TDNXQU5fNDc0WmlkOEt5UURXdkVybE5HN01fWTRSdVdZVW9zX2JqQ3JmUVJYU21OYjg5"
}
Explore Dataset Explore Row

Entry Information