Row 76349

Row ID: 76349 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 76349 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

# Hi everyone,

I'm currently working on a project where I need to run inferences using various large language models (LLMs) on a medium-sized dataset. The dataset consists of two types of inputs:

1. **Big Questions**: These are scenario-type questions with a lot of words. 2. **Multiple Statements**: Each question is followed by multiple statements that can be either true or false.

Here's my setup:

* **Operating System**: Windows Server 2019, RAM:512GB (GPU not enabled due to Tensorflow incompatiblity) * **Programming Language**: Python

I’m looking for advice on the following points:

1. **Best Practices**: What are the best practices for running LLM inferences on such a dataset? Any tips to ensure accuracy and efficiency? 2. **Python Libraries/Tools**: Which Python libraries or tools would you recommend for managing and running these inferences? I’m currently considering using libraries like Hugging Face's Transformers. 3. **Optimization Techniques**: Are there any specific techniques to optimize the performance of LLMs on a Windows Server environment? 4. **Handling Large Texts**: How should I handle the large input texts efficiently? Are there any specific preprocessing steps or configurations I should be aware of? 5. **Comparing Results**: What metrics or methods would be best to compare the inference results from different LLMs effectively?

I appreciate any insights or experiences you can share. Thanks in advance!

>As of now I have tried MLFLOW to compare small LLM like Falcon-7B, Hermes, Orca, LLama3. But I do not get good accuracy. But when I use the same models in GPT4all, I get good responses with justification.

Sample code:

# Concatenate the question and statement input_text = question + " " +"tell if the following statement is true or false:" + statement print(input_text) generated_answer = llm(input_text, max_length=1024)[0]['generated_text'] print(generated_answer)

Example Question: (I have many such questions: I am using JSON format)

>The Process of Photosynthesis:

>Photosynthesis is the process by which green plants, algae, and some bacteria convert light energy, usually from the sun, into chemical energy stored in glucose. During photosynthesis, plants take in carbon dioxide (CO2) and water (H2O) and, using sunlight, produce glucose (C6H12O6) and oxygen (O2). This process is crucial for the survival of plants and for providing oxygen to the Earth's atmosphere.

>For each of the statements 1-4, indicate on the answer sheet whether the statement is true or false:

>

>2. Oxygen is a waste product of photosynthesis and is harmful to plants.

>3. Carbon dioxide and water are the raw materials used in the process of photosynthesis.

>4. Photosynthesis occurs in animals as well as plants.

FieldValue
text # Hi everyone, I'm currently working on a project where I need to run inferences using various large language models (LLMs) on a medium-sized dataset. The dataset consists of two types of inputs: 1. **Big Questions**: These are scenario-type questions with a lot of words. 2. **Multiple Statements**: Each question is followed by multiple statements that can be either true or false. Here's my setup: * **Operating System**: Windows Server 2019, RAM:512GB (GPU not enabled due to Tensorflow incom…
label r/gpt4
dataType post
communityName r/GPT4
datetime 2024-05-24
username_encoded Z0FBQUFBQm5Lak1peGxlQW1KajhPanBEQkNyRUhWSlB5QzNNcnVsZnRBemxwU3BOb3Y0SVZZQm5LTl9HeVhJVXo0dVNtMnVXRFI3TzQ2T1ZqUHo0NFh4eExZejZGMzdSWG55bW1QODZaeERPQzV6aVYzV3llZFU9
url_encoded Z0FBQUFBQm5Lak96OFoyUk5UbjNGN2dQY2dfdUEzdjhWSHIydjh1UGl3dXFSUGZ5aVhaZzVoQkJVN1AtQTNCM3NfYm9ZcFA0MzRCT0xEbkVwMnhPbTVIdUR1SGNrYlZPeHZveFEweExhb0hDUlREOUVKaVZ5TUthT1RKMEpFVlJuM0ZoU0l2UXgtNW15TDJycGVfMUVDallrSEFkTXZyVlFuNHNGQ3N5US1IVWs4UEhVVXdaS2k0dDN2SWdwWnBNMEJXckY4WWZsRGRr

Raw Record

{
  "text": "# Hi everyone,\n\nI'm currently working on a project where I need to run inferences using various large language models (LLMs) on a medium-sized dataset. The dataset consists of two types of inputs:\n\n1. **Big Questions**: These are scenario-type questions with a lot of words.\n2. **Multiple Statements**: Each question is followed by multiple statements that can be either true or false.\n\nHere's my setup:\n\n* **Operating System**: Windows Server 2019, RAM:512GB (GPU not enabled due to Tensorflow incompatiblity)\n* **Programming Language**: Python\n\nI’m looking for advice on the following points:\n\n1. **Best Practices**: What are the best practices for running LLM inferences on such a dataset? Any tips to ensure accuracy and efficiency?\n2. **Python Libraries/Tools**: Which Python libraries or tools would you recommend for managing and running these inferences? I’m currently considering using libraries like Hugging Face's Transformers.\n3. **Optimization Techniques**: Are there any specific techniques to optimize the performance of LLMs on a Windows Server environment?\n4. **Handling Large Texts**: How should I handle the large input texts efficiently? Are there any specific preprocessing steps or configurations I should be aware of?\n5. **Comparing Results**: What metrics or methods would be best to compare the inference results from different LLMs effectively?\n\nI appreciate any insights or experiences you can share. Thanks in advance!\n\n>As of now I have tried MLFLOW to compare small LLM like Falcon-7B, Hermes, Orca, LLama3. But I do not get good accuracy. But when I use the same models in GPT4all, I get good responses with justification.\n\nSample code:\n\n    # Concatenate the question and statement\n                input_text = question + \" \" +\"tell if the following statement is true or false:\" + statement   \n                print(input_text)\n                generated_answer = llm(input_text, max_length=1024)[0]['generated_text']\n                print(generated_answer)\n\nExample Question: (I have many such questions: I am using JSON format)\n\n>The Process of Photosynthesis:\n\n>Photosynthesis is the process by which green plants, algae, and some bacteria convert light energy, usually from the sun, into chemical energy stored in glucose. During photosynthesis, plants take in carbon dioxide (CO2) and water (H2O) and, using sunlight, produce glucose (C6H12O6) and oxygen (O2). This process is crucial for the survival of plants and for providing oxygen to the Earth's atmosphere.\n\n>For each of the statements 1-4, indicate on the answer sheet whether the statement is true or false:\n\n>\n\n>2. Oxygen is a waste product of photosynthesis and is harmful to plants.\n\n>3. Carbon dioxide and water are the raw materials used in the process of photosynthesis.\n\n>4. Photosynthesis occurs in animals as well as plants.",
  "label": "r/gpt4",
  "dataType": "post",
  "communityName": "r/GPT4",
  "datetime": "2024-05-24",
  "username_encoded": "Z0FBQUFBQm5Lak1peGxlQW1KajhPanBEQkNyRUhWSlB5QzNNcnVsZnRBemxwU3BOb3Y0SVZZQm5LTl9HeVhJVXo0dVNtMnVXRFI3TzQ2T1ZqUHo0NFh4eExZejZGMzdSWG55bW1QODZaeERPQzV6aVYzV3llZFU9",
  "url_encoded": "Z0FBQUFBQm5Lak96OFoyUk5UbjNGN2dQY2dfdUEzdjhWSHIydjh1UGl3dXFSUGZ5aVhaZzVoQkJVN1AtQTNCM3NfYm9ZcFA0MzRCT0xEbkVwMnhPbTVIdUR1SGNrYlZPeHZveFEweExhb0hDUlREOUVKaVZ5TUthT1RKMEpFVlJuM0ZoU0l2UXgtNW15TDJycGVfMUVDallrSEFkTXZyVlFuNHNGQ3N5US1IVWs4UEhVVXdaS2k0dDN2SWdwWnBNMEJXckY4WWZsRGRr"
}
Explore Dataset Explore Row

Entry Information