Row 48515

Row ID: 48515 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 48515 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

So happy to see other people doing this too!

Llama 3 has been incredible in terms of perplexity at ridiculously low resource cost. I run the iQ3 model for some 7b finetune I have and it runs really well on a ryzen 7940HS. It is a bit *slow,* (10 toks/s), but I have it running off of a battery pack and it only pulls about 75w at max load.

My goal has been to minimize the energy usage of AI rather than maximize it but still maintain its key characteristics (knowledgeable, conversational, coherent). So far I can run this AI for 2-3 days nonstop generating on a 70,000mah/100watt battery pack. The whole system costs under $1k USD but its definitely not worth it yet.

Hopefully soon! The better these lower param models get, the easier my work gets. Cannot wait for llama 4 or whatever competitor takes its place.

FieldValue
text So happy to see other people doing this too! Llama 3 has been incredible in terms of perplexity at ridiculously low resource cost. I run the iQ3 model for some 7b finetune I have and it runs really well on a ryzen 7940HS. It is a bit *slow,* (10 toks/s), but I have it running off of a battery pack and it only pulls about 75w at max load. My goal has been to minimize the energy usage of AI rather than maximize it but still maintain its key characteristics (knowledgeable, conversational, coher…
label r/openai
dataType comment
communityName r/OpenAI
datetime 2024-05-22
username_encoded Z0FBQUFBQm5Lak1SOVdlVFJieUtuLVNmUm1ISC1QaXN1M0QwVFdETVJWaWtGZzZlUmhVcVVpQmRQdWwtbTAyam5vRkJJZm5JdzJGX256WmJBeFB2Tkx6X0xnY0V1LUM2eGc9PQ==
url_encoded Z0FBQUFBQm5Lak9oV2Rkd3B1RjcyX1VCdXVzalphUXo0MXhtZ21HanAwdlNJZmtILS1ScjdmWEMzS2dVdEdYVU1zMlZaUnlBaDJVU2xMX05ZYTlwVE1PSjdSY2cyaXBOdHVNMkVVbkhzbHdDb2s4dk5FdWVRa2R3ZF90S1pYQmJMbjdTazh5NFpIT2pKbF9xRnJwdlZ5Q05aOWxzbFY5TlRTUlVqbkhfZmJ6cUVSN2hfcTgybmR0dnNNQ3Z5N1JqbE1tYWVxcGxnYmxQNmt0UFJNWGNaQ0V6QVg4STAzS2o3dz09

Raw Record

{
  "text": "So happy to see other people doing this too! \n\nLlama 3 has been incredible in terms of perplexity at ridiculously low resource cost. I run the iQ3 model for some 7b finetune I have and it runs really well on a ryzen 7940HS. It is a bit *slow,* (10 toks/s), but I have it running off of a battery pack and it only pulls about 75w at max load. \n\nMy goal has been to minimize the energy usage of AI rather than maximize it but still maintain its key characteristics (knowledgeable, conversational, coherent). So far I can run this AI for 2-3 days nonstop generating on a 70,000mah/100watt battery pack. The whole system costs under $1k USD but its definitely not worth it yet.\n\nHopefully soon! The better these lower param models get, the easier my work gets. Cannot wait for llama 4 or whatever competitor takes its place.",
  "label": "r/openai",
  "dataType": "comment",
  "communityName": "r/OpenAI",
  "datetime": "2024-05-22",
  "username_encoded": "Z0FBQUFBQm5Lak1SOVdlVFJieUtuLVNmUm1ISC1QaXN1M0QwVFdETVJWaWtGZzZlUmhVcVVpQmRQdWwtbTAyam5vRkJJZm5JdzJGX256WmJBeFB2Tkx6X0xnY0V1LUM2eGc9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9oV2Rkd3B1RjcyX1VCdXVzalphUXo0MXhtZ21HanAwdlNJZmtILS1ScjdmWEMzS2dVdEdYVU1zMlZaUnlBaDJVU2xMX05ZYTlwVE1PSjdSY2cyaXBOdHVNMkVVbkhzbHdDb2s4dk5FdWVRa2R3ZF90S1pYQmJMbjdTazh5NFpIT2pKbF9xRnJwdlZ5Q05aOWxzbFY5TlRTUlVqbkhfZmJ6cUVSN2hfcTgybmR0dnNNQ3Z5N1JqbE1tYWVxcGxnYmxQNmt0UFJNWGNaQ0V6QVg4STAzS2o3dz09"
}
Explore Dataset Explore Row

Entry Information