Row 7926

Row ID: 7926 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 7926 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

This paper KAN: Kolmogorov–Arnold Networks [https://arxiv.org/pdf/2404.19756](https://arxiv.org/pdf/2404.19756) (digested in [https://towardsdatascience.com/kolmogorov-arnold-networks-the-latest-advance-in-neural-networks-simply-explained-f083cf994a85](https://towardsdatascience.com/kolmogorov-arnold-networks-the-latest-advance-in-neural-networks-simply-explained-f083cf994a85) ) suggests that any 2-hidden-layer MLP layers (like used in GPTs) can be approximated by KANs:

>"KANs outperform MLPs in terms of accuracy and interpretability. For accuracy, much smaller KANs can achieve comparable or better accuracy than much larger MLPs in data fitting and PDE solving."

But, can an already-pretrained GPT containing MLPs really be approximated by training a new KAN layer (plus a Low Rank parameter-efficient PEFT adapter) as suggested by this Huggingface stub: [https://huggingface.co/MartialTerran/GPTs\_by\_MLP-to-KAN-Transform](https://huggingface.co/MartialTerran/GPTs_by_MLP-to-KAN-Transform) KAN-based Compression of Pretrained GPT Models.

See also: The PyTorch implementation of Generative Pre-trained Transformers (GPTs) using Kolmogorov-Arnold Networks (KANs) for language modeling - AdityaNG/*kan*-*gpt* [https://github.com/AdityaNG/kan-gpt](https://github.com/AdityaNG/kan-gpt) and

[https://github.com/kabachuha/nanoGPKANT/blob/main/README.md](https://github.com/kabachuha/nanoGPKANT/blob/main/README.md)

FieldValue
text This paper KAN: Kolmogorov–Arnold Networks [https://arxiv.org/pdf/2404.19756](https://arxiv.org/pdf/2404.19756) (digested in [https://towardsdatascience.com/kolmogorov-arnold-networks-the-latest-advance-in-neural-networks-simply-explained-f083cf994a85](https://towardsdatascience.com/kolmogorov-arnold-networks-the-latest-advance-in-neural-networks-simply-explained-f083cf994a85) ) suggests that any 2-hidden-layer MLP layers (like used in GPTs) can be approximated by KANs: >"KANs outperform MLPs i…
label r/machinelearning
dataType post
communityName r/MachineLearning
datetime 2024-05-18
username_encoded Z0FBQUFBQm5LakwzZ1M0dTA1bWVGSWthQkpta3gyV09GZHpCc203NFB2NTlqUVlKRFJsQjlCcXM4b1Z5V0tQTWdIc2lRQ3ZHWFJLVXh5Y294dlBRTm5KLXlSRTFvZ1NIRHc9PQ==
url_encoded Z0FBQUFBQm5Lak9IZHBRd19uY3FnTG9oM3FGX1c2STZsQlhFWUNkRjhaVVdlSUp4d0pLN3dncHNpRnNDVVZVNmp4cmtGRV9MdnppYnBlNWJVelVVck9kc29VNEhRazRLR3lOZ0R0VnVYRE5TVUhNd1FpQ0VuamdybjV0VXBxcFdvbWNNVTRDaHZpUzBwOHNDenFFNHhXWjJ5S1FraDNfVWlCSzFBZVYxNjY5UHZtOHVjM2NIZDhBcG8wTVdVTFJRUlBrUVVlZTctN0lURTRPbUFubDI4SGhzZXFjTlFZZTJsZz09

Raw Record

{
  "text": "This paper KAN: Kolmogorov–Arnold Networks [https://arxiv.org/pdf/2404.19756](https://arxiv.org/pdf/2404.19756) (digested in [https://towardsdatascience.com/kolmogorov-arnold-networks-the-latest-advance-in-neural-networks-simply-explained-f083cf994a85](https://towardsdatascience.com/kolmogorov-arnold-networks-the-latest-advance-in-neural-networks-simply-explained-f083cf994a85) ) suggests that any 2-hidden-layer MLP layers (like used in GPTs) can be approximated by KANs:\n\n>\"KANs outperform MLPs in terms of accuracy and interpretability. For accuracy, much smaller KANs can achieve comparable or better accuracy than much larger MLPs in data fitting and PDE solving.\"\n\nBut, can an already-pretrained GPT containing MLPs really be approximated by training a new KAN layer (plus a Low Rank parameter-efficient  PEFT adapter) as suggested by this Huggingface stub:  [https://huggingface.co/MartialTerran/GPTs\\_by\\_MLP-to-KAN-Transform](https://huggingface.co/MartialTerran/GPTs_by_MLP-to-KAN-Transform) KAN-based Compression of Pretrained GPT Models.\n\nSee also: The PyTorch implementation of Generative Pre-trained Transformers (GPTs) using Kolmogorov-Arnold Networks (KANs) for language modeling - AdityaNG/*kan*-*gpt* [https://github.com/AdityaNG/kan-gpt](https://github.com/AdityaNG/kan-gpt)  and\n\n[https://github.com/kabachuha/nanoGPKANT/blob/main/README.md](https://github.com/kabachuha/nanoGPKANT/blob/main/README.md)",
  "label": "r/machinelearning",
  "dataType": "post",
  "communityName": "r/MachineLearning",
  "datetime": "2024-05-18",
  "username_encoded": "Z0FBQUFBQm5LakwzZ1M0dTA1bWVGSWthQkpta3gyV09GZHpCc203NFB2NTlqUVlKRFJsQjlCcXM4b1Z5V0tQTWdIc2lRQ3ZHWFJLVXh5Y294dlBRTm5KLXlSRTFvZ1NIRHc9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9IZHBRd19uY3FnTG9oM3FGX1c2STZsQlhFWUNkRjhaVVdlSUp4d0pLN3dncHNpRnNDVVZVNmp4cmtGRV9MdnppYnBlNWJVelVVck9kc29VNEhRazRLR3lOZ0R0VnVYRE5TVUhNd1FpQ0VuamdybjV0VXBxcFdvbWNNVTRDaHZpUzBwOHNDenFFNHhXWjJ5S1FraDNfVWlCSzFBZVYxNjY5UHZtOHVjM2NIZDhBcG8wTVdVTFJRUlBrUVVlZTctN0lURTRPbUFubDI4SGhzZXFjTlFZZTJsZz09"
}
Explore Dataset Explore Row

Entry Information