Row 6696

Row ID: 6696 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 6696 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

Hello all! Sharing my side project here: [https://github.com/andylolu2/simpleGEMM](https://github.com/andylolu2/simpleGEMM) !

This is an *extremely* minimalistic but fast implementation of matrix multiplication in CUDA. The source code is a single, 200-line CUDA/C++ file which implements fp16 tensor core matrix multiplication, optimised for Turing (SM75) architecture. The goal is to:

1. Write a matmul kernel that does not sacrifice performance. In fact, it's faster than PyTorch/CuBLAS if you [test it on a T4 in Colab](https://colab.research.google.com/github/andylolu2/simpleGEMM/blob/master/colab/simpleGEMM.ipynb)! 2. Make it hackable for new purposes. For example if you want to add a new custom prologue (e.g. Matmul + some reduction), just go to line 186, add your code, and recompile! Full flexibility with no C++ templating shenanigans. 3. Keep it as simple as possible. Hopefully someone learning CUDA will find this useful!

Of course, I didn't implement *everything* from scratch. Most of the this builds upon Nvidia CUTLASS's new CuTe interface for things like memory layout, data copying and using tensor core instructions.

*Aside:*

*Why not OpenAI Triton? I love triton, but sometimes it's hard to get the extra 10-20% performance if you are doing something off its main optimisation path. In fact,* [*triton's matmul for Turing GPUs is quite slow*](https://github.com/openai/triton/issues/189) *(because they mainly optimise for SM80+). I just enjoy having full control over the hardware, knowing that if I have infinite time I can squeeze very single bit of performance out.*

FieldValue
text Hello all! Sharing my side project here: [https://github.com/andylolu2/simpleGEMM](https://github.com/andylolu2/simpleGEMM) ! This is an *extremely* minimalistic but fast implementation of matrix multiplication in CUDA. The source code is a single, 200-line CUDA/C++ file which implements fp16 tensor core matrix multiplication, optimised for Turing (SM75) architecture. The goal is to: 1. Write a matmul kernel that does not sacrifice performance. In fact, it's faster than PyTorch/CuBLAS if you […
label r/machinelearning
dataType post
communityName r/MachineLearning
datetime 2024-05-12
username_encoded Z0FBQUFBQm5LakwzQ1dsbWY3bm9RNjVuSGxSeDgzd3k1cTEtd1RMdHhTdEUzTFowNjdSMmpYWW1TX0dLc1lHVGoyWkswWWsxVkRVbTRieUFqVWx4Qy1pbTFCT1k3NC1wUmc9PQ==
url_encoded Z0FBQUFBQm5Lak9HaVdqRHF5bXRIcGhRbERqN3VVTTkwdkdUaWRySlNINHVjQVBqbzJXaVlzRG1KY0EzRFJLWVRTUzZ0aElRSU9UYXQ4WFIyTVpFVFVDNEhsN1pvRnh5dzBEaXFPczNiQWVFRl8tZloxSmRhT3Juc0JfOXNYR2FGSk1Ua0hscHJYVkNBOHlxdHJLQTJvcGZwal9PY24xc3F0VFlhOWxmNl9RR1k0dlFmRE9hcUFvNWNYYktjblh6VU1ISVVtcTlBTDJDWHZuTGoxVmdFR1BxU1VrN293X29Ddz09

Raw Record

{
  "text": "Hello all! Sharing my side project here: [https://github.com/andylolu2/simpleGEMM](https://github.com/andylolu2/simpleGEMM) !\n\nThis is an *extremely* minimalistic but fast implementation of matrix multiplication in CUDA. The source code is a single, 200-line CUDA/C++ file which implements fp16 tensor core matrix multiplication, optimised for Turing (SM75) architecture. The goal is to:\n\n1. Write a matmul kernel that does not sacrifice performance. In fact, it's faster than PyTorch/CuBLAS if you [test it on a T4 in Colab](https://colab.research.google.com/github/andylolu2/simpleGEMM/blob/master/colab/simpleGEMM.ipynb)!\n2. Make it hackable for new purposes. For example if you want to add a new custom prologue (e.g. Matmul + some reduction), just go to line 186, add your code, and recompile! Full flexibility with no C++ templating shenanigans.\n3. Keep it as simple as possible. Hopefully someone learning CUDA will find this useful!\n\nOf course, I didn't implement *everything* from scratch. Most of the this builds upon Nvidia CUTLASS's new CuTe interface for things like memory layout, data copying and using tensor core instructions.\n\n*Aside:*\n\n*Why not OpenAI Triton? I love triton, but sometimes it's hard to get the extra 10-20% performance if you are doing something off its main optimisation path. In fact,* [*triton's matmul for Turing GPUs is quite slow*](https://github.com/openai/triton/issues/189) *(because they mainly optimise for SM80+). I just enjoy having full control over the hardware, knowing that if I have infinite time I can squeeze very single bit of performance out.*",
  "label": "r/machinelearning",
  "dataType": "post",
  "communityName": "r/MachineLearning",
  "datetime": "2024-05-12",
  "username_encoded": "Z0FBQUFBQm5LakwzQ1dsbWY3bm9RNjVuSGxSeDgzd3k1cTEtd1RMdHhTdEUzTFowNjdSMmpYWW1TX0dLc1lHVGoyWkswWWsxVkRVbTRieUFqVWx4Qy1pbTFCT1k3NC1wUmc9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9HaVdqRHF5bXRIcGhRbERqN3VVTTkwdkdUaWRySlNINHVjQVBqbzJXaVlzRG1KY0EzRFJLWVRTUzZ0aElRSU9UYXQ4WFIyTVpFVFVDNEhsN1pvRnh5dzBEaXFPczNiQWVFRl8tZloxSmRhT3Juc0JfOXNYR2FGSk1Ua0hscHJYVkNBOHlxdHJLQTJvcGZwal9PY24xc3F0VFlhOWxmNl9RR1k0dlFmRE9hcUFvNWNYYktjblh6VU1ISVVtcTlBTDJDWHZuTGoxVmdFR1BxU1VrN293X29Ddz09"
}
Explore Dataset Explore Row

Entry Information