Row 7836

Row ID: 7836 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 7836 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

Hey r/MachineLearning, I'm sharing a showcase on how we improved RAG accuracy on documents with visual elements such as tables and plots by using GPT-4o in both the parsing and answering stages.

It consists of several parts:

**Data indexing pipeline (incremental):**

1. We extract tables as images during the parsing process. 2. GPT-4o explains the content of the table in detail. 3. The table content is then saved with the document chunk into the index, making it easily searchable.

**Question Answering:**

Then, questions are sent to the LLM with the relevant context (including parsed tables) for the question answering.

**Results:**

Our method has outperformed traditional RAG toolkits on questions based on tables. To demonstrate this, we used a few sample questions derived from the Alphabet's 10K report, which is packed with many tables.

**Architecture diagram**: [https://github.com/pathwaycom/llm-app/blob/main/examples/pipelines/gpt\_4o\_multimodal\_rag/gpt4o.gif](https://github.com/pathwaycom/llm-app/blob/main/examples/pipelines/gpt_4o_multimodal_rag/gpt4o.gif)

**Repo**: [https://github.com/pathwaycom/llm-app/tree/main/examples/pipelines/gpt\_4o\_multimodal\_rag/](https://github.com/pathwaycom/llm-app/tree/main/examples/pipelines/gpt_4o_multimodal_rag/)

We are working to extend this project, happy to take comments!

FieldValue
text Hey r/MachineLearning, I'm sharing a showcase on how we improved RAG accuracy on documents with visual elements such as tables and plots by using GPT-4o in both the parsing and answering stages. It consists of several parts: **Data indexing pipeline (incremental):** 1. We extract tables as images during the parsing process. 2. GPT-4o explains the content of the table in detail. 3. The table content is then saved with the document chunk into the index, making it easily searchable. **Question …
label r/machinelearning
dataType post
communityName r/MachineLearning
datetime 2024-05-17
username_encoded Z0FBQUFBQm5LakwzeEd3TXoxeXpVcFhzYUVzUlRyeXl2azJXaXA3TXJQSVBhUklOTmZYaDNERUhUelpQRWhwWGRyUnNtaFEyVW4zU1FyMUZWNml4SlJKZWNaR1R1eVJsN2c9PQ==
url_encoded Z0FBQUFBQm5Lak9ITC01NENvWnZ3YUVPX0tjUzd4clkwVGYzR09LVmJnbkNaOGo2RTBhSDh4MjAwX21udk9MS21RYW5SS2NrdHFsTEM0eW5sOXlKZHN3QmduR0doSXpIQkY2RzE0SEQyb0hPeW1yVEZLbVEtOWJVTGRiS2hERDdsVzV5WVItaWFUV2c5ZE9tem1XdDh3MmNrWHZSeDhSVm8tV3YwelVrU1FhNFlEOVFBdFZVOWNScmxyeXR5V2lqMFhkd0R1YmVVNjVabWM1TUVVM0pwVzNnd3V4SlJmR2Fkdz09

Raw Record

{
  "text": "Hey r/MachineLearning, I'm sharing a showcase on how we improved RAG accuracy on documents with visual elements such as tables and plots by using GPT-4o in both the parsing and answering stages.\n\nIt consists of several parts:\n\n**Data indexing pipeline (incremental):**\n\n1. We extract tables as images during the parsing process.\n2. GPT-4o explains the content of the table in detail.\n3. The table content is then saved with the document chunk into the index, making it easily searchable.\n\n**Question Answering:**\n\nThen, questions are sent to the LLM with the relevant context (including parsed tables) for the question answering.\n\n**Results:**\n\nOur method has outperformed traditional RAG toolkits on questions based on tables. To demonstrate this, we used a few sample questions derived from the Alphabet's 10K report, which is packed with many tables.\n\n**Architecture diagram**: [https://github.com/pathwaycom/llm-app/blob/main/examples/pipelines/gpt\\_4o\\_multimodal\\_rag/gpt4o.gif](https://github.com/pathwaycom/llm-app/blob/main/examples/pipelines/gpt_4o_multimodal_rag/gpt4o.gif) \n\n**Repo**: [https://github.com/pathwaycom/llm-app/tree/main/examples/pipelines/gpt\\_4o\\_multimodal\\_rag/](https://github.com/pathwaycom/llm-app/tree/main/examples/pipelines/gpt_4o_multimodal_rag/)\n\nWe are working to extend this project, happy to take comments!",
  "label": "r/machinelearning",
  "dataType": "post",
  "communityName": "r/MachineLearning",
  "datetime": "2024-05-17",
  "username_encoded": "Z0FBQUFBQm5LakwzeEd3TXoxeXpVcFhzYUVzUlRyeXl2azJXaXA3TXJQSVBhUklOTmZYaDNERUhUelpQRWhwWGRyUnNtaFEyVW4zU1FyMUZWNml4SlJKZWNaR1R1eVJsN2c9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9ITC01NENvWnZ3YUVPX0tjUzd4clkwVGYzR09LVmJnbkNaOGo2RTBhSDh4MjAwX21udk9MS21RYW5SS2NrdHFsTEM0eW5sOXlKZHN3QmduR0doSXpIQkY2RzE0SEQyb0hPeW1yVEZLbVEtOWJVTGRiS2hERDdsVzV5WVItaWFUV2c5ZE9tem1XdDh3MmNrWHZSeDhSVm8tV3YwelVrU1FhNFlEOVFBdFZVOWNScmxyeXR5V2lqMFhkd0R1YmVVNjVabWM1TUVVM0pwVzNnd3V4SlJmR2Fkdz09"
}
Explore Dataset Explore Row

Entry Information