Row 16710

Row ID: 16710 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 16710 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

I really like AI and what it can do, it seems quite impressive. There are however some things that concern me, especially when it comes to trusting it.

So I decided to do a small test using Sudoku as it is very easy for a human to validate whether the AI gets things right or wrong.

Want to be clear, that I am well aware that ChatGPT is a language model and not trained for dealing with something like this, in fact, I asked it about it and it answered this: (I won't post the whole chat as it is very long.)

**Me:** *So Chatgpt would be bad at solving something like Sudoku?*

**ChatGPT:** *Yes, ChatGPT would not be well-suited for solving puzzles like Sudoku. Here are the reasons why:....*

*Why ChatGPT struggles with Sudoku*

*If you ask ChatGPT to solve a Sudoku puzzle, it might attempt to provide a solution based on pattern recognition and language generation rather than logical deduction. While it might generate a plausible-looking sequence of numbers, it would not reliably follow Sudoku rules, such as ensuring that each number from 1 to 9 appears exactly once in each row, column, and 3x3 subgrid*.

It gave a long explanation for why that is. So started a new chat and I asked this:

https://preview.redd.it/404j7xvkjn1d1.png?width=682&format=png&auto=webp&s=7b0fa49db562c68270ccd4c1a365e37eab85d335

I have tried this several times and it can't solve it. It gets it wrong every time. Even when I tell it where the error is, it confirms that it is an error and then it tries to correct it and it is wrong again.

The issue I have is not so much that it can't solve it, but rather that it apparently thinks what it is saying is correct.

This is the end of the long conversation:

https://preview.redd.it/4l8vuwn4ln1d1.png?width=666&format=png&auto=webp&s=66c39e9a64c927898f97b00d85e078af9cf38475

As it can be seen the Sudoku is still wrong yet it thinks it is correct. So I poke it a bit more. And even though the whole answer is not there, it is obvious that it understands the issue.

To me, this is rather critical especially if we imagine talking about much more complicated or potentially dangerous things than a simple Sudoku, if the AI doesn't know if it is wrong and apparently needs to be pushed or told that it is wrong before it agrees. I have no clue if it actually "knows" this to be true or if it is merely trying to satisfy me, exactly as it is more than happy to help me solve the Sudoku, despite it not being able to.

It is important that the AI are good at communicating, but if it gives incorrect answers and even after double checking itself still doesn't see it is wrong, then it is very concerning. So as my final post to it I wrote this:

https://preview.redd.it/zufxum7wnn1d1.png?width=666&format=png&auto=webp&s=0f47feb560e7bf49aae8b94c5ef2d1e5d305129b

This final answer is very good I think, the problem is that this should have been the very first thing it told me, I shouldn't have to spend so long getting it to this point.

As before the issue is the same, I have no clue whether it is merely trying to satisfy me, because it is "told" to do that, rather than admitting that it can't do it, or that it might have a problem with it. It seems to be working backwards adjusting to what you are saying, rather than applying its "AI" first and figuring this out on its own.

So honestly how much can you trust what it is saying when it is certain that something is correct even when it is wrong?

Does anyone else have this feeling that something isn't exactly working 100% as it should or have experienced something similar, maybe not with a Sudoko but something else?

FieldValue
text I really like AI and what it can do, it seems quite impressive. There are however some things that concern me, especially when it comes to trusting it. So I decided to do a small test using Sudoku as it is very easy for a human to validate whether the AI gets things right or wrong. Want to be clear, that I am well aware that ChatGPT is a language model and not trained for dealing with something like this, in fact, I asked it about it and it answered this: (I won't post the whole chat as it is …
label r/chatgpt
dataType post
communityName r/ChatGPT
datetime 2024-05-20
username_encoded Z0FBQUFBQm5Lakw5bXZMQ21QeG4tanhkVmhvRy1MRXRwdlI0YVBmeUloZ29FcVVwWGVadVlSMzZ1VFBISVhiTzZaeEVmNUljSTN4aWpqck5OZFR4N1pBR2tnU2xaemlvWXc9PQ==
url_encoded Z0FBQUFBQm5Lak9NYnBwWHVIYWhHWlVrakh2N1VBb2FmOE1vU2JqTC02eFY2SFpTdHhnU29FUGVQd2tnNTZydXpTbS1sb1NZclNZOGNWcXVFczE0cURnS2FITzZBWFdnSjh2eDJLVEZWUEo1SWc0aEJ6R0JaVXVLVTVFQ2V0Z2wyYVI5eXdBaUJlREFyVnZwcFllQndRMG1iWkFENElJOGxrMzFTeW5iNnpVTDBTZEc2eWdCT3A2bDRKblFDSnlEUnVwenRLNkhWeUty

Raw Record

{
  "text": "I really like AI and what it can do, it seems quite impressive. There are however some things that concern me, especially when it comes to trusting it.\n\nSo I decided to do a small test using Sudoku as it is very easy for a human to validate whether the AI gets things right or wrong.\n\nWant to be clear, that I am well aware that ChatGPT is a language model and not trained for dealing with something like this, in fact, I asked it about it and it answered this: (I won't post the whole chat as it is very long.)\n\n**Me:** *So Chatgpt would be bad at solving something like Sudoku?*\n\n**ChatGPT:** *Yes, ChatGPT would not be well-suited for solving puzzles like Sudoku. Here are the reasons why:....*\n\n*Why ChatGPT struggles with Sudoku*\n\n*If you ask ChatGPT to solve a Sudoku puzzle, it might attempt to provide a solution based on pattern recognition and language generation rather than logical deduction. While it might generate a plausible-looking sequence of numbers, it would not reliably follow Sudoku rules, such as ensuring that each number from 1 to 9 appears exactly once in each row, column, and 3x3 subgrid*.\n\nIt gave a long explanation for why that is. So started a new chat and I asked this:\n\nhttps://preview.redd.it/404j7xvkjn1d1.png?width=682&format=png&auto=webp&s=7b0fa49db562c68270ccd4c1a365e37eab85d335\n\nI have tried this several times and it can't solve it. It gets it wrong every time. Even when I tell it where the error is, it confirms that it is an error and then it tries to correct it and it is wrong again.\n\nThe issue I have is not so much that it can't solve it, but rather that it apparently thinks what it is saying is correct.\n\nThis is the end of the long conversation:\n\nhttps://preview.redd.it/4l8vuwn4ln1d1.png?width=666&format=png&auto=webp&s=66c39e9a64c927898f97b00d85e078af9cf38475\n\nAs it can be seen the Sudoku is still wrong yet it thinks it is correct. So I poke it a bit more. And even though the whole answer is not there, it is obvious that it understands the issue.\n\nTo me, this is rather critical especially if we imagine talking about much more complicated or potentially dangerous things than a simple Sudoku, if the AI doesn't know if it is wrong and apparently needs to be pushed or told that it is wrong before it agrees. I have no clue if it actually \"knows\" this to be true or if it is merely trying to satisfy me, exactly as it is more than happy to help me solve the Sudoku, despite it not being able to.\n\nIt is important that the AI are good at communicating, but if it gives incorrect answers and even after double checking itself still doesn't see it is wrong, then it is very concerning. So as my final post to it I wrote this:\n\nhttps://preview.redd.it/zufxum7wnn1d1.png?width=666&format=png&auto=webp&s=0f47feb560e7bf49aae8b94c5ef2d1e5d305129b\n\nThis final answer is very good I think, the problem is that this should have been the very first thing it told me, I shouldn't have to spend so long getting it to this point.\n\nAs before the issue is the same, I have no clue whether it is merely trying to satisfy me, because it is \"told\" to do that, rather than admitting that it can't do it, or that it might have a problem with it. It seems to be working backwards adjusting to what you are saying, rather than applying its \"AI\" first and figuring this out on its own.\n\nSo honestly how much can you trust what it is saying when it is certain that something is correct even when it is wrong?\n\nDoes anyone else have this feeling that something isn't exactly working 100% as it should or have experienced something similar, maybe not with a Sudoko but something else?",
  "label": "r/chatgpt",
  "dataType": "post",
  "communityName": "r/ChatGPT",
  "datetime": "2024-05-20",
  "username_encoded": "Z0FBQUFBQm5Lakw5bXZMQ21QeG4tanhkVmhvRy1MRXRwdlI0YVBmeUloZ29FcVVwWGVadVlSMzZ1VFBISVhiTzZaeEVmNUljSTN4aWpqck5OZFR4N1pBR2tnU2xaemlvWXc9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9NYnBwWHVIYWhHWlVrakh2N1VBb2FmOE1vU2JqTC02eFY2SFpTdHhnU29FUGVQd2tnNTZydXpTbS1sb1NZclNZOGNWcXVFczE0cURnS2FITzZBWFdnSjh2eDJLVEZWUEo1SWc0aEJ6R0JaVXVLVTVFQ2V0Z2wyYVI5eXdBaUJlREFyVnZwcFllQndRMG1iWkFENElJOGxrMzFTeW5iNnpVTDBTZEc2eWdCT3A2bDRKblFDSnlEUnVwenRLNkhWeUty"
}
Explore Dataset Explore Row

Entry Information