Row 25031

Row ID: 25031 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 25031 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

These sites have robots.txt, which they have not setup to block crowlers. No one cares about "please dont parse my data!" text when you have instrument to work with web crowlers. And they were fine with it, because otherwise Google and other search engines won't know anything about the site and won't show it in search.

If you don't like something, block all robots.

FieldValue
text These sites have robots.txt, which they have not setup to block crowlers. No one cares about "please dont parse my data!" text when you have instrument to work with web crowlers. And they were fine with it, because otherwise Google and other search engines won't know anything about the site and won't show it in search. If you don't like something, block all robots.
label r/technology
dataType comment
communityName r/technology
datetime 2024-05-21
username_encoded Z0FBQUFBQm5Lak1DVU0wamIwbkxKZFN3Wkt5a25CQTlqMF9Zd2w3eXZwUmFNY09uaEFrTFdTUlBMeVVYSndNempKUG5jeXBSek0wMHZldkxXMGdBU293UHBJc3BlWDZDVXc9PQ==
url_encoded Z0FBQUFBQm5Lak9SM1RfQXJycVI1Tmtld3l3V2JDdl9PUkQzV2MtWm1mUjk4RVdxRU9oNXFXZGlsYnBacXNKRmNTa1FUekcyMENLUkdJWTB1a0FuOHItdzRPM2lNN083cGpCZHhvUXIzMHFWaFBuUjB6cDEwN1NuVWcta3hYVjBzY2tPdjRIc2hDSndFelMwYTFmVEdQc0xwQ2stSklJRXQtTF80enNvTzlzSmthLXA0WC1XSWtSOXVtMDBfNHFxSjNQd2M2TmhjaWhXaXQzZHNCWTV1cXNaZXJSeFJIY2FwUT09

Raw Record

{
  "text": "These sites have robots.txt, which they have not setup to block crowlers. No one cares about \"please dont parse my data!\" text when you have instrument to work with web crowlers.\nAnd they were fine with it, because otherwise Google and other search engines won't know anything about the site and won't show it in search.\n\nIf you don't like something, block all robots.",
  "label": "r/technology",
  "dataType": "comment",
  "communityName": "r/technology",
  "datetime": "2024-05-21",
  "username_encoded": "Z0FBQUFBQm5Lak1DVU0wamIwbkxKZFN3Wkt5a25CQTlqMF9Zd2w3eXZwUmFNY09uaEFrTFdTUlBMeVVYSndNempKUG5jeXBSek0wMHZldkxXMGdBU293UHBJc3BlWDZDVXc9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9SM1RfQXJycVI1Tmtld3l3V2JDdl9PUkQzV2MtWm1mUjk4RVdxRU9oNXFXZGlsYnBacXNKRmNTa1FUekcyMENLUkdJWTB1a0FuOHItdzRPM2lNN083cGpCZHhvUXIzMHFWaFBuUjB6cDEwN1NuVWcta3hYVjBzY2tPdjRIc2hDSndFelMwYTFmVEdQc0xwQ2stSklJRXQtTF80enNvTzlzSmthLXA0WC1XSWtSOXVtMDBfNHFxSjNQd2M2TmhjaWhXaXQzZHNCWTV1cXNaZXJSeFJIY2FwUT09"
}
Explore Dataset Explore Row

Entry Information