Row 6663

Row ID: 6663 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 6663 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

If I want to include a list of sentences in a data set item, like this:

``` class Dataset(torch.utils.data.Dataset): def __init__(self): self.items = [ {'number': 1, 'sents': ['sent1_1', 'sent1_2']}, {'number': 2, 'sents': ['sent2_1', 'sent2_2']}, {'number': 3, 'sents': ['sent3_1', 'sent3_2']}, ] def __len__(self): return len(self.items) def __getitem__(self, idx): return self.items[idx] ```

And then use a data loader with a default collate function, like this:

`next(iter(torch.utils.data.DataLoader(Dataset(), batch_size=2)))`

The default collate function will group the sentences across items like this:

`{'number': tensor([1, 2]), 'sents': [('sent1_1', 'sent2_1'), ('sent1_2', 'sent2_2')]}`

When what I want is for the lists of sentences within each item to be kept together like this:

`{'number': tensor([1, 2]), 'sents': [('sent1_1', 'sent1_2'), ('sent2_1', 'sent2_2')]}`

What's the simplest collate function that will do this for me?

FieldValue
text If I want to include a list of sentences in a data set item, like this: ``` class Dataset(torch.utils.data.Dataset): def __init__(self): self.items = [ {'number': 1, 'sents': ['sent1_1', 'sent1_2']}, {'number': 2, 'sents': ['sent2_1', 'sent2_2']}, {'number': 3, 'sents': ['sent3_1', 'sent3_2']}, ] def __len__(self): return len(self.items) def __getitem__(self, idx): return self.items[idx] ``` And then use a data loader with a default collate function, like this: `next(iter(…
label r/pytorch
dataType post
communityName r/pytorch
datetime 2024-05-12
username_encoded Z0FBQUFBQm5Lakwzc3dLaE1OamkzMkdpTWV0UjBwVzBxeWlfUnZxTnFUclotZmJoQldJbmN1NS1rd0lUSjg4U0xrMDNGNmF4THdjVEdTWnVTc3JqSkV0SmVwVlVOVUVCMUE9PQ==
url_encoded Z0FBQUFBQm5Lak9HWVpub0txQnVPYkVWdndIdmk2dzN4N1V2b2ZlS3ZObzBORDRUeDdrX1JPRTJ6VWtMMTVJN1VkN21McmkyUTlzZXB2QVBaSlRGQnZidG5UWEdzeHJYbmZEcXNXTlRHRWszeTlWaVVjWEZXV3NBUms4UTh4emlZcjZQTC1sc3ZsMHltQncwbzR6Vlg0bnZKQi1zSkFzMWdJQnY1MHJKTC1PcXQ0VGJ5MldHQktOei1tbDd3UnYwcG5jTGs1eWVFeU93M1BVaHF5R0NZN3NqSzZQMzVkWm9aZz09

Raw Record

{
  "text": "If I want to include a list of sentences in a data set item, like this:\n\n```\nclass Dataset(torch.utils.data.Dataset):\n\tdef __init__(self):\n\t\tself.items = [\n\t\t\t{'number': 1, 'sents': ['sent1_1', 'sent1_2']},\n\t\t\t{'number': 2, 'sents': ['sent2_1', 'sent2_2']},\n\t\t\t{'number': 3, 'sents': ['sent3_1', 'sent3_2']},\n\t\t]\n\tdef __len__(self):\n\t\treturn len(self.items)\n\tdef __getitem__(self, idx):\n\t\treturn self.items[idx]\n```\n\nAnd then use a data loader with a default collate function, like this:\n\n`next(iter(torch.utils.data.DataLoader(Dataset(), batch_size=2)))`\n\nThe default collate function will group the sentences across items like this:\n\n`{'number': tensor([1, 2]), 'sents': [('sent1_1', 'sent2_1'), ('sent1_2', 'sent2_2')]}`\n\nWhen what I want is for the lists of sentences within each item to be kept together like this:\n\n`{'number': tensor([1, 2]), 'sents': [('sent1_1', 'sent1_2'), ('sent2_1', 'sent2_2')]}`\n\nWhat's the simplest collate function that will do this for me?",
  "label": "r/pytorch",
  "dataType": "post",
  "communityName": "r/pytorch",
  "datetime": "2024-05-12",
  "username_encoded": "Z0FBQUFBQm5Lakwzc3dLaE1OamkzMkdpTWV0UjBwVzBxeWlfUnZxTnFUclotZmJoQldJbmN1NS1rd0lUSjg4U0xrMDNGNmF4THdjVEdTWnVTc3JqSkV0SmVwVlVOVUVCMUE9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9HWVpub0txQnVPYkVWdndIdmk2dzN4N1V2b2ZlS3ZObzBORDRUeDdrX1JPRTJ6VWtMMTVJN1VkN21McmkyUTlzZXB2QVBaSlRGQnZidG5UWEdzeHJYbmZEcXNXTlRHRWszeTlWaVVjWEZXV3NBUms4UTh4emlZcjZQTC1sc3ZsMHltQncwbzR6Vlg0bnZKQi1zSkFzMWdJQnY1MHJKTC1PcXQ0VGJ5MldHQktOei1tbDd3UnYwcG5jTGs1eWVFeU93M1BVaHF5R0NZN3NqSzZQMzVkWm9aZz09"
}
Explore Dataset Explore Row

Entry Information