Row 5457

Row ID: 5457 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 5457 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

0

I'm trying to add a Attention Mechanism to a Decoder for the purpose of improving a Image Captioning Model. I'm using this tutorial: [Github](https://github.com/yunjey/pytorch-tutorial/tree/master/tutorials/03-advanced/image_captioning) and I'm trying to add this attention mechanism: [Github](https://github.com/sgrvinod/a-PyTorch-Tutorial-to-Image-Captioning/blob/master/models.py). The problem is that it seems like the shapes and sizes of the tensors don't match: `Shape of features: torch.Size([128, 256]) Shape of hiddens: torch.Size([128, 1, 21, 512])`

I'm trying to reshape or resize the tensors so they match via PyTorch `.resize` and `.reshape`, i've tried `.unsqueeze` & `squeeze` too but they don't change the shapes, when I do resize or reshape it appears this error:

# when I do: new_hiddens = hiddens.reshape(128, 1, 23, 256) # it says: RuntimeError: shape '[128, 1, 23, 256]' is invalid for input of size 1441792 #and when I do: new_hiddens = hiddens.resize(128, 256) #it says: requested resize to 128x256 (32768 elements in total), but the given tensor has a size of 128x1x26x512 (1703936 elements). autograd's resize can only change the shape of a given tensor, while preserving the number of elements.

Then I ask GPT and it says that maybe it's not because of the tensors shapes but how are they used, and it makes sense because they are from 2 different examples. So I hope somebody more experienced than me can help me identify where in my Attention mechanism it's expecting a different type of tensors.

class Attention(nn.Module): def __init__(self, encoder_dim, decoder_dim, attention_dim): super(Attention, self).__init__() self.encoder_att = nn.Linear(encoder_dim, attention_dim) self.decoder_att = nn.Linear(decoder_dim, attention_dim) self.full_att = nn.Linear(attention_dim, 1) self.relu = nn.ReLU() self.softmax = nn.Softmax(dim=2) def forward(self, encoder_out, decoder_hidden): att1 = self.encoder_att(encoder_out) # (batch_size, 1, attention_dim) att2 = self.decoder_att(decoder_hidden) # (batch_size, seq_len, attention_dim) att = self.full_att(self.relu(att1 + att2)).squeeze(2) # (batch_size, seq_len) alpha = self.softmax(att) # (batch_size, seq_len) attention_weighted_encoding = (encoder_out.unsqueeze(1) * alpha.unsqueeze(2)).sum(dim=1) # (batch_size, encoder_dim) return attention_weighted_encoding, alpha class DecoderRNN(nn.Module): def __init__(self, embed_size, hidden_size, vocab_size, num_layers, max_seq_length=20): super(DecoderRNN, self).__init__() self.embed = nn.Embedding(vocab_size, embed_size) self.lstm = nn.LSTM(embed_size, hidden_size, num_layers, batch_first=True) # change here self.linear = nn.Linear(hidden_size, vocab_size) self.max_seg_length = max_seq_length self.attention = Attention(hidden_size, hidden_size, hidden_size) # add attention here def forward(self, features, captions, lengths): embeddings = self.embed(captions) hiddens, _ = self.lstm(embeddings) hiddens = hiddens.unsqueeze(1) #new_hiddens = hiddens.resize(128, 256) #print("Shape of new hiddens: ", new_hiddens.shape) print("Shape of features: ", features.shape) print("Shape of hiddens: ", hiddens.shape) attn_weights = self.attention(features, hiddens) context = attn_weights.bmm(features.unsqueeze(1)) # (b, 1, n) hiddens = hiddens + context outputs = self.linear(hiddens.squeeze(1)) return outputs

FieldValue
text 0 I'm trying to add a Attention Mechanism to a Decoder for the purpose of improving a Image Captioning Model. I'm using this tutorial: [Github](https://github.com/yunjey/pytorch-tutorial/tree/master/tutorials/03-advanced/image_captioning) and I'm trying to add this attention mechanism: [Github](https://github.com/sgrvinod/a-PyTorch-Tutorial-to-Image-Captioning/blob/master/models.py). The problem is that it seems like the shapes and sizes of the tensors don't match: `Shape of features: torch.Si…
label r/pytorch
dataType post
communityName r/pytorch
datetime 2024-04-30
username_encoded Z0FBQUFBQm5LakwyVy1VTkV0dEZEeTk4N2pZLXcyZ3RPV1pjaXNISlVtT09LSjhkZmQybjlwZG03ODBCUURURWJGUkc0dEE4bkk3NzhXRy1WNDBDM3RaNnpBRmQ5TV93a0E9PQ==
url_encoded Z0FBQUFBQm5Lak9GSWJndUJyaTFhUURYYmhkaWZxRjFMbnc0NV9WNDM4V3h3a2lTcXJ1MUJGcWY5ZjI3OVBucC0zZkNMOWkyX1VtTFpoNTJGX3ZtTThHWnhuTDFqRHQtLV9Sa3dOVEQyem9GMndJWlFaX1U0SWk0V1BCYnpyUGQza2lIcHY4bUNPMU5GMnNwaEUyTWlGMkR5VXdXVmlWWjBHOEpUX2hBX1UzZGN5eWZITXVUX3ZWWXVaQktGRDlYVExSU1cyZVdUSU5J

Raw Record

{
  "text": "0\n\nI'm trying to add a Attention Mechanism to a Decoder for the purpose of improving a Image Captioning Model. I'm using this tutorial: [Github](https://github.com/yunjey/pytorch-tutorial/tree/master/tutorials/03-advanced/image_captioning) and I'm trying to add this attention mechanism: [Github](https://github.com/sgrvinod/a-PyTorch-Tutorial-to-Image-Captioning/blob/master/models.py). The problem is that it seems like the shapes and sizes of the tensors don't match: `Shape of features:  torch.Size([128, 256]) Shape of hiddens:  torch.Size([128, 1, 21, 512])`\n\nI'm trying to reshape or resize the tensors so they match via PyTorch `.resize` and `.reshape`, i've tried `.unsqueeze` & `squeeze` too but they don't change the shapes, when I do resize or reshape it appears this error:\n\n    # when I do:\n            new_hiddens = hiddens.reshape(128, 1, 23, 256)\n    # it says:\n    RuntimeError: shape '[128, 1, 23, 256]' is invalid for input of size 1441792\n    #and when I do:\n            new_hiddens = hiddens.resize(128, 256)\n    #it says:\n    requested resize to 128x256 (32768 elements in total), but the given tensor has a size of 128x1x26x512 (1703936 elements). autograd's resize can only change the shape of a given tensor, while preserving the number of elements. \n    \n\nThen I ask GPT and it says that maybe it's not because of the tensors shapes but how are they used, and it makes sense because they are from 2 different examples. So I hope somebody more experienced than me can help me identify where in my Attention mechanism it's expecting a different type of tensors.\n\n    class Attention(nn.Module):\n        def __init__(self, encoder_dim, decoder_dim, attention_dim):\n            super(Attention, self).__init__()\n            self.encoder_att = nn.Linear(encoder_dim, attention_dim)\n            self.decoder_att = nn.Linear(decoder_dim, attention_dim)\n            self.full_att = nn.Linear(attention_dim, 1)\n            self.relu = nn.ReLU()\n            self.softmax = nn.Softmax(dim=2)\n    \n        def forward(self, encoder_out, decoder_hidden):\n            att1 = self.encoder_att(encoder_out)  # (batch_size, 1, attention_dim)\n            att2 = self.decoder_att(decoder_hidden)  # (batch_size, seq_len, attention_dim)\n            att = self.full_att(self.relu(att1 + att2)).squeeze(2)  # (batch_size, seq_len)\n            alpha = self.softmax(att)  # (batch_size, seq_len)\n            attention_weighted_encoding = (encoder_out.unsqueeze(1) * alpha.unsqueeze(2)).sum(dim=1)  # (batch_size, encoder_dim)\n    \n            return attention_weighted_encoding, alpha\n    \n    \n    class DecoderRNN(nn.Module):\n        def __init__(self, embed_size, hidden_size, vocab_size, num_layers, max_seq_length=20):\n            super(DecoderRNN, self).__init__()\n            self.embed = nn.Embedding(vocab_size, embed_size)\n            self.lstm = nn.LSTM(embed_size, hidden_size, num_layers, batch_first=True)  # change here\n            self.linear = nn.Linear(hidden_size, vocab_size)\n            self.max_seg_length = max_seq_length\n            self.attention = Attention(hidden_size, hidden_size, hidden_size)  # add attention here\n    \n        def forward(self, features, captions, lengths):\n            embeddings = self.embed(captions)\n            hiddens, _ = self.lstm(embeddings)\n            hiddens = hiddens.unsqueeze(1)\n            #new_hiddens = hiddens.resize(128, 256)\n            #print(\"Shape of new hiddens: \", new_hiddens.shape)\n            print(\"Shape of features: \", features.shape)\n            print(\"Shape of hiddens: \", hiddens.shape)\n            attn_weights = self.attention(features, hiddens)\n            context = attn_weights.bmm(features.unsqueeze(1))  # (b, 1, n)\n            hiddens = hiddens + context\n            outputs = self.linear(hiddens.squeeze(1))\n            return outputs",
  "label": "r/pytorch",
  "dataType": "post",
  "communityName": "r/pytorch",
  "datetime": "2024-04-30",
  "username_encoded": "Z0FBQUFBQm5LakwyVy1VTkV0dEZEeTk4N2pZLXcyZ3RPV1pjaXNISlVtT09LSjhkZmQybjlwZG03ODBCUURURWJGUkc0dEE4bkk3NzhXRy1WNDBDM3RaNnpBRmQ5TV93a0E9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9GSWJndUJyaTFhUURYYmhkaWZxRjFMbnc0NV9WNDM4V3h3a2lTcXJ1MUJGcWY5ZjI3OVBucC0zZkNMOWkyX1VtTFpoNTJGX3ZtTThHWnhuTDFqRHQtLV9Sa3dOVEQyem9GMndJWlFaX1U0SWk0V1BCYnpyUGQza2lIcHY4bUNPMU5GMnNwaEUyTWlGMkR5VXdXVmlWWjBHOEpUX2hBX1UzZGN5eWZITXVUX3ZWWXVaQktGRDlYVExSU1cyZVdUSU5J"
}
Explore Dataset Explore Row

Entry Information