Row 8357

Row ID: 8357 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 8357 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

I'm trying to implement CBOW algorithm using PyTorch. I know the hidden layer is the embedding of the targeted word and it's dimension is equal to dimension I want my embeddings to be in. It's quite difficult for me to understand when to get the embeddings. **Is it that after back-propagation I again need a forward pass to get the correct hidden layer output, or is it something else?** Also, please correct me if I'm wrong anywhere.

Following is the CBOW class implementation.

class CBOW (Module): def __init__(self, in_channel: int, out_channel : int, winSize : int): super().__init__() self.N = in_channel self.V = out_channel self.lin1 = Linear(in_features= self.N, out_features= self.V) self.lin2= Linear(in_features=self.V, out_features= self.N) self.softmax= Softmax(dim=1) def forward(self,input : torch.Tensor): assert len(input.shape) == 2, "Input recieved is not in correct dimension" assert input.shape[1] == self.N, "Word feature vector is not matching" input = self.lin1(input) embeddings = torch.mean(input, dim=0, keepdim= True) out = self.lin2(embeddings) return self.softmax(out) def backward(self, prediction : torch.Tensor, target : torch.Tensor): assert prediction.shape == target.shape , f"Input shapes not matching\nPrediction shape : {prediction.shape}\nTarget shape : {target.shape}" loss_fn = MSELoss() loss = loss_fn(prediction, target) loss.backward() return loss

FieldValue
text I'm trying to implement CBOW algorithm using PyTorch. I know the hidden layer is the embedding of the targeted word and it's dimension is equal to dimension I want my embeddings to be in. It's quite difficult for me to understand when to get the embeddings. **Is it that after back-propagation I again need a forward pass to get the correct hidden layer output, or is it something else?** Also, please correct me if I'm wrong anywhere. Following is the CBOW class implementation. class CBOW (Mo…
label r/machinelearning
dataType post
communityName r/MachineLearning
datetime 2024-05-19
username_encoded Z0FBQUFBQm5Lakw0VFlrTm5JMURkR00zZHdWY1dRZUQzM05FTjVtbjJBZTNUNUVXOEJLV2NaTE1aUG43RG54OTl6blNXbHgzWHJEVFNZeXVYYkZBSklMSjkxcWI1UW52SlE9PQ==
url_encoded Z0FBQUFBQm5Lak9IUjI5SmNLZXJoUUNIQTlmd3ZvZWVja25DaWFVZ25nMUQ5UlJlSkY1S0dUUWxMUGJEUjZvRXBhS1pIN0U3ekM5MmQ3Z1Z0VkxJZjJfQUlya3dpejB0cVo1TW1Zd3FfR0YwdFBJdDd0Q0hMU0RrMk13M1Via1dud2pZcVVNclBFX296cnYwb01qT0xOUEJ0b2poZXo2Ti1oenhMaGg2b2xFY19INlI2TU1xWV9WVWp3ajBzSVZlenNnd1VWdTJLOGhFaXdGVUJKSEpxUzZweVFQOF9SRkM2UT09

Raw Record

{
  "text": "I'm trying to implement CBOW algorithm using PyTorch. I know the hidden layer is the embedding of the targeted word and it's dimension is equal to dimension I want my embeddings to be in. It's quite difficult for me to understand when to get the embeddings. **Is it that after back-propagation I again need a forward pass to get the correct hidden layer output, or is it something else?** Also, please correct me if I'm wrong anywhere.\n\nFollowing is the CBOW class implementation.\n\n    class CBOW (Module):\n        def __init__(self, in_channel: int, out_channel : int, winSize : int):\n            super().__init__()\n            self.N = in_channel\n            self.V = out_channel\n    \n            self.lin1 = Linear(in_features= self.N, out_features= self.V)  \n            self.lin2= Linear(in_features=self.V, out_features= self.N)\n            self.softmax= Softmax(dim=1) \n       \n        def forward(self,input : torch.Tensor): \n            assert len(input.shape) == 2, \"Input recieved is not in correct dimension\"\n            assert input.shape[1] == self.N, \"Word feature vector is not matching\"\n    \n            input = self.lin1(input)\n            embeddings = torch.mean(input, dim=0, keepdim= True)\n            out = self.lin2(embeddings)\n            return self.softmax(out)\n        \n        def backward(self, prediction : torch.Tensor, target : torch.Tensor):\n            assert prediction.shape == target.shape , f\"Input shapes not matching\\nPrediction shape : {prediction.shape}\\nTarget shape : {target.shape}\"\n            loss_fn = MSELoss()\n            loss = loss_fn(prediction, target)\n            loss.backward()\n            return loss",
  "label": "r/machinelearning",
  "dataType": "post",
  "communityName": "r/MachineLearning",
  "datetime": "2024-05-19",
  "username_encoded": "Z0FBQUFBQm5Lakw0VFlrTm5JMURkR00zZHdWY1dRZUQzM05FTjVtbjJBZTNUNUVXOEJLV2NaTE1aUG43RG54OTl6blNXbHgzWHJEVFNZeXVYYkZBSklMSjkxcWI1UW52SlE9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9IUjI5SmNLZXJoUUNIQTlmd3ZvZWVja25DaWFVZ25nMUQ5UlJlSkY1S0dUUWxMUGJEUjZvRXBhS1pIN0U3ekM5MmQ3Z1Z0VkxJZjJfQUlya3dpejB0cVo1TW1Zd3FfR0YwdFBJdDd0Q0hMU0RrMk13M1Via1dud2pZcVVNclBFX296cnYwb01qT0xOUEJ0b2poZXo2Ti1oenhMaGg2b2xFY19INlI2TU1xWV9WVWp3ajBzSVZlenNnd1VWdTJLOGhFaXdGVUJKSEpxUzZweVFQOF9SRkM2UT09"
}
Explore Dataset Explore Row

Entry Information