Row 5995

Row ID: 5995 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 5995 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

I'm trying to multiply the parameters of one model (model A) by a scalar $\\lambda$ to get another model (model B) which has the same architecture as A but different parameters. Then I feed a tensor into model B and get the output. I want to calculate the gradient of the output on $\\lambda$ but the .backward() method doesn't work. Specifically, I try to run the following program:

import torch import torch.nn as nn class MyBaseModel(nn.Module): def __init__(self): super(MyBaseModel, self).__init__() self.linear1 = nn.Linear(3, 8) self.act1 = nn.ReLU() self.linear2 = nn.Linear(8, 4) self.act2 = nn.Sigmoid() self.linear3 = nn.Linear(4, 5) def forward(self, x): return self.linear3(self.act2(self.linear2(self.act1(self.linear1(x))))) class WeightedSumModel(nn.Module): def __init__(self): super(WeightedSumModel, self).__init__() self.lambda_ = nn.Parameter(torch.tensor(2.0)) self.a = MyBaseModel() self.b = MyBaseModel() def forward(self, x): for para_b, para_a in zip(self.a.parameters(), self.b.parameters()): para_b.data = para_a.data * self.lambda_ return self.b(x).sum() input_tensor = torch.ones((2, 3)) weighted_sum_model = WeightedSumModel() output_tensor = weighted_sum_model(input_tensor) output_tensor.backward() print(weighted_sum_model.lambda_.grad)

And the printed value is None.

I wonder how can I get the gradient of weighted\_sum\_model.lambda\_ to optimize this parameter?

I tried various ways to get the parameters of weighted\_sum\_model.b but they all did't work. And I visualized the computation graph of WeightedSumModel, on which there is only b but not a and lambda.

FieldValue
text I'm trying to multiply the parameters of one model (model A) by a scalar $\\lambda$ to get another model (model B) which has the same architecture as A but different parameters. Then I feed a tensor into model B and get the output. I want to calculate the gradient of the output on $\\lambda$ but the .backward() method doesn't work. Specifically, I try to run the following program: import torch import torch.nn as nn class MyBaseModel(nn.Module): def __init__(self)…
label r/pytorch
dataType post
communityName r/pytorch
datetime 2024-05-06
username_encoded Z0FBQUFBQm5Lakwycm8zblpjTi1sV1A2M2lJakMxUzRIX0RlNTRXT2x1X1I5eEJ3RU9TWGlEeWdSV3l5NGhKODA3VHNEdDk2NUVKSkFnTlhTay0zWmhtYmNvS1lCaU56Zmc9PQ==
url_encoded Z0FBQUFBQm5Lak9HQlk1VmRiOWswc1ljREZtMmx5ckxkcGJNdWZLMVZheS12UW5RMkl2cS1tNHFDY2xrN0Y2cVJvT0NlZm1BSDJUYmxnNTkyZnRSUS1BN0xLMm55cHlDTm1CQkM4TmN2UmgzT3Z3ZThFYm1waGtUUXI2Mkl0U3l0Zm56YTI4emdvZlVLeHk0X1ZKclNQTjlpRU80U1VtVm13RUVUNGxBeGpGbEN6dnBGMW5pb3hGMHVhWG4tQ1dHWGxTZzZnUUMtZ3VkVXNFcndJRDY4WElMUms5SW91SmdiUT09

Raw Record

{
  "text": "\n\nI'm trying to multiply the parameters of one model (model A) by a  scalar $\\\\lambda$ to get another model (model B) which has the same  architecture as A but different parameters. Then I feed a tensor into  model B and get the output. I want to calculate the gradient of the  output on $\\\\lambda$ but the .backward()  \n method doesn't work. Specifically, I try to run the following program:\n\n    import torch\n    import torch.nn as nn\n    \n    class MyBaseModel(nn.Module):\n        def __init__(self):\n            super(MyBaseModel, self).__init__()\n            self.linear1 = nn.Linear(3, 8)\n            self.act1 = nn.ReLU()\n            self.linear2 = nn.Linear(8, 4)\n            self.act2 = nn.Sigmoid()\n            self.linear3 = nn.Linear(4, 5)\n        def forward(self, x):\n            return self.linear3(self.act2(self.linear2(self.act1(self.linear1(x)))))\n    \n    class WeightedSumModel(nn.Module):\n        def __init__(self):\n            super(WeightedSumModel, self).__init__()\n            self.lambda_ = nn.Parameter(torch.tensor(2.0))\n            self.a = MyBaseModel()\n            self.b = MyBaseModel()\n        def forward(self, x):\n            for para_b, para_a in zip(self.a.parameters(), self.b.parameters()):\n                para_b.data = para_a.data * self.lambda_\n            return self.b(x).sum()\n    \n    input_tensor = torch.ones((2, 3))\n    weighted_sum_model = WeightedSumModel()\n    output_tensor = weighted_sum_model(input_tensor)\n    output_tensor.backward()\n    \n    print(weighted_sum_model.lambda_.grad)\n\nAnd the printed value is None.\n\nI wonder how can I get the gradient of weighted\\_sum\\_model.lambda\\_ to optimize this parameter?\n\nI tried various ways to get the parameters of weighted\\_sum\\_model.b  but they all did't work. And I visualized the computation graph of  WeightedSumModel, on which there is only b but not a and lambda.",
  "label": "r/pytorch",
  "dataType": "post",
  "communityName": "r/pytorch",
  "datetime": "2024-05-06",
  "username_encoded": "Z0FBQUFBQm5Lakwycm8zblpjTi1sV1A2M2lJakMxUzRIX0RlNTRXT2x1X1I5eEJ3RU9TWGlEeWdSV3l5NGhKODA3VHNEdDk2NUVKSkFnTlhTay0zWmhtYmNvS1lCaU56Zmc9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9HQlk1VmRiOWswc1ljREZtMmx5ckxkcGJNdWZLMVZheS12UW5RMkl2cS1tNHFDY2xrN0Y2cVJvT0NlZm1BSDJUYmxnNTkyZnRSUS1BN0xLMm55cHlDTm1CQkM4TmN2UmgzT3Z3ZThFYm1waGtUUXI2Mkl0U3l0Zm56YTI4emdvZlVLeHk0X1ZKclNQTjlpRU80U1VtVm13RUVUNGxBeGpGbEN6dnBGMW5pb3hGMHVhWG4tQ1dHWGxTZzZnUUMtZ3VkVXNFcndJRDY4WElMUms5SW91SmdiUT09"
}
Explore Dataset Explore Row

Entry Information