Row 3838

Row ID: 3838 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 3838 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

I wrote a pytorch data loader which used to return data of shape `(4,1,192,320)` representing the 4 samples of single channel image, each of size `192 x 320`. I then used to unfold it into shape `(4,15,64,64)` (Note that `192*320 = 15*64*64`). Resize it to shape `(4,15,64*64)`. And then finally apply my FFN which used to return tensor of shape `(4,15,256)`. (FFN is just first of several neural network layer in my whole model. But lets just stick to FFN for simplicity.) This is the whole code:

import torch import torch.nn as nn from torchvision import transforms from torch.utils.data import Dataset, DataLoader class FFN(nn.Module): def __init__(self, in_dim, out_dim, dropout=0.1): super(FFN, self).__init__() self.linear = nn.Linear(in_dim, out_dim) self.dropout = nn.Dropout(dropout) self.relu = nn.ReLU() def forward(self, x): x = self.linear(x) x = self.relu(x) x = self.dropout(x) return x class DummyDataLoader(Dataset): def __init__(self): super().__init__() self.transforms = transforms.Compose([ transforms.ToPILImage(), transforms.Resize((192, 320)), transforms.ToTensor() ]) def __len__(self): return 10000 # return dummy length def __getitem__(self, idx): frame = torch.randn(192,380) frame = self.transforms(frame) return frame dataset = DummyDataLoader() dataloader = torch.utils.data.DataLoader(dataset, batch_size=4, shuffle=False) frames = next(iter(dataloader)) print('Raw: ', tuple(frames.shape)) unfold = torch.nn.Unfold(kernel_size=64, stride=64) unfolded_ = unfold(frames) unfolded = unfolded_.view(unfolded_.size(0),-1,64,64) print('Unfolded: ', tuple(unfolded.shape)) unfolded_reshaped = unfolded.reshape(unfolded.size(0), -1, 64*64) ffn = FFN(64*64, 256, 0.1) ffn_out = ffn(unfolded_reshaped) print('FFN: ', tuple(ffn_out.shape))

This outputs:

Raw: (4, 1, 192, 320) Unfolded: (4, 15, 64, 64) FFN: (4, 15, 256)

Now, I realized, I also need to implement sliding window. That is, In each iteration, data loader wont just return single frame but multiple frames based on sliding window size, so that the model will learn inter-frame relation. If window size is 5, it will return 5 frames. To implement this, I just changed `__getitem__` from:

def __getitem__(self, idx): frame = torch.randn(192,380) frame = self.transforms(frame) return frame

to:

def __getitem__(self, idx): frames = [torch.randn(192,380) for _ in range(5)] transformed_frames = [self.transforms(frame) for frame in frames] return torch.stack(transformed_frames)

But the code started giving me following error:

Raw: (4, 5, 1, 192, 320) --------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) d:\workspaces\my-project\my-project-win-stacked.ipynb Cell 19 line 6 57 print('Raw: ', tuple(frames.shape)) 59 unfold = torch.nn.Unfold(kernel_size=64, stride=64) ---> 60 unfolded_ = unfold(frames) 61 unfolded = unfolded_.view(unfolded_.size(0),-1,64,64) 62 print('Unfolded: ', tuple(unfolded.shape)) File ~\AppData\Roaming\Python\Python311\site-packages\torch\nn\modules\module.py:1511, in Module._wrapped_call_impl(self, *args, **kwargs) 1509 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc] 1510 else: -> 1511 return self._call_impl(*args, **kwargs) File ~\AppData\Roaming\Python\Python311\site-packages\torch\nn\modules\module.py:1520, in Module._call_impl(self, *args, **kwargs) 1515 # If we don't have any hooks, we want to skip the rest of the logic in 1516 # this function, and just call forward. 1517 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks 1518 or _global_backward_pre_hooks or _global_backward_hooks 1519 or _global_forward_hooks or _global_forward_pre_hooks): -> 1520 return forward_call(*args, **kwargs) 1522 try: 1523 result = None File ~\AppData\Roaming\Python\Python311\site-packages\torch\nn\modules\fold.py:298, in Unfold.forward(self, input) 297 def forward(self, input: Tensor) -> Tensor: --> 298 return F.unfold(input, self.kernel_size, self.dilation, 299 self.padding, self.stride) File ~\AppData\Roaming\Python\Python311\site-packages\torch\nn\functional.py:4790, in unfold(input, kernel_size, dilation, padding, stride) 4786 if has_torch_function_unary(input): 4787 return handle_torch_function( 4788 unfold, (input,), input, kernel_size, dilation=dilation, padding=padding, stride=stride 4789 ) -> 4790 return torch._C._nn.im2col(input, _pair(kernel_size), _pair(dilation), _pair(padding), _pair(stride)) RuntimeError: Expected 3D or 4D (batch mode) tensor with possibly 0 batch size and other non-zero dimensions for input, but got: [4, 5, 1, 192, 320]

As you can see, the data loader now returns data of shape `[4, 5, 1, 192, 320]` in each iteration. But it fails in next step of unfolding, as it seem to expect 4D tensor for batch mode. But data loader returned 5D tensor. I believe, each step in my model pipeline (several FFNs, encoders and decoders) will fail if I return such 5D tensor from data loader as they all be expecting 4D tensor for batch mode.

**Q1.** How we can combine batching and windowing without breaking / revamping existing model, or revamping is inevitable?

**Q2.** If I revamping model is inevitable, how do I do it, such that it will involve minimal code changes (say for example for above model, which involves unfolding and FFN)?

FieldValue
text I wrote a pytorch data loader which used to return data of shape `(4,1,192,320)` representing the 4 samples of single channel image, each of size `192 x 320`. I then used to unfold it into shape `(4,15,64,64)` (Note that `192*320 = 15*64*64`). Resize it to shape `(4,15,64*64)`. And then finally apply my FFN which used to return tensor of shape `(4,15,256)`. (FFN is just first of several neural network layer in my whole model. But lets just stick to FFN for simplicity.) This is the whole code: …
label r/pytorch
dataType post
communityName r/pytorch
datetime 2024-03-25
username_encoded Z0FBQUFBQm5LakwxZjZGS1pkX3dBQmwxWWhmOEd2YjVrM3Qyc3Zkb0pyWlVDcTRMaGJqV2lkZzlxM3R1MEhpdjg0dkZXRF9TOTItYURoVG94amhPZGZpZS1xbUdQSTY5LWhqcUxwTXFPb1VXSWVfRDI1X1I1aWs9
url_encoded Z0FBQUFBQm5Lak9FRi1qWGJ0NUxZNElURnMtQnFoT0xKelIwTDVEMHAzbVF1ODJxd0x6R0p2WDl0dDF3blNZNEkzTEwxR2FBQkg2OUVvTmxPT0NKQVQxOUR3QjJSVHYyc2dYN1RRU1N4SnlBdlBQeVpQZDU4cU9nZmY0eVI4UFg0ZHhyVlJ4ZVRyS0tTYVV0NGVhM2tja0tCWHBHbnZ2Z1pqWnlWZVNTaXFQbWwzY0Rtbkp3UFNURHU3QUl6enpDbjJiQ2hpaGJMZm1YdmZ3cFd4dzI3RkdGZnNnMHBLVF96dz09

Raw Record

{
  "text": "I wrote a pytorch data loader which used to return data of shape `(4,1,192,320)` representing the 4 samples of single channel image, each of size `192 x 320`. I then used to unfold it into shape `(4,15,64,64)` (Note that `192*320 = 15*64*64`). Resize it to shape `(4,15,64*64)`. And then finally apply my FFN which used to return tensor of shape `(4,15,256)`. (FFN is just first of several neural network layer in my whole model. But lets just stick to FFN for simplicity.) This is the whole code:\n\n    import torch\n    import torch.nn as nn\n    from torchvision import transforms\n    from torch.utils.data import Dataset, DataLoader\n    \n    class FFN(nn.Module):\n        def __init__(self, in_dim, out_dim, dropout=0.1):\n            super(FFN, self).__init__()\n            self.linear = nn.Linear(in_dim, out_dim)\n            self.dropout = nn.Dropout(dropout)\n            self.relu = nn.ReLU()\n    \n        def forward(self, x):\n            x = self.linear(x)\n            x = self.relu(x)\n            x = self.dropout(x)\n            return x\n    \n    class DummyDataLoader(Dataset):\n        def __init__(self):\n            super().__init__()\n            self.transforms = transforms.Compose([\n                        transforms.ToPILImage(),\n                        transforms.Resize((192, 320)),\n                        transforms.ToTensor()         \n                    ])\n        \n        def __len__(self):\n            return 10000 # return dummy length\n        \n        def __getitem__(self, idx):\n            frame = torch.randn(192,380)\n            frame = self.transforms(frame)\n            return frame\n        \n    dataset = DummyDataLoader()\n    dataloader = torch.utils.data.DataLoader(dataset, batch_size=4, shuffle=False)\n    frames = next(iter(dataloader))\n    print('Raw: ', tuple(frames.shape))\n    \n    unfold = torch.nn.Unfold(kernel_size=64, stride=64)\n    unfolded_ = unfold(frames)        \n    unfolded = unfolded_.view(unfolded_.size(0),-1,64,64)\n    print('Unfolded: ', tuple(unfolded.shape))\n    \n    unfolded_reshaped = unfolded.reshape(unfolded.size(0), -1, 64*64)\n    ffn = FFN(64*64, 256, 0.1)\n    ffn_out = ffn(unfolded_reshaped)\n    print('FFN: ', tuple(ffn_out.shape))\n\nThis outputs:\n\n    Raw:  (4, 1, 192, 320)\n    Unfolded:  (4, 15, 64, 64)\n    FFN:  (4, 15, 256)\n\nNow, I realized, I also need to implement sliding window. That is, In each iteration, data loader wont just return single frame but multiple frames based on sliding window size, so that the model will learn inter-frame relation. If window size is 5, it will return 5 frames. To implement this, I just changed `__getitem__` from:\n\n    def __getitem__(self, idx):\n        frame = torch.randn(192,380)\n        frame = self.transforms(frame)\n        return frame\n\nto:\n\n    def __getitem__(self, idx):\n        frames = [torch.randn(192,380) for _ in range(5)]\n        transformed_frames = [self.transforms(frame) for frame in frames]\n        return torch.stack(transformed_frames)\n\nBut the code started giving me following error:\n\n    Raw:  (4, 5, 1, 192, 320)\n    \n    ---------------------------------------------------------------------------\n    RuntimeError                              Traceback (most recent call last)\n    d:\\workspaces\\my-project\\my-project-win-stacked.ipynb Cell 19 line 6\n         57 print('Raw: ', tuple(frames.shape))\n         59 unfold = torch.nn.Unfold(kernel_size=64, stride=64)\n    ---> 60 unfolded_ = unfold(frames)        \n         61 unfolded = unfolded_.view(unfolded_.size(0),-1,64,64)\n         62 print('Unfolded: ', tuple(unfolded.shape))\n    \n    File ~\\AppData\\Roaming\\Python\\Python311\\site-packages\\torch\\nn\\modules\\module.py:1511, in Module._wrapped_call_impl(self, *args, **kwargs)\n       1509     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]\n       1510 else:\n    -> 1511     return self._call_impl(*args, **kwargs)\n    \n    File ~\\AppData\\Roaming\\Python\\Python311\\site-packages\\torch\\nn\\modules\\module.py:1520, in Module._call_impl(self, *args, **kwargs)\n       1515 # If we don't have any hooks, we want to skip the rest of the logic in\n       1516 # this function, and just call forward.\n       1517 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n       1518         or _global_backward_pre_hooks or _global_backward_hooks\n       1519         or _global_forward_hooks or _global_forward_pre_hooks):\n    -> 1520     return forward_call(*args, **kwargs)\n       1522 try:\n       1523     result = None\n    \n    File ~\\AppData\\Roaming\\Python\\Python311\\site-packages\\torch\\nn\\modules\\fold.py:298, in Unfold.forward(self, input)\n        297 def forward(self, input: Tensor) -> Tensor:\n    --> 298     return F.unfold(input, self.kernel_size, self.dilation,\n        299                     self.padding, self.stride)\n    \n    File ~\\AppData\\Roaming\\Python\\Python311\\site-packages\\torch\\nn\\functional.py:4790, in unfold(input, kernel_size, dilation, padding, stride)\n       4786 if has_torch_function_unary(input):\n       4787     return handle_torch_function(\n       4788         unfold, (input,), input, kernel_size, dilation=dilation, padding=padding, stride=stride\n       4789     )\n    -> 4790 return torch._C._nn.im2col(input, _pair(kernel_size), _pair(dilation), _pair(padding), _pair(stride))\n    \n    RuntimeError: Expected 3D or 4D (batch mode) tensor with possibly 0 batch size and other non-zero dimensions for input, but got: [4, 5, 1, 192, 320]\n\nAs you can see, the data loader now returns data of shape `[4, 5, 1, 192, 320]` in each iteration. But it fails in next step of unfolding, as it seem to expect 4D tensor for batch mode. But data loader returned 5D tensor. I believe, each step in my model pipeline (several FFNs, encoders and decoders) will fail if I return such 5D tensor from data loader as they all be expecting 4D tensor for batch mode.\n\n**Q1.** How we can combine batching and windowing without breaking / revamping existing model, or revamping is inevitable?\n\n**Q2.** If I revamping model is inevitable, how do I do it, such that it will involve minimal code changes (say for example for above model, which involves unfolding and FFN)?",
  "label": "r/pytorch",
  "dataType": "post",
  "communityName": "r/pytorch",
  "datetime": "2024-03-25",
  "username_encoded": "Z0FBQUFBQm5LakwxZjZGS1pkX3dBQmwxWWhmOEd2YjVrM3Qyc3Zkb0pyWlVDcTRMaGJqV2lkZzlxM3R1MEhpdjg0dkZXRF9TOTItYURoVG94amhPZGZpZS1xbUdQSTY5LWhqcUxwTXFPb1VXSWVfRDI1X1I1aWs9",
  "url_encoded": "Z0FBQUFBQm5Lak9FRi1qWGJ0NUxZNElURnMtQnFoT0xKelIwTDVEMHAzbVF1ODJxd0x6R0p2WDl0dDF3blNZNEkzTEwxR2FBQkg2OUVvTmxPT0NKQVQxOUR3QjJSVHYyc2dYN1RRU1N4SnlBdlBQeVpQZDU4cU9nZmY0eVI4UFg0ZHhyVlJ4ZVRyS0tTYVV0NGVhM2tja0tCWHBHbnZ2Z1pqWnlWZVNTaXFQbWwzY0Rtbkp3UFNURHU3QUl6enpDbjJiQ2hpaGJMZm1YdmZ3cFd4dzI3RkdGZnNnMHBLVF96dz09"
}
Explore Dataset Explore Row

Entry Information