Row 4685

Row ID: 4685 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 4685 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

I implemented a convolutional layer and it turned out to be too slow. Since it takes too much time even to complete an epoch, I haven't been able to see if the network backpropagates correctly, it doesn't throw any errors but don't know if the loss gets down. Other parts of the code are fine, just this one needs some correction. Any help would be appreciated, thank you.

github repo: [https://github.com/Pranavhc/Deep-Learning-from-Scratch](https://github.com/Pranavhc/Deep-Learning-from-Scratch)

import numpy as np from scipy.signal import fftconvolve class Conv2D(Layer): """ A 2D Convolution Layer Parameters: * n_filters: number of filters * filter_shape: tuple (height, width) * input_shape: tuple (channels, height, width) * padding: True = "same" / False = "valid" """ def __init__(self, n_filters:int, filter_shape: tuple[int, int], input_shape:tuple, padding:bool=True) -> None: self.n_filters = n_filters self.filter_shape = filter_shape self.input_shape = input_shape self.padding = padding def initialize(self, optimizer: Optimizer) -> None: """intialize the layer parameters""" f_height, f_width = self.filter_shape channels = self.input_shape[0] limit = 1/np.sqrt(f_height * f_width * channels) self.weights = np.random.uniform(-limit, limit, size=(self.n_filters, channels, f_height, f_width)) self.bias = np.random.uniform(-limit, limit, size=(self.n_filters, 1)) # save state of the optimizer for parameters of this layer self.W_opt = copy.copy(optimizer) self.b_opt = copy.copy(optimizer) def forward(self, input: np.ndarray, train:bool=True) -> np.ndarray: self.input = input padding = "same" if self.padding else "valid" output = fftconvolve(input, self.weights, mode=padding) output = output + self.bias.reshape((1, self.n_filters, 1, 1)) return output def backward(self, output_gradient: np.ndarray) -> np.ndarray: padding = "same" if self.padding else "valid" # calculate gradients weights_gradient = fftconvolve(self.input, output_gradient, mode='valid') bias_grad = np.sum(output_gradient, axis=(0, 2, 3), keepdims=True) input_gradient = fftconvolve(output_gradient, self.weights, mode=padding) # update parameters self.weights = self.W_opt.update(self.weights, weights_gradient) self.bias = self.b_opt.update(self.bias.reshape((1, self.n_filters, 1, 1)), bias_grad) return input_gradient

FieldValue
text I implemented a convolutional layer and it turned out to be too slow. Since it takes too much time even to complete an epoch, I haven't been able to see if the network backpropagates correctly, it doesn't throw any errors but don't know if the loss gets down. Other parts of the code are fine, just this one needs some correction. Any help would be appreciated, thank you. github repo: [https://github.com/Pranavhc/Deep-Learning-from-Scratch](https://github.com/Pranavhc/Deep-Learning-from-Scratch) …
label r/deeplearning
dataType post
communityName r/deeplearning
datetime 2024-04-20
username_encoded Z0FBQUFBQm5LakwxbFFhYTR4ZDFxeUNmNkxKSHdaWkdtRmhQblRmd2w0anBFRjhLYXJvWVFabUxwQnFPS2gxUzJkUFFncndsdVR5UkY0R25peWROSzRQWm9Dckd6NXEtdWUtaEQxVUZGdl9YdU1OY2VTQ05aR289
url_encoded Z0FBQUFBQm5Lak9GdV8teGtKX1UyUnIyWDE3WGttaF9UU195WkJrc09sZF9qN0JJWDZTU0VCWWc3SmpJNmxNUjF1bGdaVlBYQ0JyZXc5dWlsTGNtc3VMUVhCQkkxNURfREdYX3RnOGU1VWRER1RsbGZtcno0czY3cF9NYmhZbThpTlNTeFhYcFFNZXp6SDhZN3lVLVBxYW9IdDBXMVFlYUctdkVMczhYRGsyMG1XS3M2cHY1SlZhYjZhdHpIUGk4eHUzQnFkNW5TcHFh

Raw Record

{
  "text": "I implemented a convolutional layer and it turned out to be too slow. Since it takes too much time even to complete an epoch, I haven't been able to see if the network backpropagates correctly, it doesn't throw any errors but don't know if the loss gets down. Other parts of the code are fine, just this one needs some correction. Any help would be appreciated, thank you.\n\ngithub repo: [https://github.com/Pranavhc/Deep-Learning-from-Scratch](https://github.com/Pranavhc/Deep-Learning-from-Scratch)\n\n    import numpy as np\n    from scipy.signal import fftconvolve\n    \n    class Conv2D(Layer):\n        \"\"\" A 2D Convolution Layer\n    \n        Parameters:\n        * n_filters: number of filters\n        * filter_shape: tuple (height, width)\n        * input_shape: tuple (channels, height, width)\n        * padding: True = \"same\" / False = \"valid\"\n        \"\"\"\n        def __init__(self, n_filters:int, filter_shape: tuple[int, int], input_shape:tuple, padding:bool=True) -> None:\n            self.n_filters = n_filters\n            self.filter_shape = filter_shape\n            self.input_shape = input_shape\n            self.padding = padding\n            \n        def initialize(self, optimizer: Optimizer) -> None:\n            \"\"\"intialize the layer parameters\"\"\"\n    \n            f_height, f_width = self.filter_shape\n            channels = self.input_shape[0]\n    \n            limit = 1/np.sqrt(f_height * f_width * channels)\n    \n            self.weights = np.random.uniform(-limit, limit, size=(self.n_filters, channels, f_height, f_width))\n            self.bias = np.random.uniform(-limit, limit, size=(self.n_filters, 1))\n        \n            # save state of the optimizer for parameters of this layer\n            self.W_opt = copy.copy(optimizer) \n            self.b_opt = copy.copy(optimizer) \n    \n        def forward(self, input: np.ndarray, train:bool=True) -> np.ndarray:\n            self.input = input\n            padding = \"same\" if self.padding else \"valid\"\n    \n            output = fftconvolve(input, self.weights, mode=padding)\n            output = output + self.bias.reshape((1, self.n_filters, 1, 1))\n    \n            return output\n        \n        def backward(self, output_gradient: np.ndarray) -> np.ndarray:\n            padding = \"same\" if self.padding else \"valid\"\n    \n            # calculate gradients\n            weights_gradient = fftconvolve(self.input, output_gradient, mode='valid')\n            bias_grad = np.sum(output_gradient, axis=(0, 2, 3), keepdims=True)\n            input_gradient = fftconvolve(output_gradient, self.weights, mode=padding)\n            \n            # update parameters\n            self.weights = self.W_opt.update(self.weights, weights_gradient)\n            self.bias = self.b_opt.update(self.bias.reshape((1, self.n_filters, 1, 1)), bias_grad)\n    \n            return input_gradient\n\n",
  "label": "r/deeplearning",
  "dataType": "post",
  "communityName": "r/deeplearning",
  "datetime": "2024-04-20",
  "username_encoded": "Z0FBQUFBQm5LakwxbFFhYTR4ZDFxeUNmNkxKSHdaWkdtRmhQblRmd2w0anBFRjhLYXJvWVFabUxwQnFPS2gxUzJkUFFncndsdVR5UkY0R25peWROSzRQWm9Dckd6NXEtdWUtaEQxVUZGdl9YdU1OY2VTQ05aR289",
  "url_encoded": "Z0FBQUFBQm5Lak9GdV8teGtKX1UyUnIyWDE3WGttaF9UU195WkJrc09sZF9qN0JJWDZTU0VCWWc3SmpJNmxNUjF1bGdaVlBYQ0JyZXc5dWlsTGNtc3VMUVhCQkkxNURfREdYX3RnOGU1VWRER1RsbGZtcno0czY3cF9NYmhZbThpTlNTeFhYcFFNZXp6SDhZN3lVLVBxYW9IdDBXMVFlYUctdkVMczhYRGsyMG1XS3M2cHY1SlZhYjZhdHpIUGk4eHUzQnFkNW5TcHFh"
}
Explore Dataset Explore Row

Entry Information