Row 2141

Row ID: 2141 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 2141 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

I'm trying to train "manually" a tensorflow network, but the dependence of the loss on the parameters is the following (I will talk about two networks, the one I want to train is NET1):

* Given some input, NET1 gives me an output * The output from NET1 are imposed as weights of NET2 that, let's say, gives an output "u" * The loss is computed as some function of "u" * Now, I want to compute the gradient of the loss with respect to the weights of NET1.

However, the gradients I compute are always zeros.

I tried with the following approach:

def train_step(self, input_weights): with tf.GradientTape(persistent=True) as tape: pred_weights = self.NET1(input_weights) weights = self.transform_weights_from_array(pred_weights) for j in range(len(weights)): self.NET2.weights[j].assign(weights[j]) u = self.NET2(SOME_INPUT) loss = tf.reduce_sum(tf.math.abs(u)) gradients = tape.gradient(loss, self.NET1.trainable_variables, unconnected_gradients=tf.UnconnectedGradients.ZERO)

where "transform\_weights\_from\_array" is the following:

​

def transform_weights_from_array(self, w_arr): W = self.NET2.weights w_shaped = [] k = 0 for i, arr in enumerate(W): n = 1 for dim in arr.shape: n *= dim w_shaped.append(tf.reshape(w_arr[k:k + n], arr.shape)) k += n return w_shaped

it simply transforms the weights from the vector shape to the list shape.

However, the gradients are not computed as I would have expected.

FieldValue
text I'm trying to train "manually" a tensorflow network, but the dependence of the loss on the parameters is the following (I will talk about two networks, the one I want to train is NET1): * Given some input, NET1 gives me an output * The output from NET1 are imposed as weights of NET2 that, let's say, gives an output "u" * The loss is computed as some function of "u" * Now, I want to compute the gradient of the loss with respect to the weights of NET1. However, the gradients I compute are al…
label r/tensorflow
dataType post
communityName r/tensorflow
datetime 2023-06-30
username_encoded Z0FBQUFBQm5LakwwNDF3cXFTellHaUlUWV9ldEpaeWRVNzV0aFB2ckpLNEx0NVZQUzVsZVhCVGdWOUhHa0x1NzdyM1RJMkRtOEhUWUk1WXUwNkJkSjlacmVNU3VNYXIxRWc9PQ==
url_encoded Z0FBQUFBQm5Lak9FTEZfSGNYMkFIdWdtMkp1d2NoTUdWVzNRbjVlLUkzQ2Y4WEdMRU5EQktGeVN4cThmRzIyLTVxS0NEQzctM2tLMUlPMGpvOHBiTXhNRnJKeVZrTTgzU1g3aTFzMTY4VmRxSktsS0U4ZktETHJZMGFwYy0zTmVybW1FalJKR3RNeG9pdlUtVFlZUzFNd0FrVFBqNktVdGQ5WHlUaVk2aDZyLWVEY2M5Z3JpWUNZTnl1STVYeWpBbVJnNVB2Qk45MEFOT2FvV1JoNTJSWWRGYnEtLXN5cnhjdz09

Raw Record

{
  "text": "I'm trying to train \"manually\" a tensorflow network, but the dependence of the loss on the parameters is the following (I will talk about two networks, the one I want to train is NET1):\n\n*  Given some input, NET1 gives me an output\n*  The output from NET1 are imposed as weights of NET2 that, let's say, gives an output \"u\"\n*  The loss is computed as some function of \"u\"\n*  Now, I want to compute the gradient of the loss with respect to the weights of NET1.\n\nHowever, the gradients I compute are always zeros.\n\nI tried with the following approach:\n\n    def train_step(self, input_weights):\n    \n       with tf.GradientTape(persistent=True) as tape:\n           pred_weights = self.NET1(input_weights)\n    \n           weights = self.transform_weights_from_array(pred_weights)\n           for j in range(len(weights)):\n               self.NET2.weights[j].assign(weights[j])\n    \n           u = self.NET2(SOME_INPUT)\n           loss = tf.reduce_sum(tf.math.abs(u))\n    \n       gradients = tape.gradient(loss, self.NET1.trainable_variables,\n                                 unconnected_gradients=tf.UnconnectedGradients.ZERO)\n    \n\nwhere \"transform\\_weights\\_from\\_array\" is the following:\n\n​\n\n    def transform_weights_from_array(self, w_arr): \n    \n        W = self.NET2.weights\n        w_shaped = []\n        k = 0\n        for i, arr in enumerate(W):\n            n = 1\n            for dim in arr.shape:\n                n *= dim\n            w_shaped.append(tf.reshape(w_arr[k:k + n], arr.shape))\n            k += n\n        return w_shaped\n\nit simply transforms the weights from the vector shape to the list shape. \n\nHowever, the gradients are not computed as I would have expected.",
  "label": "r/tensorflow",
  "dataType": "post",
  "communityName": "r/tensorflow",
  "datetime": "2023-06-30",
  "username_encoded": "Z0FBQUFBQm5LakwwNDF3cXFTellHaUlUWV9ldEpaeWRVNzV0aFB2ckpLNEx0NVZQUzVsZVhCVGdWOUhHa0x1NzdyM1RJMkRtOEhUWUk1WXUwNkJkSjlacmVNU3VNYXIxRWc9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9FTEZfSGNYMkFIdWdtMkp1d2NoTUdWVzNRbjVlLUkzQ2Y4WEdMRU5EQktGeVN4cThmRzIyLTVxS0NEQzctM2tLMUlPMGpvOHBiTXhNRnJKeVZrTTgzU1g3aTFzMTY4VmRxSktsS0U4ZktETHJZMGFwYy0zTmVybW1FalJKR3RNeG9pdlUtVFlZUzFNd0FrVFBqNktVdGQ5WHlUaVk2aDZyLWVEY2M5Z3JpWUNZTnl1STVYeWpBbVJnNVB2Qk45MEFOT2FvV1JoNTJSWWRGYnEtLXN5cnhjdz09"
}
Explore Dataset Explore Row

Entry Information