Row 5770

Row ID: 5770 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 5770 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

Hello everyone!

Lately I've been trying to finetune a BERT multilingual model, I always had it set to Tensorflow 2.8 but a few hours ago I decided to update it to Tensorflow 2.16.

The wait times per epoch were always around 30 minutes, however since updating it to Tensorflow 2.16 the training time per epoch has increased to over an hour. Is there certainly an issue with my python code or is this expected?

Update: Since I figured it might be important, this is probably the most important part (Tensorflow wise) of my code:

def create_bert_model(bert_model, MAX_LENGTH, NUM_CLASSES):     input_ids_layer = tf.keras.layers.Input(shape=(MAX_LENGTH,), dtype=tf.int32, name='ids')     attention_mask_layer = tf.keras.layers.Input(shape=(MAX_LENGTH,), dtype=tf.int32, name='mask')     bert_output = bert_model(input_ids_layer, attention_mask_layer).last_hidden_state     net = tf.keras.layers.Dropout(0.1)(bert_output)     net = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(NUM_CLASSES, activation='softmax'))(net)     return tf.keras.Model(inputs=[input_ids_layer, attention_mask_layer], outputs=net) def compile_bert_model():     optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5)     loss = tf.keras.losses.CategoricalCrossentropy(from_logits=False)     metrics = tf.metrics.CategoricalAccuracy()         classifier_model.compile(optimizer=optimizer, loss=loss, metrics=[metrics]) def train_bert_model(epochs):         classifier_model.fit(         train_dataset,         validation_data=validation_dataset,         epochs=epochs,         callbacks = tf.keras.callbacks.EarlyStopping(         monitor='val_categorical_accuracy',         mode='max',         verbose=0,         patience=3,         restore_best_weights=True     ))

FieldValue
text Hello everyone! Lately I've been trying to finetune a BERT multilingual model, I always had it set to Tensorflow 2.8 but a few hours ago I decided to update it to Tensorflow 2.16. The wait times per epoch were always around 30 minutes, however since updating it to Tensorflow 2.16 the training time per epoch has increased to over an hour. Is there certainly an issue with my python code or is this expected? Update: Since I figured it might be important, this is probably the most important par…
label r/tensorflow
dataType post
communityName r/tensorflow
datetime 2024-05-03
username_encoded Z0FBQUFBQm5LakwyOXExc2RWLXZnY2N6bzhpQjZDZGZTWlVjTkJGVmNLWEVacDUxYWpFNEVZRElzdXQ0Z0Vvbi1EWG5OTVU2RDN1cnoxbWtvbGJmOS1SOFppZVBwYnhBSEE9PQ==
url_encoded Z0FBQUFBQm5Lak9HTGY3d3BrS1dEQzVpY2Q5TmJKQWF2SndvYmhNTk00WHlCT3ZYcFJTYks4Tm80a09uNjdsMk1vaEJFS2ZjVHdrbTNaLVNSVkVUaTdzQ3BITjBicmoxcjIyZFE0cjNrbzdTOVFEWDZ3N3VhZFBWNFRGMElwT3NQakpMdUh2dTVHaThfYnYwZkRxblB2NGFNMzA4TElFck4tbVU4REd4ZUdqamF6NTlINzdDQmhtQmtqUlZwbnFpRTIwZ21Bd18tX1Bk

Raw Record

{
  "text": "Hello everyone!\n\nLately I've been trying to finetune a BERT multilingual model, I always had it set to Tensorflow 2.8 but a few hours ago I decided to update it to Tensorflow 2.16.\n\nThe wait times per epoch were always around 30 minutes, however since updating it to Tensorflow 2.16 the training time per epoch has increased to over an hour. Is there certainly an issue with my python code or is this expected?\n\nUpdate:  \nSince I figured it might be important, this is probably the most important part (Tensorflow wise) of my code:\n\n    def create_bert_model(bert_model, MAX_LENGTH, NUM_CLASSES):\n        input_ids_layer = tf.keras.layers.Input(shape=(MAX_LENGTH,), dtype=tf.int32, name='ids')\n        attention_mask_layer = tf.keras.layers.Input(shape=(MAX_LENGTH,), dtype=tf.int32, name='mask')\n        bert_output = bert_model(input_ids_layer, attention_mask_layer).last_hidden_state\n        net = tf.keras.layers.Dropout(0.1)(bert_output)\n        net = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(NUM_CLASSES, activation='softmax'))(net)\n        return tf.keras.Model(inputs=[input_ids_layer, attention_mask_layer], outputs=net)\n    \n    def compile_bert_model():\n        optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5)\n        loss = tf.keras.losses.CategoricalCrossentropy(from_logits=False)\n        metrics = tf.metrics.CategoricalAccuracy()\n        \n        classifier_model.compile(optimizer=optimizer, loss=loss, metrics=[metrics])\n    \n    def train_bert_model(epochs):\n            classifier_model.fit(\n            train_dataset,\n            validation_data=validation_dataset,\n            epochs=epochs,\n            callbacks = tf.keras.callbacks.EarlyStopping(\n            monitor='val_categorical_accuracy',\n            mode='max',\n            verbose=0,\n            patience=3,\n            restore_best_weights=True\n        ))\n\n  \n",
  "label": "r/tensorflow",
  "dataType": "post",
  "communityName": "r/tensorflow",
  "datetime": "2024-05-03",
  "username_encoded": "Z0FBQUFBQm5LakwyOXExc2RWLXZnY2N6bzhpQjZDZGZTWlVjTkJGVmNLWEVacDUxYWpFNEVZRElzdXQ0Z0Vvbi1EWG5OTVU2RDN1cnoxbWtvbGJmOS1SOFppZVBwYnhBSEE9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9HTGY3d3BrS1dEQzVpY2Q5TmJKQWF2SndvYmhNTk00WHlCT3ZYcFJTYks4Tm80a09uNjdsMk1vaEJFS2ZjVHdrbTNaLVNSVkVUaTdzQ3BITjBicmoxcjIyZFE0cjNrbzdTOVFEWDZ3N3VhZFBWNFRGMElwT3NQakpMdUh2dTVHaThfYnYwZkRxblB2NGFNMzA4TElFck4tbVU4REd4ZUdqamF6NTlINzdDQmhtQmtqUlZwbnFpRTIwZ21Bd18tX1Bk"
}
Explore Dataset Explore Row

Entry Information