Row 6363

Row ID: 6363 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 6363 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

At a job interview, I was asked how would I help a model generalize to new data, if no labels are available. I went with try to get labels from the old model, clean up as best you can, use ensemble prediction between the stable model trained on lots of clean data and a moving window model trained on less data that isn't as clean.

The company was instead looking for a solution as follows. I have not seen this done and am wondering if this is genius or is not going to work. What are your thoughts, community?

Train an autoencoder + classifier on old data with labels. Update the autoencoder on new data without labels, keep the classifier.

To me, it seems like the classifier has to be trained together with the autoencoder. If you retrain the encoder, even on the same data, the weights can enter a different local minimum. There is no guarantee that an object that is encoded as 0.7, 0.3, 0.2 by one encoder instance is not encoded as 0.2, 0.3, 0.7 by the next. So, no labels - no go. Or is this done in practice? \*goes on to Google how Netflix classifies new movies\*

FieldValue
text At a job interview, I was asked how would I help a model generalize to new data, if no labels are available. I went with try to get labels from the old model, clean up as best you can, use ensemble prediction between the stable model trained on lots of clean data and a moving window model trained on less data that isn't as clean. The company was instead looking for a solution as follows. I have not seen this done and am wondering if this is genius or is not going to work. What are your thought…
label r/deeplearning
dataType post
communityName r/deeplearning
datetime 2024-05-09
username_encoded Z0FBQUFBQm5LakwyaHJHbkZwb095eXRoSW9ma1lKRzlUX3Y0MXlMWjNhQ211VGlaTGVoZ1BOc2JWWW9RVUc1VHNzR2lWaGNPX1M5dDR5dzlodXFvbnRtLUV3SFJPREhlQXc9PQ==
url_encoded Z0FBQUFBQm5Lak9Hd1REV0NRTGcxc3ltVElyV2psQ0xWUjhlRVdWNkJBOWdfZnFPbENiUmpOdUFnMmpsLWRmZWljVEpHaTV6TFNhQi1xekgyaXpESzhmbVI0THZpM3FkcGRHRXpnaWJXNVpXMTFiNnFPMk1QS3k4OHBfMmVQbDQ4TFgxNDJfdm1veFVoUHQyRWMtY2c1Tnpab0ZyaTB4anBsSXBmd2J0Z25obVZBZkw1QjZEekxSdmdCaVA4MjgzVnJ2TGF6Rkpmc0VhNmNLamZuYkFFRzFJbTNpczZQcGZ2QT09

Raw Record

{
  "text": "At a job interview, I was asked how would I help a model generalize to new data, if no labels are available. I went with try to get labels from the old model, clean up as best you can, use ensemble prediction between the stable model trained on lots of clean data and a moving window model trained on less data that isn't as clean. \n\nThe company was instead looking for a solution as follows. I have not seen this done and am wondering if this is genius or is not going to work. What are your thoughts, community? \n\nTrain an autoencoder + classifier on old data with labels. Update the autoencoder on new data without labels, keep the classifier. \n\nTo me, it seems like the classifier has to be trained together with the autoencoder. If you retrain the encoder, even on the same data, the weights can enter a different local minimum. There is no guarantee that an object that is encoded as 0.7, 0.3, 0.2 by one encoder instance is not encoded as 0.2, 0.3, 0.7 by the next. So, no labels - no go. Or is this done in practice? \\*goes on to Google how Netflix classifies new movies\\*",
  "label": "r/deeplearning",
  "dataType": "post",
  "communityName": "r/deeplearning",
  "datetime": "2024-05-09",
  "username_encoded": "Z0FBQUFBQm5LakwyaHJHbkZwb095eXRoSW9ma1lKRzlUX3Y0MXlMWjNhQ211VGlaTGVoZ1BOc2JWWW9RVUc1VHNzR2lWaGNPX1M5dDR5dzlodXFvbnRtLUV3SFJPREhlQXc9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9Hd1REV0NRTGcxc3ltVElyV2psQ0xWUjhlRVdWNkJBOWdfZnFPbENiUmpOdUFnMmpsLWRmZWljVEpHaTV6TFNhQi1xekgyaXpESzhmbVI0THZpM3FkcGRHRXpnaWJXNVpXMTFiNnFPMk1QS3k4OHBfMmVQbDQ4TFgxNDJfdm1veFVoUHQyRWMtY2c1Tnpab0ZyaTB4anBsSXBmd2J0Z25obVZBZkw1QjZEekxSdmdCaVA4MjgzVnJ2TGF6Rkpmc0VhNmNLamZuYkFFRzFJbTNpczZQcGZ2QT09"
}
Explore Dataset Explore Row

Entry Information