Row 31677

Row ID: 31677 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 31677 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

Are you an author?

> Since we aim to extensively tune XGboost and Catboost for their best performances, we increase the number of estimators/ iterations (i.e., the number of decision trees) from 2000 to 4096 and the number of tuning iterations from 100 to 500, which give a more stringent setting and better performances.

This section made me raise an eyebrow. Most of the time with XGBoost you don’t see more than a few hundred trees. 2000 to 4096 trees is a ton. Also, tuning the number of trees is OK but I really prefer letting early stopping do its magic so that the number of trees is an adaptive parameter based on the loss during training. Last but not least it sounds like there’s a lot of parameter tuning happening here, and tree ensembles really don’t benefit all that much from parameter tuning, but they often perform worse. Given sufficient compute, some parameter combinations will appear to do better simply based on chance, when in reality the model is overfitting to the training data. Happens even with XGBoost.

I fear this paper may be committing a classic and widespread error, which is to misuse XGBoost / CatBoost and thus any other method looks better in comparison.

In 2023 and 2024, it is well known that some of those other models (like SAINT) aren’t all that competitive either.

FieldValue
text Are you an author? > Since we aim to extensively tune XGboost and Catboost for their best performances, we increase the number of estimators/ iterations (i.e., the number of decision trees) from 2000 to 4096 and the number of tuning iterations from 100 to 500, which give a more stringent setting and better performances. This section made me raise an eyebrow. Most of the time with XGBoost you don’t see more than a few hundred trees. 2000 to 4096 trees is a ton. Also, tuning the number of trees …
label r/machinelearning
dataType comment
communityName r/MachineLearning
datetime 2024-05-21
username_encoded Z0FBQUFBQm5Lak1HWHBIU2VXTVpCaC1NbDRQSFhKdEVWY0YxaWlEalVGT2pPTDZRRmJTQkFta21GR0FHRFpQNkhHb0JUUkpIZ2hPdS1YYVRZLUlrMFdXNlgyYjQ4X2d6TlE9PQ==
url_encoded Z0FBQUFBQm5Lak9XTTQ3emdydzdrbFpmWkl5d01TNEd0dmJEY3J4enkzc2RnVVAwZV9uQy1qeGVWQktCMko4WmV4MUVTQTNVd0hINS1fWE1aM2hPczh4UlIyX3dEWnJSUHVHRWd2ZUVyd09sMVIwSkgxV1g2TmVVc1B6VUNQTFk2TjRFbVNITUpqVmlTMWhXWWsxT2FEWlRBOVVWcHhxbFU0U2lfNlg3eHdDR1lxdkdFWkl2aFJqMDRQQXd6VWJJUmtIR1FKRWRhMlVuOTYxMkw5MUNwR0gyZUZBajZhaDJTVFlMMVdLejVHaTYyQmkxcUl3WjdiND0=

Raw Record

{
  "text": "Are you an author?\n\n> Since we aim to extensively tune XGboost and Catboost for their best performances, we increase the number of estimators/ iterations (i.e., the number of decision trees) from 2000 to 4096 and the number of tuning iterations from 100 to 500, which give a more stringent setting and better performances.\n\nThis section made me raise an eyebrow. Most of the time with XGBoost you don’t see more than a few hundred trees. 2000 to 4096 trees is a ton. Also, tuning the number of trees is OK but I really prefer letting early stopping do its magic so that the number of trees is an adaptive parameter based on the loss during training. Last but not least it sounds like there’s a lot of parameter tuning happening here, and tree ensembles really don’t benefit all that much from parameter tuning, but they often perform worse. Given sufficient compute, some parameter combinations will appear to do better simply based on chance, when in reality the model is overfitting to the training data. Happens even with XGBoost.\n\nI fear this paper may be committing a classic and widespread error, which is to misuse XGBoost / CatBoost and thus any other method looks better in comparison.\n\nIn 2023 and 2024, it is well known that some of those other models (like SAINT) aren’t all that competitive either.",
  "label": "r/machinelearning",
  "dataType": "comment",
  "communityName": "r/MachineLearning",
  "datetime": "2024-05-21",
  "username_encoded": "Z0FBQUFBQm5Lak1HWHBIU2VXTVpCaC1NbDRQSFhKdEVWY0YxaWlEalVGT2pPTDZRRmJTQkFta21GR0FHRFpQNkhHb0JUUkpIZ2hPdS1YYVRZLUlrMFdXNlgyYjQ4X2d6TlE9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9XTTQ3emdydzdrbFpmWkl5d01TNEd0dmJEY3J4enkzc2RnVVAwZV9uQy1qeGVWQktCMko4WmV4MUVTQTNVd0hINS1fWE1aM2hPczh4UlIyX3dEWnJSUHVHRWd2ZUVyd09sMVIwSkgxV1g2TmVVc1B6VUNQTFk2TjRFbVNITUpqVmlTMWhXWWsxT2FEWlRBOVVWcHhxbFU0U2lfNlg3eHdDR1lxdkdFWkl2aFJqMDRQQXd6VWJJUmtIR1FKRWRhMlVuOTYxMkw5MUNwR0gyZUZBajZhaDJTVFlMMVdLejVHaTYyQmkxcUl3WjdiND0="
}
Explore Dataset Explore Row

Entry Information