Row 7324

Row ID: 7324 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 7324 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

My team is tidying up code and data to prepare a model for use in production. There are some diverging opinions on the best method to join data. A couple of people favor using one very large SQL query in ODBC that joins as much as possible in database before importing it. A couple of others favor importing a four or five tables and then joining them in the stats package. I'm of the latter opinion and curious if anyone could tell me whether there is a right or wrong way in this circumstance.

The way I see it, doing a handful of joins in R makes it easier to understand the data for the whoever is updating the model in the future. I'm slightly worried that future people working on this will just take a magical dataframe that contains everything for granted and not question where the data is coming from or how it is generated. And if a problem does develop, a huge SQL query seems more difficult to troubleshoot. There's also one table/dataframe that needs to be heavily manipulated before being joined in a way I don't think SQL was really designed for.

As for advantages to the big ol' SQL query method, I would think it might be more robust? And then obviously all code after is more parsimonious which is always positive.

Any thoughts?

FieldValue
text My team is tidying up code and data to prepare a model for use in production. There are some diverging opinions on the best method to join data. A couple of people favor using one very large SQL query in ODBC that joins as much as possible in database before importing it. A couple of others favor importing a four or five tables and then joining them in the stats package. I'm of the latter opinion and curious if anyone could tell me whether there is a right or wrong way in this circumstance. Th…
label r/datascience
dataType post
communityName r/datascience
datetime 2024-05-15
username_encoded Z0FBQUFBQm5LakwzZWV6NHMtdlM0bW9SWnVrbG9FR3B0OUM0VERqX0VXN3Y2NGxmX2F1aks0WEdhZ1A4b29YSDVSNTdfMkVRelE4Vm1NWGpzU3hSaGljcTlDeDNIZDNPaGc9PQ==
url_encoded Z0FBQUFBQm5Lak9Ib1dnUlRTTVBTR3RacnhaVWtlMHdveHN2SG1IS09OOUYzUVBZT2ZwWGdCQzB5enFMckVrMm1Ja0RCSW9IOXZybFZwQjRCSE5pckZTT184Wldwa2UzVVZjR2pkQktUbFFDcnVkUlAtWFc2SHVFZW8xdkFaVWYxb1BTU0dsMEpVaHRTVFdOVktEMEs3SnpJVVlqRmFBR3EtclRsNWdBLU15aHZfMjFyWDJfaUpHUEZiY3lCV2dFdFRvT2ZPY0VfcmVHTTMwMGZfNVlOX2QyYV92dnZnVGg5Zz09

Raw Record

{
  "text": "My team is tidying up code and data to prepare a model for use in production. There are some diverging opinions on the best method to join data. A couple of people favor using one very large SQL query in ODBC that joins as much as possible in database before importing it. A couple of others favor importing a four or five tables and then joining them in the stats package. I'm of the latter opinion and curious if anyone could tell me whether there is a right or wrong way in this circumstance. \n\nThe way I see it, doing a handful of joins in R makes it easier to understand the data for the whoever is updating the model in the future. I'm slightly worried that future people working on this will just take a magical dataframe that contains everything for granted and not question where the data is coming from or how it is generated. And if a problem does develop, a huge SQL query seems more difficult to troubleshoot. There's also one table/dataframe that needs to be heavily manipulated before being joined in a way I don't think SQL was really designed for. \n\nAs for advantages to the big ol' SQL query method, I would think it might be more robust? And then obviously all code after is more parsimonious which is always positive.\n\nAny thoughts?",
  "label": "r/datascience",
  "dataType": "post",
  "communityName": "r/datascience",
  "datetime": "2024-05-15",
  "username_encoded": "Z0FBQUFBQm5LakwzZWV6NHMtdlM0bW9SWnVrbG9FR3B0OUM0VERqX0VXN3Y2NGxmX2F1aks0WEdhZ1A4b29YSDVSNTdfMkVRelE4Vm1NWGpzU3hSaGljcTlDeDNIZDNPaGc9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9Ib1dnUlRTTVBTR3RacnhaVWtlMHdveHN2SG1IS09OOUYzUVBZT2ZwWGdCQzB5enFMckVrMm1Ja0RCSW9IOXZybFZwQjRCSE5pckZTT184Wldwa2UzVVZjR2pkQktUbFFDcnVkUlAtWFc2SHVFZW8xdkFaVWYxb1BTU0dsMEpVaHRTVFdOVktEMEs3SnpJVVlqRmFBR3EtclRsNWdBLU15aHZfMjFyWDJfaUpHUEZiY3lCV2dFdFRvT2ZPY0VfcmVHTTMwMGZfNVlOX2QyYV92dnZnVGg5Zz09"
}
Explore Dataset Explore Row

Entry Information