Row 5100

Row ID: 5100 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 5100 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

When we talking about the deep learning theory, what theory we really need? We also say that deep learning is a blackbox, we do not have a solid theory for deep learning. But what kind of deep learning theory you think we really need to have. If it is just principle of modern neural network, I think there are already such theory like universal approximation theorem, and backpropagation automatical derivation theory.

So what kind of aspect you think we need to have some more theory about it.

For myself, I think we need a theory that can have theoretical analysis and tell why one kind of network has better performance than another.

It is very like the some circuit theory. We know these voltage and current theory Ampere theorem. But it can not help us too much to design better circuit. But those theory can analysis why one circuit is more stable than other (like circuit zero-pole analysis), why one circuit has less power consumption and more efficient than other. I think we need that kind of theory. If we know more about these case, when we design the architecture, the theory can direct us.

Of course, everyone want to know why one kind of network works, or like why one kind of mechanism work so well. But for me those detailed analysis may be a rabbit hole. Because I feel like for every detail thing it is hard to analysis it, we can only discuss this thing by case study.

However, what we need is something more general theory. Like during the second industry revolution, there is new machine developed everyday, people even try to make a perpetual motion machine. It is like what we are facing now, new network emerging everyday, people want to create some human-level intelligence.

To analysis every specific internal combustion engine is impossible, one is different from another, but there is general something in common. Like we can summarize the thermodynamics 3 laws. which predict the perpetual motion machine is impossible. And we get the idea of entropy production. And although the combustion engine differ one from another, all the combustion engine follow Carnot Cycle. With the help of Carnot Cycle we can analysis the efficiency of the different combustion engine. And this Carnot Cycle can direct the engine design, like if you want to increase the efficiecy you need have more compression or something (I can remember detail, but you know what I mean).

The same for the communication, before Shannon publish his theory, engineer invent hundreds of thousand of transmitter and receiver. But they did not realize that the key for improve the noise-signal ratio is coding and bandwith.

Without these theory, all engineer is like blind person walking in the dark night, you can find something by touching, but you will never know the right way.

FieldValue
text When we talking about the deep learning theory, what theory we really need? We also say that deep learning is a blackbox, we do not have a solid theory for deep learning. But what kind of deep learning theory you think we really need to have. If it is just principle of modern neural network, I think there are already such theory like universal approximation theorem, and backpropagation automatical derivation theory. So what kind of aspect you think we need to have some more theory about it. F…
label r/deeplearning
dataType post
communityName r/deeplearning
datetime 2024-04-26
username_encoded Z0FBQUFBQm5LakwyXzdFT1U1MmdCR0pHaTNjRWFhZkpJRVhXR25uMVdQVWNUUElJcWlKMENCRUVkMEJQdGFTaldHV3ByVS1vcUJnNmJXaUd5R3pxNG9WMXJ5Y0liREJyb0E9PQ==
url_encoded Z0FBQUFBQm5Lak9GbzRLTEtjSFZWa2JiV1Y2b3FBMnNXd3pCUlVKNnp3RjJhRkxsTU9iNC1lZkF2S21EZm9wVDhjNklxdC00MEdpenJDMkMyWlRKME1rSFJ2TDJmMWRjUlZJN2l3c2NBYWZHRm56YlV3VkVXWl9tbFV4eTQ5WWEyd1lvTTJ2a1hWUDQyX21nSkJiUm16WGRhdWM3YWxpS2dKUzM0VTJFVG1JemVOQWRQWUhUeHdBQTAwcDI3ZThKTVpyb2ZRTjBqUzhY

Raw Record

{
  "text": "When we talking about the deep learning theory, what theory we really need?  We also say that deep learning is a blackbox, we do not have a solid theory for deep learning. But what kind of deep learning theory you think we really need to have. If it is just principle of modern neural network, I think there are already such theory like universal approximation theorem, and backpropagation automatical derivation theory.\n\nSo what kind of aspect you think we need to have some more theory about it.\n\nFor myself, I think we need a theory that can have theoretical analysis and tell why one kind of network has better performance than another.\n\nIt is very like the some circuit theory. We know these voltage and current theory Ampere theorem. But it can not help us too much to design better circuit. But those theory can analysis why one circuit is more stable than other (like circuit zero-pole analysis), why one circuit has less power consumption and more efficient than other. I think we need that kind of theory. If we know more about these case, when we design the architecture, the theory can direct us.\n\n\n\n\n\nOf course, everyone want to know why one kind of network works, or like why one kind of mechanism work so well. But for me those detailed analysis may be a rabbit hole. Because I feel like for every detail thing it is hard to analysis it, we can only discuss this thing by case study.\n\nHowever, what we need is something more general theory. Like during the second industry revolution, there is new machine developed everyday, people even try to make a perpetual motion machine. It is like what we are facing now, new network emerging everyday, people want to create some human-level intelligence.\n\nTo analysis every specific internal combustion engine is impossible, one is different from another, but there is general something in common. Like we can summarize the thermodynamics 3 laws. which predict the perpetual motion machine is impossible. And we get the idea of entropy production. And although the combustion engine differ one from another, all the combustion engine follow Carnot Cycle. With the help of Carnot Cycle we can analysis the efficiency of the different combustion engine. And this Carnot Cycle can direct the engine design, like if you want to increase the efficiecy you need have more compression or something (I can remember detail, but you know what I mean).\n\nThe same for the communication, before Shannon publish his theory, engineer invent hundreds of thousand of transmitter and receiver. But they did not realize that the key for improve the noise-signal ratio is coding and bandwith.\n\nWithout these theory, all engineer is like blind person walking in the dark night, you can find something by touching, but you will never know the right way.",
  "label": "r/deeplearning",
  "dataType": "post",
  "communityName": "r/deeplearning",
  "datetime": "2024-04-26",
  "username_encoded": "Z0FBQUFBQm5LakwyXzdFT1U1MmdCR0pHaTNjRWFhZkpJRVhXR25uMVdQVWNUUElJcWlKMENCRUVkMEJQdGFTaldHV3ByVS1vcUJnNmJXaUd5R3pxNG9WMXJ5Y0liREJyb0E9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9GbzRLTEtjSFZWa2JiV1Y2b3FBMnNXd3pCUlVKNnp3RjJhRkxsTU9iNC1lZkF2S21EZm9wVDhjNklxdC00MEdpenJDMkMyWlRKME1rSFJ2TDJmMWRjUlZJN2l3c2NBYWZHRm56YlV3VkVXWl9tbFV4eTQ5WWEyd1lvTTJ2a1hWUDQyX21nSkJiUm16WGRhdWM3YWxpS2dKUzM0VTJFVG1JemVOQWRQWUhUeHdBQTAwcDI3ZThKTVpyb2ZRTjBqUzhY"
}
Explore Dataset Explore Row

Entry Information