Monday, March 31, 2008

Streaks of thought: Streak 16

It's important to experience extremes to truly have stakes and hence understand the importance of balance.

Scott's research advice as a grad student was: "to understand something we need to have stakes in it". Also independently I came up with a thesis about learning being triggered from "losing something". Of course doesn't having stakes mean having something to lose!

Sunday, March 23, 2008

Faith in truth

Science without religion is lame, religion without science is blind... Underlying the effort of science is the religious belief in truth and understanding. These were kind of responses from Einstein when he was repeatedly asked about his religious beliefs.

Having a religious belief is like having faith. There's no objective reason for chosing a faith in something except for interest in survival in a group. Life seems to be liking regularities, patterns which constitute provable truths. The apparent reason for this liking is efficiency because of finiteness. Imagine, if were no regularities then the "program of evolution" has no chance of exploiting structures in the problems of life. This would mean the evolution would be very slow, unimaginably slow! Even the central question in computer science is about limits of efficient computation in the physical world! Scott's thesis and other major theoretical works in CS show lot of evidence that it is very hard to compute efficiently without exploiting structures of problems. Hence provable truths which are about regularites and repeatability atleast in statistical sense is a proper thing to have faith in, in the interest of efficient survival.

Thursday, March 20, 2008

Learning and changing

About two years ago, I had a post on learning from losing. There I mentioned about one situation that triggers learning. It's almost always true that learning is triggered from losing something or to avoid losing something. That something can be anything like say starting from survival to satisfaction gained by understanding Nature.

Life can be viewed as a function of several parameters. There are two aspects in optimizing the function. First identifying the parameters then adjusting the values of the parameters. The more parameters one identifies or in other words the higher ones' awareness is, the better he can adjust or try to adjust the parameters to optimize the value of the function. Everyone is typically endowed with certain level of awareness by default as part of intellectual heritage. Then there are some noble ones who add to our intellectual heritage.

Adjusting the parameters is also called learning. Now based on general observations in the population different people find it hard to learn different parameters. Why is that? Let's try to see. Typically to learn a parameter, we need to compute gradients in the parameter space. The gradients are the places of change. The harder it is to compute gradient along a certain parameter (detect changes) the harder it is to adjust that parameter in the direction of convergence. See we know that not everyone can perceive all relevant parameters let alone gradients along those. Again even if one does perceive gradients the function might not be convex along that parameter and it might require more sophistication of zig-zagging (most people adapt this approach) or trying to convexify (some spiritually inclined adapt this approach) to avoid local minima. This is why change could be hard sometimes.

Hardness of learning a parameter depends on the VC dimension of the training examples for that parameter. The complexity of the learning algorithm required is inversely proportional to the VC dimension. The higher the VC-dimension the easier the algorithm can be. Being aware of this helps us in admitting that some parameters can be easier to learn while some may not. VC-dimension can also be viewed as the number of training examples that the algorithm can shatter that is give zero training error. So what can we psychologically do to keep our learning in life simple or optimal? We have to try to adjust the labels for our training data (experiences) so that we don't unnecessarily increase the requirements on learning algorithms. We rather are better off spending our resources on some parameters which genuinely need hard algorithms. Here genuinely meaning for independent samples in successful and noble population.

Monday, March 17, 2008

Health: Portions

Almost as if I am compensating for having such a long gap in my health posts: here's a quick sequel to my last one.

Disproportionate amounts of food intake in terms of quality and quantity can be detected with the following litmus test. The ideal reaction from eating should be turning active not turning sleepy. Sleepiness is a sign of drop in metabolism. Eating ideally should increase metabolism not decrease. Both, over eating and under eating (like being anorexic) trigger this and such repeated training of drop in metabolism will have serious short term and long term consequences. In the short term ones productivity goes down because of various reasons like lacking attention span etc. Serious health issues can show up in the long run, like organs getting damaged because of constant "under performance".

Not giving into the "call for sleep" by consciously doing something active is not a solution as that will result in negative consequences again both in the short and long run. Fighting sleep is not encouraged. What needs to be consciously adjusted is the portions of intake: both in quality and quantity. Quantity is easier to control but what is important is taking care of the portions of quality nutrition intakes. Eating labeled food is a very good practice in this regard.

Wednesday, March 12, 2008

Health: Food and feelings

After a real long gap in the series of my posts on health since my last one, I intend to discuss some connections I experienced between food and feelings.

The connections between eating patterns and emotional states are well studied in society. Usually the results that are most popularly known are the ones that say how having emotional instability instigates eating disorders. To mitigate eating disorders psychological advice is usually used to reduce the perception of insecurities and induce the perception of one's purpose. Leaving aside serious medical conditions the most common eating disorders are usually because of bad habits.

Alright now comes my main thesis on the connection between food and feelings. Actually this kind of correlation is preached by many spiritual communities, but I wanted to write this because I thought I experienced this especially because of long enough exposure to good American food after eating in India for 20 years. See, I see a chicken-egg relationship between the two. Feelings are produced because of various chemical reactions inside our bodies. Food is one of the biggest modes of injecting new chemicals and triggering chemical reactions inside our bodies. So if ones feelings are bad it might actually be because of the food intakes he is having. The short term effects are pretty clear, for e.g. taking sugar shots drives you a bit crazy, consuming alcohol makes you drowzy. What goes undercover are the long term effects (besides clogging your physical infrastructure like plumbing).

Food is supposed not just to provide nutrition but to provide it in an efficient way. The more efficient way the food fuels us the better our emotional and physical state will be. Signs of inefficient fueling is too much stubborness in adapting tastes to less spicier and "less tasty" foods. Taste is too abstract and subjective that it has to be controlled by us, not control us. So for better health, both physical and emotional, we need good and efficient food and of course with better health the good food habits get reinforced in turn.

A general note: Cultures that control abstract and subjective issues typically produce leaders. Cultures that are controlled by abstract and subjective issues typically end up as followers as they can be "controlled by leaders". Think about it, isn't it obvious from history and differences between western and eastern cultures.

Sunday, March 09, 2008

Conferences and corporations

In academia productivity is measured in terms of quality and quantity of publications one generates. In industry productivity is measured in terms of money one generates. I intend to analyse in this post the chief characteristic difference between academic and industrial efforts and how this characterisation demands for balance between the two type of efforts. Similar demands for balance in different walks of life arise from different characterizations as well. Almost always it's the variety of perspectives that automatically demands for balance if we are to survive "long enough".

The main characterization is that the motivation underlying the effort in academia is "first assimilate and then deploy" while that in industry is "first deploy and then assimilate". Now which strategy is optimal? Human progress is gradual and continuous and for us to actually survive and measure the progress we need to switch between both strategies: one cannot wait until we answer all the questions about an idea to be deployed. Think about it, science would not exist otherwise! Similarly one cannot just jump into conclusions without understanding the soundness and limitations of an idea. Think about it, science would not be science as we see today otherwise. So we naturally need a balance between both kinds of motivations similar to that in my means and ends post.

Now for maintaining such a balance the conferences and corporations which are main arenas for both the efforts need to be designed carefully to fuel both types of motivations. Conferences should primarily be motivated to support the academic effort which means they have to provide for ideas which need time to be studied, analysed, assessed, assimilated and then deployed. Similarly corporations, to take on deploying the ideas so that they can be disseminated and become "accepted as a norm". Lately conferences (in many areas of CS) seem to be driven overly by motivations underlying industrial effort. This is partly because of increased short-sightedness of public funding agencies and increased reliance on corporations. This makes papers on "industrially hot topics" get more accepted in the conferences. This year's CVPR results kind of ascertain that as well. This kind of change is also becoming an increasing concern even in theory community.

Usually such crisis points occur in cycles and these are the points to check for the balance. I once presented a paper by Christos Papadimitriou addressing such crisis in databases theory. Since conferences are usually run as peer-reviewed and mostly non-profit, issues like conflicts of interests etc. play more important role than in corporations. Checks on corruption in academic community are more important because it is typically "assumed" not to exist the machinery for such checks is not given so much weight. It is important to remember that investigations in academia are mainly curiosity driven excercise not frustration driven. This should be supported but not exploited to make it go otherway around.

Wednesday, March 05, 2008

Streaks of thought: Streak 15

Neils Bohr once said: "It is wrong to think that the task of physics is to find out how nature is. Physics concerns what we can say about nature."
The founding father of Computer Science, Alan Turing also came up with a test for artificial intelligence based on observations alone. Scott's post about Turing's philosophy about machines and morals is worth a read. Also (after reminded by Scott's post) according to Aumman's theorem we have to behave based on observable priors to be considered "rational".

Einstein effectively was unproductive after generalizing his theory of relativity "for invariance results among accelerating frames" because of his trouble accepting the uncertainities underlying Nature's laws. I intend to add a full post about uncertainties but for this post I would like to say that all these statements suggest us to use concrete evidence in all walks of life instead getting drained by intangible correlations woven by emotions.