Monday, July 21, 2008

Streaks of thought: Streak 20

There is a simple difference between having multiple personalities and having multiple personality disorder. All superheroes like Batman (The Dark Knight is awesome: super entertainment package that appeals most), Spiderman, Superman etc. have multiple personalities that they maintain in an orderly fashion. It's when they cannot handle them in order that they run into trouble.

Many of us also have several moods. Having moods is not a bad thing (in fact if we don't have moods life becomes trivial) but having them out of order is not desirable with limited resources around. Mood distribution and personality of a person are correlated. Having strong moods gives you some sort of extra strength but "strong moods come with strong responsibilities".

Sunday, July 20, 2008

Faith in machines

In many of my previous posts I stressed, efficiency and local perspectives. Such emphasis can be traced back to faith in machines which reflects the faith of theoretical computer science community that polynomial amount of resources is efficient while exponential is not.

I love studying machines as the results can so much help understand ourselves (modulo communicable understanding). Computer Science provides a nice unified way for studying machine characteristics. This reminds me I have to finish one of my posts on language of thoughts :) It has a rough precursor which I posted about 2 years ago.

Practical and efficient statistics relies on Bayesian and Markovian principles which allow principled way of working by understanding limits of machines. It's amazing to know that lot of lower bound results in complexity theory and almost entire applied statistics can be traced back to the contributions of Markov brothers! All the extensions are non-trivial but behind the non-trivial efforts of later generations was the motivation based on faith in practicality or in more crisp words, machines.

Monday, July 14, 2008

Streaks of thought: Streak 19

For a country to be a dream of many, it has to build and evolve a system that can foster hope so that a common person becomes courageous enough to work towards her/his dreams without requiring her/him to handle exponential amount of resources on tangential problems. Basically it should be able to breed dreams efficiently. America has been doing a really great job in this regard.

Friday, July 11, 2008

Importance of being normal

In my previous post I discussed how variance in abilities and needs in a local neighborhood is necessarily a result of efficient resource management by Nature for sustenance of life. Then how change is essential to keep the variance (multiple hypotheses) over time to avoid getting stuck in local optima. I also mentioned that being around normal value is important.

In statistics filtering is a problem of estimating posterior of a random variable given observations correlated with the variable over time. Life can almost be treated as a random variable with some moments. In a more global perspective it's hard to characterize these moments and hence the posterior is usually represented using just random samples. I had a related post about six months ago. But our lives are mostly dominated by local perspectives. Actually if we had true global perspectives all the time we would be super natural!

In statistics a very popular technique of estimating a complex non-linear probability distribution of a random variable is non-parametric kernel density estimation. Intuitively it says that any complex distribution can be approximated using sum of Gaussians (or normal distributions). Let's say if we can track these individual Gaussians then we automatically track the overlaying complex distribution. Kalman filters are useful when the Gaussians undergo linear changes that is the mean and variance of the Gaussian only undergo linear transformations. For reasonable non-linear changes there are linear approximations resulting in extended Kalman filters. But for highly non-linear transformations the approximations made in extended Kalman filters are not good enough. Hence people developed unscented Kalman filter which is a combination of sampling based and closed form trackers. The key elements in unscented Kalman filters are the a set of sample around the mean of the distribution. These "sigma points" are the ones that undergo non-linear transformations which can then lead to recovering the necessary moments! See it's quite important to be around the normal distribution especially in the era of highly non-linear changes to actually "participate/contribute" in successful propogation of moments.

Sunday, July 06, 2008

Need for change

Most of us agree that change is important yet hard. All of us can understand the benefits of change but the reason it seems hard is partly not having an "causal understanding" of why it is important. In this post I try to analyze why change is necessary for our survival. For that first I generalize the perspective of change to be more than just variance in temporal dimension of life. This means change characterizes any variance in the needs in any of our effective neighborhood. For e.g. it could be variance in the tastes of your roommates or variance in the goals of your friends etc. Having this generalized perspective, as I will try to argue, helps to see that variance in temporal dimension is not much harder than issues like tolerances etc. and that variance is essential.

Let's see why we need variance in the first place. There is high correlation between the composition of various chemicals in the body to the personalities and behaviors we manifest. Nature which has limited resources. For life to be persistent it is important that life can sustain on variety of resources so that Nature can efficiently "refill the resources". Based on the refilling abilities of Nature our bodies evolved to incorporate variance in the needs. We are not that variant in terms of needs for oxygen for e.g. since Nature seems to be very efficient in that resource regeneration. Hence the variance in our needs can be roughly traced to the variance in the availability of resources in our Natural neighborhood. Since Nature cannot handle all (varied) needs of all the life in a global way efficiently it decentralizes need fulfilling activities into the life forms itself. In other words the body chemical compositions are evolved in such a way as to locally have a cycle of supply and demand: starting from the most obvious examples, some are male, some are female, some have strong feelings about environment, some have strong feelings about high energy colliders creating black holes (see here), some are interested in making money, some are interested in education, some are spiritual, some are materialistic, some are good in theory, some are good in practice etc. etc. So to summarize we can think of local variances in needs is an efficient design of Nature for long-term sustenance of life. Something like: For a rope to be strong the individual fibers and yards and then strands have to be intertwined with friction among them. One other important thing with variance is that variance has to be "normal" locally so as to have the benefit of decentralization otherwise it would demand redundant effort. For e.g. if the friction between the fibers or yarns or strands is too high the rope might self-destruct under it's own friction without additional effort.

Now since the cycles of supply and demand usually are formed locally we might get stuck at local optima (which would be evident by diminishing returns in the cluster etc.). Change in temporal dimension would shuffle around the neighborhoods and gives us a chance to get out those local optima and form new cycles. Eventually we hope to find global optimum configuration. But as it is well known such optimization problems though can be "solved" require exponential amount of time in principle. So enjoy the journey and don't be scared to enter new cycles. It also helps to keep in mind an important property of stochastic optimization methods that not every move is better than the previous move which precisely is its strength. Of course blind (ignorant) change is not great since there are lot of probabilities (based on evidences, priors and likelihoods) you could compute to figure out the types of changes (moves) to reduce the mixing time.

I would ask the readers to pay attention to the words "efficient" and "local" as they carry the central message. Of course what all I discussed above (or in general in this blog) is not always new but is based on original thought. Lot of economists study such behaviors professionally and it would greatly help design your lives better by reading papers in such fields (I don't though): that's why the mathematicians who study and contribute to understanding such patterns are eligible Nobel prizes.