Skip to content

Unrepeatable

One of the pillars of the scientific method is repeatability. It should be possible for an experiment performed in one lab under certain parameters to be repeated under exactly the same conditions in any other lab.

Part of this requires at least some parameters that are controlled and easy manipulated by the scientist. From wikipedia on Dependent and Independent variables:

Dependent variables and independent variables refer to values that change in relationship to each other. The dependent variables are those that are observed to change in response to the independent variables. The independent variables are those that are deliberately manipulated to invoke a change in the dependent variables.

So at least a few parameters need to be easily controlled and manipulated. This allows nice graphs where one axis represents the dependent variable and the other the independent variable.

The result is a plethora of studies where men and mice are put on treadmills and made to walk or jog at a constant pace. The resulting hormone response, weight change, etc. are studied against varying speed, distance, time and so on. These are easily repeatable and result in nice graphs.

The problem is that most natural systems behave as far from equilibrium systems with a lot of “novelty” and enormous variation in the systems. Predators describe levy-distributions in their energy expenditures and predatory ranges. These distributions have highly unstable means and variances (if they exist at all). A levy-distributed range looks like this:

levyflight.png

There are large jumps that are unique. The mean is finite but unstable and converges only with extremely large data sets. The variance doesn’t even exist. So a hypothesis along the lines of:

Levy-flights with infinite variance result in a specific response in other variables.

This can be tested by experimentation, but said experiments will be very difficult to replicate. The sample mean jumps all over the place. There really isn’t any meaningful parameter to be used as an independent variable. Nice graphs are out of the question. Asking people to vary their energy expenditure based on a levy-distribution on a treadmill will result in a lot of data, but no two experiments will be alike - that’s the whole point of the levy-distribution. Even two successive experiments at the same lab will be different (else they aren’t really following a levy-distribution).

I’m not saying the type of studies being done now are useless, I’m just pointing out that there’s a fundamental problem with certain types of hypothesis. This wouldn’t be a problem except that so many systems in nature exhibit this levy-distributed behavior. And there’s a huge hole in experimentation because the graphs don’t come out nice and the experiments aren’t repeatable.

Fractal Productivity?

From Zen and the Art of Motorcycle Maintenance:

When the paper came due she didn’t have it and was quite upset. She had tried and tried but she just couldn’t think of anything to say.

He had already discussed her with her previous instructors and they’d confirmed his impressions of her. She was very serious, disciplined and hardworking, but extremely dull. Not a spark of creativity in her anywhere. Her eyes, behind the thick-lensed glasses, were the eyes of a drudge. She wasn’t bluffing him, she really couldn’t think of anything to say, and was upset by her inability to do as she was told.

It just stumped him. Now he couldn’t think of anything to say. A silence occurred, and then a peculiar answer: “Narrow it down to the main street of Bozeman.” It was a stroke of insight.

She nodded dutifully and went out. But just before her next class she came back in real distress, tears this time, distress that had obviously been there for a long time. She still couldn’t think of anything to say, and couldn’t understand why, if she couldn’t think of anything about all of Bozeman, she should be able to think of something about just one street.

He was furious. “You’re not looking!” he said. A memory came back of his own dismissal from the University for having too much to say. For every fact there is an infinity of hypotheses. The more you look the more you see. She really wasn’t looking and yet somehow didn’t understand this.

He told her angrily, “Narrow it down to the front of one building on the main street of Bozeman. The Opera House. Start with the upper left-hand brick.”

Her eyes, behind the thick-lensed glasses, opened wide. She came in the next class with a puzzled look and handed him a five-thousand-word essay on the front of the Opera House on the main street of Bozeman, Montana. “I sat in the hamburger stand across the street,” she said, “and started writing about the first brick, and the second brick, and then by the third brick it all started to come and I couldn’t stop. They thought I was crazy, and they kept kidding me, but here it all is. I don’t understand it.”

Haven’t really seen this approach emphasized anywhere else though. I mostly see tips about “drawing up an outline first.”

Underpriced Options, Competition, and Fear

I have written about the role of competition and salience before. I argued that the presence of stiff competition can mean that the option is at least priced correctly, if not overpriced. It’s a matter of looking at how skewed the demand-supply distribution is. If demand is much higher than supply (stiff competition), it’s highly likely that the option is priced appropriately or overpriced. I’m talking about real-world options here.

On the other hand, if supply exceeds demand, the situation is ripe for underpriced or “cheap” options. This means that you would know beforehand that there’s a good chance that the ROI is positive. In the case with overpriced options, there is no clear indication beforehand that it’s rational to enter the transaction at all (but salient options with high payoffs will tend to draw hordes of competitors).

I was going through Ben Casnocha’s archives when I read this:

The main thing I’ve learned in my seven years studying and doing presentations is that the standard for presentations / public speaking in the professional world is low, and as such it’s easy to be seen as great. When most people suck at something, all you have to do is suck less. And when I discovered I had a natural knack for communication, I identified speaking / communicating / presenting as one of my natural strengths that, if built upon, could become an unstoppable strength thanks not only to my own capabilities but because of how I would be perceived relative to the masses.

Since public speaking is supposed to be one of the top feared things amongst the population, there is little competition to speak of (relative to less feared things). And since this fear keeps the average quality low, a small edge can be positively leveraged easily.

In competitive environments, a small loss of edge will mean leverage in the opposite direction - being tossed out of the pool. Here, a small edge means an increase in positive leverage.

For me this is exciting because I’m always on the lookout for investing in “cheap” options. I now have a criterion that I can use to ferret out such opportunities. Keeping an eye out for things that people fear.

Of course, it’s very likely that I will initially fear the same things, but if the worst-case scenario is clipped, I will invest in training to overcome the fear. Because I know that I will only have to improve by a little to gain a positively leveraged edge that will cost very little to acquire.

Distributional Entropy, Information, and Fat-Tails

My latest essay goes deeper into the connection between entropy, information, and kurtosis.

I discuss the behavior of kurtosis and entropy in unconstrained probability distributions and investigate the behavior of sample kurtosis in time-series with non-decreasing kurtosis.

The essay: Distributional Entropy, Information, and Fat-Tails.

I will discuss how infinite kurtosis is manifested in the real world in future posts.

Entropy and Kurtosis

I varied the tail-exponent of a Student’s-T distribution, taking it from a near-gaussian distribution to a near-cauchy, fat-tailed distribution and noted the relationship between kurtosis and information entropy. Here’s the plot:

Kurtosis vs. Entropy

The implications are interesting. More on that later.

Earthquakes as Rare Events

The Mercury News is reporting a new study by the US Geological Survey that predicts “California faces a 99.7 percent chance of big quake by 2037.”

How the notion of “probability” is interpreted in these studies is a mystery. The power-law tails on these processes have a very low exponent and the ability to calibrate the probability in the tails shrinks very fast as one moves down the tail.

D. A. Freedman and P.B. Stark at Berkeley looked at this problem in a paper titled “What is the chance of an earthquake?” This 2001 paper studied an earlier similar forecast.

From the paper, on the probability forecast:

There is no straightforward interpretation of the USGS probability forecast. Many steps involve models that are largely untestable; modeling choices often seem arbitrary. Frequencies are equated with probabilities, fiducial distributions are used, outcomes are assumed to be equally likely, and sub jective probabilities are used in ways that violate Bayes rule.

…and:

Philosophical difficulties aside, the numerical probability values seem rather arbitrary.

They conclude with this:

Probabilities are a distraction. Instead of making forecasts, the USGS could help to improve building codes and to plan the government’s response to the next large earthquake. Bay Area
residents should take reasonable precautions, including bracing and bolting their homes as well as securing water heaters, bookcases, and other heavy objects. They should keep first aid supplies, water, and food on hand. They should largely ignore the USGS probability forecast.

Invest in preparation instead of building models and forecasting rare events. This false precision is dangerous if it informs decision making.

Learning in Complex Systems

We cannot learn from history in complex systems, except at the meta level (we can learn that we cannot learn, for example). This follows once one accepts that causality is impossible to pin down given an observation of a phenomenon.

Ericsson et al show that learning, and becoming an expert, requires:

  1. A tight relationship between action and consequence - i.e causality needs to be discoverable
  2. Quick and clear feedback on actions
  3. Repeatability

Let us consider whether a brain surgeon passes these criteria. Causality is indeed easily discoverable, nicking an artery in the theatre will make it clear that a mistake has been made - there is a tight relationship between action and consequence. The feedback is often immediate, at least in terms of the surgical procedure itself. Since there is relatively minor variability between human brains, and since the operation theatre is a controlled environment, the entire process of “brain surgery” is highly repeatable.

This makes brain surgery a classic domain for learning from the past and allows the creation of experts.

Now let us look at managing the environment, (enough people are picking on economists now, so I’ll skip them), and see if it is possible to learn from the past and become an expert in this domain. Are the three criteria satisfied?

First, causality is very difficult to identify. Small changes in any number of parameters, ranging from climate to prey availability can have large and sudden impact on predator populations. Given a phenomenon, it hard to say what caused it. If a human action is involved, outcomes are unpredictable and can spiral out of control when coupled with external variations (see the many instances of predator introduction for hilarious examples) .

Second, the results can take years to manifest themselves. The chain of events precipitated by an action can cascade into a series of relatively small changes until it shows up as a big impact a long way down the daisy-chain. Feedback on actions is far from immediate. Often, the people initiating the action are no longer even in charge anymore. Attempts to control Wolf populations in Yellow Stone National Park had impacts decades down the line.

Third, the environment is not controlled. Conditions that held once need not hold after a period of time has passed. Rivers dry up, predator-prey populations change all the time, and climate changes make any assumptions of stationary processes unjustified. Actions that worked earlier need not work now, because the underlying mechanisms may have changed.

All three conditions for learning and expertise are violated. Under these conditions, it is not possible to learn from the past and become an expert.

An “expert” at managing the environment can only be someone who takes a very fallible approach and shows humility and respect in the face of the unknown.

Other domains where experts cannot be present by the nature of the system itself: The movie industry, publishing, economics, stock-markets, climate, politics, wars, … the list goes on.

Seven Minute Silence and Causality

Ever notice that even in pretty large groups a sudden hush can fall as everyone stops talking at the same time? I’ve noticed this in large classrooms filled with boisterous kids to formal dinners with great conversationalists.

When it does occur, the group usually just looks around, smiles, and moves on. The din of chatter fills the air again. The members of the group would find it impossible to find a specific cause for the sudden silence. In fact, every individual in the group who is a part of this process knows that there is no specific cause - they don’t even think about looking for one.

Now consider a blind man at the same table. He can only observe the sounds and the atmosphere but cannot see. He is in no position to rule out a large number of possible causes. While the rest of the non-blind people can see and confirm that nothing extraordinary occurred to cause the sudden silence, he cannot do this and so his mind will race to figure out what the cause was.

He will consider many hypothesis: maybe somebody spilt a drink, maybe somebody did something inappropriate, maybe a large group of waiters showed up to serve them, maybe the lights went out for a fraction of a second and came back on …

He will not consider that there is no cause. He will not consider that the question “Why?” is simply unanswerable.

In the case above, us humans are participants in the process so we have some insight into the lack of causality. In many other endeavors we are simply external observers or lack the self-awareness to consider the lack of causality. Socio-economic processes are highly complex and distributed. It is very possible that these are environments where “Why?” is unanswerable. But we may be like blind men looking for causes where none exist.

In any situation where a large investment is being made based on an answer to the question “Why?,” it may be prudent to ask if that question is answerable to begin with. Historians, economists and scientists who study history looking for non-proximate causes may be looking for an answer that does not exist (or is unknowable, they are epistemologically equivalent).

Tradeoff between Volatility and Kurtosis

I’ve written before on the topic of stability and robustness to sudden shocks. In mathematical terms, any attempt to squeeze the second moment (volatility) of the distribution will push mass into the tails and drastically increase the fourth moment (kurtosis)

standard_symmetric_pdfs.png

You can see above that the probability distributions with fatter tails have “thinner” bodies. While sampling these distributions, measures such as variance, standard deviation, and volatility will appear lower than for the ones from thin-tailed distributions (note how “thick” the thin-tailed distributions are), giving the appearance of lower risk and volatility.

Since probabilities need to sum up to unity, there is no way to get around this constraint. Squeezing the second moment will increase the fourth.

This is a tradeoff that needs to be considered carefully. At the moment decisions only take into account the second moment and we’ve put in place processes that tend to minimize the second moment without heed to what is happening to larger moments.

Win-Win in Zero-Sum Games

Zero-Sum games with results being decided far into the future can trick participants into thinking they’ve just entered a win-win transaction where both of them come out ahead. This is similar to a conman “overpaying” for goods with counterfeit money - both the conman and the merchant go home happy. The merchant is in for a rude surprise only after he tries to use that money. The pain he experiences is in direct proportion to the happiness he felt earlier.

Derivatives transaction, stock market transaction with a fixed money-supply, and forex speculation are all zero-sum games. Derivatives are the most interesting because they let traders push the uncertainty out far into the future. Consider two traders who enter a derivatives transaction with an expiration date two years out. If the market is illiquid, they can both happily “mark to model” and book paper profits if they use different models. In fact, since they entered the transaction in the first place, they probably are using different models.

Even “mark to market” transactions are susceptible to model-error, just less so. The market as a whole is an average of many models, any sharp change in this average will expose participants to model error. Even if the merchant was paid in real dollars for the asset, he can still lose out if rampant inflation sets in and the asset’s value doesn’t change but the cash in his hands is steadily becoming worthless. The model that the market used for currencies changed and the change did him in.

As in the counterfeit money transaction, the person who is assuming a tightly-constrained game (ludic fallacy) will emerge as the loser. False premises are usually the root problem (these dollar bills are real, cash will be worth the same tomorrow, this cannot happen in this market, …).

In fact, the presence of win-win players in what is known to be a zero-sum game can be used as a diagnostic tool to prepare for massive reality-checks in the future. Hedge Funds and Investment Banks all tend to trade with each other. And they particularly enjoy derivatives transaction. If most hedge funds end up showing stellar returns for an extended period of time, it shouldn’t come as a big surprise that at least some of them have been borrowing from destiny.