Yet another “theta class” problem .. those that are worth automating to a degree but are simply too complex to package up. The barrier in this case is the problem-specific amount of thought that is needed, up front.

Let me explain.

Suppose you have a complex problem. You wish to estimate the likely effects of some action (introducing a new product is the paradigmatic exemplar, but of course where you can add you can subtract – delete a product- or expand or contract the product line, or modify product features).

You don’t have a lot of data, and so you have to make some assumptions, and you need to work through the effects of those assumptions.

You want an answer to “what if” (we took the proposed course of action), but you ALSO want an analysis of the sensitivity of the outcomes to the assumptions : hence, “explicated reasoning” – you make assumptions, but those assumptions are explicit and are in the form of parameters to the model. So you can vary them, and see what happens to the outcomes. You can even work backwards to see what sort of assumptions would be required to support a given outcome .. if the assumptions are too heroic, then the outcome is not well supported in logic.

OK, all this is abstract. Let me concretize it a bit.

Macro Flow modelling has been around in a variety of guises in a number of years.

Urban and Hauser popularized it in the context of new product introduction, where trial and repeat purchase are the driving factors. Have a look at the (rather poorly scanned) diagrams attached – you can get the general idea, even though there is a fair bit of hand waving involved.

This is flow modelling, “flowcasting” as we call it. The basic idea is that you start with some number, some “pool”, and then successively refine that down to an estimate of an outcome (eg a product demand estimate) based on some “filters”, some “parameters” .. the important point being that those parameters are fully explicated, can be argued about and debated, and the influences of varying those can be seen by a simple recomputation of the model.

This is not rocket science, it is just simply quantified common sense – or more accurately, a sensible quantification of the processes at work.

It is not too hard to visualize this, either. I have written code that will allow users to add “flowchart-like” decision boxes to a graphical representation of the process and the parameters. Like those Urban and Hauser scanned diagrams, only much nicer and allowing interaction with the flowchart, introducing extra decision nodes etc.

Let me be unrealistic for a moment, to play with the ideas.. I’ll get more serious in due course.

Suppose I wanted to sell Australian beer in China. The adult male population in China is 1.2 billion (I am making these figures up), and we think that maybe 30% of those drink beer. 80% live in rural areas, and we estimate our probability of getting distribution to those areas via our strategic partner at around 5%. The Chinese market for beer is very heavily price driven (and most regular consumption amongst the target market takes place at less than 20c per bottle) – we estimate a drop off from 20c to 40c of 100%. But there is a small urban (about 2% of all urban males) upwardly mobile professional class who are increasingly (increase at 20 per annum from a 1% base) drinking foreign brands. Competition from Canada in Shanghai is strong, elsewhere weaker. Chinese prefer the heavy sweet style of Asian beers to the lager style that we produce. We expect transnational/global heavyweights to spend $400m on internet advertising in the next 3 years, and that to have an awareness effect of nearly 70% in the target market.

How much beer will we sell, and is the strategy to sell beer in China supportable by any reasonable set of assumptions?

Right. Well, there are lots of assumptions there (parameters that could be varied).. but this is not too complex a scenario to be put into a macro flow model. The benefit of so doing would be that we can then directly see the effects of varying our assumptions.

We could, if we were smart enough, incorporate our uncertainty about the parameters (the assumptions) in some more formal framework.. perhaps a multinomial distribution for discrete parameters, some heavy tailed stable distribution for continuous parameters.

At the end of the day, we are looking for “sensible support”. After playing with this graphical process model representation and changing the parameters, does it look likely that the outcome will be desirable?. Or is our uncertainty about what is going on likely to swamp everything, make the good outcomes about as likely as the bad - in which case we could try and find out more, refine the parameter estimates.

An example, closer to home.

Suppose we are an investment bank, and are planning to launch a new product. The product is complex, will appeal primarily to high net worth individuals and sophisticated investors, and will be marketed primarily through financial advisors. We have an existing, but loose relationship, with these financial advisors and other gatekeepers (trustees of superannuation funds) and some database records of their past purchases – which may give us some clue as to their likelihood of taking up this product. In the past we have used seminars as the prime advisor education tool, and these seminars have been promoted through email, mail brochures and selected mass media. We have some estimate of the likely appeal of these seminars, and the likely turnout. The complexity of the product is of some concern and we are unsure as to what the take-on-board rate might be amongst advisors, and the subsequent conversion rate is also unknown. It may also cannibalize existing product sales. Success is also rather contingent upon a few big players coming on board, not quite the 80/20 Pareto rule but they are not to be ignored – indeed, there may be a domino effect.

If the product was to be successful, if we were to get adequate distribution and take up, it would be highly profitable to us.

How might we assess this with a macro flow model?

Essentially the logic of the market and the decision process is described in the above. ‘All’ we have to do is formalize it, isolate the key parameters (turnout rates, subsequent recommendation rates, conversion rates given recommendation, $ per conversion, the size of the advisor pool, the number of qualifying clients per advisor etc etc), display this ‘market acceptance process’ in a graphical flowcast model and get acceptance of this, run the flowcasts varying the parameters and assumptions across reasonable ranges … to get a distribution of likely outcomes. Piece of cake.

Note what we have achieved here. Without external research or primary data collection and just by explication of what we believe is the market process and our beliefs about the key parameters, we have a simulation of what might happen and of how the outcome assessments are to our assumptions.

A very good starting point. Of course we would definitely use primary research and other forms of modelling (sales as a function of product characteristics) to calibrate parts of this, get better estimates of the parameters etc, all in the interests of convergent validity.

Another example.

Recently in Australia there has been some controversy about an advertising campaign aimed at increasing the number of “long haul” tourists, those visitors from origins such as the UK, Europe, North America .. not the closer countries such as New Zealand. The campaign was aired at great expense, the prior research came up with the slogan “Where the Bloody Hell Are You?” ..

Let’s ignore what might be good or bad about the campaign and the creative execution thereof. And just look at how macro flow modelling might have helped us assess the likely payoff of such a campaign.

It just so happens that some years ago I did do some development of macro flow models in the long-haul travel market. The obvious issues are some form of awareness/top of mind .. what proportion of UK holiday planners include Australia in their consideration set? And to what extent does price eliminate Australia from further consideration? How many always choose the cheapest? How many do repeat visitation? How many of the target market have already visited Australia?

We have, right there, the beginnings of a parameterized macro flow model.

A further issue is the nature of the dependent variable. Are we simply trying to attract visitors .. or dollars? Will the campaign be successful only amongst low budget (but long stay) backpackers.. or will it attract short stay high spenders. Obviously there are many unknowns, but starting with a macro flow model and reasonable parameter estimates, then using research to get better estimates of some of the critical factors, is at least a disciplined and reasoned approach, one where the assumptions and the implications thereof are on display for all interested parties to evaluate and tweak.

This is going to be a short blog because, although I am interested in heavy tailed distributions, it is not my area of expertise and I don’t want to pretend to any special competence or insights. However, I thought you might share my interest .. so, read on.

Ok, heavy tails .. simply means that more of the probability mass is concentrated in the tails of the distributions than in the normal/gaussian distribution.

So, its a relative term, and means that we are more likely to be “surprised” than we think we are (based on our adsorbed gaussian expectations).

Sort of surprise about surprises.

Or shocked about shocks .. we can be very unpleasantly surprised (“shocked”) if something happens (the “shock”) if we dramatically underestimated the probability (even just in our heads, in unspoken and unexplicated assumptions about the shape of the likelihood) of some extreme event – and it happens.

If there is a lot of money attached to this, then it can be a very shocking event indeed. (The question of whether we can, in reality, insure against such shocks we will leave for another time).

So, I was idly perusing a document entitled “Stable Distributions: Models for Heavy Tailed Data” by John Nolan .. it is somewhat heavy going and a definite work in progress, you can googlefind it on the net.

Take out of it what you will, but I was struck by a few things ..

• firstly, that there are sometimes good theoretical reasons for expecting non Gaussian Stable model (Stable distributions being a class of probability distributions that allow skew ness and heavy tails), as in the hitting times for a Brownian motion yielding a Levy distribution.
• secondly, just how different even the simplest of heavy tailed distributions and the Gaussian really really are. This REALLY REALLY matters in practice. A Gaussian (normal distribution) has little probability mass above 3 (standard deviations). Remember the 1.96 sigma rule (2.5% beyond those tails), then 2.58 sigma ..we are down to nothing. But in a Cauchy distribution which looks like a normal distribution (ie is symmetric) we have something like 100 times the probability mass in the tails as we do in the normal. In a sample of data from a Cauchy distribution and a sample of data from a Gaussian distribution, there will be MORE THAN ONE HUNDRED TIMES AS MANY VALUES ABOVE 3 (standard deviations) in the former compared to the latter. VERY NASTY SURPRISES are MUCH MORE COMMON THAN YOU THINK.
• The Levy distribution is a skewed to the right (or left) distribution with all the probability mass concentrated above (below) zero, and even heavier tails than the Cauchy.
• Stable distributions are a generalization of the above (Gaussian/Normal, Cauchy, Levy) allowing for varying degrees of skewness and tail heaviness but they are analytically intractable : this is not such a big deal these days, plenty of grunt power available

OK, if I was heavily into decision making in fields that involved mega bucks and I wanted to avoid nasty surprises, I would be watching this like a hawk. And try learning how to do some sensible implementations.

Finance is the obvious candidate, but get this : “Outliers are Prevalent in Social Variables”. Nicholas Taleb argues this in “The Black Swan: Why Don’t We Learn that We Don’t Learn?” (again, you can googlefind this, or better yet/as well buy his book “Fooled By Randomness”).

I am not going to quote him, I would rather you read him direct – his works are very accessible and very stimulating. Just one point : some of the notions underlying Stable distributions go back a long way to Pareto and thence forward to Mandelbrot (he of the fractals and chaos fame).

I guess it depends on your sets of responsibility and interests as to how much this is relevant to you. There are of course many more immediate and accessible problems to be studied, but to me there is a flavour of an “edge” in this.. if I can study it and do something with it, yes I think it will be profitable (and not just in the sense of insurance against large losses).

fwiw

some implementations (in R) at Jim Lindsey’s site http://popgen.unimaas.nl/~jlindsey/rcode.html

Probability functions and generalized regression models for stable distributions http://popgen.unimaas.nl/~jlindsey/rcode/stable.zip