“Probability does not exist”

de Finetti (1946)

What did the scholar de Finetti mean by this? Why does it matter?

  1. Motivation
  2. Examples (to follow in next post)
  3. Applications (to follow that)

Section One: Motivation

Here, I look at Bayesian subjective or conditional probability as extended logic. I compare it with orthodox, frequentist ad-hoc statistics. I look at the pros and cons of probability, utility and Bayes’ logic and ask why it is not used more often.

In the title, I have used a quote of de Finetti, a Roman scholar known for his intellect and beautiful writing. He meant that your probability of an event is subjective (up to a point) rather than objective. That probability does not exist in the same way that the ‘ether’ that scientists thought existed before the Michelson-Morley experiments of the early 20th century.

Probability is relative not objective. It is a function of your state of knowledge, the possible options you are aware of, and the observed data that you may have, and which you trust. When these have been used up we equivocate between the alternatives. We do this in the sense that we choose our probability distribution so as to use all the information we have, not throwing any away, and so as not to add any more ‘information’ that we do not have. As you find out more or get more data, you can update your probabilities. This up-to-date probability distribution is one of your key tools for making decisions. Many people don’t write it down. Such information may be tacit. If you try to work with probability, it is likely that you may not be using the above logic, i.e. probability theory. You may be making decisions by some other process: be aware!

Recently, I attended a lecture by the Nobel Laureate, Professor ‘t Hooft. He won the prize in the late 1990s for his work with a colleague on making a theory of subatomic particle forces make sense. At the lecture, he expounded a newer theory in which everything that happens is determined in advance. This is called the ‘1/N expansion’. It’s just a theory.

Why did I tell you that? Well, let us go back to de Finetti. Since we can never know all the ‘initial conditions’ in their minute detail, then our world is subjective, based on our state of knowledge, and this leads to other theories, including that of probability logic, which is my topic here.

As human beings, we find this situation really tricky. There may be false intuition. There may be ‘groupthink’. Alternatives may be absent from the calculations. The famous ether experiment mentioned above is an example of the great majority of top scientists (physicists), in fairly modern times, believing in something that turned out later literally to be non-existent, like the Emperor’s New Clothes.

In the ‘polemic’ section of his paper about different kinds of estimation intervals (1976), the late, eminent physicist, E T Jaynes, wrote ‘…orthodox arguments against Laplace’s use of Bayes’ theorem and in favour of “confidence Intervals” have never considered such mundane things as demonstrable facts concerning performance.’

Jaynes went on to say that ‘on such grounds (i.e. that we may not give probability statements in terms of anything but random variables*), we may not use (Bayesian) derivations, which in each case lead us more easily to a result that is either the same as the best orthodox frequentist result, or demonstrably superior to it’.

*In his book, de Finetti avoids the term ‘variable’ as it suggests a number which ‘varies’, which he considers a strange concept related to the frequentist idea of multiple or many idealised identical trials where the parameter we want to describe is fixed, and the data is not fixed, which viewpoint probability logic reverses.

Jaynes went on: ‘We are told by frequentists that we must say ‘the % number of times that the confidence interval covers the true value of the parameter‘ not ‘the probability that the true value of the parameter lies in the credibility interval‘. And: ‘The foundation stone of the orthodox school of thought is the dogmatic insistence that the word probability must be interpreted as frequency in some random experiment.’ Often that ‘experiment’ involves made-up, randomised data in some imaginary and only descriptive rather than prescriptive model. Often, we can’t actually repeat the experiment directly or even do it once! Many organisations will want a prescription for their situation in the here-and-now, rather than a description of what may happen with a given frequency in some ad hoc and imaginary model that uses any amount of made-up data.

Liberally quoting again, Jaynes continues: ‘The only valid criterion for choosing is which approach leads us to the more reasonable and useful results?

‘In almost every case, the Bayesian result is easier to get at and more elegant. The main reason for this is that both the ad hoc step of choosing a statistic and the ensuing mathematical problem finding its sampling distribution are eliminated.

‘In virtually every real problem of real life the direct probabilities are not determined by any real random experiment; they are calculated from a theoretical model whose choice involves ‘subjective’ judgement…and then ‘objective’ or maximum entropy calibration of what we don’t know. Here, ‘maximum entropy’ simply means not putting in any more information once we’ve used up all the information we believe we actually have.

‘Our job is not to follow blindly a rule which would prove correct 95% of the time in the long run; there are an infinite number of radically different rules, all with this property. Things never stay put for the long run. Our job is to draw the conclusions that are most likely to be right in the specific case at hand; indeed, the problems in which it is most important that we get this theory right or just the ones where we know from the start that the experiment can never be repeated.’ (See blog three in this series for some application sectors.)

‘In the great majority of real applications long run performance is of no concern to us, because it will never be realised.’

And finally, E T Jaynes said ‘the information we receive is often not a direct proposition, but is an indirect claim that a proposition is true, from some “noisy” source that is itself not wholly reliable’. The great Hungarian logician and problem-solver Polya deals with such situations in his 1954 works around plausible inference.

Most people are happy to use logic when dealing with certainty and impossibility. This is the standard framework for trillions of pounds worth of electronic devices, for example. Where there is uncertainty between these extremes of logic, let us use the theory of probability as extended logic.

I will next post a second letter here, giving examples of how probability logic works, as compared to frequency statistics.

If you’d like to contact me about the above letter, please write to ‘teiresaas’ at ‘cantab’ dot ‘net’

CIR > Bayes Task Group > Letter 1 (of 3)

Applications of Bayesian decision logic in business

Reading time: 5 minutes scan and reflect

This letter, the third in the series, and a shorter one(!) will focus on applications in business, generalising out from the examples given in the previous letter.

The following is a short list, in alphabetical order, of applications of the approach described in my letters:

  • Acceptance testing for HVM products and services
  • A/B Testing – Is A more desirable/higher value than B?
  • Cricket analysis
  • Customer arrival and resources management
  • Customer lifetime and value
  • Economics and econometrics
  • Environmental regulation fulfilment, controls
  • Game theory strategies
  • Geopolitics
  • Human resources selection/hire/no-hire decisionmaking
  • Healthcare: Medical diagnosis, disease/infection risk, vaccination and course of action
  • Healthcare: therapeutics: to treat or not to treat
  • Cancer prognosis
  • Justice, jurisprudence, case proposition proof or denial via weights of evidence
  • Law, generally
  • Marketing focus decisionmaking
  • Military applications: strategy, defence systems, offence, purchasing, logistics, war/peace games
  • Mineral resource prospecting and archeology
  • Negotiation
  • Preference ranking and prioritisation in logic
  • Political research
  • Policy analysis
  • Pricing strategy
  • Profitability (utility) of options
  • Psychometrics
  • Quality control
  • Quantifying confirmation via evidence
  • Strategy or tactical decision logic
  • Supply chain management and logistics
  • Understanding bias
  • Verification (confirmation or infirmation)
  • Technical diplomacy
  • Venture Capital, valuation estimation for investment choices
  • Weather, forecasting and severity

If you are in any of the areas above or in things apparently parallel, and might like to try decisionmaking using extended probability logic, please let me know! It’d great to catch up. Please send an email, to the author of this blog, via: ‘teiresaas’ which is at ‘cantab’ dot ‘net’

CIR > Bayes Task Group > Letter 3 (of 3)





‘By 2020, we’ll all be Bayesians’

2: Examples and use cases for probability logic in business and governmental organisations

This is the second letter. In the previous letter, I gave an introduction to and motivations for making decisions with probability logic and contrasted it with the ad hoc frequentist approach. We have seven decades of experience confirming what probabilists have said, centuries ago, such as Bernoulli, Bayes, and Laplace. In the following letter, I’ll just give a list of use cases for making decisions with Bayesian subjective-objective probability logic.

Intuition often fails us. When it breaks down under probability logic, should we should blame and reject the logic, or look at and update our intuition?

I will discuss examples, by E T Jaynes and others, in which a wide range of Bayesian probability logical techniques are shown side-by-side in the same paper to be superior or similar with simpler mathematics to ad hoc, frequentist approaches. These are, with lighthearted labelling:

  1. All swans are white
  2. You do play dice
  3. Rain or shine?
  4. What happens next?
  5. To treat or not to treat?
  6. Why not split the difference?
  7. How big is it?
  8. How long will it last?
  9. Controlling the quality
  10. Where is it?

In the foreword of de Finetti’s book ‘Theory of Probability’, a book dubbed destined to be ‘one of the great books of the world’, D V Lindley suggested that ‘by 2020 we’ll all be Bayesians‘.

As of 2023, this has turned out not to be the case. If he and many others unwilling to change their own views to fit the prevailing narratives are right, it is still a great opportunity in a wide range of fields.

  1. All swans are white. The statement that ‘all swans are white’ is susceptible to surprise. We do not know everything about ‘our world’, or put another way, our state of knowledge is that of one world but another real world is what exists. If we take the statement at its face value, it seems logically equivalent to ‘all non-white things are non-swans’. We next see and note that a bird is not white, and is not a swan, which agrees with and is in support of our theory! But if we have our ‘world’ where there are 1,000,000 birds of which 10,000 are swans all of which are white, but the ‘real’ world has 3,000,000 birds of which 1,500,000 are white swans and the rest are black swans. Then since the evidence against ‘our world’ from the observation of a white swan is odds of 1/2 / 10000/1000000 : 1 = 50:1 I.e. actually the observation makes the ‘real’ world alternative much more likely. It is about what alternatives we consider and what information we start with. If you want to pull the wool over people’s eyes, and over your own, don’t let them run Worlds 2, 3, and 4, et cetera.
  2. Playing dice. If a die is rolled N times and the average number of spots facing upwards comes up as 4.5, then we may for sufficiently high N consider the die might be biased, as we would expect a fair die to average 3.5.
    But what then are the probabilities for this die for scoring any given total, i.e. from 1 through 6?
    Well, we can use what is called the ‘maximum entropy principle‘ which equivocates between the options that give us this average value of 4.5, in a way that does not put any more information that we didn’t have into the answers. It can also be called the ‘minimum added entropy principle’. There is a calculation to do, but the unique result gives us a range of values from ‘probability of rolling a ‘1’ = just over 1/20, through to ‘probability of rolling a ‘6’ = just over 1/3.
    These numbers are uniquely determined to give the system the maximum entropy or least assumed information, and that entropy score is lower than that of the fair die of average score 3.5, whose probabilities for any of the results 1 through 6, we know intuitively and correctly to be uniformly 1/6 in each case.
    The constraint on the system of a bias to a higher score of 4.5 on average provides some new information and therefore reduces the overall entropy.
    In my opinion, the probabilities given of 1/20 for a ‘1’ and markedly differently 1/3 for a ‘6’ are not entirely intuitive, although one expects that higher scores on the biased die would increase in probability. We just don’t know intuitively by how much. This principle is really nice because it uniquely gives us the actual ‘best’ probabilities of the options. If our intuition does not narrow down the probabilities reasonably well, then in cases of decisions where a great amount of value is at stake, this argues strongly for taking decisions with the help of this principle, which is entirely unbiased or ‘apolitical’ .
  3. The weatherman. A classic example of how Bayesian inference can give a radically better answer than frequency statistics is that of the weatherman. It is also another example of how intuition can mislead us. This example was also given by Jaynes in 1976.

    This particular weatherman predicts the weather correctly (for simplicity, ‘rainy’ or ‘sunny’) half the time. But from the data, we notice that if he simply predicts ‘sunny’ every time, he is right 75% of the time, from a frequency perspective! Jaynes posed the question: should we relieve the weatherman of his duties, and take on this frequency statistician?

    Well, by looking at the accuracy or inaccuracy of the resulting predictions using a measure of the ‘disorder’ in the resulting sequences (that disorder is the ‘entropy’ we encountered above) and another concept called the ‘joint distribution’ of the actual and predicted weather, we easily show that over the course of a year of 365 forecasts, the frequency statistician’s approach of ‘always predict sunny’ turns out to make the situation worse by a factor of 5 x 1075 while the weatherman improved it by a factor of 3 x 1013 and never predicts sunny when it turns out to be wet. But if you bought into the frequency stats approach, that worse case what you’d go for. I know which I’d want!
  4. What happens next? Bayesian logic also has methods of testing one hypothesis against another. The basic principle is that of Laplace and Bayes from 250-200 years ago, tried and itself tested and common sense: add in the information we already have, our subjective state of knowledge of the situation, and then proceed to equivocate completely or ‘maximally’ fairly between the options that can now happen under the constraints of our subjective knowledge. Using this (‘Laplace Rule of Succession’) approach to the ‘what happens next’ question, seems to have provoked much resistance over time, but it is still unclear why, when this resistance is up against logic.

    Let us take an example. Our past data is 18 positive cases of a certain event X happening in a given environment R, which I won’t specify, out of 21 examples. We now look at some more data but wherein the category of environment is say, M, rather than R, and for this ‘M’ data, there are 3 positive cases of that same certain event X happening out of 9.

    The question we choose to ask is, what are the odds of getting up to and including 3 cases of X out of the next 9 data points, if we started from having 18 from 21 data points? Underlying this, is there something fundamentally different about environment M that is different from that of R?

    Well, you can do this, using an extended form of that Laplace Rule of Succession. You basically look at the number of ways out of the total number of ways from a continued Bernoulli trial set that we would see 3 cases in the next 9 given what went ‘before’ in the 21. Do this and you find that the odds are 1: 146 against this happening. You may then conclude that the environment M probably has something different about it from that of R. And in a decisionmaking context, if the outcome X was ‘bad’, then you might want to recommend changing the environment to be like M, and conversely, if the outcome X was ‘good’, then vice versa, stick with environment R. If such were in your gift.

    If we had wanted to give a statement of the accuracy of this result, it is again very simple for the Bayesian. We have effectively assumed nothing about the prior distribution, i.e. it is called a ‘vague prior’. We have effectively estimated the limiting frequency of the outcome and there is an equally simple formula for the accuracy of that outcome (the variance) in terms of the frequency and the number of trials. But trying to treat this problem as a ‘confidence intervals’ problem turns it from a simple ‘homework’ problem to a difficult ‘research project’, according to E T Jaynes, as ‘we require a new series of tables and charts’. He goes on to explain how the more elegant Bayesian approach tends to yield ‘confidence’, or rather, credibility ranges slightly narrower than those of standard techniques.
  5. To treat or not to treat? We turn now to a very powerful and often non-intuitive area, medical intervention, causation and diagnosis, etc. I follow J Williamson (Source: Williamson, “Objective Bayesianism”), here.

    A good example is that of cancer, recurrence and therapy. The physician and patient must first judge on recurrence, and then on whether to enter into therapy. Judgement is basically a decision problem. We can create simple 2×2 matrices of desirability of outcomes given what actually is the case. First, for example, we have the cases ‘recurrence’ and ‘no recurrence’, and, say, we have the ‘acts’ or ‘judgements’ that there will be ‘recurrence’ or ‘no recurrence’. We set the desirabilities of getting it right to be +1 in each case. We set the ‘getting it wrong’ cases as follows: judging recurrence but there is none to have a desirability of -4 and judging no recurrence but there is actually recurrence has a desirability of -1.

    Now we select the judgement that maximises our ‘utility’. Since the utility is just the sum of the probability of a case times the desirability of the case, given the above, we can calculate that we should choose recurrence if the probability of recurrence is greater than 5/7.

    Now, we consider another 2×2 utility matrix, this time judging to carry out therapy or not vs case of recurrence or not. The case recurrence but having chose not to do therapy is very bad, desirability -20 say. Choosing therapy but the case was no recurrence is also bad, desirability -4 say. Choosing therapy when there was recurrence is good, desirability +5 say, and not choosing therapy when there was no recurrence is also good, +1 say. With these desirabilities, we again want to choose therapy if it has higher utility than not choosing therapy, and this is the case if the probability of recurrence is greater than 1/6. If the case of having recurrence when we have judged no therapy gets worse, ie has a more negative desirability, then the probability threshold for the recurrence must also have tightened to a lower level: i.e. we would only judge no therapy if the probability for our patient is even lower than 1/6th.

    Similarly, if medical science now has a therapy that is less onerous, when we recalculate our decision, the decision logic tells us that we shall want now to choose in favour of therapy if the probability of recurrence is lower than before, all else being the same.

    In other words, the utility threshold for the probability of recurrence being too high will get lower, when either falsely choosing therapy is less bad, or falsely not choosing therapy is worse. If the therapy were no skin off a person’s nose – no time, cost, inconvenience, discomfort – then we might all end up having the treatment because the utility of treatment kicks in when the probability of recurrence is close to zero. I.e. A case of ‘Why wouldn’t you?’ It is sometimes surprising how following through on a simple Bayesian problem can generate ideas commercial or otherwise…

    These probabilities are strictly Bayesian choices. Often, agents (decisionmakers) will not choose the ‘choiceworthy’ option. It appears that when there are two stages in a decision like this, to make the ‘right’ choice, we cannot merge beliefs (because then we can have situations where we judge non-recurrence but we also judge therapy), so we instead merge evidence. This leads us to work out the extent to which we (Bayesians) should believe a proposition, when a given numerical majority of ‘experts’ or consultants with an aggregated reliability threshold (above 1/2) support the proposition. More about that later…
  6. Why not split the difference? The CMO of a large aerospace company is looking to choose between two marketing strategy firms A and B, which have been asked to submit evidence for their work for other similar large companies around lifetime of customers. Marketing strategy firm A presents 9 examples with a mean and standard deviation lifetime of (24.0 +/- 4.36) years, in a normal distribution. Strategy firm B presents 4 similar large companies which had (28.57 +/- 3.70) years. In other regards, the CMO cannot tell the companies apart, but the stakes are high with customers averaging mutiple billion dollars a year in mean value. She instructs two mathematically competent colleagues independently to assess the situation, one using frequentist stats and the other, a Bayesian approach.

    The frequentist colleague looks at the variances and checks with a Fisher-test (don’t worry) that at 95% confidence level, they are the same and pools the data for a new estimate of the variance. With this new figure, he applies the Gosset-test (again, don’t worry) and finds that at the 90% confidence level, the sample doesn’t favour one firm over the other and reports back with this. This is what is taught in business schools.

    The colleague chosen to look at this from a Bayesian perspective focuses on the question: is the customer lifetime of B greater than the customer lifetime of A? She therefore looks at the probability that that of B is greater than that of A.

    This is done by multiplying the two probability distributions together and summing up first all the ‘probability mass’ where the lifetime of B is above that of A and then summing that result up over all value of the lifetime of A from zero to infinity. She obtains a result of odds of over 10 to 1 that the lifetime of B’s customers is indeed greater than those of A. But she goes further having bumped into her frequency-method colleague in the coffee space, and having been told that he had pooled the variances of the two cases. She now does the same and finds that the Bayesian approach now gives odds that the lifetime of B’s customers is longer than those of A are 17:1 on. This approach is not typically taught in business schools.

    It turned out that the frequency colleague had used an ‘equal-tails’ test, which actually looks at the possibility, despite the data suggesting otherwise, that A might turn out fall into the 5% extreme end of the distribution on the other side of B!

    This extreme example, implies that the process driven large company insisting on frequentist approaches and ignoring Bayesian analyses, could choose a partner wherein the performance is worth tens of billions lower value. It shows the adhockeries of the frequentist methods and how the method can be and often is misapplied.
  7. How big is it? Suppose we are in a military conundrum. We have some data about two kinds of missile defence module: type I deploys to a variance in accuracy of 2.2 metres squared and we have tested this 31 times, while type 2 modules deploy to 1.35 metres dispersion using 61 tests. How strong is the evidence for the superiority of type 2 over type 1? Well, on the face of it, the evidence looks clear but the proponent of frequency statistics using a two-tailed test at the 95% confidence level on the variances comes back with a negative, and so again the suggestion is not to differentiate between the two samples with respect to the variances.

    Using the Bayesian approach, similarly to our example 6 just above, the result is advice to the decision-makers that the odds in favour of the type 2 modules are 22.5 to 1.

    This advice is in a useful form! It was clear from inspection of the data that the type 2 modules were superior but the analysis should tell us by how much, i.e. in quantitative terms, so that our decision-making team can decide and report clearly and concisely to whomever it may concern on this matter, and importantly, they can act accordingly. In this case, such action might be to protect the lives of millions of countrywomen and men. A given population should hope that such methods are already being applied in the optimal way…
  8. How long will it last? In this example, we find that the frequency approach is unfathomable: we’ll see that such an approach seems out of reach. In contrast, the Bayesians have a lovely, elegant and straightforward common sense solution.

    We are now interested in the lifetime of a particular industrial machine. We choose a value x0, say, and we want to use the available data and our subjective state of knowledge and belief to determine the probability that the mean life is greater than our chosen value x0. We’ll set an acceptance probability and we’ll see if the machine passes the test.

    We are able to conduct a test on n similar machines for a period of time t, and we find r of them have failed within the test run time t. I.e. We have this much time in which we can run these tests and can get hold of or have made only n machines.

    In frequency stats, we can set a critical number of fails C, and we will accept the machine only if we see fewer than C fail. There is a binomial sum formula in terms of numbers of fails from zero to r, our critical fail lifetime parameter and our number of units tested, n.

    In this approach, we shall need to run a certain number of tests in order to obtain assurance that our fail rate is lower than our critical level with a given ‘significance level’, such as 90%. Now if we can only test for a fraction of our critical fail time, the inverse of the fail rate, say 1% with say 3 critical fails, then we shall need to test a very large number of machines (many hundreds of them).

    If we are talking about some high value manufactured complex product, like a space rover, things can become impossible very quickly. Suppose we need to obtain a result within an actual time period of 5% of our critical lifetime, and are wed to the standard 95% significance test, then we must build 97 space vehicles to test! The situation would be turned back by the executives to the statisticians, and if they were only able to use standard frequency methods, there might be a strong desire and great spent resource for resolving the problem, but only a slight ‘give’ after much similar effort, and the situation is still untenable.

    The Bayesian approach here uses the actual times of failures that is ‘thrown away info’ in the frequentist method. The prior knowledge of engineers and makers in development is also not thrown away.

    The type of information the Bayesian approach gives us is also much more exact and useful: what is the probability that this space vehicle will last longer than the need on the one-off mission? What is the probability of exactly r fails in that time t?

    The Bayesian formula tells us exactly this, and involves the same sum derived by Bayes in his original 1763 paper “..Towards Solving a Problem in the Doctrine of Chances.”

    Assuming no prior information, the much simpler and higher relevance Bayesian formula is seen to be the same as that of the frequentist approach, using sequential, actual fail times to our direct advantage, we avoid pessimism and we answer a much more pertinent question for our client.

    Performance of both methods turns out to be similar if there are few failures, which can by chance be the case, of course, but if all or too many units fail, our frequency method tells us that we must reject the vehicle, even if our ‘real’ lifetime could turn out to be many times greater than our critical level. The Bayesian test doesn’t go wrong in this way and gives us a usable, common sense result, i.e. the probability that this vehicle will last longer than our critical lifetime. The Bayesian test can be improved further to give us the simple formula for the probability that a given vehicle will last longer than a given time, as a function of the fail-free test time, the average test fail time and another ‘subjective’ or expertise-led time of life which our engineers had reason to believe in at the outset.

    In summary, unlike those of the frequentist approach, the Bayesian test gave us common sense results. These results hold in various conditions, needing only achievable new observable data. They take prior information into account, as well as update test data and then provide the analysis that does not introduce arbitrary assumptions into results, which results are expressed in a useful and intuitive way.

  9. Control the quality! Not everyone is aware, even in these days of consumerism and fast fashion, that certain products, perhaps more than we might think, are specifically designed to last reliably until the warranty runs out, and then to break or decay with a probability distribution not unlike that of a radioactive isotope, the shorter the half-life, the better for the provider, depending on their approach and ethos. This may take the form of, say, inhibitors added in quantities that are calculated to run out after a given time. Also, the seller can then offer a premium longer warranty that is priced to make the strategy remain in profit, using the above knowledge.

    What the Bayesian can do in this situation, is to obtain with a preassigned probability that the parameter representing the ‘half-life’ lies within a range which he or she calculates and gives to the ‘client’.

    For those with a mathematical bent, both frequentists and Bayesians in this case can start from a ‘truncated exponential distribution with a “location” parameter’ corresponding to the time when the product is almost certain not to break down for the above reasons.

    The Bayesian can simply assume she doesn’t know anything in advance about how long the product will work, from that truncation time to infinity. The frequentist has to find a sampling distribution for the parameter, and this turns out not to be possible without numerical methods, and the project becomes difficult very quickly, even with a small sample. In contrast, the Bayesian can see the answer almost by inspection, i.e. the shortest possible range that contains the desired probability in the resulting ‘posterior’ probability distribution. Further, the frequentist’s estimated range actually makes no sense as it lies in a place which is not possible, i.e. before the truncation (inhibitors have worn out) has ended, a consequence of having chosen a very small data set in order to find a solution. In the idealised long run, the frequentist method will work and will be close to the Bayesian one.

    Technically, this is due to the requirement for complete statistics being present in the ad hoc confidence interval approach. The Bayesian method, dating back 300 years, automatically includes all relevant information in the problem and gives a working common sense answer.

  10. Where is it? This is a tale of cases where we have fairly sparse data and of the importance of not throwing any away.

    [Cauchy and risk of of frequency estimators: motto: don’t throw away info and focus on the next not the long run…]

    We keep in mind a real war-and-missiles specific case. Suppose our intelligence team has set out an overlay map with constant intervals across it along one ‘axis’ across the field of ‘view’. We only have two data points, corresponding to the possible exact origin of a missile launcher. In this case, it is obvious that the frequentists’ penchant of talking about the ‘long-run’ is not going to cut it, since it may be all over if the first strike in the vicinity fails. We must put our best foot forward in our very next action. As in the majority of cases, the long run performance is never realised. It’s a highly dynamic situation.

    In the orthodox approach, we are torn at the point of determining an estimator. Taking the average of the two datapoints seems common sense, but any choice will yield the same length 90%-confidence interval and in the long run, the resulting interval will yield the same quality of results for those choices.

    In our specific case at hand, though, if we choose either point, then the other point worries us greatly. Let us assume that the two sources are sufficiently far apart, such that if we choose one and attack it, and we are wrong, the other can comfortably retaliate.

    In the Bayesian case, there is a unique optimal ‘estimator’. But it is also clear we have another, independent piece of information that we can easily use in the Bayesian approach, i.e. the separation of the two data points.

    Without introducing the mathematics following from this, we see that this additional information improves the analysis, but the specific sample used will result in greatly varying confidence interval lengths, and if the data points happen to be further apart, then the long run can be wrong much too often, for example the case can be wrong 90% of the time, for a 90% confidence interval. This is obviously not good for decisionmaking.

    By a further clunky method, an orthodox statistics process can be set up to average 90% correct in the long run, using a technique used by Jaynes called the “uniformly reliable” interval. This method yields a tighter interval for distances between the points that are below average. But again at the top end it has to be more conservative than the confidence interval. But it is much better than the original method which was poor for the majority of cases of data point differences.

    Now over to the Bayesian approach…a much simpler and more elegant piece of mathematics gives us back the same answer as the ‘uniformly reliable’ interval, using a completely vague uninformative prior distribution for the location parameter, i.e. the position of the enemy launcher.

    If you are at an organisation for which you wish to make rational and optimal decisions, using your best subjective state of knowledge and the data at hand and nothing but the data at hand, however ‘big’ or however sparse, then if you are not applying probability logic in your core and wider decisions you are highly likely in the long run to be throwing away value in your business, and typically, the larger the business, the more so.

    In the next, third letter, I will list some application or use case areas of the Bayesian approach in business.

You can write to us at ‘teiresaas’ at ‘cantab’ dot ‘net’, I’d love to hear from you about topics related to the above letter.

CIR > Bayes Task Group > Letter 2 (of 3)










Programmes Cleanpower Smart Grids 2019 Cambridge 1-2 July

Summit Guide: 10th anniversary Cleanpower Smart Grids Conference 2019
Events home: www.cir-strategy.com/events
Direct fast tickets: https://mysplink.com/cir/shop
Formal Registration: http://www.cir-strategy.com/c4ir/cpsg19/register/#shop
1 July Day 1 – Technology Executive Briefing Day 10:30am – 4:30pm
10.30-11.00 Registration and networking
Session 1 11.00-13.00 Introduction to clean power and smart grid energy systems
11.00 – 11.15 Gavin Jones – Chair’s Opening Remarks
11.15 – 11.45 Erwin Frank-Schultz, IBM, CTO Energy, Environment and Utilities “Energy Systems and Digital Twins”
11.45 – 12.00 Mash-Hud Iqbal, Partner, Marks and Clerk “IPR and Energy innovation”
12.00 – 12.10 David Richardson, Innovation Lead – Energy Systems, InnovateUK “PFER Programme and smart local energy systems design”
12.10 – 12.35 Sylvain Vittecoq, CTO, CyanConnode “The benefits of RF mesh networks for smart metering, smart grid and IoT”
12.35 – 13.00 Discussion & Q&A
led by Gavin Jones

13.00 – 14.00 Lunch

Session 2 14.00 – 15.15 Energy storage & battery technologies
14.00 – 14.20 Professor Vasant Kumar, Cambridge University “A brief sprint through battery science”
14.20 – 14.40 Dr Rumen Tomov, CJET Ltd “Novel storage and battery materials”
14.40 – 15.00 Daniella Sanchez-Lopez, Cambridge University Research Fellow, “Li supply chains and battery tech”
15.00 – 15.15 Panel moderated by Professor Vasant Kumar, Cambridge University Materials Science, leading battery scientist and author, with Ian Ellerington, The Faraday Institution & and the Space Catapult (invited) Mining Li in Cornwall (panellist)

Tea break 15.15 – 15.45

Session 3 15.45 – 17.00 Data, ML & security technology
15.45 – 15.55 Mike Handley, PolyChord, “A grounding in ML”
15.55 – 16.05 Eric Topham, Business Development, T-DAB “The ML Use Case for Energy Optimisation”
16.05 – 16.20 Dr Natalie Lowery, Energy Systems Catapult “Modelling Energy Systems”
16.20 – 16.40 Dr Andrew Tsonchev, Director of Technology, Darktrace Industrial, “Using AI for Real-Time Threat Detection across OT & IT”
16.40 -17.00 Session Q&A led by Gavin Jones
16.55 Close for Day

18.30 – 21.00 Dinner evening
18.30 reception drinks networking for 19.00 – 21.00 Dinner hosted by CIR at Christ’s College Cambridge

Day 2 – Innovation Conference 09:00 – 17:00
09.00 – 09.30 Registration and networking
Session 1 09.30 – 11.00 Introduction: energy system futures and innovation and threats
09.30 – 09.40 Conference Introduction & Chair’s Introduction: Jeremy Nicholson, Alfa Energy Corporate Affairs & VP/Chair IFIEC Europe
09.40 – 10.00 Richard Smith, Head of Commercial, National Grid – “Operating the GB transmission system, carbon free, by 2025: opportunities/challenges”
10.00 – 10.15 Anser Shakoor, ABB, “Future of the energy systems mix”
10.15 – 10.30 Victoria Doherty, QinetiQ, “Power, storage and human factors in cybersecurity”
10.30 – 10.50 Emily Orton, CMO, Darktrace, “Cybersecurity and energy infrastructure and devices” Keynote
10.50 – 11.10 Panel with moderator Pamela Taylor, Taylor Macpherson

Coffee break 11.00 – 11.40

Session 2 11.40 – 13.00 Grids, connectivity innovation
11.40 – 12.00 Dr Sean Cochrane, Head of Technical Sales, CyanConnode, “Commercialising an IoT communication platform”
12.00 – 12.15 Nick Merricks, Landis+Gyr, “Grid edge intelligence in the smart grid”, securely.”
12.15 – 12.30 Moixa Energy, CEO Simon Daniel “Connectivity and energy management of smart grids and off grids”
12.30 – 12.40 Jim Lott, Technical Lead – Energy Systems Catapult “Prospering from the Energy Revolution”
12.40 – 13.00 Panel with moderator Gavin Jones and panellist Jane Lucy, CEO, Labrador

13.00 – 14.00 Lunch Networking

14.00 – 15.20
Session 3 Storage / battery innovation and commercialisation
14.00 – 14.05 Ian Ellerington, Head of Tech Transfer, The Faraday Institution – “Introduction & the importance of storage”
14.05 – 14.20 Richard Druce, Associate Director of Energy, Environment & Infrastructure, NERA “Distributed energy resources and how to monetise them”
14.20 – 14.35 Georgina Dingley, Business Development, Anesco, “Utility scale energy storage”
14.35 – 14.45 Dr Gleb Ivanov, CEO, Sigma Lithium “Anode technology for advanced energy storage, Li primary and rechargeable batteries”
14.45 – 15.00 Dr Athan Fox, CEO, Aurelius Environmental “Out of the furnace and into the leaching tank”
15.00 – 15.20 Panel moderated by Isobel Sheldon, UK Battery Industrialisation Centre (confirmed)

Tea break 15.20 – 15.45

Session 4 Clean energy policy & strategy – final panel
15.45 – 17.00
15.45 – 16.00 Centrica/SSE Strategy/Grids Speaker tbc
16.00 – 16.15 Sally Fenton MA, Innovation Manager, Dept of BEIS, “Delivering innovation to the Grand Challenge of Clean Growth”
16.15 – 16.30 Akshay Kaul, Director, Systems & Networks, Ofgem
16.30 – 16.55 Panel with moderator Judith Ward, Sustainability First
16.55 – 17.00 Chair closing remarks by Jeremy Nicholson, Alfa Energy and Gavin Jones
CLOSE OF EVENT

2019 Advisory Committee – CPSG
Dr Justin Hayward MBA, CIR (Executive Chair)
Maxine Frerk, Grid Edge Policy
Gavin Jones
Jeremy Nicholson, Alfa Energy & VP Chair IFIEC Europe
Professor Vasant Kumar, Cambridge University
Ian Ellerington, Head of Tech Transfer, Faraday Institution

10th Anniversary Cleanpower Smart Grids Conference Expo

This summer, the Cleanpower & Smart Grids Conference Summit 2019, an innovation event on the Grand Challenge of clean growth, reaches its 10th birthday in Cambridge.

The conference will approach the innovation challenges of clean energy systems (both generation and smart grids), energy storage and efficiency via connectivity, AI and data and materials and with cyber-security in mind also using ML.

1-2 July 2019

10th anniversary Cleanpower Smart Grids Conference 2019 #CPSG19

Innovations in energy, digital connectivity & ML/MI & data to solve the Grand Challenge of Clean Growth
Clean Growth is certainly one of the key Grand Challenges. It has perhaps never been more important to focus on clean energy systems in a whole-system way and to consider how to move forward with a fully sustainable, robust and affordable global energy system. Please join us for this 10th anniversary celebration of this conference series Cleanpower and Smart Grids, where we will try to make progress towards these objectives. We’ll be focusing not only on clean power generation such as renewables and on digital connectivity and cybersecurity for smart grids, through companies like CyanConnode and Darktrace, but also on energy storage and battery technology and energy efficiency through a range of companies such as Moixa.

– Justin Hayward, Director of Conference Summit, CIR

Day 1: Executive briefing day and evening banquet dinner at King’s
Day 2: Executive conference summit

Events homepage

http://www.cir-strategy.com/events/

Ticket shop (open now with early bird pricing):
https://mysplink.com/cir/shop/

Sponsorship Opportunities
Your support in enabling this conference is much appreciated by the organiser and team.

We hope to provide great publicity and showcasing for your technology, innovation and services.

For marketing sponsorship and exhibition positions, please call CIR directly on +447720047402 and join a roster of excellent organisations that have participated since 2009.

Cambridge Investment Research(CIR) (founded 2002)
CIR has run 50 technology commercialisation conference days since 2002, in four cities in the UK including Cambridge, London and Oxford.
There have been over 4,250 attendees and over 500 speakers and the internal list reached over 50,000 in 2019, with media partners and other promotions likely to reach a relevant audience of well over 300,000 executives, investors, inventors & innovators, industrial academics and public sector senior staff.

 

5th HVM New Materials Conference Expo 6-7 November 2019 Cambridge, UK

CIR has launched its 5th HVM & New Materials 2019 event in association with Cambridge Graphene Centre and Manchester Graphene Engineering Innovation Centre and National Graphene Institute on large industrial sector applications. This conference series since 2013 is independent of all other conference, market intelligence and corporates.
You are welcome to sign up now as a delegate, or call us about a stand presence or sponsorship. The team is now getting in touch in particular with sponsors seeking to partner and market with this prestigious international conference series. Some exciting growth companies are already confirmed. The dates for the diary are

6-7 November 2019 in Cambridge with a banquet dinner on 6th – always a highlight of CIR Conferences.

Innovations in advanced & functional materials and composites including graphenes; HVM; digital connectivity; ML and AI and data to solve the Grand Challenges of future mobility & transport, ageing population and clean growth.

Events information pages
http://www.cir-strategy.com/events

Ticket shop (all pricing is lower than at this successful summit in 2017!)
https://mysplink.com/cir/shop/

For sponsorship marketing and exhibition please call 07720047402

CIR plans to take time to build the best and most coherent and valuable agenda yet, with an executive briefing day which will update on relevant sectors prior to the dinner and in advance of the conference summit talks the following day.

5th HVM GNM 2019 Conference Summit – in association with Cambridge Graphene Centre and Manchester Graphene Engineering Innovation Centre following on from HVM Conferences since 2002 and Graphene New Materials Conferences since 2013, this executive conference summit will bring together technology innovators and entrepreneurs to solve grand challenges.

CIR has run 50 technology commercialisation conference days 2002 – 2017 in four cities in the UK. There have been over 4,250 attendees and over 500 speakers and the internal list reached 75,000 in 2017, with media partners and other promotions likely to reach a relevant audience of well over 300,000 executives, investors, inventors & innovators, industrial academics and public sector senior staff.

 

Top 10 Reasons to attend 15th Anniversary HVM and 4th Graphene New Materials Conference Summits 3 November 2017

Book early here to secure tickets

1. Network with over 100 senior executives in sectors such as (1) aerospace & defence, space; (2) automotive; (3) electronics & displays; (4) sensors & devices; (5) energy (6) energy storage & heat transfer; (6) printing & packaging; (7) healthcare & biomedical.

2. Visit leading exhibitors from all over the world to see the latest in HVM and New Materials technology and to form and enhance relationships with buyers, suppliers and other key players in the new materials and HVM value networks

3. Be among the first to structure how to overcome the barriers in the HVM 4IR and new materials value network in order for your company to thrive, by attending the exclusive masterclass alongside industry leaders and led by a world-class coach

4. Gain media exposure via all physical events and online content e-news

5. Keep up to date and “juice your brain” on the latest topics in the industry at the business conference by listening to over 30 market-led talks that have been requested by past delegates through many detailed surveys

6. Dine at the beautiful King’s College with high-level decision makers, industry leaders and experts to continue conversations and further expand your network

7. See labs and departments opportunities to connect with Cambridge University

8. Cambridge is not only home to one of the most historic and successful universities in the world, but is also one of the most well established science & technology clusters in the UK with over 1,500 tech companies based here, 14 of which are worth over £1bn

9. CIR the strategic consulting firm, has 15 years’ experience organising over 48 high-quality, market demand led events

10. The new materials and HVM industry4.0 races are on – attend this unique and highly valuable co-located double conference to make sure you build your knowledge and value network & don’t get left behind

ARM the tech leader gives sessions at Smart Grids Conference Cambridge 19-20 June

Book fast | Book medium pace with more info | All C4IR events ticket shop | SGCP17 Event BrochureEventbrite with all concessionary, all-event & 1-day-only tickets.

We’re delighted to welcome the technology cluster leader ARM to the conference. ARM have come to dominate smartphone chip design in recent years after founding under 25 years ago in Cambridge. They are attacking server and IOT segments and also have interests in investment at the conference.

Dr Amyas Philips  of ARM IOT, covers maintaining leadership in a rapidly developing market with equally rapid technology evolution is a growing multidisciplinary challenge for both small and multinational enterprises.

Session Synopsis
Technology providers are rapidly maturing their various IoT platforms, all offering to solve the engineering problems of transitioning your business to being data-driven, so that you can focus on what you do best: energy generation, storage and distribution.  It’s de rigueur to acknowledge that exploiting data is as big a challenge as getting it in the first place – and to leave it at that.  Instead of doing that, I will look at the challenge from a digitising business’ point of view and show that the problem can be collapsed into a few key technology and business model decision points, with the options available at each.

Dr Amyas Philips will work with a colleague from IBM to deliver a fascinating technology session on 19 June.

ARM will deliver the Keynote at the start of 20 June and ARM’s ventures colleagues will give a short investment landscape talk also on 20 June.

3 weeks to go to this event and it will be very interesting and timely indeed, one not to miss! In order to attract startups, we have a few tickets available for the Business Conference on 20 June at just £149 pv(please email us or buy these & a full set of special options via Eventbrite here (+ card fee)), courtesy of ARM sponsorship of the conference. ARM, the global chip design ecosystem leader returns to SGCP17. Along with ARM & UtilityWise, sponsors also include NERA and Cambridge Consultants and media partners Pan European Networks & Cambridge TV.

Among expert industrial speakers are UK Power Networks (speaking on Microgrids), RWE (speaking on market efficiency and policy), EON (speaking on markets and math modelling them), UtilityWise (transitive grid); and from industrial academia Prof Andrew Cruden (speaking on energy storage technologies and applications).

Confirmed speakers
Dr Amyas Philips, Technology Director ARM Holdings plc, Mike Dimelow, ARM Accelerated Ventures
Andrew Strong, Business Development Manager, Cambridge Consultants
Joanna Hubbard, COO, Electron – Blockchain
Alex Bak, Founder, LightFi
Simon Daniel. CEO, Moixa Energy
Jon Ferris, Strategy Director, UtilityWise
Gavin Jones, GJC, Co-founder of Smart Grids GB
Sally Fenton, Innovation Manager, BEIS (formerly DECC)
Dr Erwin Frank-Schultz, Master Architect, IBM
Nick Easton, Future Whole Systems Programme Leader, National Grid
Jeremy Nicholson, Senior Adviser, Energy Intensive User Group, EEF
Professor Jerker Delsing, Project Co-ordinator, EU Arrowhead Smart Grids Sweden
Professor Andy Cruden from Energy Storage & Applications, an EPSRC initiative
Richard Druce,
Associate Director, NERA
Professor Peter Sharratt, Head of Strategy, WSP Global, London
Christos Keramisanos, Power Systems Senior Engineer, UK Power Networks
Harsh Pershad, Lead Technologist – Energy projects, Innovate UK
Greg Payne, Business Modeller, E.ON plc
Ben Willis, Corporate Development Strategy Manager, RWE
Ofgem 
Anant Prakash BP plc – Energy Outlook

Come and meet over 30 speakers and expert moderators over two days with a conference dinner overnight. Our pricing is simple and bookable easily on mobile or desktop. There are just two types of ticket: with or without event hotel. If you can only attend one day or the other, please call us on 01223303500 and we can arrange booking for this if available.

If you buy a ticket at £495 for the 2 days with dinner – you can also take a free table-top banner stand. There is a large subsidy for investible scale-ups with under 4 staff via ARM – consultants not included in this offer. These and other, larger innovation companies may also take an included 90 second pitch in plenary on 20 June with their ticket. With two tickets for a total of £990, you can build a large 3m stand at the venue for the 2 days & have a pitch. With 3 tickets, at £1,485 you can also include a speaking role (if agreeable and a fit to programme)Silver Sponsorship: 4 tickets at a total of £1,980 you obtain the above with months of active marketing online presence.

Book now and exhibit your great products and services free with your ticket/s!

 Conference Outline 

This 8th high level conference expo (with lead event sponsor ARM) brings together innovators, kit manufacturers with smart grids project buyers. It develops themes requested by global participants in previous series conferences since 2009, and new trends, drivers, innovations, solutions, the global market & value network.

Dr Justin Hayward, Director of C4IR added: “Just 3 weeks to go now to this event and we are delighted with the programmes and attendee mix! We have a VC & ARM-led innovation pitching session on the 20th at the business conference and a high quality delve into basics, technologies, markets and innovation on the 19th in the briefing day.”

Book fast | Book medium pace with more info | All C4IR events ticket shop | SGCP17 Event BrochureEventbrite with all incl concessionary tickets. Or call C4IR Maya on 07720 047 402 to book or discuss event.