The other day at Nature there was an article called Neurobiology: The currency of guessing where there was a small box describing how to use log-likelihood ratio (LogLR) to make an optimal guess from collected statistics and new measurements. The article is actually about a research that found a similarity in the technique and the way some neurons work…

I’ve been wondering: isn’t it possible to deduce the formulae for this LogLR technique from information-theoretical concepts? The calculations of entropy and mutual information are all based on the logarithms of probabilities…

I fiddled with the equations a bit, and managed to rewrite the criterion of the example in that article based on the Kullback-Leibler divergence, the entropy of the probability of rain, and another term that still lacks an interpretation…

Does anybody here believe that it might be possible to find at least an approximate criterion, based purely on information theory, that would work like the LogLR technique for making predictions?…

The calculations I did until now seem to depend on the fact that the variables are binary. This makes the probability of one possibility be 1 less the probability of the other, and this results in a subtraction of logarithms and consequent division of probabilities… I can show what I did if anyone is interested…