How to measure confidence with a fraction/ratio?

BeeCuz

New member
Joined
Jun 6, 2016
Messages
26
I have a fraction where the numerator and the denominator are intermittently increased, a tally expressed as successes/failures. There is a notion of confidence associated with it, in that higher numbers represent larger sample sets.

I'd like to measure that confidence somehow. It looks sort of like a standard deviation might fit the bill maybe, but how?
 
I have a fraction where the numerator and the denominator are intermittently increased, a tally expressed as successes/failures. There is a notion of confidence associated with it, in that higher numbers represent larger sample sets.

I'd like to measure that confidence somehow. It looks sort of like a standard deviation might fit the bill maybe, but how?

Please explain your question more fully; an example, at least, would help. It is not clear in what way the numbers are "intermittently increased", what counts as success, and what you mean by "confidence".
 
My application is part of an AI neural net that stores judgments in memory, used in a video game.

An example is an assessment of the outcome of firing a type of weapon against a type of enemy. Did it kill the enemy? This success/failure test results in a tally mark to the question every time, either a new success or a new failure. The accumulation of trial results eventually settles into a pattern closely approximating a theoretical ideal model.

During the first few runs random noise is a problem, but as the successes and failures accumulate, the fraction successes/(successes+failures) settles into a good measure. What I'm interested in is this propensity for error based on limited sample size, and how it reduces as the count of samples increases.

A method of storing short-term memories vs longer-term memories is to put a different ceiling on the total tallies in two different nodes. One stores every trial and the other stores only the most recent x trials. Where there is a significant difference in the proportion is an indicator that something has recently changed, an alert for the AI to behave differently than the total accumulation would suggest.
 
Last edited:
Top