When the probability of an outcome is affected by the preceding outcome

Here is a problem I have been tackling. I need to make this an equation in Excel, but I have been driving myself nuts over it for over 24 hours now and feel like I need to reach out for help. Any input is most welcome, explanations invaluable! Thanks!

"There is a pellet dispenser which releases 2 pellets a second from a reservoir consisting of 90% white and 10% black pellets. Whenever a black pellet is released, for the next 5 seconds the dispenser releases pellets from a second reservoir consisting of 95% white and 5% black pellets. If a black pellet is released from the second reservoir, the timer resets and the dispenser stays on the second reservoir for another 5 seconds. On average, how long does the dispenser stay on the second reservoir in a 5 minute time frame?"

I'm trying to solve this with zero terminology knowledge but some digging shows me that this might be a Markov Chain. Is it? Are they solvable with variables in Excel?

woodsielord said:

Here is a problem I have been tackling. I need to make this an equation in Excel, but I have been driving myself nuts over it for over 24 hours now and feel like I need to reach out for help. Any input is most welcome, explanations invaluable! Thanks!

"There is a pellet dispenser which releases 2 pellets a second from a reservoir consisting of 90% white and 10% black pellets. Whenever a black pellet is released, for the next 5 seconds the dispenser releases pellets from a second reservoir consisting of 95% white and 5% black pellets. If a black pellet is released from the second reservoir, the timer resets and the dispenser stays on the second reservoir for another 5 seconds. On average, how long does the dispenser stay on the second reservoir in a 5 minute time frame?"

Click to expand...

What is the rate of release from the second reservoir? Answer would be different if there were a drop every second of that 5 seconds as compared to a drop once during that 5 seconds. However, I did sort of get started so here it is:

The way I would approach it but then I'm not so great at probability problems so I would hope for other comments also.

Will need to fix, but assume 100 pellets to start with [as number of pellets get very large, we get a 10% chance each time rather that the reduced percent if there are a 'small' number of pellets to start with]. The probability that a black pellet will be released is 1 minus the probability that both pellets will be white = 1 - \(\displaystyle \frac{90}{100}\frac{89}{99}\). So, on average, every 2 seconds about 0.19091 pellets are released from the main reservoir or about 9.545% of the time a black pellet is released from the main reservoir. During that 9.545% of the time Whoops - got stuck.

NOTE: We are assuming that the reservoirs are being refilled each time with the same type of pellets which were released in order to keep the probabilities the same each time. Either that or the population of black and white pellets is so large that depletion makes no significant difference in the probability over the length of time of 'the experiment'. We are also assuming that the second drop has a memory, i.e. suppose we had a drop a black pellet from the main reservoir and during the time of the five seconds of release from the second reservoir, we have another drop of a black pellet from the main reservoir. Then that additional 5 seconds would just get tacked on to the end of the first five seconds and so forth. That would also apply to the secondary release.

Ishuda said:

What is the rate of release from the second reservoir?
...
We are assuming that the reservoirs are being refilled each time with the same type of pellets which were released in order to keep the probabilities the same each time. Either that or the population of black and white pellets is so large that depletion makes no significant difference in the probability over the length of time of 'the experiment'. We are also assuming that the second drop has a memory, i.e. suppose we had a drop a black pellet from the main reservoir and during the time of the five seconds of release from the second reservoir, we have another drop of a black pellet from the main reservoir. Then that additional 5 seconds would just get tacked on to the end of the first five seconds and so forth. That would also apply to the secondary release.

Click to expand...

The rate of release is fixed, two pellets per one second.
The reservoir size is insignificant (infinite).
The drops have a so-called memory only in-so-far as blacks. The dispenser knows to switch only 5 seconds after the last black pellet.
The timer doesn't get an additional 5 seconds for a black drop, it gets reset to 0. So if a black pellet drops right after a black pellet, the second dispenser still has 5 seconds to go, not 9.5.
I'm stuck as well

woodsielord said:

The rate of release is fixed, two pellets per one second.
The reservoir size is insignificant (infinite).
The drops have a so-called memory only in-so-far as blacks. The dispenser knows to switch only 5 seconds after the last black pellet.
The timer doesn't get an additional 5 seconds for a black drop, it gets reset to 0. So if a black pellet drops right after a black pellet, the second dispenser still has 5 seconds to go, not 9.5.
I'm stuck as well :smile:

Click to expand...

Just ramblings since it looks like it is a mixed process. That is the drop from the main reservoir is a continuing process and does not transition but, letting A_j be the process 'j seconds to go for second reservoir' We have
A_j -> A_j-1 = 0.9025; A_j -> A₅ = 0.0975, j = 1, 2, 3, 4, 5
A₀ -> A₀ = 1.00
and this process is only dependent on where it is at the present time, not what has happened in the past. This would result in a sparse 6X6 transition matrix (each row would have only two non zero entries at most) so raising the matrix to a power might not be too difficult - an Excel program could be built although some non-usual options might have to be used, i.e. automatic computation and order of computation options to mention two. If this converged, you would then have a percentage of time spent in each process. Since the first process (drop from the main reservoir) would start this "chain" 19% of the time (= 1 - 0.9 * 0.9) this would give you a total time spent when the second reservoir would be 'on'

Looking at
http://en.wikipedia.org/wiki/Markov_chain
we see that flipping a coin could also be modeled as a Markov chain but it is much easier to do the modeling the 'old fashioned way'. It may be that this problem could be tackled in an easier fashion also.

Ishuda said:

Looking at
http://en.wikipedia.org/wiki/Markov_chain
we see that flipping a coin could also be modeled as a Markov chain but it is much easier to do the modeling the 'old fashioned way'. It may be that this problem could be tackled in an easier fashion also.

Click to expand...

Aha! You're right. I can just calculate the likelihood of 5 seconds of straight whites after a black, and go from there. I'll look into that.