I want to thank you for your reply and the effort to answer my question! I am still confused.
Let's look at it as follows. Ignore the 80% part for now.
Why does the following make sense? (amount insured)/(value of house) * (loss by fire) = amount paid by insurance company.
JeffM indicated that (loss by fire)/(value of house) * (amount insured) = amount paid by insurance company, but I doubt that an insurance company would use this calculation but perhaps one would.
OK. Let's take this in steps.
As denis said, it is a mathematical fact that
\(\displaystyle \dfrac{a}{b} * c = \dfrac{a}{b} * \dfrac{c}{1} = \dfrac{ac}{b} = \dfrac{c}{b} * \dfrac{a}{1} =\dfrac{c}{b} * a.\)
In short, \(\displaystyle \dfrac{a}{b} * c = \dfrac{c}{b} * a.\)
Second, you may be mixed up about order of operations of inline expressions.
\(\displaystyle a / b * c \text { means } \dfrac{a}{b} * c, \text { not } \dfrac{a}{bc}.\)
Inline notation is a minefield of potential confusion. Parentheses are critical.
\(\displaystyle a / (b * c) \text { means } \dfrac{a}{b * c}.\)
You are of course correct that \(\displaystyle \dfrac{a}{bc} \not \equiv \dfrac{c}{ab},\)
but no one is saying that.
Third, let's go back to the logic of coinsurance.
Guy says to insurance company, "Insure my house worth 80,000 against fire." Insurance company says "Ok, that's a premium of 100 a year." The house is completely destroyed by fire, and the guy presents three appraisals from reputable appraisers saying the house is worth 800,000. Guy demands 800,000 in payment because the insurance company insured the house against fire damage. The insurance company says, "Wait a minute. You bought insurance for a house you said was worth 80,000, not 800,000, and you only paid premiums on a house worth 80,000."
What do you think the insurance company should pay? If you answered 800,000, how long will anybody be able to offer insurance?
You might say 80,000, but the insurance company might respond that there was fraud in the inducement and that therefore they owe nothing because the courts should punish fraud as a matter of public polcy.
Now keep the same facts except assume that the guy declared a value of 500,000 rather than 800,000. It is a lot harder to claim fraud. It may have been simple mistake. Most people don't pay an appraiser when they buy an insurance policy. Saying that the insurance company should pay 500,000 when they insured the house for 500,000 doesn't hurt the insurance company: that was the risk that they were paid for.
So without coinsurance, the insurance company pays
\(\displaystyle \text {actual loss} * \dfrac{\text {declared value}}{\text {actual value}}.\)
Does that make sense?
But the insured wants the actual loss if one occurs, not a fraction. Nor does the insured want to pay a premium for a lot more than the house is worth. (Nor does the insurance company want people over-insuring and burning down their insured properties: moral hazard.) So the insurance company offers a deal. You the insured make a reasonable estimate of the value of your house, but don't overinsure it. In return, we shall charge a premium based on that estimate. If it turns out you have a loss and your estimate is in fact reasonable but somewhat low, we shall pay the actual loss. But if your estimate is a way out of line, we are going to go through this convoluted process.
If the coinsurance rate is 80% and your estimate is 80,000, but the value of your house is 800,000, we are going to take 80% of 800,000, which is 640,000. That's what you should have declared the house to be worth. But you said it was worth only 80,000. The degree of deficiency is
\(\displaystyle \dfrac{80,000}{640,000} = 12.5\%.\)
So we will only pay 12.5% of the actual loss.
Let
\(\displaystyle p = \text {payment due;}\)Let
\(\displaystyle a = \text {actual loss on insured property;}\)Let
\(\displaystyle c = \text {coinsurance rate as a decimal, not a percentage;}\)Let
\(\displaystyle d = \text {declared value of insured property; and}\)Let
\(\displaystyle m = \text {market value of insured property.}\)
Then this is what we get
\(\displaystyle \text {If } c * m \ge d, \text { then } p = a, \text { but}\)
\(\displaystyle \text {if } c * m < d, \text { then } p = a * \dfrac{d}{c * m}.\)
The c * m is what the insured should have declared the property to be worth.