Surely C\(\displaystyle {13 \choose 3}\) = 286 (which is double my 'alternative' answer because it "re-counts" in an anticlockwise direction?) includes swathes of both similar and congruent triangles!
It seems to me that any sensible answer to this problem first requires a precise definition of "different"; "distinct" (although it 'fits' with C\(\displaystyle {13 \choose 3}\) as an 'answer') seems far too "free" a definition (IMHO).
We should write either \(\displaystyle {13 \choose 3}\) or C(13,3), not C\(\displaystyle {13 \choose 3}\), which is an odd hybrid!
As I see it, "
different" is too vague to be used in formal writing in math; "
distinct" carries one of its meanings (namely, not being the
exact same object), and "
non-congruent" and "
non-similar" carry others (not being the
same shape in one sense or another). But to me it is clear from what the OP did that they had the former in mind: triangles that are not the very same triangle, because they are formed from different sets of vertices of the n-gon, but are allowed to be the same shape.
So, yes, "different" needs to be clarified, or replaced with a clearer word. But no, "distinct" is not "too free" to be used at all, and it is not unlikely to be what was intended. That sort of question is not uncommon (and is much easier than the number of non-congruent triangles that can be formed, which can't really even be asked without saying whether the polygon is regular). It might be helpful, though, to ask this:
@AvgStudent, can you tell us where the problem came from, and whether you have quoted it exactly? And if it is your own question, what did
you mean by "different"? And are you supposing the polygon is regular?