Anecdotal evidence should only be used as a basis for permissive policies. Anecdotal evidence is not sufficient justification for restrictive policies.
In recent discussions about Target's policy for transgender individuals using store bathrooms, I've seen repeated arguments based on the perceived risk of having biological men in the women's bathroom or vice versa. These arguments are either speculative "men could abuse girls" or anecdotal "This guy abused a woman by claiming he was transgender".
Speculative or anecdotal evidence is reasonable in circumstances where the desired policies are permissive. For example, suppose a hospital has a policy barring non-family members from visiting a patient after hours. A single example of a boy who wants to see his best friend who is dying of cancer would be sufficient to justify making the hospital's policy more permissive. Similarly, a family member getting into a fight with a patient after hours would not be sufficient to justify banning all family members from visiting after hours.
Let's do another example, graphically this time:
If a policy is going to restrict people, its merits should be demonstrable across an appropriately large sample of the affected population. In the case of Target's bathroom policy, barring transgender individuals from using the bathroom of their identified gender would require demonstrative proof (via experimentation or analysis) of the risks of allowing transgendered people bathrooms of their choice.
This doesn't mean we should run experiments to see how long it takes for someone to be abused. It means that we should look for statistically significant risk based on the data we already have about abuse when calling for restrictions in policy.