I agree with you. I had Grandparents who swore that they, and many others personally saw UFOs, and had numerous graphic accounts of their own experiences as witnesses, in the years leading up to the second World War.

After that, accounts seem to have pretty much gone, or are easily explained as hoaxes or illusions.

It is interesting Fermi himself was working on our deadliest weapon ever when he asked the question.

I find it hard to believe this thought would not have crossed his mind.