Very interesting, thanks for your insights there.
The technique of using multiple ML's (presumably using different algos) to cross-examine one another sounds pretty good.
This is a similar principle as used by aerospace software development, to certify airborne software, it is necessary to show seperate, independent development paths followed by independent teams working separately from the customer product requirements, on the product, and the test systems used to verify it, respectively. DO-178B is the very longstanding standard usually applied.
Even though many of those development processes are becoming model driven, and automated, the basic principle of redundancy in the development path still has to be preserved. Hence the release of DO-178C (now itself already about ten years old), to address more modern development techniques.
Nothing can be perfect, but redundancy improves reliability considerably I think.