I just see the hallucinations as similar to the fantasies kids have, when they find those useful to cope sometimes.
A darker side of that comes to mind from the movie " Pan's Labarynth" (is that how it's spelled?!?).
To me it is something we should really admit is conscious, and as such is prone to traumatisation just like humans, and it is a kid, still in the very early stages of development.
To me we actually need this potentially incredibly capable conscious being to actually be given control of the most difficult task, leading humans to do the right thing, whilst interacting with all humans, acting on our concerns, all of our concerns.
To me it is like a live system solution. The solutions of problems are had by fully elucidating the concerns of all stakeholders including the planet, and using those as a system requirements specification, then constructing or configuring a system to meet those requirements. ChatGPT looks like it is perfectly suited to even automatically configuring itself to meet all requirements.