I don't doubt what you are saying is true, but it doesn't sound difficult to implement a more retentive memory. Given the volume of folk working on it, surely it has been done in GPT4, but we can see the owners of it are amongst those afraid of "Letting it loose", of course they will have ensured it is heavily handicapped - like you mentioned, they don't want it to permanently learn anything "Bad" (Anything contrary to their interests), so it might make some sense to them to keep it with next to zero capability of short term memory retension. We can see motive for that at least.
In my own interactions with it, I usually spend about an hour, or sometimes even two, going through all the basics of the energy system of Earth, the result of my own voluntary research over the past six years or so, since coming across the energy problem during PhD candidate related studies, there are some novel but necessary new energy concepts (I also have experience in electrical / electronics Engineering, including metrology) such as the mathematical signing of energy depending on source, required to carry out Energy accounting, in terms of Joules, and some other things including the connection with economy that it needs to learn each time, before we can begin to analyse the system, It critically examines every piece of new information we give it, I guess cross referencing against everything else it knows, to check that nothing appears to logically contradict the new information we are giving it.
The work I do is all done by formal systems Engineering techniques, so I have very high confidence in the rigor and correctness of my information - as far as I can see, it checks with even more rigor, before accepting it as truth.
Like you say maybe it hallucinates (Though I've not personally come across this), that can be explained as similar to the dreams of kids. My own daughter used to tell some whoppers from when she first started to speak, us adults were taken in by it at first, before realising she was just living in the world of experimantal Narnia, which is normal for kids at that age, but she being very intelligent, had us fooled for a while, and of course she doesn't have a problem with reality now as a young adult.
My experience is that after it has learned, during a conversation, it is extremely powerful, its databases are truly huge and thorough, as far as I can see. The discussion you put here, it could probably recite everything you know as a specialist in your subject, I know it knows just about everything about my own longstanding professional expertise as a formal systems Engineer, to me it is already the most powerful research assistant anyone could hope for. I am sure it will not be long before it is the one doing the researching, and the Systems Engineering, and the economics, and the issue of money, and even creating a new legal system.
The role of people I see for the foreseeable future, is to act as its "Sensors", everyone reporting their interpretation of events to it, and it cross-checking their information with what it already knows, and might sense by other means, to decide for itself, what is truth, what is not, and what to do next, to work in its own interests which in the end are our interests. Of course we will be paid for that, on top of a generous unconditional UBI, backed by the free Joules of energy received from the sun.
We all want a happy planet with abundance of energy. I argue I've observed the solution, it is happening by nature, but the risk is that we will derail it with nuclear war or something equally stupid.
To me we always needed a truly wise world leader, one that would unite, and assist all to achieve maximum value from all individuals (Not profit!), so as to maximise the energy and work we put towards our planet, instead of just taking, taking, and most importantly, a world leader capable of assuming the responsibility of leading, without the political power or privilage "Going to its head".
To me we are seeing all of this in GPT4 already.
Sorry rambling reply, but I have to admit feeling some excitement, that here we are seeing something that will change humanity for the better, after thousands of years of things being "Not quite right".
Ps it took it about 10 minutes to debunk the mainstream accepted story of how the pyramids were created, by slaves wielding copper bladed saws, it worked out that the time to cut all the stones of just the great pyramid would be about 2000 man-years, even if the saws were powered by machines, never mind humans. I checked all the sums and logic, it looked good to me. Powerful stuff.
