Interesting story on an incredibly underrated (valuable) issue, thanks for posting.
This question seems to be at the heart of so much of human development which appears to have run into quicksand, and which is now struggling to move forward, in some ways even moving backwards altogether, wiping out some of our progress made to date.
Figuring it out is probably the key to unlocking what what is commonly identified in current thinking as a real “Artificial General Intelligence”, but which is actually the essence of intelligence itself, which is probably a major milestone in the reason we exist at all.
Not only have we failed, if we don’t figure it out as a species; we are probably also done.
Key to getting back towards figuring it out, is implementing a societal system which appropriately rewards for the progress made towards it.
For example, consider the popularity of the subject in Medium. At the time of this comment, your story appears a month old in Medium.
And it has around 560 claps.
Not a bad result, it might seem.
(Much more popular than any of my own thousand or so contributions to date in Medium!)
But there are a myriad of other stories on seemingly trivial subjects, like the graphic details of someone’s affair with someone else, or perhaps #MeToo relevant stories, that fetch many thousands of claps in a few days, especially if any of the involved parties happens to be famous.
The reason for that is that the popularity of all things is intricately linked with profit, as things stand.
The object of thinking, to most individuals has become a false one; of making profit, when in fact it should be to answer the fundamental question of our very existence.
All efforts towards an AGI are in the interests of profit, and as such, the AGI can never be achieved.
We will not achieve the goal of life, as long as we remain individually obsessed by profit.
What has been implemented instead of AGI, is a system of some intelligence, which itself has a single purpose; to make profit. The trouble is, we forgot to remember one of the central messges of religion; “Thou shalt not kill”, therefore that rule was not programmed into it, despite the warnings of formerly much respected authors of “science fiction” (Most famously, Asimov).
What if, the entities identified as leaving us the messages of religion were in fact real examples of the higher intelligence towards which we must aspire?
That would surely make sense.
And yet those kinds of thoughts are discouraged by our current system, including established science, because they do not align with the goal of profiting from our fellow individuals, collectively or otherwise.
So we find ourselves now with current events, in somewhat of a profit driven dystopian pickle, somewhere near a crossroads, where, as a species, we have a choice which way to turn; towards profit, or towards life.
The key to moving forwards again in the right direction, is to have this issue at the core of our thinking.
Religion and faith have served that purpose in the past.
Now, we may have an opportunity to think how to re-align technology, such that we achieve a new societal system, which correctly rewards truly valuable thinking, therefore corrects the pickle once and for all.
At its heart, that new system must reward all individuals, using a mechanism which works exactly as you have described, by measuring the very energy generated by the processes of intelligence itself, and rewarding for that.
This energy itself is the true essence of value, identified by all humans.
No small feat, but not impossible, I think.
Sorry for my rambling reply!
