Insights from computer game to create construction, destruction based neural network to make Siri learn like human

In order to make Siri learn like human learn isn’t the best way be to see from how human learn to play computer game?

When human play computer game, then in the beginning nothing learnt, random gameplay, overtime what happen is, that if one gets a positive feedback / construction / success, then the technique be learnt, while if one gets a negative feedback / destruction / failure, then the technique be changed, the word technique here would mean a combination of keys, like move forward -> jump -> dive …

Another thing be that it take time to change technique, for example if one learns a particular combination of keybinds for a move, and then one day decides to change the keybinds, then one is not able to play with the new keybinds immediately, it takes time to update a technique, and for some duration one would keep using the same old keybinds.

Another thing be that one would not consider the real world facts in a game, for example if asked do you want to eat this candy, then in the game world, the game developer could program the rules, that your health would be negative if you eat candy, so one would avoid eating candy in game, as it has destructive output, while in the real world, eating a candy would give good taste, that is a constructive outcome, so, one would do it in the real world, that is our actions are based on this notion of success / failure.

One more insight be that, on a positive feedback, the combination of keys only get strongly learnt, like, one get used to a particular combination of keys, if they have had a positive feedback, using the same combination multiple times, that is if a particular set of moves was X->Y->Z, then this combination would be strongly learnt, while on a negative feedback, it lead to combination of keys, getting changed, like, if one used to have a particular set of moves X->Y->Z, and it lead to failure, then they would update it to something like X->Z->Y, or break certain combination like X->Z.

Is there a term for this type of neural network?

I’m not an expert in Reinforcement Learning, but your description seems to point towards this field of machine learning, which uses rewards as a training signal and thus can be used to learn how to play video games.

One interesting thing in my opinion is the creation of rewards.
Let’s stick to video games, since you’ve mentioned them. While some “moves” might give a negative reward after their execution (you might lose life or points in any sense), they might be beneficial in the long run and you might even win the game with them.
Also, it’s always funny to watch when some agents are focusing on the “wrong” rewards while playing games, such as here:

The learned movement of grabbing these “turbo” boxes gave apparently more points than winning the game and was thus considered to be better. :wink:

This is also an interesting article from OpenAI:

hello, thanks for your reply, I do not get this idea from a reinforcement learning paper, I get this idea while searching about binary neural network, I read a paper on creating unorganized neural network, that is this paper, https://weightagnostic.github.io/papers/turing1948.pdf, the technique you provide a link to, use, floating point parameter, and use backpropagation, but in the paper I link, it use binary parameter, and NAND gate. The paper describes that simplest way to model infant learning be through a binary neural network and that an infant learn through a pleasure-pain system, where pleasure keeps the system as it is, that is no change, while pain leads to alter the system, as I write in my article above (in my article infant be equivalent to a player who is new to a computer game).

Furthermore, in this paper, https://academic.oup.com/mind/article-pdf/LIX/236/433/30123314/lix-236-433.pdf, the paper describe Turing test, that is, if Siri be able to have a conversation like human, then Siri be able to do all tasks that human do, or in other words, language be the most complex task that we do.

Also, one insight be that, we do not learn language by reading words only in that language, for example, if I want to learn a language like Spanish, and someone gives me a book, in which every single word be written in Spanish, then after beginning to read the book, I will not have idea what any word means, as I do not know the meaning of any Spanish word, and end up in a recursive state of, ‘ok, but what does that mean?’, in order for me to learn even a single Spanish word, someone will either have to tell the menaing of that word in a language that I have already learnt, or show me an image (for our first language, we see images). This learning technique, is common in language learning apps, like Duolingo. This again give hint towards, a link between word->word, or word->image, or even image->image being created for learning.

Furthermore, what be a text task where, an experiment be carried to create a binary unorganized neural network, I change this https://pytorch.org/tutorials/beginner/text_sentiment_ngrams_tutorial.html, but here again, all the word be in one language, and no image as input, so, computer be in state of ‘ok, but what does that mean?’, do there be an alternative task?

Furthermore, another insight be that, our actions are always based on this notion of success / failure, the example that you share, where the player, goes for the ‘wrong’ rewards, is still going for success.

Consider a person who has played, a game 100 times, and in each of the 100 times they played that game, they won, they are so much used to winning that they are bored.
For the first 100 games, the notion of success, always meant winning.
that is,

first 100 games -> success -> win the game

In the 101st game, the person is bored, now, they decide to create a funny clip / a meme, and share it with others, now the notion of success, changes from winning the game to creating a funny clip, that is,

101st game -> success -> create a funny clip

or another scenario could be that, in the 102nd game, the person says that now I would want to give my friend a win (assuming it be a multi player game, and their friend is one of the opponents), so the notion of success, would change from winning a game, to giving friend a win, that is,

102nd game -> success -> give friend a win

but in every game, the person is still going for success, for example, in the 101st game, they would consider the scenario of winning, and not being able to create a funny clip as a failure, that is,

101st game -> failure -> no funny clip, but won

or in the 102nd game, they would consider it a failure, if their friend lost, that is,

102nd game -> failure -> friend lost

In the 103rd game, the person might set the notion of success, to be the first person to get eliminated, and if they survive longer than anyone else, then that would be a failure, that is,

103rd game -> success -> first person to get eliminated
103rd game -> failure -> survived longer than others

but, in the all of the games, the person attempts at success, what they consider as success keeps changing.

Furthermore, another insight be that, one word on its own does not have a meaning, in order for a word to have a meaning, there must be some more words (or image).
But in the current state, the embedding based approach in text experiments, keep updating the so called representation of each word, as if it has nothing to do with any other word, and as if each word be isolated on its own, I think this be a big problem, which do not consider that words be connected to each other.

in order to further elaborate these insights, i start noting down human actions by observing people play computer games, it has become feasible to do that, as people have started life-logging their time spent in a computer generated environment.

it appears to me that this notion of construction, destruction happens on a recursive manner, plus it appears that we start placing words near construction or destruction side, that is something like this,

in the beginning,

construction/success                                          destruction/failure

no patterns learnt, empty, just like an infant does not come preloaded with any words, they learn it overtime.
let us say that the infant is learning two words, ‘good’ and ‘bad’, they would accordingly place it near
either of the ends, that is, overtime something like this happens,

construction/success  <->  good                      bad   <-> destruction/failure

one thing to note here is that, someone who has not learnt English language would not give any response, when we say a word like ‘good’ or ‘bad’ to them, because again they have not placed it near construction/destruction.

it it happens for a couple of words, then it must be happening for every single word that we learn, we can again see this from games, so anything that gives health, one would place it near construction side, anything that leads to loss of health would be placed near destruction side, something like this,

construction/success  <-> fruits                              poison <-> destruction/failure
                      <-> healing                            falling <->
                      <-> vegetables                            fire <->

it would even depend on the game, so if in any game, fruits lead to loss of health, then accordingly we will place it on the destruction side.

plus, if there is a linking system, then these patterns would look something like this,

construction  <-> fruits <-> eating <-> poison  <-> destruction/failure
              <-> healing              falling  <->
              <-> water                    fire <->

or every pattern that we learn, is either towards a constructive side, or a destructive side.

i found this on watching people play games, and noting down the phrases they say, on a win, people start saying phrases like,

i am the best player
too good
thats awesome
what can i say, i am just so good
i am insane
easy win, lets go

while on a loss, people start saying phrases like,

oh my god, no i failed
this game is triggering me today
get rid of this game mode
i hate this game
that was terrible
how unlucky is that

one thing here, is that, win and a loss happens on a recursive manner, so if playing a game like football, then there would be an ultimate win/loss, that is for the entire game, and then within the game, there would be win/loss for scoring a goal, making a good pass, that is mini games within a game.

plus what i found, is that, since we learn, so we start creating these learnt representations for game modes also, consider a person is playing a game X, and it has 10 mini games, something like this,

entire game - X

mini games - A B C D E F G H I J

after playing the game once, the results were something like this,

entire game - X - win

mini games -           A    B   C   D    E   F    G   H   I  J
                      Win Loss Win Loss Win Loss Win Win Win Win

now, in our brain, we create a learnt representation, again based on construction/desctruction, so,

construction <-> X <-> A, C, E, G, H, I, J        B, D, F <-> destruction

the next time, the player hops on to play the same game, for the ones in which they won, (that is the learnt representation is near the construction side), even before playing the game, they say phrases like,

this is easy
i know the technique
i am decent at this one

while for the ones in which they lost, (that is the learnt representation is near the destruction side), even before playing the game, they say phrases like,

i hate this one so much
this one is so scary
everytime i play this i am stressed

now, one thing here, is that, even the phrases that they say, at a certain stage in their life, they learnt those phrases, so if i was watching a person who speaks in a different language, then they would not say English language phrases, they would say the phrases in the language that they learnt in their life.

and again like in these phrases, words are either towards construction or towards destruction,

construction <-> i am decent at this one                      scary <-> destruction
             <-> i know the technique                          hate <->

furthermore, there are these notions of fear, anger, laughter, so, i start noting down the events that lead to such reflexes, it appears to me that even these are happening based on the construction/destruction learnt representations, for example, an event like,

flying a helicopter, and the helicopter is not going down

would lead to fear reflex, because, there is a destruction based future prediction,

construction                                          helicopter not going down <-> fall down <-> destruction

but i will have to observe people for some more time, and note the actions that lead to such reflexes.

also, from what i have read is that, there are regions in the human brain responsible for notions of fear, that is amygdala, but right now, i cannot say that it is related to creating this notion of destruction, we will have to wait, to see a proper digital reconstruction of a human brain in a computer, based on this article, https://en.wikipedia.org/wiki/Blue_Brain_Project, they are planning to do it by 2023.

i continue the same task of noting down people actions and behaviors as they play a computer game.
one thing to note here, is that the number of hours people spend in a gaming environment is increasing (on a global basis), so we can say that if not all, then definitely a lot of areas of ones brain must be getting triggered during this time, that keeps one playing on a continuous manner.

it appears to me, that along with creating a construction, destruction based linking system, we also do an act of scaling things up/down on a regular basis.
by scaling things i mean the following,

scaled down version | scaled up version
-----------------------------------------------------
its just a new helicopter added | OMG, they added a whole new helicopter, this changes the entire game
playing same game at lower fps, resolution | playing same game at higher fps, resolution
holy | hoooooolyyyyyyy (this happens for a lot of phrases like, no wayyyyy, that is insaaaaane)
regular | legendary
no caps | CAPS

that is some part of our brain appears to be creating this illusion, that something is bigger/smaller than something else.
if we add it to the construction, destruction picture above then it would look something like this,

                                         scaled up
construction                                                                     destruction
                                         scaled down

so, for a big fruit vs a small fruit (assuming both are on the constructive side)

                                               scaled up
                                      <->      big
construction <-> fruit                                                                     destruction
                                      <->      small                                     
                                               scaled down

the more a pattern is towards the scaled up side, the stronger impact it has towards either construction, or destruction.
again in order to give a proof that something like this is happening, one would have to see a live 3d model in vr, that shows which regions are getting triggered as we carry a particular task.

i think the notion of duration needs to be described in a different dimension, because each word we learn has a duration, in the example above, it would look like,

short duration | long duration
--------------------------------------
no way | no wayyyyyyyy
insane | insaaaaaane
holy | hoooooolyyyyyy

again we see that words in the longer duration side would have a stronger impact towards either construction/destruction.
if we add it to the picture above, then our learning system would now look like a cube, with the 6 sides corresponding to construction, destruction, scaled up, scaled down, long duration, short duration.
and the learning system task is to put words at the correct place.
the cube would look something like this,

for every word, the question is where do we place it, that is what our system learns overtime, let us see some examples,

insane -> towards construction + scaled up + medium duration
insaaaaane -> towards construction + scaled up + long duration
good -> towards construction + medium scale + medium duration
goooooood -> towards construction + medium scale + long duration

the highest constructive impact would of a word that is

towards construction + scaled up + long duration

something like,

insaaaane

the highest destructive impact would work in the opposite manner.
plus for a linking system, thinking about one word would trigger nearby words placed in the cube.

i started finding the statistics, again because of advances in brain scanning techniques, these statistics are getting revealed, these are the statistics that I found, these might change slightly in the future as there is a higher resolution scan.

from one video,

from a more recent video I found this,

now, one thing to note here, is that multiple companies are again creating brain inspired chips, for example,

IBM -> TrueNorth
Intel -> Loihi

and so on, that follow a non Von Neumann architecture, plus the chips of the future work at lower power consumption, again because human brain works at a lower power consumption, and people are busy emulating human brain functionality into chip.

plus, for neural network, it appears that there are only a couple of hyper-parameters here, that is the number of neurons, number of synapses, rest everything our system learns, so one would want to avoid using fixed values in the learning system. so fixed value for a task of normalization, or something like a fixed value for dropout/pruning rate, one would want to avoid.

I continue the same task of investigating how do we learn, in my second article above, i write that an infant is equivalent to a player who is new to a computer game, but there is a difference, let me describe what the difference is.

Suppose we are comparing the learning process of an infant vs lets say a 15 year old in 2020, (there might be some future technology that enables for faster learning, but let us compare based on how things are in 2020).

Suppose both of our players hop on to a game, in the game, since real world facts do not work the same way, so, lets say there is a game in which the game developer changed the counting system, so, instead of a real fact like ‘the number 2 is greater than the number 1’, the game developer changed it to ‘the number 1 is greater than the number 2’.

Now, in order to learn this fact, the 15 year old, who has already learnt counting in the real world, will have to forget what they have learnt, or for a linking system, they will have to break existing links, and create new links, while the infant (assuming has not learnt counting in the real world), will not have to break any links, and only needs to create new links, the infant will be like ‘1 is greater than 2, ok, what is wrong with that’.

In this case, the infant is at an advantage as compared to the 15 year old for learning this fact.

Whereas, suppose that both of our players hop on to a game, in which a lot of real world facts work the same way, lets say a game of football. Here, the 15 year old (assuming they have learnt how to play football in the real world) would be at an advantage, since they already have learnt representations of the game, and football works the same it does in the real world, in the game world, so they do not need to break any links, and most of the facts they have learnt work the same way. Whereas the infant, same as the previous case, will have to create new links.

In the second case, the 15 year old is at an advantage as compared to the infant.

I continue the same task of noting down human actions and behavior to understand how do we learn.

One thing to note here, is that, the output of a brain scan, is subject to how strong of a computer is used in the background, so brain scanning techniques started somewhere around 1970s, as computers before 1970 were not strong enough, the output of such scans keeps evolving, like around 1970s one would get a 2d black and white image, which evolved to 2d colored image, to 2d black and white video, to 2d colored video, that is where we are in 2020, so a brain scan output is a 2d colored video at a certain resolution, over the upcoming years, it would evolve, to being a 3d colored video, to being a live 3d colored video.

This concept applies even for other body part scans, so if you search for when did ct scan, or heart scan started, all of it would be around 1970s, because before that computers were not strong enough. So, this leads to biology simulations, again revealing insights to how lungs work, how heart works and so on. But let us keep it to brain, because human brain is a net of neurons, the future of neural networks is headed towards creating a digital net of neurons in vr. As vr is an alternative 3d space, everything people do in the real 3d space, keeps getting programmed in the alternative 3d space over the upcoming years.

So, I categorize the tasks that people do based on reflexes that get triggered, primarily humor, anger, and fear. And let us see if this is related to construction, destruction or a linking system, or do we have any further insights from these.

One thing to note here, is that this is again learning based, so, lets say an expert comedian in one language, if asked to say a joke in a language they have not learnt, then the expert comedian will not be able to do that, and will run short of words, if anything they will be able to give facial expressions only, because they have not learnt any word in the alternative language. So to express humor with words, need to learn words, same thing to express anger with words, need to learn words, and so on.

humor
--------------------------------------
opponent team got zero points
thank you anonymous
griefing people
ez qual after ending last
reached highest level, still no win
70 year ago paper, lol
instead of lawn mower, use hand tool
i am the best player in the world, gets eliminated first
losing whole game, won last second
thinking about what someone else would be thinking on a comment
looking at a person, with park background
person who is known for raging, does not rage
the wifi better start wearing a mask
right right right right right
tell ai to stop stream sniping me
take care of fire to vtuber
digital flower will grow if you water it
someone else lost
watch someone laughing
none of the shots hit, player survives
viewer is sleeping while watching world cup
this is what people in bird box saw (to cats movie trailer)
harley quinn bald image
every 60 sec in Africa a minute passes, together we can stop this
someone else opens chest, someone else takes the item
character size shrinking/enlarged
anger
--------
own teammate blocking
winning whole game, lost at last second
on getting eliminated
someone eliminated, and got eliminated themselves
person did not do something, they could have easily done
something that is so obvious, not done
repeated failure
fear
---------------
only two players remaining
teacher could ask question
seeing a snake
light turning off one by one
phone might fall down
sees clown
about to see results
going against a rare skin player

these are some that i noted down. i will add some more to it for a few more reflexes. and then see if there is some insight.

I’m always reading your interesting thoughts, but have to be a bit picky regarding the medical imaging claims.
Even today CT and MRT scans are displayed as black and white images on special monitors, which can “properly” display all necessary levels of gray. That being said, you can find colored images, e.g. via functional MRT images, where the color might denote the flow direction or speed of the liquid (blood) in the vessel. While color codes can be helpful (as seen in the fMRT example) often a grayscale image (on a medical monitor) is more beneficial and not necessarily limited by the compute power your machine has. :wink:
The most important aspect is what exactly do you want and need to see in the image.
Medical software usually comes with some presets using special “windows” to display certain tissue classes. E.g. if you are only interested in the lungs, you would select the “lungs window” so that the proper voxel intensities will be scaled to the full monitor range.

i think it depend on the pixel density, or the number of pixels within an inch, it have a bigger impact than color, so a video that is black and white but at a higher pixel density would be preferred over a video that is colored but at lower pixel density.

plus this pixel density thing applies in a couple of more areas, for example, evolution of a game avatar, lets say Mario, goes something like this,

that is Mario is increasing in pixel density, that is what Mario is evolving in, also the bits of information represented by Mario is increasing with evolution. this is one of the reasons, for an increase in the number of hours people spend while playing computer games. also you will see something like vtuber, digital human because of this pixel density thing, because real world avatar stays the same in terms of pixel density (in 2020), computer generated avatar keeps increasing, so, in 2020, it has become comparable (although still in 2d), so comes the notion of vtuber, digital human.

but let us keep it to brain, i added a few more points to the list i made above.
let us see if there be an insight from this list, or does it relate to our learning system.

let us begin with fear, for the case,

seeing a snake

in this case, this event leads to fear, because there is a learnt representation of snake that is near the destruction end, that is,

construciton                                                  snake <->  poison <->  destruction

but if the same snake was in a game, and looked just like the real world snake, (this is possible, as a 3d model of a snake that looks just like real world snake is possible), and the game developer reprogrammed the rules of the game, that the snake would give health, instead of take away health, then, snake shifts to the construction side, now,

construciton <-> poison <-> snake                                                   destruction

in this case, one would not feel fear, when they see the snake in game, so they will move towards the snake, instead of going away from the snake, similar thing one can say for

sees clown

for

phone might fall down

here again, the fear is because of the future prediction,

phone -> fall down -> phone gets destroyed

plus the learnt representation of a phone is towards constructive end, if the word phone is replaced by an object for which the learnt representation is towards destructive end, then fear would go away.
Or, again, if the phone was in a game, and the game developer changed the rules, that phone leads to reduction in rewards, then one will not feel fear, for it falling down, or it getting destroyed.

for the phrases

teacher could ask question

and

about to see results

the future prediction is

teacher could ask question -> will not be able to give answer
about to see results -> results will not be satisfactory

that leads to fear
whereas another set of future prediction would be,

teacher could ask question -> know the answer to the question -> give answer
about to see results -> know that the results will be satisfactory

in this case, the person will not feel fear.

it is as if a person is making a prediction for the scale/rank for their learnt representation that someone else has, that is one would be afraid of not being able to answer because it would lead to lowering their scale/rank with respect to the teacher.
again results will not be satisfactory would lead to lowering their scale/rank that people around them have given to them, that is a person would assume that people around them have a learnt representation for them, and in this learnt representation, they are at a certain scale/rank, and if the results will not be satisfactory, then this scale/rank lowers.

its like if there are five words

regular
common
rare
epic
legendary

then each of them have a scale, higher is better

regular - 1
common - 2 
rare - 3
epic - 4
legendary - 5

so, in our learning system we give these to words.

                             <->     legendary - 5
                             <->     epic - 4 
construction <-> fruit       <->     rare - 3                                                                    destruction
                             <->     common - 2
                             <->     regular - 1

infant does not know numbers also, so at a certain stage we create a learnt representation for these numbers also.

again the one on the up side has a stronger impact towards construction/destruction

for the phrase,

light turning off one by one

maybe notion of darkness is inclined towards destruction, and it keeps scaling up in darkness as lights get turned off one by one, something like,

construction                           completely       <->         darkness <-> destruction
                                       slightly         <->

for

only two players remaining

or

seeing a player with a rare skin

again the learnt representation of the player the person is going against, would have a scale, and if they are going against, then the other player would aim at eliminating them, so its on the destructive side.

construction                                             rare skin player    <-> destruction
                                                         common skin player  <->

there are couple more things related to evolution of Mario, Mario is not subject to natural evolution, it is what we would call statistical evolution for Mario, plus again Mario keeps increasing in bits of information (like KB to MB to GB), and more of the things that real world avatar do, keep getting programmed into Mario. so this would lead to Mario being able to interact like a human, or the same thing for Siri, that is what is described as passing Turing Test.

So, we will see, a computer generated avatar, in vr, (because by the time it happens, the number of interactions in vr would have increased), that interacts like a human, but does not exist in the real world, just like, if Siri starts interacting like human, then Siri exists in an alternative n-dimensional space. But if we have an understanding of how human brain works, then you would see that same behavior programmed into a real world entity that is again subject to statistical evolution. This also leads to solving brain related diseases like Alzheimer, Parkinsons, for real world avatars.

Again the notion of life and death does not work for a computer generated avatar exactly as it does for a real world avatar, the first appearance of Mario was in July 9, 1981, so saying Mario is 39 year old is fine, but comparing Mario with a real world human that is 39 year old does not make any sense, for Siri, it was October 12, 2011, so saying Siri is about to be 9 year old is fine, but comparing Siri with a real world human who is 9 year old, again does not make any sense.

All of the bits of information represented by a computer generated avatar is backed up, this is not yet true for a real world avatar.

but let us discuss about humor, do we any insight from humor.

for the case,

opponent team got zero points

one would place

construction <-> points  <-> highest one can get in that game                                                 opponent team <-> destruction
                         <->  0 (would be at the bottom)

again opponent team is towards destruction, because they attempt at eliminating.

so here, we are linking, something from the destructive side with something that is the least scaled towards the constructive side.

for the case,

reached highest level, still no win
construction <-> win (more) <-> level (more)                                              destruction
             <-> win (less) <-> level (less)

again linking something that is scaled down towards construction, with something that is scaled up

i am the best player in the world, gets eliminated first
construction <-> best player                                         eliminated first      <-> destruction
             <-> other players                                       eliminated second <->

linking something from the scaled up construction end, with scaled up destruction end.

losing whole game, won last second

this is more like happiness, rather than humor, but it triggers a laugh reflex.
it appears like a sudden transition from destructive side towards constructive side.

person who is known for raging, does not rage
c <-> no rage (although it would be more like neutral)           rage <-> person 1 <-> d
                                                                      <-> person 2
                                                                      <-> person 3

and person 1 does not rage, so linking from scaled up destructive side, towards constructive side

digital flower will grow if you water it

there is a learnt fact that real world flower grows (gets scaled up) if you water it
but nothing will happen to a digital flower on water.

the character size shrinking/enlarged appears to be similar to light turning off one by one for fear, but everything is on the constructive side, if you see an avatar that is a threat enlarge in size would lead to fear
for

someone else opens chest, someone else takes the item

the learnt representation for chest is towards construction, scaled up, attempting at linking with something that is constructive, scaled up, but could not.

viewer is sleeping while watching world cup

again the learnt representation for world cup is constructive, scaled up, something like a national cup would be at a lower scale, here one is not attempting at linking with something that is constructive, scaled up. if in the above case, we changed it, someone opens the chest, and fell asleep that might again trigger humor, because chest is constructive, scaled up.

the wifi better start wearing a mask

this one is somewhat similar to digital flower will grow if you water it
here again, its like an attempt at linking with something that is towards constructive side, scaled up.

take care of fire to vtuber

this again feels like an attempt at linking with something that is towards constructive side, scaled up,

tell ai to stop stream sniping me

here again, ai does not stream snipe, so its like a failed link.

we have some hints like it appears to be linking things that are scaled up either towards construction/destruction, or a transition from top right destruction to bottom left construction end.