Onto movement, Mr Abash said ‘movement is the area in which the inferential nature of reality really shines through.’ For this example he used an illusion (pictured) that causes what’s known as a motion after-effect (MAE). It is caused when visual neurons react to moving objects or stimulus
- Comments were made by Michael Abrash, chief scientist from Facebook-owned Oculus at the social network’s annual F8 conference
- He said we only infer the real world ‘based on the ‘sparse’ data we receive
- This is proved by the way optical illusions trick our senses
- As a result, he said virtual reality ‘done right’ can be observed as reality
Classic science fiction film The Matrix depicts a future in which humans experience the world in a simulated reality.
But if Facebook has its way, this fiction could become fact.
Michael Abrash, chief scientist from Facebook-owned virtual reality (VR) experts Oculus said The Matrix provides the best sense of what virtual reality could someday be like.
And he used optical tricks to prove that we are merely ‘inference machines’ and the world we see now is already an illusion.
The comments were made at the social network’s annual F8 conference in San Francisco.
‘While science fiction novels gave me the conceptual framework for thinking about VR, it was The Matrix that made me believe in it,’ Mr Abrash said.
‘Even though it was based on technology that won’t exist for decades, if ever, The Matrix gave me a deep sense of what VR could someday be like.
‘Not only how real it could be, but also how exciting it would be to bend and stretch that reality.’
In particular, he quoted a speech made by character Morpheus.
Summing up what’s truly unique about VR, the character, played by Laurence Fishburne, says: ‘What is real? How do you define real?
‘If you are talking about what you can feel, what you can smell, what you can taste and see, then ‘real’ is simply electrical signals interpreted by your brain.’
‘Unlike Morpheus, I’m not offering you a choice,’ said Mr Abrash. ‘No matter what choice you pick, we’re all headed down the rabbit hole together.’
‘While science fiction novels gave me the conceptual framework for thinking about VR, it was The Matrix that made me believe in it,‘ Mr Abrash said. He quoted Morpheus (played by Laurence Fishburne centre) who said in the franchise: ‘What is real? ‘Real’ is simply electrical signals interpreted by your brain’
‘Unlike Morpheus, I’m not offering you a choice. No matter what choice you pick, we’re all headed down the rabbit hole together,’ Mr Abrash said. This made references to pills offered to Neo in The Matrix. He used the illusion (pictured) to show that our brain already infers what’s real based on the limited data it receives
The red and blue colours in the previous image are simply what a viewer’s brains perceive, based on the rest of the information around them. The pills are actually grey (pictured). And Mr Abrash said that even when a person knows the pills are grey, they still see them as red or blue
This refers to the pills offered to Neo in The Matrix. If he takes the blue pill the story ends and he is told he will wake up in his bed and ‘believe whatever he wants to believe’.
Taking the red pill means he stays in The Matrix.
Mr Abrash continued that most people focus on the ‘virtual’ in virtual reality, but we should be focusing on the latter.
‘Morpheus made two critical points with his sentence: our conscious minds never actually interact with the real world and that we interact with sensors on our eyes, ears and tongue, and throughout our body. This is just a very small subset of the real world.’
He gave the example of vision. Humans only have three colour sensors, we can’t see infrared or ultraviolet and we have a blind spot in each eye.
‘Our visual data is actually astonishingly sparse and even if we were able to accurately record and process every photon that reaches our eyes, we’d still have too little data to be able to reconstruct the world accurately,’ he said.
He used the recent black and blue/white and gold dress as an example.
This illusion was repeated with a Rubik’s Cube in which the blue tiles on a yellow background were highlighted alongside the yellow tiles on the blue background (pictured)
The true colours of the tiles is shown. ‘Your visual system isn’t interested in whether the photons coming from a tile on a random image are red or blue or grey.’Knowing that didn’t keep anyone from being eaten by lions on the Savannah. What it is interested in is identifying potentially relevant features, in the real world’
‘Our visual system takes its best guess and sends that to the conscious mind,’ he continued.
‘The way that the brain compensates for the limited data it receives is by maintaining a model of the real world that it constantly updates as new data comes in.
HOW THE MOTION AFTER-EFFECT ILLUSION WORKS
The motion after-effect (MAE) can be explained by changes in visual neurons that respond in certain ways to moving parts within an image.
In the brain, there are cells are tuned to respond to different features and directions of an image or stimulus.
For example, there are cells in the brain that are sensitive to motion in a clockwise direction, but there are also cells that are sensitive to motion in the opposite direction.
When there is no movement within an image, these cells produce roughly the same response.
But, in the case of a simple MAE illusion, as the circles spin in a clockwise motion, the cells that are sensitive to this direction use energy and become tired.
When the spinning stops, the cells sensitive to motion in an anti-clockwise direction take over and become active – in a bid to restore the balance.
This causes even stationary items to look like they’re moving in that opposite direction for a short time.
‘And it is that model, not the real world, that you experience and trust implicitly. We are inference machines, not objective observers.
‘The fact that reality is whatever your mind infers from the nerve impulses sent to sensors, based on the model of the world, is at the heart of what makes VR different and more powerful than anything that has ever come before.’
He then gave three examples that demonstrate how this inference models breaks down.
The first showed a red and blue pill on hands that were shown on a yellow background.
He used this to reveal that the colours of the pills are the same shade of grey, and the red and blue colours that people see are simply what their brains perceive, based on the rest of the information around them.
And even when a person knows that the pills are grey, they still see them as red or blue.
This was repeated with a Rubik’s Cube in which the blue tiles on a yellow background were highlighted alongside the yellow tiles on the blue background.
‘Your visual system isn’t interested in whether the photons coming from a tile on a random image are red or blue or grey.
‘Knowing that didn’t keep anyone from being eaten by lions on the Savannah.
‘What it is interested in is identifying potentially relevant features, in the real world, under a variety of conditions.
‘Your visual system constantly corrects for the colours in the scene. It is reverse engineering reality rather than just recording it. The colours seen are your brain’s “best guess.”’