The novel approach allowed the team to push the limits of mind-reading technologies by seeing if the decoder could translate the thoughts of the participants as they watched silent movies, or just imagined stories in their heads. In both cases, the decoder was able to decipher what the participants were seeing, in the case of the movies, and what subjects were thinking as they played out brief stories in their imaginations.
The decoder produced more accurate results during the tests with audio recordings, compared to the imagined speech, but it was still able to glean some basic details of unspoken thoughts from the brain activity. For instance, when a subject envisioned the sentence “went on a dirt road through a field of wheat and over a stream and by some log buildings,” the decoder produced text that said “he had to walk across a bridge to the other side and a very large building in the distance.”