Abstract：Based on the deep neural network features and EEG responses at the visual cortex of rhesus monkeys, an adaptive"brain-in-the-loop" fusion model of image representation is proposed, it is demonstrated that the brain-machine fusion can provide complementary information and enhance the performance of deep neural network model. Two kinds of brain-machine fusion computation mode based on brain response reconstruction and sharing representation space are proposed to accomplish the“brain-in-the-loop modeling and brain-out-of-the-loop application”, the application scenario of brain-machine fusion model is broadened. The effectiveness of sharing representation is proved by visualization method of feature saliency, a new thought is provided to make full use of neural response of brain visual representation.
2024, Vol. 10 
