AIの記憶形成メカニズムは脳のメカニズムと驚くほど似ていることが判明(AI’s memory-forming mechanism found to be strikingly similar to that of the brain)

ad

2023-11-30 韓国基礎科学研究院(IBS)

AIの記憶形成メカニズムは脳のメカニズムと驚くほど似ていることが判明(AI’s memory-forming mechanism found to be strikingly similar to that of the brain)(a) Diagram illustrating the ion channel activity in post-synaptic neurons. AMPA receptors are involved in the activation of post-synaptic neurons, while NMDA receptors are blocked by magnesium ions (Mg²⁺) but induce synaptic plasticity through the influx of calcium ions (Ca²⁺) when the post-synaptic neuron is sufficiently activated. (b) Flow diagram representing the computational process within the Transformer AI model. Information is processed sequentially through stages such as feed-forward layers, layer normalization, and self-attention layers. The graph depicting the current-voltage relationship of the NMDA receptors is very similar to the nonlinearity of the feed-forward layer. The input-output graph, based on the concentration of magnesium (α), shows the changes in the nonlinearity of the NMDA receptors.

◆IBSの研究チームが、AIモデルと人間の脳の海馬の記憶処理に共通点を見出しました。発見により、AIシステムのメモリー統合に新たな視点が提供され、Transformerモデルが脳のNMDA受容体と類似したプロセスを使用していることが明らかになりました。
◆この知見を基に、AIモデルの学習が神経科学の原則で説明できる可能性が浮かび上がり、低コストで高性能なAIシステムの開発に寄与するとともに、脳の仕組みに関する洞察が得られることが期待されています。

<関連情報>

トランスフォーマーは長期記憶のためにNMDA受容体の非線形性を必要とします Transformer needs NMDA receptor nonlinearity for long-term memory

Dong-Kyum Kim, Jea Kwon, Meeyoung Cha, C. Justin Lee
OpenRebview  Published: 02 Feb 2023

Abstract

The NMDA receptor (NMDAR) in the hippocampus is essential for learning and memory. We find an interesting resemblance between deep models’ nonlinear activation function and the NMDAR’s nonlinear dynamics. In light of a recent study that compared the transformer architecture to the formation of hippocampal memory, this paper presents new findings that NMDAR-like nonlinearity may be essential for consolidating short-term working memory into long-term reference memory. We design a navigation task assessing these two memory functions and show that manipulating the activation function (i.e., mimicking the Mg2+-gating of NMDAR) disrupts long-term memory formation. Our experimental data suggest that the concept of place cells and reference memory may reside in the feed-forward network layer of transformers and that nonlinearity plays a key role in these processes. Our findings propose that the transformer architecture and hippocampal spatial representation resemble by sharing the overlapping concept of NMDAR-like nonlinearity.

生物工学一般
ad
ad
Follow
ad
タイトルとURLをコピーしました