Paper List
-
SpikGPT: A High-Accuracy and Interpretable Spiking Attention Framework for Single-Cell Annotation
This paper addresses the core challenge of robust single-cell annotation across heterogeneous datasets with batch effects and the critical need to ide...
-
Unlocking hidden biomolecular conformational landscapes in diffusion models at inference time
This paper addresses the core challenge of efficiently and accurately sampling the conformational landscape of biomolecules from diffusion-based struc...
-
Personalized optimization of pediatric HD-tDCS for dose consistency and target engagement
This paper addresses the critical limitation of one-size-fits-all HD-tDCS protocols in pediatric populations by developing a personalized optimization...
-
Realistic Transition Paths for Large Biomolecular Systems: A Langevin Bridge Approach
This paper addresses the core challenge of generating physically realistic and computationally efficient transition paths between distinct protein con...
-
Consistent Synthetic Sequences Unlock Structural Diversity in Fully Atomistic De Novo Protein Design
This paper addresses the core pain point of low sequence-structure alignment in existing synthetic datasets (e.g., AFDB), which severely limits the pe...
-
MoRSAIK: Sequence Motif Reactor Simulation, Analysis and Inference Kit in Python
This work addresses the computational bottleneck in simulating prebiotic RNA reactor dynamics by developing a Python package that tracks sequence moti...
-
On the Approximation of Phylogenetic Distance Functions by Artificial Neural Networks
This paper addresses the core challenge of developing computationally efficient and scalable neural network architectures that can learn accurate phyl...
-
EcoCast: A Spatio-Temporal Model for Continual Biodiversity and Climate Risk Forecasting
This paper addresses the critical bottleneck in conservation: the lack of timely, high-resolution, near-term forecasts of species distribution shifts ...
Transformer的神经科学
Institute of Cognitive Science, University of Osnabrück | Department of Neuroscience, Erasmus MC
30秒速读
IN SHORT: 提出了Transformer架构与皮层柱微环路之间的新颖计算映射，连接了现代AI与神经科学。
核心创新
- Methodology Introduces a granular mapping where cortical columns correspond to transformer tokens, rather than whole cortical areas mapping to ANN layers.
- Biology Hypothesizes that laminar pathways (L4→L2/3 as Values, L1 feedback as Queries, tangential streams as Keys) implement self-attention-like multiplicative gating.
- Theory Frames the cortical column as a reusable module performing context-dependent routing and local recoding, analogous to a transformer block.
主要结论
- 皮层柱可以映射到具有特定层状对应的Transformer块：L4→L2/3作为Values（前馈驱动），L1反馈作为Queries（自上而下的上下文），L2/3/L5中的切向流作为Keys。
- 皮层中的乘法相互作用（增益调制、树突非线性）可能实现了自注意力的计算主题，实现了上下文相关的路由，而无需字面的QKV矩阵。
- 这一映射生成了可验证的假设：例如，丘脑驱动仅占约10-15%的突触，计算由皮层内上下文主导（类似于注意力权重覆盖输入嵌入）。
摘要: 神经科学长期以来为人工神经网络的发展提供了信息，但现代架构的成功反过来提出了一个问题：现代网络能否教会我们关于大脑功能的经验？在这里，我们检查了皮层柱的结构，并提出Transformer为皮层微环路组织的多个元素提供了自然的计算类比。我们并非声称Transformer方程在皮层中的字面实现，而是开发了Transformer操作与层状皮层特征之间的假设映射，使用这一类比作为分析和讨论的导向框架。这一映射使我们能够更深入地研究上下文选择、内容路由、循环整合和层间转换如何分布在皮层电路中。通过这样做，我们生成了关于层状特化、上下文调制、树突整合、振荡协调和皮层柱有效连接性的一系列广泛预测和可实验验证的假设。这一提议旨在作为一个结构化的假设，而非对皮层计算的确定性描述。将Transformer操作和皮层结构学置于一个共同的描述框架中，可以锐化问题，揭示新的功能对应关系，并为系统神经科学和现代AI之间的互惠交流开辟一条富有成效的途径。更广泛地说，这一视角表明，在计算组织层面比较大脑和架构可以为两者带来真正的洞见。