Paper List
-
GOPHER: Optimization-based Phenotype Randomization for Genome-Wide Association Studies with Differential Privacy
This paper addresses the core challenge of balancing rigorous privacy protection with data utility when releasing full GWAS summary statistics, overco...
-
Real-time Cricket Sorting By Sex A low-cost embedded solution using YOLOv8 and Raspberry Pi
This paper addresses the critical bottleneck in industrial insect farming: the lack of automated, real-time sex sorting systems for Acheta domesticus ...
-
Training Dynamics of Learning 3D-Rotational Equivariance
This work addresses the core dilemma of whether to use computationally expensive equivariant architectures or faster symmetry-agnostic models with dat...
-
Fast and Accurate Node-Age Estimation Under Fossil Calibration Uncertainty Using the Adjusted Pairwise Likelihood
This paper addresses the dual challenge of computational inefficiency and sensitivity to fossil calibration errors in Bayesian divergence time estimat...
-
Few-shot Protein Fitness Prediction via In-context Learning and Test-time Training
This paper addresses the core challenge of accurately predicting protein fitness with only a handful of experimental observations, where data collecti...
-
scCluBench: Comprehensive Benchmarking of Clustering Algorithms for Single-Cell RNA Sequencing
This paper addresses the critical gap of fragmented and non-standardized benchmarking in single-cell RNA-seq clustering, which hinders objective compa...
-
Simulation and inference methods for non-Markovian stochastic biochemical reaction networks
This paper addresses the computational bottleneck of simulating and performing Bayesian inference for non-Markovian biochemical systems with history-d...
-
Assessment of Simulation-based Inference Methods for Stochastic Compartmental Models
This paper addresses the core challenge of performing accurate Bayesian parameter inference for stochastic epidemic models when the likelihood functio...
Transformer的神经科学
Institute of Cognitive Science, University of Osnabrück | Department of Neuroscience, Erasmus MC
30秒速读
IN SHORT: 提出了Transformer架构与皮层柱微环路之间的新颖计算映射，连接了现代AI与神经科学。
核心创新
- Methodology Introduces a granular mapping where cortical columns correspond to transformer tokens, rather than whole cortical areas mapping to ANN layers.
- Biology Hypothesizes that laminar pathways (L4→L2/3 as Values, L1 feedback as Queries, tangential streams as Keys) implement self-attention-like multiplicative gating.
- Theory Frames the cortical column as a reusable module performing context-dependent routing and local recoding, analogous to a transformer block.
主要结论
- 皮层柱可以映射到具有特定层状对应的Transformer块：L4→L2/3作为Values（前馈驱动），L1反馈作为Queries（自上而下的上下文），L2/3/L5中的切向流作为Keys。
- 皮层中的乘法相互作用（增益调制、树突非线性）可能实现了自注意力的计算主题，实现了上下文相关的路由，而无需字面的QKV矩阵。
- 这一映射生成了可验证的假设：例如，丘脑驱动仅占约10-15%的突触，计算由皮层内上下文主导（类似于注意力权重覆盖输入嵌入）。
摘要: 神经科学长期以来为人工神经网络的发展提供了信息，但现代架构的成功反过来提出了一个问题：现代网络能否教会我们关于大脑功能的经验？在这里，我们检查了皮层柱的结构，并提出Transformer为皮层微环路组织的多个元素提供了自然的计算类比。我们并非声称Transformer方程在皮层中的字面实现，而是开发了Transformer操作与层状皮层特征之间的假设映射，使用这一类比作为分析和讨论的导向框架。这一映射使我们能够更深入地研究上下文选择、内容路由、循环整合和层间转换如何分布在皮层电路中。通过这样做，我们生成了关于层状特化、上下文调制、树突整合、振荡协调和皮层柱有效连接性的一系列广泛预测和可实验验证的假设。这一提议旨在作为一个结构化的假设，而非对皮层计算的确定性描述。将Transformer操作和皮层结构学置于一个共同的描述框架中，可以锐化问题，揭示新的功能对应关系，并为系统神经科学和现代AI之间的互惠交流开辟一条富有成效的途径。更广泛地说，这一视角表明，在计算组织层面比较大脑和架构可以为两者带来真正的洞见。