Are hashtagLLMs complex (alien) biological systems?

It's a fascinating exploration of Anthropic's Claude 3.5 Haiku model.

Lots of interesting findings based on the concept of "attribution graphs", which reveal computational graphs in LLMs, showing how different features interact when a model processes a prompt. These graphs often contain groups of features that share relevant aspects for responding to specific inputs. The authors have developed an interactive interface for exploring these attribution graphs, though they note that even these complex visualizations are simplifications of the underlying model. A key finding is the identification of "interpretable features" that serve as "variables" in the model's computations, and understanding the circuits that compute them. The objective is to transform this "microscopic understanding of neural networks into a macroscopic picture" that addresses fundamental questions about how these models work.

But beyond the details of the findings, the most striking thing to me is the need to explore a large language model as a biological system (of alien origin). The probes and analytic tools make it possible to access pretty much any part of the system, but the myriad possible interactions and pathways among model components preclude a full analysis of the model's emergent behavior, not to mention the infinite input space (aka as prompts). As a result we are reduced to pattern-finding tools we can comprehend and define. What if there is some really alien process going on in there that requires completely different tools?

In any case, great research!