How to compose LLMs adaptively

Fun preprint I just came across through the ICLR 2025 submissions site (link). Even though it is still a preprint, there have been extensive reviews.

👏 First, I think the authors from Google, Google DeepMind, Google Cloud and University of Washington did a great job of summarizing the current approaches for composing LLMs, such as Mixture of Experts (MoE) [query routed to ONE expert], Learning-to-Fuse [requires a lot of training data], model arithmetic [pre-determined notion of composition] and how their approach differs: adaptive composition with little new training data.

🦕 The second thing I like, obviously, is the use of a kind of evolutionary algorithm, although I have always objected to the use of "hashtag#swarm" in the name of this particular algorithm, Particle Swarm Optimization (PSO). PSO is a very close cousin to a number of CONTINUOUS optimization methods (particle filtering, Covariance Matrix Adaptation Evolution Strategy aka CMA-ES) and performs of type of approximate Bayesian computation (ABC). It is evolutionary in the sense that it maintains a population of solutions and alters them depending on their performance.

⚡ Third, because it is a continuous optimization method, it can be applied to the model weights directly and not, as has been the case with recent uses of evolutionary algorithms, to prompt optimization (EvolMerge, PromptBreeder, EvoPrompt). As first glance it seems like optimizing the weights of LLMs is a much more formidable task than optimizing prompts but the key finding of the paper is that it might not be the case.

💡 There is a lot more that could be said. But let me conclude that this is an interesting and clear paper (but do read the appendices) that deserves to be examined and its methods tested. This kind of approach sets the stage for SMALLER, "specialized" LLMs that can be composed on the fly, emphasis on the word SMALLER (obviously), as in, much smaller. Lastly, given the apparent small size of the required training data, active learning with a human in the loop may not be out of reach, a big deal for Eric Bonabeau.