MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery

This research introduces MLEvolve, a self-evolving multi-agent framework specifically engineered for automated machine learning algorithm discovery. The framework aims to overcome critical limitations observed in current machine learning engineering (MLE) agents, including inter-branch information isolation, memoryless search paradigms, and insufficient hierarchical control mechanisms, all of which hinder effective long-horizon optimization. MLEvolve employs large language model (LLM) agents within a comprehensive, end-to-end algorithm discovery pipeline. A central methodological contribution is the adaptation of tree search into Progressive Monte Carlo Graph Search (MCGS), which enables robust cross-branch information flow via graph-based reference edges. This progressive search strategy dynamically shifts from broad exploratory phases to more focused exploitation, guided by an entropy-inspired scheduling

mechanism. To foster continuous agent evolution and learning, MLEvolve integrates Retrospective Memory, comprising a cold-start domain knowledge base and a dynamic global memory for efficient task-specific experience retrieval and reuse. For stable and sustained long-horizon iteration, the framework further distinguishes strategic planning from code generation through the implementation of adaptive coding modes. Experimental evaluations on MLE-Bench indicate that MLEvolve achieves state-of-the-art performance across multiple dimensions, including average medal rate and valid submission rate, demonstrating its efficacy in advancing automated ML algorithm discovery.

Message: Hi FE, I need help with “MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery” in “Electrical Engineering”