Previous Next
Scroll down to
explore more

The LION Manifesto:
Learning and Intelligent Optimization for self-improving Artificial Intelligence

Optimization for faster and more effective Machine Learning

Machine Learning for online and offline customization of Optimization

Artificial Intelligence is booming, but the wild transformative power of the current technoscience lies in the strong coupling between Optimization and Machine Learning.

Optimization drives Machine Learning, and Machine Learning improves Optimization by exploiting data produced while searching for better and better solutions: a rocketing spiral of continuously self-improving AI, along with automated (data-driven) creativity and innovation. This autonomy is a source of prodigious power and responsibility.

The recognition of this symbiosis motivated the LION conference (Learning and Intelligent Optimization) two decades ago. We want to continue this union and encourage research in the following directions.

Machine Learning for Optimization

- Use the “learning from data” (Machine Learning) paradigm to build digital twins (models/simulators/surrogates) of reality to be used by optimization schemes. It is the interconnection of optimization and software models that allows the current exponential acceleration in problem-solving.

- Raise the level of automation in problem-solving by:

  • Selecting the proper algorithm to apply to a given problem (algorithm selection)
  • Self-tuning algorithm parameters (meta-parameters / hyper-parameters) to a given problem
  • Reactively adapting meta-parameters of an optimization scheme while it runs, in a kind of “algorithmic learning on the job” (Reactive Search Optimization).

- Automate the creation of new algorithms through Reinforcement Learning, often through Deep Learning of internal representations.

- Increase the robustness and avoid the fragility of optimization to deal with noise, stochasticity, and partial knowledge of most real-world problems, hence considering both aleatoric and epistemic uncertainties.

- Solve problems with human paradigm. Human problem-solving leans on learning and rapid adaptation: optimization metaphors should follow more human learning-on-the-job abilities, rather than insects, viruses, etc. (the original LION logo hints at this human orientation)

- Learn to optimize through human feedback. If the measurable goal is not fully specified, human feedback from a set of initial solutions can fine-tune the function(s) to be optimized (e.g., in multi-objective optimization dealing with the search for trade-off solutions).

Optimization for Machine Learning

- Scientists are well aware that optimization algorithms are the motor under the hood of Artificial Intelligence. Simple variations of gradient descent fuel the current successes of AI in dealing with language models (automated translation, ChatGPT…), images, etc.

- Investigate more efficient optimization algorithms tuned for Machine Learning to decrease the “brute-force” attitude of many recent applications, requiring enormous amounts of hardware resources and energy, with environmental implications.

- Reach more automation in defining complex chains leading from row data to features, Machine Learning architectures, training, evaluation, deployment, and continuous adaptation (AutoML is a notable example)

- The emergence of Deep Learning (e.g., for Large Language Models) promises big steps towards automating ML and developing sintelligent ystems (almost) without programming.

- Optimization (e.g., Mixed-Integer Linear Programming) can be used for analysing and verifying ML systems and ensuring that relevant properties are satisfied (like resistance to attacks)

LION for Science

- Science is based on reproducibility and falsifiability. In many cases, the success of an optimization (or Machine Learning) algorithm relies on a human in the loop (“human, too human”). Human creativity seeds ideas, but if the availability of the original scientist is essential to reproduce results (because of poorly documented choices, fine-tuning, and strategic choice of partial results to present) science degrades to luck, “will to power”, and “publish or perish” drives. We solicit that also the process of selecting algorithms, and fine-tuning hyperparameters becomes an integrated part accompanying the presentation of a new method.

LION for business and society

- The convergence of Optimization and learning from data considered in LION creates a qualitative landscape change in techno-science. Businesses, society, and politicians must become aware of the underlying technology, its strengths and limitations, to take back our destiny, freedom, and human centrality in this accelerated evolution.

Optimization on steroids via Machine Learning.

Realizing the vision of extreme automation and self-improving Artificial Intelligence.

Leaning from data.
 

Machine Learning to create digital twins for improving complex problems and fine-tuning Optimization.

Reactive Optimization.
 

Mathematical Optimization can tune itself and be applied to most real-world problems, including noisy and ill-defined ones.

Fueling industrial innovation.

LION methods are the core of the current qualitative change of landscape in techo-science.

Resources

(preliminary - to be completed)

Contacts

Prof. Roberto Battiti
DISI - Università of Trento, Italy
Via Sommarive 5, 38123, Trento

Email: roberto.battiti ((AT)) unitn.it