A Clockwork Sundial

Demonstrating our ideas is fundamental for us, and the most effective demonstration is the one that people can touch and see. This is why we sometimes resort to physical systems to explain some key concepts. To this end, a little hardware can go a long way.

As a side effect, our drawers are crowded with objects that you wouldn’t expect from a software-oriented business: Lego bricks, electronic components, nuts, bolts, and citrus-flavored hard sugar candies (thanks, Patrizia!)

With such plenty of hardware, there comes a time when some of us feels a simple need (say, he would really like a clock on his desk) and decides that the best course of action is to build it himself. This guy’s reasoning goes like this:

(A) The most natural clock in the world is the sundial (a perfectly legit assertion.)
(B) I like nature (who doesn’t?)
(C) Therefore: the clock on my desk must be a sundial.

Aristotle himself would agree that (C) logically follows from (A) and (B). The trouble, however, begins when a few subtle shortcomings of sundials must be addressed. At that point, Occam’s Razor is swiftly abandoned in favor of Rube Goldberg ; the drawers are opened and the contraption spree begins.

For example, as we all know the Sun is a rather inconstant source of light, even more so from inside an office; better replace it with a high-brightness LED. The main difference between an LED and the sun, however, is that the latter moves (albeit apparently), while an LED does not, unless forced to. Therefore, the LED will be mounted on a drinking straw stitched to a servo with a length of copper wire, and the servo is driven by an Arduino microcontroller.

After solving a series of challenges, the prototype is finally and proudly sitting on its creator’s desk, although its upcoming withdrawal for some modifications has been announced (its creator says that what he really needs is an alarm clock). Below you can see a video of the current prototype incarnation.


Information graphics is at least 143 years old

Post title - LIONblog

Napoleon's withdrawal from Russia, a painting by Adolph Northen, image from wikipedia,

I was a bit shocked when I considered that one of the first works - and possibly one of the most artistic ones - in information graphics dates back to 1869, when Charles Joseph Minard produced his famous "Carte figurative des pertes successives en hommes de l'Armée Française dans la campagne de Russie 1812-1813", a flow map on the subject of Napoleon's disastrous "French invasion of Russia (1812).

To celebrate - we are now 200 years from Napoleon's French invasion - I reproduced that historic information graphics with our Grapheur.

Post title - LIONblog
LIONlab software allows us to reproduce Minard's map and to do something more: to represent the temporal evolution of the events (you should get a sample by clicking on the animated GIF image above). This is done by the usage of arrows and Grapheur's sweeping visualization: time is selected as the variable to be "swept" and a movie in time is created by clicking the "advance" button in the control window.


Biology as Reactivity and responsive software.

Post title - LIONblog

A recent article in Comm. ACM explores the connection of biology with reactive systems to better understand living systems . I could not find a better motivation for our effort in Reactive Search Optimization .

"One way to explain a certain class of complex dynamical systems is to view them as highly concurrent reactive systems. We argue that this perspective is a natural fit for many biological systems. A reactive system is characterized by the way it responds to its inputs, as they arrive over time, sequentially, or concurrently. The system's behavior and outputs are not just a function of the input values but also of the order in which the inputs arrive, their arrival times, speeds, and locations, and so forth."

"Biological systems are also highly adaptive, to both internal and external changes; they use signals coming in from receptors and sensors, as well as emergent properties to fine-tune their functioning. This adaptivity is another facet of the reactivity of such systems."

If we succeed at transferring some of the robustness and responsiveness of biological systems into problem-solving and decision-making tools we obtain software which is adapting to the decision maker and not decision makers who must adapt to rigid and inflexible software.

  • 1-Source: Communications of the ACM Vol. 54 No. 10, Pages 72-82 10.1145/2001269.2001289


US representatives: "Who is the most Page-ranked of them all?"

Post title - LIONblog

"Mirror, mirror on the wall / Who is the fairest of them all?"
We are in a time of quantitative evaluations obtained by mining relationships . This is particularly true for the web (did you hear about Google?) but also for social networks (did you head about Facebook and Twitter?). The more your connections the more your value, your rank . Last week I was in a public commission to select a winning professor among seventy participants and the discussions about how and when to use bibliometric data to evaluate candidates was very hot, as you can imagine. BTW: this kind of evaluations must be taken cum grano salis, Albert Einstein could have lost a competition based on quantitative bibliometric data.

To relax a bit, I decided to apply the above concepts to the entire set of US representatives. The social-network rank ("PageRank") of each representative depends on sponsoring many bills which are in turn co-sponsored by representatives of high rank.

I went to the public website opencongress.organd mined the contained data. ...and the winner is: Republican Scott DesJarlais. he has the highest Page-rank of US representatives. Apparently, he is working with many people of different parties.
After the various representatives are mapped onto a two-dimensional surface by trying to keep highly-interacting representatives together (interacting means signing the same law proposals), one obtains a galaxy showing entities and relationships. Below the personal network of Rep. Ronald Paul.

Post title - LIONblog

Warning: ranking people by social network analysis is fun but it must be interpreted with additional context information which may not be visible from these raw data. Use with moderation.


Ten push-ups for each "excellence"

Post title - LIONblog

After reading "The Ten Most Annoying Management Terms Of 2011" I observed there is one term missing: Excellence. Each time I hear this word mentioned in business documents, a red alarm goes off in my mind. When I am a guest of an hotel mentioning "our staff is dedicated to reaching the highest excellence levels, bla bla bla" usually the plumbing does not work, and the air conditioning is as noisy as an airplane on a runway during takeoff. And, if you complain with the manager, they have no idea about how to deal with that. If you are excellent you do not go around bragging about it, if you are not, each reference to the word "excellent" makes you loose one point on the perceived quality scale. My good resolutions for 2012: each time I feel the need to utter the word "excellent" ... ten push-ups. Either I improve our communication strategy or at least I loose some weight.


Mutual information coming of age in 2012 ?

Post title - LIONblog

An interesting paper[2] of this period refers to the wide applicability of techniques based on information theory to detect arbitrary relationships between variables. It took seventeen years from my '94 paper [1] about using Mutual Information to identify informative features to reach this wider level of adoption.

It is absolutely clear – but still unknown to a large public - that a parameter may have zero correlation with respect to another parameter, while still containing the complete information needed to predict it!

The cause of the problem: correlation measures only linear relationships. The lesson: do not use correlation between variables if you suspect nonlinear relationships in your system. It is like using a hammer for driving screws.

Mutual information and related measures can identify relationships which would remain hidden by limiting your tools to linear ones.

If you are interested in predictions, you may throw away very informative, and therefore very useful, variables.


Stomach burning and software

Post title - LIONblog

Yesterday I was discussing with a colleague of mine after an abundant Xmas meal. According to him “business intelligence is not adopted in small companies because managers act by their gut feelings and they would not trust software to help them in their decisions”.

At first I was inclined to attribute this conservative approach to the short-sightedness of managers, apparently too arrogant and proud to admit that they could be helped by data mining and business intelligence software. Then a ray of light entered my dusty brain (some moderate alcohol during the meal helped the transformation).

Could it be that it is not the managers but a lot of BI software that is too arrogant, or actually better said, developed by people who are too arrogant to acknowledge that “gut feels”, like all decisions based on sub-symbolic paths (not really “irrational” but “not explicitly rational”) are crucial in most business processes?

Could it be that many managers still rely largely on “gut feel” and factors such as intuition and experience because they feel that many software programs are too rigid and too superficial to deal with the complexities of most business decisions?

Could it be that our effort in “Learning and Intelligent OptimizatioN” is related to allowing some of the “gut feels” to be transferred to the software through learning processes guided by the decision maker?


NoSQL: want to discover structure? Do not assume structure is given!

Post title - LIONblog

Procrustes had an iron bed in which he invited every passer-by to spend the night, and where he set to work on them with his smith’s hammer, to stretch them to fit. If the guest proved too tall, Procrustes would amputate the excess length. Procrustes continued his reign of terror until he was captured by Theseus, travelling to Athens along the sacred way, who “fitted” Procrustes to his own bed.

In relational databases data are organized with a clear structure of rows and columns, the columns being given specific and fixed names.

If data are unstructured at the beginning and you are looking for hidden or non-obvious structure, this looks like a contradiction: How can one use such a fixed structure to search for unknown structure?

This is the case for many recent developments related to social networks, web communities, recommendation systems, bioinformatics, etc.

According to the nosql movement:
“NoSQL Next Generation Databases mostly address some of the points: being non-relational, distributed, open-source and horizontal scalable. The original intention has been modern web-scale databases….Often more characteristics apply as: schema-free, easy replication support, simple API, eventually consistent / BASE (not ACID), a huge data amount, and more.”

Graph databases are an example. If the original data is represented as a graph of nodes and edges (relationships between pairs of nodes) one should use a database appropriate to this structure and not force the structure to fit an SQL database.


The Day Evolutionary Computation Took Over the World

Post title - LIONblog

“Come in!”

Professor Benson’s voice came muffled through the heavy wooden door. Josh, the shy college freshman who had knocked moments before expecting no answer, took two deep breaths of encouragement and entered the office. The professor gestured him to take a seat and kept typing for a while, then relaxed and turned to the student, waiting patiently for him to speak.

Uncomfortably perched on the edge of his chair, Josh realized that he had never tried to rehearse this meeting in his mind, and for once in his life he got directly to the point.

“Sir, Wh… Why such a low mark? I answered all questions… I’m Josh Brown,” he added, seeing that the professor was opening the folder with the test sheets, “and you gave me a C…”

“Josh,” professor Benson’s voice had a grave tone, “Structural Mechanics isn’t just about giving the right answers to a test. I must evaluate your true understanding of the subject. Do you have the right mindset for a civil engineer? This is what I’m trying to answer. And so far my answer is ‘C’.” He put a sheet on the desk. “Tell me, what’s this?”

Josh recognised his test sheet and described what he saw. “Two loads are applied to the same point. The resulting force is the sum of the…”

“This is not how it works, Josh.” Professor Benson’s voice took an annoyingly patronizing accent. “That’s what your grandfather would do. Today we know better than ‘adding’ things: our deep understanding of nature and of its inherent stochasticity allows us to look at problems and to solve them in a much more satisfactory way.”

“Sir, I’m not sure I understand…”

“Your solutions look obsolete, I could barely understand some of them, as if you came from anoher time.”

“I think I can explain this. I have grown in a rather secluded community founded by people who refuse any technological advancement that happened after 2011.”

“I heard about your group: they stick with VLSI computers, gasoline cars… I think you abandoned progress fifty years too soon.”

Josh nodded. “I got bored of my people, so I decided to try college. So far, everything went fairly well: I had a lot of physics and maths back in my high school, so I thought that your course would be an easy match for me. When you describe a physical system I understand everything. But when you show how to solve an exercise, I get lost. You seem to take a very long and convoluted path to get to an answer that I obtain in a moment with very simple maths…”

Professor Benson was startled by a sudden realization: “Maths… You mean ‘mathematics’! A deterministic, but largely innatural method of associating numbers to entities so that ‘algebraic’ and ‘analytic’ manipulations could be used to derive new numbers for other entities.” His plain voice suggested that he was merely repeating a definition that he had learned long before.

“Josh,” Benson’s condescending tone gained new strength, “as I said before, you lost fifty years of progress. Fifty years have passed since ‘mathematics’ was abandoned in favor of a more modern approach. You know, ‘mathematics’ was fundamentally flawed. Nature is stochastic, and by developing our problem solving skills in a more natural way we get to much better solutions. Did you learn about genetics, in high school?”

“Of course: genes are the basic units of heredity, which is the basis of biological evolution.” Josh was offended: was the professor questioning his basic scientific knowledge?

“And nature,” Benson continued, “has been improving the system for at least a billion years. What hubris, from the part of men, to believe that their puny ‘mathematics’, developed over a mere thousand years, would challenge the power of genetics, whose crossover and mutation paradigms modeled life itself over the eons!” Benson’s voice got louder and louder, and an embarassed silence followed.

Josh felt overwhelmed. “So, by mimicking genetics, your approach is to solve the problem by trying a first guess, then you change it…”

“…actually, I apply mutations…” Benson pointed out.

“…then, if you are not satisfied, take two wrong…”

“Suboptimal,” Benson corrected.

“Sorry… you take two suboptimal solutions, somehow mix them…”

“Actually, I breed new candidate solutions by choosing good candidates and crossing them over.”

“OK, then you repeat the process…”

“I explore the future generations,” Benson was clearly annoyed from having to correct his student over those elementary details.

“…until you decide it’s over.”

“More precisely, until the time budget for solving the problem expires. Nowadays, the art of determining the correct time budget for every kind of problem is explained in high school, and we don’t have the time to go over it again in university courses. You see, having a single methodology to solve all kinds of problem is a clear advantage over a discipline, like ‘mathematics’, that claimed to be universal, but carried the burden of having to be adapted to every new problem in a possibly different way.”

Josh protested: “But… Look at my answer to the test: adding two forces is much faster, and gives the correct answer! The best possible answer!”

“You are trying to compare two methods that are uncomparable by definition.” Professor Benson had the feeling that he was talking to a child taking his first lesson in Evolutionary Methods. “Crossover and mutation capture the true esssence of nature in a way that ‘addition’ and ‘multiplication’, being mere thought constructs, cannot. Answers obtained by ‘mathematical’ means are artificial. By using ‘mathematics’ you waste a lot of effort in trying to forge a… I believe the word is ‘equation’… that has no intuitive resemblance to the problem you are trying to solve. Time and energy that could be spent on searching for a solution are dilapidated into building a ‘mathematical’ equivalent of the problem, before being able to generate an answer. This is simply not natural, therefore unreliable.”

“But we have a brain!” Josh protested again. “We are capable of abstract thought, we can find analogies, we don’t need to go blindly after the solution!”

“Neural networks and other antiquate sub-symbolic methods are covered by epistemologists in post-graduate seminars, if you are interested in that kind of stuff.” Benson continued: “Look, you cannot compare evolution and mathematics, because they exist on very different grounds. In order to set a common ground, you would need to ask yourself how would nature be if it laid out an ‘equation’ instead of crossing genes,” he smirked, clearly amused at the idea.

“Just try harder and get an A in the next module: crossovers between ‘A’ and ‘C’ marks often breed good final results.” Professor Benson dismissed Josh with a sincere sympathetic smile.

Josh politely thanked him for his time. He stood up and went to the door, noticing how it squeaked when he closed it behind his back. Clearly, an evolutionary carpenter had allocated a low time budget to its design.

Josh realized that he didn’t really want to become an engineer. Later, he discovered that the replacement of mathematics with genetics did not harm all disciplines in the same way. He became a respected meteorologist.


Welcome to the LIONblog

Post title - Welcome to the LIONblog

This blog is about Learning and Intelligent OptimizatioN, a combination of modeling, machine learning and optimization.
Why do you need that?
Consider a simple toy example: you want to identify your optimal partner (well ...not so "toy" after all). Your two reasonable objectives to maximize are beauty and intelligence. The problem is not trivial because objectives are mutually incompatible, impossible to find a partner with both maximum intelligence and maximum beauty! Now: imagine that a software vendor wants to sell you his product. According to his words :"You just need to specify your weights for beauty and intelligence and then my software will work and identify the best girlfriend or boyfriend for you". Do you buy this software? Will you be able to give weights and then be happy with the partner that the software will identify for you? Think for a moment (without asking your current partner of course). No? What is the problem? Specifying details about how to combine objectives without looking at some concrete results is puzzling, hard, complicated, and may lead to disappointing results.

Most probably you would like to proceed in the following way:

  1. Give some preliminary values (maybe just by guessing)
  2. Examine preliminary results
  3. Revise your expectations and modify the problem definition

Effective design/problemsolving/optimization is an iterative process with a lot of learning involved! Stay tuned!