Quickly scanning the paper I did not find any actual comparison of metaheuristics and their behaviour, sans the table 1 which lists their possible parameters.
It's all well and good to list sensible sounding principles "for meaningful comparison of metaheuristics", but I was kind of expecting to see those fine principles applied in meaningful examples.
yes, from a skim read, there didn't seem to be a lot of meat here.
lots of the "popular metaheuristics" as described won't be anything approaching state of the art, and i doubt their test bed considers problem instances that are hard or of industrial relevance. so the paper sort of seems to be about a method to compare the performance of uninteresting algorithms on uninteresting problems, without doing any comparison.
tangentially, in contrast, here's something from a vendor of commercial discrete optimisation software (from 2016):
> Pasco Shikishima, a Japanese leading food company (¥165 billion, equivalently $1.6 billion, in annual revenue), has chosen LocalSolver to solve what is probably the largest real-life optimization problem in the world.
> MIP solvers needed the original problem to be decomposed into almost twenty subproblems, each one solved separately, then leading to a poor global solution
> Pasco's supply chain involves 15 factories in Japan, each one with several production lines, and more than 100 distribution centers. Pasco's catalog contains more than 1,000 products. 900,000 orders have to be executed each day in Pasco's factories. For each order, Pasco has to decide where and when to produce it. Moreover, Pasco has to decide where to source raw materials and which routes to deliver distribution centers. The goal is to minimize production and distribution costs over several days of horizon, while respecting production and distribution capacities. Here is the scale of a LocalSolver instance solved by Pasco to plan the next 3 days: 32,670,717 expressions (that is, intermediate variables) including 8,307,431 binary decisions, 991,251 constraints, 16 lexicographic-ordered objectives. This model was solved in 3 minutes of running time on a modern but standard server, including input and output processing times.
this probably would have been a pretty fun project to work on, given that there was already an incumbent optimisation system in production, so a lot of the required prep-work and IT infrastructure for being able to gather and clean the input data for planning would have been in place. a sub-percentage improvement in optimality of the solution at this scale might translate into tens of millions of dollars of net profit for the company.
Very interesting. Is there any hint as to how much LocalSolver costs? If they're doing 3 minute runs for major optimisation jobs for massive companies on a single server, I imagine the "contact us for pricing" should give me a hint that it's quite a lot.
From memory I believe their competition (gurobi etc) would charge somewhere in the range of $10k-$100k per year for a server, perhaps toward the upper end of the scale if the server is used to solve problems for a number of clients. I haven't paid attention to pricing for this for a few years.
It's all well and good to list sensible sounding principles "for meaningful comparison of metaheuristics", but I was kind of expecting to see those fine principles applied in meaningful examples.