Evolutionary Computation Competition 2018
based on benchmark problem of Lunar Lander Landing Site Selection
December 8, 2018 (Saturday) 9: 00-12: 00
Fukuoka, Japan
Organizer: The Japanese Society for Evolutionary Computation (JSEC) and Japan Aerospace Exploration Agency (JAXA)
Notices
- Description of the constraint is modified.
(before) All constraints are written in the form g(x) >= 0, meaning that when g(x) is less than 0, the constraints are not satisfied.
(after ) All constraints are written in the form g(x) > 0, meaning that when g(x) is greater than 0, the constraints are satisfied.
Because the consecutive shade days is an integer, the normalized constraint becomes discrete functions.
As a result, the normalized consecutive shade days can be 0.05. This is the reason for the modification.
- The algorithms and results of participating teams were uploaded on the web (12/17/2018)
- Evolutionary Computation Competition 2018 was successfully held. Thank you all (12/8/2018)
- The evaluation criteria for the single-objective optimization part have been revised. (10/24/2018)
- Post-processing program (R script) has been released. (10/23/2018)
- Since there was a bug in the evaluation module for single-objective optimization, the evaluation module was updated. (10/12/2018)
- Evolutionary Computation Competition 2018 page opened. (10/1/2018)
The purpose of the competition
The Evolutionary Computation competition 2018 will be held on the first day of the Evolutionary Computing Symposium 2018 organized by JSEC.
This year we use the benchmark problem based on Lunar lander landing site selection probided by JAXA.
Although this problem has a small number of design variables (longitude and latitude), it has a very strong nonlinearity and severe constraints.
We have two categories in the competition: single-objective optimization category and multi-objective optimization category.
website of the last year cometition (in Japanese)
Venue and access
Check the website of Evolutionary Computation Symposium 2018 (in Japanese).
The competition will be held on the first day (12/8) in the morning at the venue of the Evolutionary Computation Symposium 2018.
Schedule
Competition announcement and application start: Monday, October 2, 2018
Application deadline: November 19, 2018 (Monday) (No deadline extension)
Deadline for data submission: November 26, 2018 (Monday) (No deadline extension)
Competition Date: December 8, 2018 (Saturday)
How to apply
Participation fee is free.
Fill out the following and apply by e-mail by November 19, 2018 (Monday).
Subject: Evolution Computation Competition 2018 Application
Body: Enter the applicant's name, affiliation, e-mail address,
and whether the application is for a single-objective or multi-objective optimization category.
If the applicant is a student, include the name of the supervisor.
Send to: ec2018-competition@flab.isas.jaxa.jp
How to submit data
Send the following data by email by November 26, 2018 (Monday).
Subject: Evolution Computation Competition 2018 (single-objective or multi-objective)
Send to: ec2018-competition@flab.isas.jaxa.jp
Program
9: 00-9: 10 Explanation of the purpose of competition by Akira Oyama (Japan Aerospace Exploration Agency)
9: 10-11: 10 Presentation from applicants (planned for 5-10 minutes per applicant)
11: 10-11: 40 Summary and discussion of results by Akira Oyama (Japan Aerospace Exploration Agency)
11: 40-12: 00 Award ceremony
Award
The winner of each category (single-objective optimization and multi-objective optimization) receives award from the Japanese Society for Evolutionary Computation.
Result
We had nine teams in the single-objective optimization category and five teams in the multi-objective optimization category. Thank you!
[Single-objective optimization category participants]
| 余俊，李宇豪，高木英行 | 九州大学 |
| 古川雄大，小野景子 | 龍谷大学 |
| 開発拓也，渡邉真也 | 室蘭工業大学 |
| 串田淳一 | 広島市立大学 |
| 原田智広 | 立命館大学 |
| 加藤拓也，小野功 | 東京工業大学 |
| Pereira Junior, JAIR，CLAUS Aranha | 筑波大学 |
| 岩瀬拓哉，高玉圭樹 | 電気通信大学 |
| 深瀬貴史，橋本龍一，増山直輝，能島裕介，石渕久生 | 大阪府立大 |
| 鎌田一樹，青木勇輔，小野功 | 東京工業大学 |
| 宮本将英，中田涼介，渡邉真也 | 室蘭工業大学 |
| 磯林知志，大伴周也，原田智広，ターウォンマットラック | 立命館大学 |
| ユーリラヴィナス，Claus Aranha | 筑波大学 |
If you are interested in the algorithms used by each participating team, please see the following materials.
Introductory material on optimization algorithms of participating teams (in Japanese)
The copyright of the materials is held by the participants, so secondary use in any way is prohibited.
The winners of this year's competition are
[Single-objective optimization category] Takuya Kato and Isao Ono (Tokyo Institute of Technology)
Total communication time 0.98，Number of objective function evaluations to reach to the final solution 5168
[Multi-objective optimization category] Kazuki Kamata, Yusuke Aoki and Isao Ono (Tokyo Institute of Technology)
Hypervolume 0.941
Congratulations!
Details of the results of the competition is in the following document
Introduction material of optimization results (in Japanese)
The copyright of the material is held by Akira Oyama (JAXA). Please contact us for secondary use.
Competition conditions
(1) Benchmark problem: we use the "Lunar Lander Landing Site Selection Problem". Reference (in Japanese)
Constraints are
continuous shade days < 0.05
and
landing point inclination angle < 0.3.
The design variables are longitude and latitude, and the search range is [0,1] respectively.
In the case of a single-objective optimization, it is a problem of minimization of (-1) x (the total communication time), this corresponds to maximization of the total communication time.
In the case of multi-objective optimization, there are three objectives:
minimizing the number of continuous shaded days, minimization of (-1) x (the total communication time) (i.e. maximizing the total communication time), and minimizing the tilt angle.
In this competition, we don't consider the probability of existance of water ice near the landing site.
(2) The maximum number of objective function evaluations is 30,000.
(3) To generate the initial population, use a general method such as random generation in the entire design space,
or arranging them based on such as design of experiments.
(4) Judgement of winner
- In the single-objective optimization category, the algorithm with the largest total communication time is the best.
The value of the solution with the largest total communication time among all the solutions evaluated during the optimization process is used.
(Not the largest total communication time in the final generation group)
If there are multiple algorithms (teams) with the same total communication time,
the algorithm (team) with the least number of solution evaluations required to obtain the final solution is the best (added on 10/24/2018)
- In the multi-objective optimization cateogry,
the algorithm with the largest hypervolume (hereinafter referred to as HV) value is the best.
The HV value is not the HV of the final generation, but HV of the all solutions evaluated during the entire optimization process.
When calculating the HV value, use the reference point (1,0,1) (in the order of consecutive shade days, total communication time, and tilt angle).
(5) 21 trials with different initial population or random number of optimization algorithm should be executed.
The median value of total communication time (single objective) or HV value (multi-objective) is evaluated.
How to run the evaluation module
Same approach for both single-objective optimization and multi-objective optimization
1. Download and unzip the following files.
[moon_sop.zip]
Evaluation module for single-objective optimization
[moon_mop.tgz]
Evaluation module for multi-objective optimization
[sample.zip] Calculation result example
[DB.zip] Landing point database (about 10GB)
Landing point database is about 10GB in size. If the download is not successful, please use the link below.
sharepoint
dropbox
If it is still difficult, you can download the contents of DB.zip divided into 3 parts from the following.
If you create a folder named DB and unzip these files in it, you will have the same environment as DB.zip.
continue_night_nml_full.zip
slope_nml_full.zip
total_comm_nml_full.zip
2. Create an executable file of the evaluation module from the source code.
Compile with VS2017 for Windows or compile with make command for Linux.
We don't check Mac, but we think it should work.
3. Place the DB folder and executable file in the same folder,
Then you can run it with
./moon_sop (path with pop_vars_eval.txt)
or
./moon_mop (path with pop_vars_eval.txt)
4. Upon successful completion, pop_objs_eval.txt and pop_cons_eval.txt are generated.
pop_vars_eval.txt
Search points to be read by the evaluation module.
Each row correspoinds to each individual.
Each column separated by tabs represents each design variable.
The first column is d1 = longitude and the second column is d2 = latitude.
Both are normalized by [0,1].
pop_objs_eval.txt
Objective function values of all points evaluated by the module.
Each row coresponds to each individual.
Each column separated by tabs is each objective function.
From the left,
[Single-objective optimization] f1 = (-1) x (Total communication time)
[Multi-objective optimization] f1 = continuous shade days, f2 = (-1) x (total communication time), f3 = inclination angle
The order of the rows (individuals) is consistent with pop_vars_eval.txt.
pop_cons_eval.txt
Constraint function values of all points evaluated by the module.
One row correspoinds to one individual.
Each column separated by tabs is each constraint value.
From the left
C1 = consecutive shade days,
C2 = tilt angle
The order of the rows (individuals) is consistent with pop_vars_eval.txt.
All constraints are written in the form g(x) > 0, meaning that when g(x) is greater than 0, the constraints are satisfied.
Data to be submitted (single-objective optimization category)
(s1) Objective function value of the optimal solution (median of 21 trials) (follow format s1)
(s2) Optimization history (median of 21 trials) (follow format s2)
(s3) Objective function value of the optimal solution (Best value of 21 trials) (follow format s3)
(s4) Optimization history (best value of 21 trials) (follow format s4)
[Format s1]
- File format is csv file.
- The file name is s_opt_***. csv (*** is the group name written in single-byte alphabet).
- From the left, each column should be f1 (= (-1)x(total communication time)), c1 (= continuous shade days),
c2 (= tilt angle), d1 (= longitude), d2 (= latitude).
[Format s2]
- File format is csv file.
- The file name is s_his_***.csv (*** is the group name written in single-byte alphabet).
- From the left, each column should be the number of generations, the number of evaluations,
f1 of the optimal solution of that generation (= (-1)x (total communication time)),
c1 of the optimal solution of that generation (= consecutive shade days),
c2 of the optimal solution of that generation ( = Inclination angle),
the number of solutions that satisfy the constraints in that generation.
Use -1 if not appropriate.
[Format s3]
- File format is csv file.
- The file name is s_bst_***.csv (*** is the group name written in single-byte alphabet).
- From the left, each column should be f1 (= (-1)x(total communication time)), c1 (= continuous shade days),
c2 (= tilt angle), d1 (= longitude), d2 (= latitude).
[Format s4]
- File format is csv file.
- The file name is s_bst_his_***.csv (*** is the group name written in single-byte alphabet).
- From the left, each column should be the number of generations, the number of evaluations,
f1 of the optimal solution of that generation (= (-1)x (total communication time)),
c1 of the optimal solution of that generation (= consecutive shade days),
c2 of the optimal solution of that generation ( = Inclination angle),
the number of solutions that satisfy the constraints in that generation.
Use -1 if not appropriate.
Data to be submitted (multi-objective optimization category)
(m1) HV value (median of 21 trials)
(m2) Design variables, constraints, objective functions, etc. of all non-dominated solutions (excluding dominated solutions)
that are used in calculating the HV value (median of 21 trials) (follow format m2)
(m3) History of optimization (median of 21 trials) (follow format m3)
(m4) HV value (best value of 21 trials)
(m5) History of optimization (best value of 21 trials) (follow format m5)
[Format m2]
-File format is csv file.
-The file name is m_prt_***.csv (*** is the group name written in single-byte alphabet).
-From the left, each column is f1 (= continuous shade days), f2 (= (-1)x(total communication days)),
f3 (= tilt angle), c1 (= continuous shade days), c2 (= tilt angle), d1 (= longitude) ), D2 (= latitude),
the number of generations for which the solutions were obtained, and the number of evaluations for which the solutions were obtained.
Use -1 if not appropriate.
[Format m3]
-File format is csv file.
-The file name is m_his_***.csv (*** is the group name written in single-byte alphabet).
-From the left, each column should be
the number of generations, the number of evaluations, the HV value of that generation,
the HV value of all solutions evaluated up to that generation,
the number of feasible solutions that satisfy all the constraints in that generation.
Use -1 if not appropriate.
[Format m5]
-File format is csv file
-The file name is m_bst_***.csv (*** is the group name written in single-byte alphabet).
-From the left, each column should be the number of generations, the number of evaluations,
the HV value of that generation, the HV value of all solutions evaluated up to that generation,
the number of feasible solutions that satisfy all the constraints in each generation.
Use -1 if not appropriate.
Post-processing tool
(1) Download the program (R script) for selection of the non-dominated solutions and evaluation of HV value from the output files.
Post-processing program (R script)
[R official site]
You may use your own program as well.
Other
(1) We request the competition participants to make a presentation of about 5 minutes at the competition venue.
In the presentation, include
- Explanation of the algorithm used (including devised points)
- Results
(2) Please note that we will compare the data submitted by everyone in the "Summary of Results".
(3) The data submitted by you may be used for writing a review article.
If there is a problem, let us know before the competition.
Contact information
ec2018-competition@flab.isas.jaxa.jp