Optimizer using Natural Evolution Strategies

NaturalEvolutionStrategiesOptimizer

class l2l.optimizers.naturalevolutionstrategies.optimizer.NaturalEvolutionStrategiesOptimizer(traj, optimizee_create_individual, optimizee_fitness_weights, parameters, optimizee_bounding_func=None)[source]

Bases: l2l.optimizers.optimizer.Optimizer

Class Implementing the separable natural evolution strategies optimizer in natural coordinates as in:

Wierstra, D., Schaul, T., Peters, J., & Schmidhuber, J. (2008). Natural evolution strategies. In Evolutionary Computation, 2008. CEC 2008.(IEEE World Congress on Computational Intelligence) (pp. 3381-3387).

Glasmachers, T., Schaul, T., Yi, S., Wierstra, D., & Schmidhuber, J. (2010). Exponential natural evolution strategies. In Proceedings of the 12th annual conference on Genetic and evolutionary computation (pp. 393-400).

Wierstra, D., Schaul, T., Glasmachers, T., Sun, Y., Peters, J., & Schmidhuber, J. (2014). Natural evolution strategies. In Journal of Machine Learning Research, 15(1) (pp. 949-980).

In the pseudo code the algorithm does:

For n iterations do:

  • Sample individuals z from multinormal search distribution with parameters mu, sigma

    s <- sample from N(0,1) z <- mu + sigma * s

  • If mirrored sampling is enabled, also sample individuals with opposite perturbations s

    z <- [mu + sigma * s, mu - sigma * s]

  • evaluate individuals z and get fitnesses F_i(z)

  • Update the parameters of the search distribution as

    mu_{t+1} <- mu_{t+1} + eta_mu * sigma * sum(F_i * s_i) sigma_{t+1} <- sigma_t * exp(eta_sigma / 2 * sum(F_i * (s_i ** 2 - 1))

  • If fitness shaping is enabled, F_i is replaced with the utility u_i in the previous step, which is calculated as:

    u_i = max(0, log(n/2 + 1) - log(k)) / sum_{k=1}^{n}{max(0, log(n/2 + 1) - log(k))} - 1 / n

    where k and i are the indices of the individuals in descending order of fitness F_i

Parameters

  • traj (Trajectory) – Use this trajectory to store the parameters of the specific runs. The parameters should be initialized based on the values in parameters

  • optimizee_create_individual – Function that creates a new individual. All parameters of the Individual-Dict returned should be of numpy.float64 type

  • optimizee_fitness_weights – Fitness weights. The fitness returned by the Optimizee is multiplied by these values (one for each element of the fitness vector)

  • parameters – Instance of namedtuple() NaturalEvolutionStrategiesParameters containing the parameters needed by the Optimizer

post_process(traj, fitnesses_results)[source]

See post_process()

end(traj)[source]

See end()

NaturalEvolutionStrategiesParameters

class l2l.optimizers.naturalevolutionstrategies.optimizer.NaturalEvolutionStrategiesParameters(learning_rate_mu, learning_rate_sigma, mu, sigma, mirrored_sampling_enabled, fitness_shaping_enabled, pop_size, n_iteration, stop_criterion, seed)

Bases: tuple

Parameters

  • learning_rate_mu – Learning rate for mean of distribution

  • learning_rate_sigma – Learning rate for standard deviation of distribution

  • mu – Initial mean of search distribution

  • sigma – Initial standard deviation of search distribution

  • mirrored_sampling_enabled – Should we turn on mirrored sampling i.e. sampling both e and -e

  • fitness_shaping_enabled – Should we turn on fitness shaping i.e. using only top fitness_shaping_ratio to update current individual?

  • pop_size – Number of individuals per simulation.

  • n_iteration – Number of iterations to perform

  • stop_criterion – (Optional) Stop if this fitness is reached.

  • seed – The random seed used for generating new individuals

property fitness_shaping_enabled
property learning_rate_mu
property learning_rate_sigma
property mirrored_sampling_enabled
property mu
property n_iteration
property pop_size
property seed
property sigma
property stop_criterion