MomentumGradientAscent
Related Doc:
package optim
Inertia parameter
compute the ascent direction.
compute the ascent direction.
Current batch gradient
Wrapper for accelerated gradient ascent utilities
Deffinitions for algebraic operations for the apropiate data structure, e.g. vector or matrix. .
updated parameter values
Use fromSimplex method from WeightsTransformation
Use fromSimplex method from WeightsTransformation
mixture weights
Compute full updates for the model's parameters.
Compute full updates for the model's parameters. Usually this has the form X_t + alpha * direction(X_t) but it differs for some algorithms, e.g. Nesterov's gradient ascent.
Current parameter values
Current batch gradient
Wrapper for accelerated gradient ascent utilities
Deffinitions for algebraic operations for the apropiate data structure, e.g. vector or matrix. .
updated parameter values
Alternative method to set step size's shrinkage rate.
Alternative method to set step size's shrinkage rate. it will be automatically calculated to shrink the step size by half every m iterations.
positive intger
this
Step size
Step size
Minimum allowed learning rate.
Minimum allowed learning rate. Once this lower bound is reached the learning rate will not shrink anymore
Rate at which the learning rate is decreased as the number of iterations grow.
Rate at which the learning rate is decreased as the number of iterations grow. After t iterations the learning rate will be shrinkageRate^t * learningRate
Use toSimplex method from WeightsTransformation
Use toSimplex method from WeightsTransformation
valid mixture weight vector
Shrink learningRate by a factor of shrinkageRate
Shrink learningRate by a factor of shrinkageRate
Implementation of gradient descent with momentum
As formulated in Goh, "Why Momentum Really Works", Distill, 2017. http://doi.org/10.23915/distill.00006