8:00 – 4:30 ASRC MKT
- Still working on the nomad->flocking->stampede slide. Do I need a “dimensions” arrow?
- Labeled slides. Need to do timings – done
- And then Aaron showed up, so lots of reworking. Done again!
- Put the ONR proposal back in its original form
- An overview of gradient descent optimization algorithm
- Gradient descent is one of the most popular algorithms to perform optimization and by far the most common way to optimize neural networks. At the same time, every state-of-the-art Deep Learning library contains implementations of various algorithms to optimize gradient descent (e.g. lasagne’s, caffe’s, and keras’ documentation). These algorithms, however, are often used as black-box optimizers, as practical explanations of their strengths and weaknesses are hard to come by. This blog post aims at providing you with intuitions towards the behaviour of different algorithms for optimizing gradient descent that will help you put them to use.