What is Gradient Descent?
Gradient Descent is a method to minimize the cost
The graph of

Let’s assume that the initial value of

We are going to calculate the gradient of

By repeating the process above until

What is Stochastic Gradient Descent?
Stochastic Gradient Descent compensates for the disadvantage of Gradient Descent, which cannot be implemented when there is no single convex region.
Gradient Descent trains on the entire dataset and then updates the weights, whereas Stochastic Gradient Descent updates the weights each time a row of the dataset is trained, as follows:
