This is a Linear Regression Calculator

Input a set of x values and corresponding y values, and inital w and b values. The program will try to find the optimal w and b value such that w*x+b=y for as many pairs of (x,y) values as possible. Yhats are the outputs given by the formula using the current parameters, the goal is for them to be as close to the actual y-values are possible.

The data displays multiple visualizations of the process:

1. The gradient square shows the costs at various w and b values, where both range from -10 to 10 and the x-axis is w and the y-axis is b. A darker square is a lower cost. The lime path shows the different (w,b) combinations the program tests as it tries to find the optimal combination, which on the gradient would be where the darkest pixel is. The lime path gets whiter and whiter the more iterations it undergoes.

2. The two parabolic graphs show the costs when one parameter is varied from -10 to 10 while the other is kept at 0. The left graph is w, the right graph is b.

3. The giant column of text boxes are the various w and b values the program tests.

4. The graph shows the cost of each combination of w and b values per iteration. It should reduce and then plateau at the lowest cost. When the cost goes below a certain threshold, the graph will scale the cost up, which is why it looks like multiple discontinous curves. Each curve is ten times smaller numerically than the one preceding it.

Linear Regression (and most other regressions) work by calculating the total cost of a set of parameters using a unique cost function. This cost function relies on the parameters of whatever formula the program is trying to regress to (in this case, the formula is w*x+b=y, where w and b are the parameters). The goal is to find the values of the parameters that minimize cost. This is done via gradient descent: Find the derivative of the cost function with respect to a parameter, and apply it to the parameter, which basically pushes it towards where the minimum cost is. There's often a learning rate involved to decide how strong of a push the derivative applies, but I created an algorithm that optimizes the learning rate so that it's the biggest possible without breaking the program. By applying gradient descent through a few thousand iterations (for this program I've selected 2000 as the max number of iterations), it is possible to minimize cost and get the optimal set of parameters.