How dumb is your smartphone?
16/09/2023
Have you ever wonder the phone your reading this on is just as dumb as it was yesterday... Unless it learns.
Yes, They can learn and this phenomenon is know to us as
MACHINE LEARNING
Okay okay, before you either close this tab or run over to get a book, Know that im no expert but, i can assure you that by the end of this blog you will have a slightly better understanding about whats all this fuzz about. So without any futher ado
I WELCOME YOU TO
Lets say you are given
[0, 0]
[1 , 2]
[2, 4]
[3, 6]
[4, x]
and if you were ask what will x be, you could just take a look and be like 8? and YES!!
Now doing the same with your laptop or phone may be useless
Cause it wont understand our language. we need to talk to it using the universal language...
Before you being to throw random calculus and functions at it, remember how did you figure out the answer.
You might have tried doing something with the inputs that gets you closer to the outputs
Y = X * W
Y : Output
X : Input
W : something we are willing to find
With that out of the way, lets actually make this tin box do some learning
# Do the obvious
Give this tin box somewhere to start and whats better than giving this W a random value
float w = rand()
float y = w * x
Now iterate through the inputs and get the output using this w
Worse or bad, we have a starting point!
Lets look at the output values
Actual : 00.00000 Expected : 00.000000
Actual : 07.107210 Expected : 02.000000
Actual : 14.214420 Expected : 04.000000
Actual : 21.321630 Expected : 06.000000
Boy oh boy, we are way off
We need a way to tell our tin box how badly are we calculating...
But... we cant just yell it, we need math
So how do we do it?
# Do the obvious
Calculate the distance
It is easy to just find the difference between actual and expected values
float y = x * w
float d = y - expected_value
But we need to consider all of the inputs and thats why we will be needing to store all of the differences together
float result = 0;
loop(inputs in data)
--float y = inputs * w
--float d = y - expected_value
--result += (d * d)
result /= length(data)
The reason we are squaring the distances is because it could be negative and also cause squaring will help magnify even the smallest error. At last we find the mean of all the distances squared and added up
DONT FREAK OUT!
You had just calculated something know as a cost function
Now your tin box knows exactly how far is it from being correct
Lets see how far we are
cost : 79.78347259
Imagine, if y was close enough to expected_value
the distance would be near to 0
and so will be the result too!
All we have to do now is to drive this cost function near to ZERO
But How?! well i hope you like maths cause...
CALCULUS WARNING !!
If you had paid enough attention in you M-1 class
you may know something about finite difference which is used to approximate derivatives
If that made no sense, Dont worry
You may know this
The derivative tells us about the growth of the function
Since we want to minimize the value, we will need to move in the direction opposite to that of the growth
fun cost(w):
--float result = 0;
--loop(inputs in data)
----float y = inputs * w
----float d = y - expected_value
----result += (d * d)
--result /= length(data)
--return result
float eps = 1e-3
float rate = 1e-3
float dw = (cost(w + eps) - cost(w)) / eps
w -= dw * rate
eps (0.001) is that small change that tends to zero
rate (0.001) decides how much of a step should we take in the direction opposite to that of the growth.
cost(w) : 79.78347259
cost(w) : 79.30495521
YES!!! The cost is decreased
How about iterating in a loop and then checking in on the new w we will end up with
for(iterations=500):
--float dw = (cost(w + eps) - cost(w)) / eps
--w -= dw * rate
cost(w) : 79.78347259
cost(w) : 5.018469931
Holy freaking moly, Its approaching to zero
Lets iterate some more...
for(iterations=1500):
--float dw = (cost(w + eps) - cost(w)) / eps
--w -= dw * rate
cost(w) : 79.78347259
cost(w) : 0.000068932
Now if we were to calculate the output with this new w
we would get some pretty great outputs
float w = rand()
float y = w * x
SCORE CARD
Actual : 00.00000 Expected : 00.000000
Actual : 02.04610 Expected : 02.000000
Actual : 04.00920 Expected : 04.000000
Actual : 06.00384 Expected : 06.000000
# who W?
You know, that W we were trying to find is known as Weight
It determines how strong the input from one neuron influences the output of the connected neuron
# who neuron?
Its that messy diagram we were thinking of,
It is consist of an input, weight and an output. Although there are alot of other things such as activation function and bias
But for today, all you need to know is neuron is the smallest block of a Neural Network
# who neural network?
She is the girl next door...
You see, At times the hardest part isnt doing it,
its to not trying to start doing it
So if you wanna know more and more just get a book and start reading about it
