Let’s Learn ML

A gentle introduction to ml and writing one's own classifiers using python.

Dear reader,

At the very beginning of mankind, we predicted whether or not the food was edible or not. How ? let me tell you when a substance was being eaten by an animal mainly dears, and goats if the animal died the people got to know that, okay so this is not edible they just plucked it out from the place and threw them away in the barren piece of land somewhat like the dumping grounds we have today and if the animal survived they all became happy and celebrated that day and they collected that fruit or vegetable or leave as much as they could.

Now in this few last century machines also have started classification and analysis. it is so fascinating to see machines predict things. we don't also know but in our daily life, we are using machine learning and artificial intelligence to perform tasks for us. checking the weather, robotic hands doing work in a factory, computers doing billions and billions of instructions in a minute ever wonder how these things are done.

In this tutorial, I would be using python to code our own classifier from scratch and learn very basic machine learning.

Our brain is very good at emotions and processing and doing work properly in a step by step manner. but computers excel from us because they can do extreme calculations. Machine learning gives machines a brain with intelligence about a particular task and artificial intelligence automates all the tedious and long works for us and makes the lives of people easier. In this blog, I will cover a very basic 2 part of Machine Learning Regression and classification. it is enough to get started. Eg- Project HAESolve is a project that helps people around the globe with machine learning and artificial intelligence.

Our first simple problem

you are given a task to find the price of land given sq. miles.

you have got a piece of simple data with sq. miles and price and now you have to find the price of the land given the square miles

sq. miles, price100, 20
200, 40
400, 60
500, 80

python provides 3 fantastic libraries namely NumPy, pandas, Scikit-Learn

NumPy helps with algebra, pandas help with reading data, and editing and formatting, and Scikit-Learn helps in making predictive models and machines.

So, let’s start some coding for the problem.

import pandas as pd
import sklearn.linear_model.linear_regression as model
import numpy as np
# the most common data file types are .csv, .tsv, and .xlsx / .xls
# pandas provides classes for reading these our file is in format .csv so we will use pd.read_csv("data.csv") here
df = pd.read_csv("data.csv")
df.head(1) #show a small batch of data here only 1 row but as our data is small we dont need it
df.columns # shows the columns here sq. miles, price
# every model / function has a dependent and independent value that is the f(x) = y
x = df["sq. miles"] # the independent variable or on which the out put depends
y = df.drop(["sq. miles"], axis=1)# dependent variable and drop function drops a row when axis = 0 and column when axis = 1
lin_reg_model = model() # a linear regression predicts growth and fall or a particular value given 1 or more instances
# an sklearn linear regression model consists of 3 important functions fit(for fitting data), predict(for predicting data), score(to find how accurate or correct your model is)
lin_reg_model.fit(x,y)
# now model is ready to be tested
lin_reg_model.predict([10]) #meaning of this line is when the sq. miles is 10 what will the price be here it will be 2 but the result will not be 2 it will be very very close to 2 like 1.982377 but not 2
# now lets predict all the values
pred = lin_reg_model.predict(x)
lin_reg_model.score(y,pred) # in linear regression score works in this way it at first subtracts the pred from y and then squares it and finds the average or mean, that means all the value in y is subtracted by the pred and then the whole array is squared and you get all the values of the array squared and then finding average of the whole array.

And voila solved the first problem feeling proud of yourself to wait there is more to cover and solve. feeling the breeze through your hair and power in hand I know and it will be more fun while you solve the first exercise that you will get at the end of the blog.

The great flower mystery

You are a programmer appointed to do a task.

There is a great mystery an owner of a garden has to give 2 drops of medicine on A flower but there are 6 flowers in the garden where there are B flowers also now you have to classify A and B flowers to help that person.

data for us

weight, type
20, 0
10, 1
25, 0
30, 0
5, 1
2, 1
40, 0

let’s go…

import pandas as pd
import numpy as np
import sklearn.linear_model.logistic_regression as model
df = pd.read_csv("../data.csv")
df.head()# the default no. of rows is 5
print(df.coumns)
x = df["weight"]
y = df.drop("weight",axis=1)
log = model().fit(x,y) #as the sklearn model is a class you can also fit while creating the object
# now let's directly predict all the flowers and score our accuracy
y_pred = log.predict(x)
print(y_pred)
log.score(y,y_pred) # here score works a little differently here you just check how many times it has predicted properly and return the percentage of (how many times correct/no. of elements in y) X 100

And, congo! you are done solving your task as a programmer. Feeling very happy right. I know now fasten your seat belts and get ready for the race to begin:-

Get, Set, Go ……

The best part of the blog our own regression and classification models.

let’s code

import numpy as np
class LinearRegression:

def fit(self, X, y, lr = 0.001, epochs=10000, verbose=True, batch_size=1):
X = self.add_bias(X)
self.weights = np.zeros(len(X[0]))
for i in range(epochs):
idx = np.random.choice(len(X), batch_size)
X_batch, y_batch = X[idx], y[idx]
self.weights -= lr * self.get_gradient(X_batch, y_batch)
if i % 1000 == 0 and verbose:
print('Iterations: %d - Error : %.4f' %(i, self.get_loss(X,y)))

def predict(self, X):
return self.predict_(self.add_bias(X))

def get_loss(self, X, y):
return np.mean((y - self.predict_(X)) ** 2)

def predict_(self, X):
return np.dot(X,self.weights)

def add_bias(self,X):
return np.insert(X, 0, np.ones(len(X)), axis=1)

def get_gradient(self, X, y):
return -1.0 * np.dot(y - self.predict_(X), X) / len(X)

def evaluate(self, X, y):
return self.get_loss(self.add_bias(X), y)

Now let’s understand it thoroughly inside our brain the processing happens in the following way.

Image for post
Image for post
A simple neural network showing the processing and prediction inside the brain

Our brain takes the input and multiplies it with a weight, now weight for the neuron is like the amount of processing needed, and bias is added to the data.

So we at first start with a random set of weight and a random bias and then keep changing them this procedure is called gradient descent. here is a visualization of step by step gradient descent.

Image for post
Image for post
Step by Step gradient descent

There are many ways to calculate gradient descent. and gradient descent is often called the optimization process as it optimizes your results and by adjusting the weights and bias.

Now we can program linear regression and do some great stuff, play around with the code soon I will be adding my NumPy and pandas series blogs on medium.

Logistic Regression sounds logit

code:-

import numpy as np

import matplotlib.pyplot as plt

class MultiClassLogisticRegression:

def __init__(self, n_iter = 10000, thres=1e-3):

self.n_iter = n_iter

self.thres = thres

def fit(self, X, y, batch_size=64, lr=0.02, rand_seed=4, verbose=False):

np.random.seed(rand_seed)

self.classes = np.unique(y)

self.class_labels = {c:i for i,c in enumerate(self.classes)}

X = self.add_bias(X)

y = self.one_hot(y)

self.loss = []

self.weights = np.zeros(shape=(len(self.classes),X.shape[1]))

self.fit_data(X, y, batch_size, lr, verbose)

return self

def fit_data(self, X, y, batch_size, lr, verbose):

i = 0

while (not self.n_iter or i < self.n_iter):

self.loss.append(self.cross_entropy(y, self.predict_(X)))

idx = np.random.choice(X.shape[0], batch_size)

X_batch, y_batch = X[idx], y[idx]

error = y_batch - self.predict_(X_batch)

update = (lr * np.dot(error.T, X_batch))

self.weights += update

if np.abs(update).max() < self.thres: break

if i % 1000 == 0 and verbose:

print(' Training Accuray at {} iterations is {}'.format(i, self.evaluate_(X, y)))

i +=1

def predict(self, X):

return self.predict_(self.add_bias(X))

def predict_(self, X):

pre_vals = np.dot(X, self.weights.T).reshape(-1,len(self.classes))

return self.softmax(pre_vals)

def softmax(self, z):

return np.exp(z) / np.sum(np.exp(z), axis=1).reshape(-1,1)

def predict_classes(self, X):

self.probs_ = self.predict(X)

return np.vectorize(lambda c: self.classes[c])(np.argmax(self.probs_, axis=1))

def add_bias(self,X):

return np.insert(X, 0, 1, axis=1)

def get_randon_weights(self, row, col):

return np.zeros(shape=(row,col))

def one_hot(self, y):

return np.eye(len(self.classes))[np.vectorize(lambda c: self.class_labels[c])(y).reshape(-1)]

def score(self, X, y):

return np.mean(self.predict_classes(X) == y)

def evaluate_(self, X, y):

return np.mean(np.argmax(self.predict_(X), axis=1) == np.argmax(y, axis=1))

def cross_entropy(self, y, probs):

return -1 * np.mean(y * np.log(probs))
Image for post
Image for post
Our neural net structure

here in the above picture logit score means our x X weights + bias. we just follow our steps followed in linear regression only add a small part that is an activation function or the function which helps in converting the logit scores into a probabilistic map or the probability of the object falling into the following category and here the optimization is a cross-entropy loss. Rest everything is the same for logistic regression.

So, I can see that beam. You just finished learning the basics of machine learning.

Now time for some exercise:-

1) given a peice of data with year and no. of patient 
find out that how many patient will be there in the year 2025
2) given blue or red flowers with their petal length and petal width find out the flower type the petal length is 2,2 and 4.5,2

data for exercise 1 (copy the following data and save it as data.csv and complete the program yourself):-

No_Of_Patients, year
1260000000,1999
1240000000,20001250000000,20011270000000,20021300000000,20031290000000,20041190000000,20051200000000,20061280000000,2007

data for exercise 2 (copy the following variable and complete the exercise):-

data = pd.dataframe([[3,1.5,1],[2,1,0],[4,1.5,1],[3,1,0],[3.5,.5,1],[2,.5,0],[5.5,1,1],[1,1,0]])#creates a data frame object

Please do share your result with me at goswamidivyosmi@gmail.com.

Thank you.

Please show your appreciation with some claps because of each clap counts.

Written by

A digital nomad's account wandering through the physical and cyber city discovering himself.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store