Essentials of Machine Learning Algorithms (Python and R)

До загрузки: 30 сек.



Благодарим, что скачиваете у нас :)

Если, что - то:

  • Поделится ссылкой:
  • Документ найден в свободном доступе.
  • Загрузка документа - бесплатна.
  • Если нарушены ваши права, свяжитесь с нами.
Формат: pdf
Найдено: 13.08.2020
Добавлено: 06.10.2020
Размер: 0.51 Мб

Broadly, there are 3 types of Machine Learning Algorithms..
1. Supervised Learning
How it works: This algorithm consist of a target / outcome variable (or dependent variable) which is to be predicted from a given set of pr edictors (independent
variables). Using these set of variables, we generate a function that map inputs to desired outputs. The tra ining process continues until the model achieves a
desired level of accuracy on the training data. Examples of Supervised Learning: Regression, Decision Tree , Random Forest , KNN, Logistic Regression etc.
2. Unsupervised Learning
How it works: In this algorithm, we do not have any target or outcome variable to predict / estimate. It is used for clustering population in different groups, which
is widely used for segmenting customers in different groups for specific intervention. Examples of Unsu pervised Learning: Apriori algorithm, K -means.

3. Reinforcement Learning:
How it works: Using this algorithm, the machine is trained to make specific decisions. It works this way: the machine is exposed to an envi ronment where it trains
itself continually using trial and error. This machine learns from past experience and tries to capture t he best possible knowledge to make accurate business
decisions. Example of Reinforcement Learning: Markov Decision Process
List of Common Machine Learning Algorithms
Here is the list of commonly used machine learning algorithms. These algorithms can be app lied to almost any data problem:
1. Linear Regression
2. Logistic Regression
3. Decision Tree
4. SVM

5. Naive Bayes
6. kNN
7. K-Means
8. Random Forest
9. Dimensionality Reduction Algorithms
10. Gradient Boosting algorithms
1. GBM
2. XGBoost
3. LightGBM
4. CatBoost
1. Linear Regression
It is used to estimate real values (cost of houses, number of calls, total sales etc.) based on continuous variable(s). Here, we establish relationship between
independent and dependent variables by fitting a best line. This best fit l ine is known as regression line and represented by a linear equation Y= a *X + b.
The best way to understand linear regression is to relive this experience of childhood. Let us say, you ask a child in fifth grade to arrange people in his class by
increasin g order of weight, without asking them their weights! What do you think the child will do? He / she would likely look (visual ly analyze) at the height and
build of people and arrange them using a combination of these visible parameters. This is linear regr ession in real life! The child has actually figured out that height
and build would be correlated to the weight by a relationship, which looks like the equation above.
In this equation:
Y – Dependent Variable a – Slope X – Independent variable b – Interc ept
These coefficients a and b are derived based on minimizing the sum of squared difference of distance between data points and regression line.
Look at the below example. Here we have identified the best fit line having linear equation y=0.2811x+13.9 . Now using this equation, we can find the weight,
knowing the height of a person.

Linear Regression is of mainly two types: Simple Linear Regressio n and Multiple Linear Regression. Simple Linear Regression is characterized by one independent
variable. And, Multiple Linear Regression(as the name suggests) is characterized by multiple (more than 1) independent variab les. While finding best fit line, yo u
can fit a polynomial or curvilinear regression. And these are known as polynomial or curvilinear regression.
Python Code
#Import Library
#Import other necessary libraries like pandas, numpy...
from sklearn import linear_model
#Load Train and Test datasets
#Identify feature and response variable(s) and values mus t be numeric and numpy arrays

x_train=input_variables_values_training_datasets y_train=target_variables_values_training_datasets
x_test=input_variables_values_test_datasets
# Create linear regression object linear = linear_model.LinearRegression()
# Train the model using the training sets and check score linear.fit(x_train, y_train ) linear.score(x_train, y_train)
#Equation coefficient and Intercept print('Coefficient: \n', linear.coef_) print('Intercept: \n', linear.intercept_)
#Predict Output
predicted= linear.predict(x_test)
R Code
#Load Train and Test datasets
#Identify feat ure and response variable(s) and values mus t be numeric and numpy arrays
x_train < - input_variables_values_training_datasets y_train < - target_variables_values_training_datasets x_test < -
input_variables_values_test_datasets x < - cbind(x_train,y_train)
# Train the model using the training sets and check score linear < - lm(y_train ~ ., data = x)

summary(linear) #Predict Output
predicted= predict(linear,x_test)

2. Logistic Regression
Don’t get confused by its name! It is a classification not a regression algorithm. It is used to estimate discrete values ( B inary values like 0/1, yes/no, true/false )
based on given set of independent variable(s). In simple words, it predicts the probability of occurrence of an event by fitting data to a logit function . Hence, it is
also known as logit regression . Since, it predicts the probability, its output values lies between 0 and 1 (as expected).
Again, let us try and understa nd this through a simple example.
Let’s say your friend gives you a puzzle to solve. There are only 2 outcome scenarios – either you solve it or you don’t. Now imagine, that you are being given wide
range of puzzles / quizzes in an attempt to understand wh ich subjects you are good at. The outcome to this study would be something like this – if you are given
a trignometry based tenth grade problem, you are 70% likely to solve it. On the other hand, if it is grade fifth history ques tion, the probability of ge tting an answer
is only 30%. This is what Logistic Regression provides you.
Coming to the math, the log odds of the outcome is modeled as a linear combination of the predictor variables.
odds= p/ (1 -p) = probability of event occurrence / probab ility of no t event occurrence ln(odds) = ln(p/(1 -p))
logit(p) = ln(p/(1 -p)) = b0+b1X1+b2X2+b3X3....+bkXk
Above, p is the probability of presence of the characteristic of interest. It chooses parameters that maximize the likelihood of observing the sample values rather
than that minimize the sum of squared errors (like in ordinary regression).
Now, you may ask , why take a log? For the sake of simplicity, let’s just say that this is one of the best mathematical way to replicate a ste p function. I can go in
more details, but that will beat the purpose of this article.

Python Code
#Import Library
from sklearn.linear_model import LogisticRegression #Assumed you have, X (predictor) and Y (target) for train ing data
set and x_test(predictor) of test_dataset
# Create logistic regression object model = LogisticRegression()
# Train the model using the training sets and check score model.fit(X, y) model.score(X, y)
#Equation coefficient and Intercept print('Coefficient: \n', model.coef_) print('Intercept: \n', model.intercept_)
#Predict Output
predicted= model.predict(x_tes t)
R Code
x < - cbind(x_train,y_train)
# Train the model using the training sets and check score logistic < - glm(y_train ~ ., data = x,family='binomial')
summary(logistic) #Predict Output predicted= predict(logistic,x_test)

Furthermore..

There are many different steps that could be tried in order to improve the model:
including interaction terms removing features regularization techniques using a non -linear model

3. Decision Tree
This is one of my favorite algorithm and I use it quite frequently. It is a type of supervised learning algorithm that is mostly used
for classification problems. Surprisingly, it works for both categorical and continuous dependent variables. In th is algorithm, we split the population into two or
more homogeneous sets. This is done based on most significant attributes/ independent variables to make as distinct groups as possible. For more details, you
can read: Decision Tree Simplified .

source: statsexchange

In the image above, you can see that population is classified into four different groups based on multiple attributes to iden tify ‘if they will play or not’. To split the
population into different heterogeneous groups, it uses various technique s like Gini, Information Gain, Chisquare, entropy.
The best way to understand how decision tree works, is to play Jezzball – a classic game from Microsoft (image below). Essentially, you have a room with moving
walls and you need to create walls such that maximum area gets cleared off with out the balls.

So, every time you split the room with a wall, you are trying to create 2 different populations with in the same room. Decisi on trees work in very similar fashion
by dividing a population in as different groups as possible.
More : Simplified Version of Decision Tree Algorithms
Python Code
#Import Library
#Import other necessary libraries like pandas, numpy... from sklearn import tree
#Assumed you have, X (predictor) and Y (target) for train ing data set and x_test(predictor) of test_dataset
# Create tree object
model = tree.DecisionTreeClassifier(criterion='gini') # f or classification, here you can change the algorithm as g ini or
entropy (information gain) by default it is gini # model = tree.DecisionTreeRegressor() for regression
# Train the model using the training sets and check score model.fit(X, y) model.score(X, y) #Predict Output predicted=
model.predict(x_test)

R Code
library(rpart) x < - cbind(x_train,y_train) # grow tree
fit < - rpart(y_train ~ ., data = x,method="class") summary(fit) #Predict Output predicted=
predict(fit,x_test)

4. SVM (Support Vector Machine)
It is a classification method. In this algorit hm, we plot each data item as a point in n -dimensional space (where n is number of features you have) with the value
of each feature being the value of a particular coordinate.
For example, if we only had two features like Height and Hair length of an indi vidual, we’d first plot these two variables in two dimensional space where each point
has two co -ordinates (these co -ordinates are known as Support Vectors )

Now, we will find some line that splits the data between the two differently classified groups of data. This will be the line such that the distances fro m the closest
point in each of the two groups will be farthest away.

In the example shown above, the line which splits the data i nto two differently classified groups is the black line, since the two closest points are the farthest apart
from the line. This line is our classifier. Then, depending on where the testing data lands on either side of the line, that’ s what class we can cl assify the new data
as.
More: Simplified Version of Support Vector Machine
Think of this algorithm as playing JezzBall in n -dimensional space. The tweaks in the game are:
You can draw lines / planes at any angles (rather than just horizontal or vertical a s in classic game)
The objective of the game is to segregate balls of different colors in different rooms. And the balls are not moving.

Python Code
#Import Library from sklearn import svm
#Assumed you have, X (predictor) and Y (target) for train ing da ta set and x_test(predictor) of test_dataset
# Create SVM classification object
model = svm.svc() # there is various option associated wi th it, this is simple for classification. You can refer l ink ,
for mo# re detail.
# Train the model using the training sets and check score model.fit(X, y) model.score(X, y) #Predict Output predic ted=
model.predict(x_test)
R Code
library(e1071) x < - cbind(x_train,y_train) # Fitting model fit < -svm(y_train ~ ., data = x)
summary(fit) #Predict Output predicted= predict(fit,x_test)

5. Naive Bayes
It is a classification technique based on Bayes’ theorem with an assumption of independence between predictors. In simple terms, a Naive Bayes classifier assumes
that the presence of a particular feature in a class is unrelated to the presence of any other feature. For example, a fruit may be considered to be an apple if it is
red, round, and about 3 inches in diameter. Even if these features depend on each other or upon the existence of the other fe atures, a naive Bayes classifier would
consider all of the se properties to independently contribute to the probability that this fruit is an apple.

Naive Bayesian model is easy to build and particularly useful for very large data sets. Along with simplicity, Naive Bayes is known to outperform even highly
sophisti cated classification methods.
Bayes theorem provides a way of calculating posterior probability P(c|x) from P(c), P(x) and P(x|c). Look at the equation bel ow:

Here,
P(c|x ) is the posterior probability of class (target ) given predictor
(a ribute ).
P(c) is the prior probability of class .
P(x|c ) is the likelihood which is the probability of predictor given class .
P(x) is the prior probability of predictor .
Example: Let’s understand it using an example. Below I have a training data set of weather and corresponding target variable ‘Play’. N ow, we need to classify
whether players will play or not based on weather condition. Let’s follow the below steps to perform it.
Step 1: Convert the data set to frequency table
Step 2: Create Likelihood table by finding the probabilities like Overcast probability = 0.29 and probability of playing is 0 .64.

Step 3: Now, use Naive Bayesian equation to calculate the posterior probabilit y for each class. The class with the highest posterior probability is the outcome of
prediction.
Problem: Players will pay if weather is sunny, is this statement is correct?
We can solve it using above discussed method, so P(Yes | Sunny) = P( Sunny | Yes) * P(Yes) / P (Sunny)
Here we have P (Sunny |Yes) = 3/9 = 0.33, P(Sunny) = 5/14 = 0.36, P(
Yes)= 9/14 = 0.64
Now, P (Yes | Sunny) = 0.33 * 0.64 / 0.36 = 0.60, which has higher probability.
Naive Bayes uses a similar method to predict the probability of different class based on various attributes. This algorithm i s mostly used in text classification and
with problems having multiple classes.
Python Code
#Import Library
from sklearn.naive_bay es import GaussianNB #Assumed you have, X (predictor) and Y (target) for train ing data set and
x_test(predictor) of test_dataset # Create SVM classification object model = GaussianNB() # there is other distribution
for multinomial classes like
Bernoulli Naive Bayes, Refer link

# Train the model using the training sets and check score model.fit(X, y) #Predict Output predicted= model.predict(x_test)
R Code
library(e1071) x < - cbind(x_train,y_train) # Fitting model fit < -naiveBayes(y_train ~ . , data = x)
summary(fit) #Predict Output
predicted= predict(fit,x_test)

6. kNN (k - Nearest Neighbors)
It can be used for both classification and regression problems. However, it is more widely used in classification problems in the industry. K nearest neighbors is a
simple algorithm that stores all available cases and classifies new cases by a majority vot e of its k neighbors. The case being assigned to the class is most common
amongst its K nearest neighbors measured by a distance function.
These distance functions can be Euclidean, Manhattan, Minkowski and Hamming distance. First three functions are used for continuous function and fourth one
(Hamming) for categorical variables. If K = 1, then the case is simply assigned to the class of its nearest neighbor. At time s, choosing K turns out to be a challenge
while performing kNN modeling.
More: Introduction to k -nearest neighbors : Simplified .

KNN can easily be mapped to our real lives. If you want to learn about a person, of whom you have no information, you might l ike to find out about his close
friends and the circles he moves in and gain access to his/h er information!
Things to consider before selecting kNN:
KNN is computationally expensive
Variables should be normalized else higher range variables can bias it
Works on pre -processing stage more before going for kNN like outlier, noise removal
Python Code
#Import Library
from sklearn.neighbors import KNeighborsClassifier #Assumed you have, X (predictor) and Y (target) for train ing data set
and x_test(predictor) of test_dataset # Create KNeighbors classifier object model KNeighborsClassifier(n_neighbors=6) #
default value for n
_neighbors is 5
# Train the model using the training sets and check score model.fit(X, y) #Predict Output predicted= model.predict(x_test)

R Code
library(knn) x < - cbind(x_train,y_train) # Fitting model fit < -knn(y_train ~ ., data = x,k=5)
summary(fit) #Predict Output predicted= predict(fit,x_test)

7. K -Means
It is a type of unsupervised algorithm which s olves the clustering problem. Its procedure follows a simple and easy way to classify a given data set through a
certain number of clusters (assume k clusters). Data points inside a cluster are homogeneous and heterogeneous to peer group s.
Remember figur ing out shapes from ink blots? k means is somewhat similar this activity. You look at the shape and spread to decipher how ma ny different clusters
/ population are present!

How K -means forms cluster:

1. K-means picks k number of points for each cluster known as centroids.
2. Each data point forms a cluster with the closest centroids i.e. k clusters.
3. Finds the centroid of each cluster based on existing cluster members. Here we have new centroids.
4. As we have n ew centroids, repeat step 2 and 3. Find the closest distance for each data point from new centroids and get associated with n ew k -clusters.
Repeat this process until convergence occurs i.e. centroids does not change.
How to determine value of K:
In K -means , we have clusters and each cluster has its own centroid. Sum of square of difference between centroid and the data points wi thin a cluster constitutes
within sum of square value for that cluster. Also, when the sum of square values for all the clusters ar e added, it becomes total within sum of square value for the
cluster solution.
We know that as the number of cluster increases, this value keeps on decreasing but if you plot the result you may see that t he sum of squared distance decreases
sharply up to s ome value of k, and then much more slowly after that. Here, we can find the optimum number of cluster.

Python Code
#Import Library from sklearn.cluster import KMeans #Assumed you have, X (attributes) for training
data set a nd x_test(attributes) of test_ dataset # Create KNeighbors classifier object model

k_means = KMeans(n_clusters=3, random_state=0) # Train the model using the training sets and check
score model.fit(X) #Predict Output predicted= model.predict(x_test)
R Code
library(cluster) fit < - kmeans(X, 3) # 5 cluster solution

8. Random Forest
Random Forest is a trademark term for an ensemble of decision trees. In Random Forest, we’ve collection of decision trees (so known as “Forest”). To classify a
new object based on attributes, each tree gives a classification and we say the tree “votes” for that class. The forest chooses the classificatio n having the most
votes (over all the trees in the forest).
Each tree is planted & grown as follows:
1. If the number of cases in the training set is N, then sample of N cases is taken at random but with replacement . This sample will be the training set for
growing the tree.
2. If there are M input variables, a number m< these m is used to split the node. The value of m is held constant during the forest growing.
3. Each tree is grown to the largest extent possible. There is no pruning.
For more details on this algorithm, comparing with decision tree and tuning model parameters, I would suggest you to read these articles:
1. Introduction to Random forest – Simplified
2. Comparing a CART model to Random Forest (Part 1)
3. Comparing a Random Forest to a CART model (Part 2)

4. Tuning the parameters of your Random Forest model
Python
#Import Library
from sklearn.ensemble import RandomForestClassifier #Assumed you have, X (predictor) and Y (target) for train ing data
set and x_te st(predictor) of test_dataset
# Create Random Forest object model= RandomForestClassifier()
# Train the model using the training sets and check score model.fit(X, y) #Predict Output predicted= model.predict(x_test)
R Code
library(randomForest) x < - cbind(x_train,y_train)
# Fitting model
fit < - randomForest(Species ~ ., x,ntree=500) summary(fit) #Predict Output
predicted= predict(fit,x_test)

9. Dimensionality Reduction Algorithms
In the last 4 -5 years, there has been an exponential increase in data capturing at every possible stages. Corporates/ Government
Agencies/ Research organisations are not only coming with new sources but also they are capturing data in great detail.

For example: E -commerce companies are capturing more details about customer like their demographics, web crawling history, what they like or dislike, purchase
history, feedback and many others to give them personalized attention more than your nearest groce ry shopkeeper.
As a data scientist, the data we are offered also consist of many features, this sounds good for building good robust model b ut there is a challenge. How’d you
identify highly significant variable(s) out 1000 or 2000? In such cases, dimensio nality reduction algorithm helps us along with various other algorithms like Decision
Tree, Random Forest, PCA, Factor Analysis, Identify based on correlation matrix, missing value ratio and others.
To know more about this algorithms, you can read “ Beginne rs Guide To Learn Dimension Reduction Techniques “.
Python Code
#Import Library
from sklearn import decomposition
#Assumed you have training and test data set as train and test
# Create PCA obeject pca= decomposition.PCA(n_components=
k) #default value of k =min(n_sample, n_features)
# For Factor analysis
#fa= decomposition.FactorAnalysis()
# Reduced the dimension of training dataset using PCA train_reduced = pca.fit_transform(train) #Reduced the
dimension of test dataset
test_reduced = pca.transform(test) #For more detail on this, please refer this link .

R Code
library(stats) pca < - princomp(train, cor = TRUE) train_reduced < - predict(pca,t rain) test_reduced
<- predict(pca,test)

10. Gradient Boosting Algorithms
10.1. GBM
GBM is a boosting algorithm used when we deal with plenty of data to make a prediction with high prediction power. Boosting i s actually an ensemble of learning
algorithms which combines the prediction of several base estimators in order to improve robustne ss over a single estimator. It combines multiple weak or average
predictors to a build strong predictor. These boosting algorithms always work well in data science competitions like Kaggle, AV Hackathon, CrowdAnalytix.
More: Know about Boosting algorithms in detail
Python Code
#Import Library
from sklearn.ensemble import GradientBoostingClassifier #Assumed you have, X (predictor) and Y (target) for train ing data
set and x_test(predictor) of test_dataset # Create Gradient Boosting Classifier object model=
GradientBoostingClassifier(n_estimators=100, learn ing_rate=1.0, max_depth=1, random_state=0) # Train the model using the
training sets and check score model.fit(X, y) #Predict Output predicted= model.predict(x_test)

R Code
library(caret) x < - cbind(x_train,y_train) # Fitting model
fitControl < - trainControl( method = "repeatedcv", number
= 4, repeats = 4)
fit < - train(y ~ ., data = x, method = "gbm", trControl = fitControl,verbose = FALSE) predicted= predict(fit,x_test,type=
"prob")[,2]
GradientBoostingClassifier and Random Forest are two different boosting tree classifier and often people ask about the difference between these two
algorithms .
10.2. XGBoost
Another classic gradient boosting algorithm that’s known to be the decisive choice between winning and losing in some Kaggle competitions.
The XGBoost has an immensely high predictive power which makes it the best choice for accuracy in events as it possesses both linear model and the tree
learning algorithm, making the algorithm almost 10x faster than existing gradient booster techniques.
Th e support includes various objective functions, including regression, classification and ranking.
One of the most interesting things about the XGBoost is that it is also called a regularized boosting technique. This helps t o reduce overfit modelling and ha s a
massive support for a range of languages such as Scala, Java, R, Python, Julia and C++.
Supports distributed and widespread training on many machines that encompass GCE, AWS, Azure and Yarn clusters. XGBoost can a lso be integrated with Spark,
Flink and other cloud dataflow systems with a built in cross validation at each iteration of the boosting process.
To learn more about XGBoost and parameter tuning, visit https://www.analyticsvidhya.com/blog/2016/03/complete -guideparameter -tuning -xgboost -with -
codes -python/ .

Python Code:
from xgboost import XGBClassifier
from sklearn.model_selection import train_test_split from sklearn.metrics import accuracy_score X =
dataset[:,0:10] Y = dataset[:,10:] seed = 1
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.33, random_state=seed)
model = XGBClassifier()
model.fit(X_train, y_train)
#Make predictions for test data y_pred = model.predict(X_test)
R Code:
require(caret)
x < - cbind(x_train,y_train) # Fitting model TrainControl < - trainControl( method = "repeatedcv", numb er =
10, repeats = 4)

model< - train(y ~ ., data = x, method = "xgbLinear", trCo ntrol = TrainControl,verbose = FALSE) OR
model< - train(y ~ ., data = x, method = "xgbTree", trCont rol = TrainControl,verbose = FALSE)
predicted < - predict(model, x_test)

10.3. LightGBM
LightGBM is a gradient boosting framework that uses tree based learning algorithms. It is designed to be distributed and effi cient with the following advantages:
Faster training speed and higher efficiency
Lower memory usage
Better accuracy
Parallel and G PU learning supported
Capable of handling large -scale data
The framework is a fast and high -performance gradient boosting one based on decision tree algorithms, used for ranking, classification and many other machine
learning tasks. It was developed under the Distributed Machine Learning Toolkit Project of Microsoft.
Since the LightGBM is based on decision tree algorithms, it splits the tree leaf wise with the best fit whereas other boostin g algorithms split the tree depth wise
or level wise rather than lea f-wise. So when growing on the same leaf in Light GBM, the leaf -wise algorithm can reduce more loss than the level -wise algorithm
and hence results in much better accuracy which can rarely be achieved by any of the existing boosting algorithms.
Also, it is surprisingly very fast, hence the word ‘Light’.
Refer to the article to know more about LightGBM:
https://www.analyticsvidhya.com/blog/2017/06/which -algorithmtakes -the -crown -light -gbm -vs-xgboost/

Python Code:
data = np.random.rand(500, 10) # 500 entities, each conta ins 10 features
label = np.random.randint(2, size=500) # binary target
train_data = l gb.Dataset(data, label=label) test_data = train_data.create_valid('test.svm')
param = {'num_leaves':31, 'num_trees':100, 'objective':'b
inary'}
param['metric'] = 'auc'
num_round = 10
bst = lgb.train(param, train_data, num_round, valid_sets=
[test_data])
bst.save_model('model.txt')
# 7 entities, each contains 10 features data = np.random.rand(7, 10) ypred = bst.predict(data)
R Code:

library(RLightGBM) data(example.binary)
#Parameters
num_iterations < - 100
config < - list(objective = "binary", metric="binary_logl oss,auc", learning_rate = 0.1, num_leaves = 63, tree_lear ner = "serial", fea ture_fraction = 0.8, bagging_freq = 5, bagging_fraction =
0.8, min_data_in_leaf = 50, min_sum_he

ssian_in_leaf = 5.0)
#Create data handle and booster handle.data < - lgbm.data.create(x)
lgbm.data.setField(handle.data, "label", y)
handle.booster < - lgbm.booster.create(handle.data, lapply (config, as.character))
#Train for num_iterations iterations and eval every 5 ste ps
lgbm.booster.train(handle.booster, num_iterations, 5)
#Predict
pred < - lgbm.booster.predict(handle.booster, x.test)
#Test accuracy
sum(y.test == (y.pred > 0.5)) / length(y.test)
#Save model (can be loaded again via lgbm.booster.load(fi lename)) lgbm.booster.save(handle.booster, filename =
"/tmp/model.
txt")

If you’re familiar with the Caret package in R, this is another way of implementing the LightGBM.
require(caret) require(RLightGBM) data(iris)
model < -caretModel.LGBM()
fit < - train(Species ~ ., data = iris, method=model, verb osity = 0) print(fit)
y.pred < - predict(fit, iris[,1:4])
library(Matrix)
model.sparse < - caretModel.LGBM.sparse()
#Generate a sparse mat rix
mat < - Matrix(as.matrix(iris[,1:4]), sparse = T) fit < - train(data.frame(idx = 1:nrow(iris)), iris$Specie s, method =
model.sparse, matrix = mat, verbosity = 0) print(fit)
10.4. Catboost
CatBoost is a recently open -sourced machine learning algorithm from Yandex. It can easily integrate with deep learning frameworks like Google’s TensorFlo w
and Apple’s Core ML.

The best part about CatBoost is that it does not require extensive data training like other ML models, and can work on a variety of data formats; not
undermining how robust it can be.
Make sure you handle missing data well before you proceed with the implementation.
Catboost can automatically deal with categorical variables without sh owing the type conversion error, which helps you to focus on tuning your model better
rather than sorting out trivial errors.
Learn more about Catboost from this article:
https://www.analyticsvidhya.com/blog/2017/08/catboost -automatedcategorical -data/ Python Code:
import pandas as pd import numpy as np
from catboost import CatBoostRegressor
#Read training and testing files train = pd.read_csv("train.csv") test = pd.read_csv("test.csv")

#Imputing missing values for both train and test train.fillna( -999, inplace=True) test.fillna( -999,inplace=True)
#Creating a training set for modeling and validation set to check model performance
X = train.drop(['Item_Outlet_Sales'], axis=1) y = train.I tem_Outlet_Sales
from sklearn.model_selection import train_test_split
X_train, X_validation, y_train, y_validation = train_test
_split(X, y, train_size=0.7, random_state=1234) categorical_features_indices = np.where(X.dtypes != np.fl oat)[0]
#importing library and building model
from catboost import CatBoostRegressormodel=CatBoostRegre ssor(iterations=50, depth=3, learning_rate=0.1, loss_func
tion='RMSE')
model.fit(X_train, y_train,cat_features=categorical_featu res_indices,eval_set=(X_validation, y_validation),plot=Tr ue)

submission = pd.DataFrame()
submission['Item_Identifier'] = test['Item_Identifier'] submission['Outlet_Identifier'] = test['Outlet_Id entifie r']
submission['Item_Outlet_Sales'] = model.predict(test)
R Code:
set.seed(1)
require(titanic)
require(caret)
require(catboost)
tt < - titanic::titanic_train[complete.cases(titanic::tita nic_train),]
data < - as.data.frame(as.matrix(tt), stringsAsFactors = T RUE)
drop_columns = c("PassengerId", "Survived", "Name", "Tick et", "Cabin")
x < - data[,!(names(data) %in% drop_columns)]y < - data[,c
("Survived")]
fit_control < - trainControl(method = "cv", number = 4,cla ssProbs = TRUE)

grid < - expand.grid(depth = c(4, 6, 8),learning_rate = 0. 1,iterations = 100, l2_leaf_reg = 1e -3, rsm =
0.95, border_count = 64)
report < - train(x, as.factor(make.names(y)),method = catb oost.caret,verbose = TRUE, preProc = NULL,tuneGrid = gri d,
trControl = fit_control)
print(report)
importance < - varImp(report, scale = FALSE)
print(importance)

Сообщить о нарушении / Abuse

Все документы на сайте взяты из открытых источников, которые размещаются пользователями. Приносим свои глубочайшие извинения, если Ваш документ был опубликован без Вашего на то согласия.