<![CDATA[Easy Guides]]> https://www.sthda.com (C) 2005-2026 PHPBoost en PHPBoost <![CDATA[Regression Analysis Essentials For Machine Learning]]> https://www.sthda.com/english/wiki/regression-analysis-essentials-for-machine-learning https://www.sthda.com/english/wiki/regression-analysis-essentials-for-machine-learning

Regression analysis consists of a set of machine learning methods that allow us to predict a continuous outcome variable (y) based on the value of one or multiple predictor variables (x).

Briefly, the goal of regression model is to build a mathematical equation that defines y as a function of the x variables. Next, this equation can be used to predict the outcome (y) on the basis of new values of the predictor variables (x).

Linear regression is the most simple and popular technique for predicting a continuous variable. It assumes a linear relationship between the outcome and the predictor variables.

The linear regression equation can be written as y = b0 + b*x + e, where:

  • b0 is the intercept,
  • b is the regression weight or coefficient associated with the predictor variable x.
  • e is the residual error

Technically, the linear regression coefficients are detetermined so that the error in predicting the outcome value is minimized. This method of computing the beta coefficients is called the Ordinary Least Squares method.

When you have multiple predictor variables, say x1 and x2, the regression equation can be written as y = b0 + b1*x1 + b2*x2 +e. In some situations, there might be an interaction effect between some predictors, that is for example, increasing the value of a predictor variable x1 may increase the effectiveness of the predictor x2 in explaining the variation in the outcome variable.

Note also that, linear regression models can incorporate both continuous and categorical predictor variables.

When you build the linear regression model, you need to diagnostic whether linear model is suitable for your data.

In some cases, the relationship between the outcome and the predictor variables is not linear. In these situations, you need to build a non-linear regression, such as polynomial and spline regression.

When you have multiple predictors in the regression model, you might want to select the best combination of predictor variables to build an optimal predictive model. This process called model selection, consists of comparing multiple models containing different sets of predictors in order to select the best performing model that minimize the prediction error. Linear model selection approaches include best subsets regression and stepwise regression

In some situations, such as in genomic fields, you might have a large multivariate data set containing some correlated predictors. In this case, the information, in the original data set, can be summarized into few new variables (called principal components) that are a linear combination of the original variables. This few principal components can be used to build a linear model, which might be more performant for your data. This approach is know as principal component-based methods, which include: principal component regression and partial least squares regression.

An alternative method to simplify a large multivariate model is to use penalized regression, which penalizes the model for having too many variables. The most well known penalized regression include ridge regression and the lasso regression.

You can apply all these different regression models on your data, compare the models and finally select the best approach that explains well your data. To do so, you need some statistical metrics to compare the performance of the different models in explaining your data and in predicting the outcome of new test data.

The best model is defined as the model that has the lowest prediction error. The most popular metrics for comparing regression models, include:

  • Root Mean Squared Error, which measures the model prediction error. It corresponds to the average difference between the observed known values of the outcome and the predicted value by the model. RMSE is computed as RMSE = mean((observeds - predicteds)^2) %>% sqrt(). The lower the RMSE, the better the model.
  • Adjusted R-square, representing the proportion of variation (i.e., information), in your data, explained by the model. This corresponds to the overall quality of the model. The higher the adjusted R2, the better the model

Note that, the above mentioned metrics should be computed on a new test data that has not been used to train (i.e. build) the model. If you have a large data set, with many records, you can randomly split the data into training set (80% for building the predictive model) and test set or validation set (20% for evaluating the model performance).

One of the most robust and popular approach for estimating a model performance is k-fold cross-validation. It can be applied even on a small data set. k-fold cross-validation works as follow:

  1. Randomly split the data set into k-subsets (or k-fold) (for example 5 subsets)
  2. Reserve one subset and train the model on all other subsets
  3. Test the model on the reserved subset and record the prediction error
  4. Repeat this process until each of the k subsets has served as the test set.
  5. Compute the average of the k recorded errors. This is called the cross-validation error serving as the performance metric for the model.

Taken together, the best model is the model that has the lowest cross-validation error, RMSE.

In this Part, you will learn different methods for regression analysis and we’ll provide practical example in R.

The content is organized as follow:

  1. Regression Analysis
  2. Regression Model Diagnostics
  3. Regression Model Validation
  4. Model Selection Essentials in R
]]> Wed, 21 Mar 2018 07:35:58 +0100 <![CDATA[ggpubr: Create Easily Publication Ready Plots]]> https://www.sthda.com/english/wiki/ggpubr-create-easily-publication-ready-plots https://www.sthda.com/english/wiki/ggpubr-create-easily-publication-ready-plots

The ggpubr R package facilitates the creation of beautiful ggplot2-based graphs for researcher with non-advanced programming backgrounds.

The current material presents a collection of articles for simply creating and customizing publication-ready plots using ggpubr. To see some examples of plots created with ggpubr click the following link: ggpubr examples.

ggpubr Key features:

  • Wrapper around the ggplot2 package with a less opaque syntax for beginners in R programming.
  • Helps researchers, with non-advanced R programming skills, to create easily publication-ready plots.
  • Makes it possible to automatically add p-values and significance levels to box plots, bar plots, line plots, and more.
  • Makes it easy to arrange and annotate multiple plots on the same page.
  • Makes it easy to change grahical parameters such as colors and labels.

Official online documentation: https://www.sthda.com/english/rpkgs/ggpubr.

ggpubr: publication ready plots

Install and load ggpubr

  • Install from CRAN as follow:
install.packages("ggpubr")
  • Or, install the latest version from GitHub as follow:
# Install
if(!require(devtools)) install.packages("devtools")
devtools::install_github("kassambara/ggpubr")
  • Load ggpubr:
library("ggpubr")
]]> Thu, 14 Sep 2017 08:38:34 +0200 <![CDATA[The Ultimate Guide To Partitioning Clustering]]> https://www.sthda.com/english/wiki/the-ultimate-guide-to-partitioning-clustering https://www.sthda.com/english/wiki/the-ultimate-guide-to-partitioning-clustering

In this first volume of symplyR, we are excited to share our Practical Guides to Partioning Clustering.

Partitioning clustering methods

The course materials contain 3 chapters organized as follow:

K-Means Clustering Essentials

Contents:

  • K-means basic ideas
  • K-means algorithm
  • Computing k-means clustering in R
    • Data
    • Required R packages and functions: stats::kmeans()
    • Estimating the optimal number of clusters: factoextra::fviz_nbclust()
    • Computing k-means clustering
    • Accessing to the results of kmeans() function
    • Visualizing k-means clusters: factoextra::fviz_cluster()
  • K-means clustering advantages and disadvantages
  • Alternative to k-means clustering

K-Medoids Essentials: PAM clustering

Contents:

  • PAM concept
  • PAM algorithm
  • Computing PAM in R
    • Data
    • Required R packages and functions: cluster::pam() or fpc::pamk()
    • Estimating the optimal number of clusters: factoextra::fviz_nbclust()
    • Computing PAM clustering
    • Accessing to the results of the pam() function
    • Visualizing PAM clusters: factoextra::fviz_cluster()

CLARA - Clustering Large Applications

Contents:

  • CLARA concept
  • CLARA Algorithm
  • Computing CLARA in R
    • Data format and preparation
    • Required R packages and functions: cluster::clara()
    • Estimating the optimal number of clusters: factoextra::fviz_nbclust()
    • Computing CLARA
    • Visualizing CLARA clusters: factoextra::fviz_cluster()

Example of plots:

K means clustering plots


Licence: Licence Creative Commons

]]> Wed, 06 Sep 2017 13:47:55 +0200 <![CDATA[Practical Guide to Principal Component Methods in R]]> https://www.sthda.com/english/wiki/practical-guide-to-principal-component-methods-in-r https://www.sthda.com/english/wiki/practical-guide-to-principal-component-methods-in-r

Introduction

Although there are several good books on principal component methods (PCMs) and related topics, we felt that many of them are either too theoretical or too advanced.

This book provides a solid practical guidance to summarize, visualize and interpret the most important information in a large multivariate data sets, using principal component methods in R.

Where to find the book:



The following figure illustrates the type of analysis to be performed depending on the type of variables contained in the data set.

Principal component methods

There are a number of R packages implementing principal component methods. These packages include: FactoMineR, ade4, stats, ca, MASS and ExPosition.

However, the result is presented differently depending on the used package.

To help in the interpretation and in the visualization of multivariate analysis - such as cluster analysis and principal component methods - we developed an easy-to-use R package named factoextra (official online documentation: https://www.sthda.com/english/rpkgs/factoextra).

No matter which package you decide to use for computing principal component methods, the factoextra R package can help to extract easily, in a human readable data format, the analysis results from the different packages mentioned above. factoextra provides also convenient solutions to create ggplot2-based beautiful graphs.

Methods, which outputs can be visualized using the factoextra package are shown in the figure below:

Principal component methods and clustering methods supported by the factoextra R package

In this book, we’ll use mainly:

  • the FactoMineR package to compute principal component methods;
  • and the factoextra package for extracting, visualizing and interpreting the results.

The other packages - ade4, ExPosition, etc - will be also presented briefly.

How this book is organized

This book contains 4 parts.

Principal Component Methods book structure

Part I provides a quick introduction to R and presents the key features of FactoMineR and factoextra.

Key features of FactoMineR and factoextra for multivariate analysis

Part II describes classical principal component methods to analyze data sets containing, predominantly, either continuous or categorical variables. These methods include:

  • Principal Component Analysis (PCA, for continuous variables),
  • Simple correspondence analysis (CA, for large contingency tables formed by two categorical variables)
  • Multiple correspondence analysis (MCA, for a data set with more than 2 categorical variables).

In Part III, you’ll learn advanced methods for analyzing a data set containing a mix of variables (continuous and categorical) structured or not into groups:

  • Factor Analysis of Mixed Data (FAMD) and,
  • Multiple Factor Analysis (MFA).

Part IV covers hierarchical clustering on principal components (HCPC), which is useful for performing clustering with a data set containing only categorical variables or with a mixed data of categorical and continuous variables

Key features of this book

This book presents the basic principles of the different methods and provide many examples in R. This book offers solid guidance in data mining for students and researchers.

Key features:

  • Covers principal component methods and implementation in R
  • Highlights the most important information in your data set using ggplot2-based elegant visualization
  • Short, self-contained chapters with tested examples that allow for flexibility in designing a course and for easy reference

At the end of each chapter, we present R lab sections in which we systematically work through applications of the various methods discussed in that chapter. Additionally, we provide links to other resources and to our hand-curated list of videos on principal component methods for further learning.

Examples of plots

Some examples of plots generated in this book are shown hereafter. You’ll learn how to create, customize and interpret these plots.

  1. Eigenvalues/variances of principal components. Proportion of information retained by each principal component.

  1. PCA - Graph of variables:
  • Control variable colors using their contributions to the principal components.

  • Highlight the most contributing variables to each principal dimension:

  1. PCA - Graph of individuals:
  • Control automatically the color of individuals using the cos2 (the quality of the individuals on the factor map)

  • Change the point size according to the cos2 of the corresponding individuals:

  1. PCA - Biplot of individuals and variables

  1. Correspondence analysis. Association between categorical variables.

  1. FAMD/MFA - Analyzing mixed and structured data

  1. Clustering on principal components

Book preview

Download the preview of the book at: Principal Component Methods in R (Book preview)

Order now



About the author

Alboukadel Kassambara is a PhD in Bioinformatics and Cancer Biology. He works since many years on genomic data analysis and visualization (read more: http://www.alboukadel.com/).

He has work experiences in statistical and computational methods to identify prognostic and predictive biomarker signatures through integrative analysis of large-scale genomic and clinical data sets.

He created a bioinformatics web-tool named GenomicScape (www.genomicscape.com) which is an easy-to-use web tool for gene expression data analysis and visualization.

He developed also a training website on data science, named STHDA (Statistical Tools for High-throughput Data Analysis, www.sthda.com/english), which contains many tutorials on data analysis and visualization using R software and packages.

He is the author of many popular R packages for:

Recently, he published three books on data analysis and visualization:

  1. Practical Guide to Cluster Analysis in R (https://goo.gl/DmJ5y5)
  2. Guide to Create Beautiful Graphics in R (https://goo.gl/vJ0OYb).
  3. Complete Guide to 3D Plots in R (https://goo.gl/v5gwl0).
]]> Thu, 24 Aug 2017 14:04:17 +0200 <![CDATA[simplyR]]> https://www.sthda.com/english/wiki/simplyr https://www.sthda.com/english/wiki/simplyr

simplyR is a web space where we’ll be posting practical and easy guides for solving real important problems using R programming language.

As we aren’t fans of unnecessary complications, we’ll keep the content of our tutorials / R codes as simple as possible.

Many tutorials are coming soon.

Topics we love include:

  • R programming
  • Biostatistics
  • Genomic data analysis
  • Survival analysis
  • Machine/statistical learning
  • Data visualization

Samples of our recent publications, on R & Data Science, are:

If you want to contribute, read this: https://www.sthda.com/english/pages/contribute-to-sthda

]]> Sat, 19 Aug 2017 23:56:23 +0200