Section 1
Preview this deck
dir('dir')
Front
| 5 | 0 | ||
| 4 | 0 | ||
| 3 | 0 | ||
| 2 | 0 | ||
| 1 | 0 |
Active users
0
All-time users
0
Favorites
0
Last updated
6 years ago
Date created
Mar 1, 2020
Cards (213)
Section 1
(50 cards)
dir('dir')
Lists the files in the directory specified
All items in the array except the third.
A[-3]
seq(0, 100, by=1)
Creates a vector of numerics from 0 to 100 that increment by 1.
Simple Variable Assignment
a = 1
Element wise arithmetic
Applying a function to every element of an array
outer(y[1,], y[1,])
The outer product of two vectors (results in a matrix)
y$name
the name variable in the list y
typeof(a)
Returns the type of variable a
x[-1,]
all rows but the first
Factor variables
Represent values from an ordered or unordered finite set
seq(0, 1, length.out = 11)
11 evenly spaced numbers between 0 and 1
Multiple Variable Assignment on a Single Line
a = 1; b = 2; c = 3
Vector
A one dimensional ordered variables of the same type.
Show example of a function X
example(X)
which(w<0.5)
returns the indices of w where their value is less than 0.5, otherwise 0
all(w < 0.5)
returns TRUE if all values of w are less than 0.5, otherwise FALSE
y = x[c(1,2),c(1,2)]
extracts the top left 2x2 of an array
as.integer(b)
Casts variable b to an integer
Start the web based help page
help.start()
any(w < 0.5)
returns TRUE if any values of w are less than 0.5, otherwise FALSE
x[2,]
accesses the entire 2nd row
Remove all variables from memory
rm(list=ls())
x = array(data = z, dim = c(4, 5))
creates a two dimensional array with 4 rows and 5 columns
Matrix transpose
A rotation of the matrix around the diagonal
Search the help pages for a specific term X
help.search('X')
Concatentation Function
c()
cbind(x[1,], x[1,])
Horizontal concatenation of vectors.
rep(3.2, times=10) function
Repeats the value 3.2 10 times.
ls.str()
Shows variables in memory and their types
y %*% y
Matrix or Inner Product of y and y
dir()
Lists the files in the current directory
List Signature
An ordered list of variable types in the list.
system('ls')
Runs the ls command on the system
x[2,3]
access the 2nd row, 3rd column of a 2d array
w[w>50] = 0
set all values of w that are greater than 50 to 0
length(A)
Returns the length of array A
Array
A multi-dimensional generalization of vectors.
is.array(w)
returns TRUE if w is an array, otherwise FALSE
Length of array A
length(A)
Set working directory to '/'
setwd('/')
List
An ordered collection of variables of possibly different types.
x <- matrix(c(1,2,3,4), nrow=2, ncol=2)
creates a matrix (or array) that is 2x2
rbind(x[1,], x[1,])
Vertical concatenation of vectors.
Variable assignment with pointer syntax
a <- 1
Remove a specific variable from memory
rm(a)
y = list(name="Mike", title="badass")
creates a list with two attributes
Dataframe
An ordered collection of lists having the same list signature.
List all variables in memory
ls()
Vector function example
y = vector(mode="logical", length=4)
Declaring a Vector
A = c(1, 2, 3)
Section 2
(50 cards)
while (b > a) { }
Repeats code block until b !> a
Variable passing
R passes variables to functions by value.
L[[1]]
Returns the first variable from the List L. If the first variable were a character, the returned type would be a character.
log(10)
natural log of 10
for(num in seq(1, 100, by=1) { print("hello world") }
A for loop that prints "hello world" 100 times.
n = c('Mike', 'Mark') a = c(30, 40) s = c(1000,2000) R = data.frame(name=n, age=a, salary=s) names(R) = c('Name', 'Age', 'Salary') R
Creates a DataFrame, essentially a table. Name Age Salary 1 mike 30 1000 2 mark 40 2000
Vectorized code
Code, like element wise operations in arrays, that avoids computations in loops in the interpreter. Vectorized code is executed natively, like in C.
unname(L)
Strips out the names of attributes in the list or dataframe.
iris$Sepal.Length[1:10]
The first 10 Sepal Length values. A vector.
save.image(file='fname')
Saves an image of the current working memory (i.e. all variables) to a file named fname.
L$name
Prints the name variable in the list L.
a = 1:10
Creates a vector of 10 integers, 1 through 10.
L['name']
Prints the property 'name' from the list L.
repeat { }
Repeats code block until break is called.
exp(0)
e raised to the power of 0, i.e. 1
a = c(1,2) b=c(1,2,3,4) c = a+b c
2 4 4 6
Iris = read.table('someFile.txt', useHeader=TRUE)
Reads someFile.txt into Iris as a dataframe and expects a header to be use as the column names.
mypower = function(bas = 10, pow = 2) { return(bas ^ pow) }
Defines a function, mypower, that takes 2 arguments, bas and pow and returns bas ^ pow. bas has a default value of 10, pow has a default value of 2.
mean(Sepal.Length)
The average of Sepal Lengths.
iris = edit(iris)
Edits the iris dataset in a spreadsheet and sets the resulting dataset to the iris variable.
dim(subset(iris, Sepal.Length < 5 & Species != 'setosa'))[1]
Returns the number of rows of the resulting subset. 2
colMeans(iris[,1:4])
The means of the first 4 columns of iris.
pi
A global variable representing pi.
b = c(1,2,3) b[5] = 5 b
1 2 3 NA 5
cat()
Prints arguments one after the other. e.g. cat("Hello", "World")
edit(iris)
Opens the dataset in a spreadsheet editor.
load('fname')
Loads working data into memory from filename fname.
iris$Sepal.Length
All Sepal Length values in the iris dataset. A vector.
subset(iris, Sepal.Length < 5 & Species != 'setosa')
Returns a subset of the iris dataframe which includes rows where the Sepal Length is less than 5 and the Species is not setosa. Sepal.Length Sepal.Width Petal.Length Petal.Width Species 58 4.9 2.4 3.3 1.0 versicolor 107 4.9 2.5 4.5 1.7 virginica
names(L)=c('a','b','c','d')
Overrides the variable names in the list L with a, b, c, d.
sink('outFile', split=TRUE)
Sends console output to the 'outFile' file AND to the console.
is.vector(a)
Returns TRUE if a is a vector, FALSE otherwise.
L = list(name='mike', age=100, no.children=2, children.ages=c(60,50))
Creates a new list with a number of properties.
iris dataset
A core package dataset that includes flower measurements of different flower species.
foo(1,2,3)
Calls function foo, passing in argument 1, 2, 3. If argument(s) are omitted, the default value for the variable is used.
foo(name='mike', age=36)
Calls function food, passing in arguments name and age. Note: order does not matter when variables are named.
exp(1)
e raised to the power of 1, i.e. 2.718282
sink('outFile')
Sends console output to the 'outFile' file, not to the console.
log10(100)
log base 10 of 100, i.e. 2
L[1]
Returns a List with the first variable from the List L in it. The returned type is a List.
summary(iris)
A useful statistical summary of the dataset iris.
dim(iris)
Returns the number of rows and columns of the iris dataset. 150 5
tail(iris, 4)
the last 4 rows of the iris dataset
history(10)
Displays the 10 most recent commands execute.
attach(iris, warn.conflicts=FALSE)
Attaches the iris dataset to the local namespace. This allows us to call attributes from it directly. i.e. we can just say Sepal.Length instead of iris$Sepal.Length.
names(L)
Prints the keys (attribute values from the list L).
system.time(function)
Displays the time it took to execute function.
head(iris, 4)
the first 4 rows of the iris dataset
R CMD SHLIB foo.c
Compiles foo.c into a foo.so object file that can be called by R, .C or .CALL functions. e.g. dynload('foo.so'); .C('foo',...)
sapply(data, function)
Applies the supplied function to the data set provided.
Section 3
(50 cards)
lines
A low-level function in the graphics package that adds a line plot to an existing graph.
low-level functions
In the graphics package, functions that edit graphs.
S = sort.int(mpg$cty, index.return=T)
Returns a sorted list of values from the mpg$cty dimension. The list contains two lists. The first is the sorted list of values. The second is the index of each value from the original data set.
title(main="Some title")
Adds a title to the current plot.
hist(data_frame$x, xlab="x label", ylab="y label", main="title", breaks=20)
Uses the graphics package to generate a histogram of the x dimension of data_frame with 20 breaks or 21 bins.
xlab
Defines the x label attribute in the plot function.
ylab
Defines the y label attribute in the plot function.
ggplot(data_frame, aes(x=x, y=..density..)) + geom_histogram(binwidth=4)
Creates a plot of the x attribute of data_frame and shows the probability distribution of the bin values on the y axis and displays as a histogram of binwidth 4.
curve(sinc, -7, 7)
Creates a line plot using the graphics package that applies values from -7 to 7 to the function sinc.
main
Defines the title string in the plot function.
title
A low-level function in the graphics package that modifies the title of a graph.
microbenchmark(function)
Displays the execution time of function in microseconds.
graphics package
The default visualization package in R. Harder to use than ggplot2 but may run faster.
legend
A low-level function in the graphics package that connects symbols, colors, and line types to descriptive strings.
breaks
In a histogram (hist) plot, the number of breaks between bins. The number of bins equals breaks + 1.
datasets package
A package of datasets that come installed by default in R.
ggplot(data_frame, aes(x=x, y=y)) + geom_point()
Creates a scatter plot with ggplot and adds point geometry to it.
.C("fooC", A, B, C, D)
Calls the fooC c program passing input A, B, C, and result D.
stopifnot(boolean)
Similar to an assert statement. Stops the program if the boolean is not TRUE.)
.Rprofile
Placed in the user's home directory, can be used to define .First and .Last functions.
lines(mpg$hwy[S$ix], lty=1)
Adds a line representing highway mileage to an existing plot using line type 1.
bin width
Good bin width balances information loss with good aggregation.
qplot
A function from the ggplot2 package that produces a scatter plot by default.
library(ggplot2)
Loads the ggplot2 library into memory.
mtcars
A dataset of car model data from the Motor Trend Magazine in 1974. Part of the datasets package.
options(expressions=500000)
Sets the maximum number of nested recursive calls to 500,000.
#pragma omp parallel for
A directive placed before a for loop in C that mult-threads the for loop. From OpenMP extension.
ggplot
A function from the ggplot2 package that returns a graphics object that may be modified by adding layers to it. Provides automatic axes labeling and legends.
high-level functions
In the graphics package, functions that produce graphs, i.e. plot, hist, or curve.
ggplot2
A visualization package that may be simpler to use than graphics. It may also run slower, however. It is based on the Grammar of Graphics by Wilkinson (2005).
diamonds
A dataset from the ggplot2 package that lists the details of 50,000 round cut diamonds.
install.packages('ggplot2') library(ggplot2)
Install the ggplot2 package and bring into scope.
Histograms vs Strip Plots
Histograms discard the ordering of the data.
A function that will print a ggplot graph.
source('h1.R')
Loads the R code in h1.R into the interpreter and runs it.
ggplot(data_frame, aes(x=x)) + geom_histogram(binwidth=1)
Creates a plot of the x attribute of data_frame and displays it as a histogram.
qplot(x = x, data=data_frame, binwidth=3, main="title")
Uses the ggplot2 package to generate a histogram from dimension x of data_frame.
Strip Plot
A plot that maps the ordered index of a data frame against a single attribute. The strip plot can expose strips or lines of similar data.
mpg
A dataset of car model data from fueleconomy.gov. Part of the ggplot2 package.
plot(S$x, type="l", lty=2, xlab="x label", ylab="y label")
Creates a line plot of the S$x dimension with line type of 2.
qplot(x=x, y=y, data=data_frame, main="title feature", geom="point")
Creates a basic scatter plot with ggplot's qplot function.
legend("topleft", c("highway mpg", "city mpg"), lty=c(1,1))
Creates a legend in the top left corner of an existing plot with the labels and line types given.
.First
Function in .Rprofile that gets executed when R starts.
dyn.load("fooC.so")
Loads the fooC.so shared object file into memory.
.Last
Function in .Rprofile that executes when R stops.
plot(x=data_frame$x, y=data_frame$y)
Using the high-level plot function from the graphics package that produces a simple scatter plot.
aes
A function that accepts data variables as arguments and is passed to ggplot.
Histogram
A one dimensional plot that groups data into bins and shows counts of those bins.
grid
A low-level function in the graphics package that adds grid lines to a graph.
faithful
A dataset of eruption times of Old Faithful in Yellowstone National Park, Wyoming, USA. Part of the datasets package.
Section 4
(50 cards)
numeric variables
variables that are real valued. difference between two numeric variables is expected to be the euclidean distance. abs(b-a)
Outlier
A data item is an outlier if it s below the alpha percentile or above the 100-alpha percentile.
na.omit(dataframe)
Returns a new dataframe that omits all rows with missing data.
IQR
Inter quartile range
Indicator Variables
Breaking out categories into separate binary variables. For example, if we had 6 height categories (A-F), we would create 5 height category binary indicators (all 0 would mean category was A). Category variable would be set to 1 for appropriate category.
Outliers
Two sources: 1.) errors in entering data (like with a human typing data in) 2.) non corrupt data but highly unlikely
Replacing missing data for MAR
Methods may introduce systematic bias into the data.
qplot(x, y, geom=c("line"))
Create a line chart with functions x and y.
geom_point()
Adds points to a ggplot plot.
MCAR
Missing Completely At Random Missingness does not depend on observed or unobserved variables.
ggsave("filename.pdf")
Saves the current graph as a pdf.
geom_line()
Adds a line to a ggplot plot.
complete.cases(dataframe)
Returns a vector whose components are FALSE for all data rows missing data and TRUE for all data rows with no missing data.
ordinal variables
variables representing measurements in a certain range R with a well defined order relation. e.g. the seasons.
kernel function
smooths a distribution
Dealing with Outliers
1.) Truncate - drop them 2.) Winsorization - reset outliers to the highest value remaining outside of the outliers 3.) Robustness - analyze data with a robust procedure
Problems with Data sets
1.) Missing Data 2.) Outliers 3.) Highly skewed data
log-log Relationship
log log plot can reveal linear relationships between variables that are otherwise hard to see. e.g. qplot(brain, body, log = "xy", data = Animals)
Sampling
Selecting a random number of rows.
sample function
sample(data, number to sample, replace=TRUE/FALSE)
Raster graphics
Lower resolution but smaller file sizes.
data shuffling
randomly mixing up data frame rows
Sampling without Replacement
Rows can only be sampled once.
Vector graphics
Higher resolution but larger file sizes.
ggplot(data_frame, aes(x=x, y=y)) + geom_line() + geom_point()
Creates a line plot with points from the x and y dimensions of data_frame.
pch, col, cex
Shape, Color, Size of points in a scatter plot
Replacing missing data for MCAR
Any of the 3 missing data techniques are OK.
na.rm
An option passed to functions that causes the function to operate only on rows without missing data.
Sampling with Replacement
When you sample, you can sample the same rows multiple times.
data partitioning
splitting data into two sets, like 75% in one and 25% in another
qplot(waiting, eruptions, data=faithful)
Scatterplot of faithful waiting vs eruptions
Standard Deviation with Outliers
First remove the most extreme values, then calculate the standard deviation.
Faceting
Displaying multiple panels in the same graph.
qplot(x, y, geom=c("line", "point"))
Creates a line plot of x and y with points.
categorial variables
variables that do not satisfy the ordinal or numeric assumption. e.g. items on a restaurant menu.
Robustness to Outliers
Means a model is not sensitive to outliers. Mean is not robust. Median is robust.
Binarization
Replacing a value with 0 or 1 based on threshold.
Box plot whiskers
Extend no more than 1.5 times the IQR away from the edges of the box
is.na(dataframe)
Returns TRUE where dataframe value is NA and FALSE otherwise.
Scatter Plot
Plots two variables against each other as points.
Smoothed Histogram
f_h : R -> R_+ f_h(x) = (1/n) * sum_i_to_n( K(x-x_of_i) )
qqplot
quantile-quantile plots
Lambda in Power Transformation
Lambda < 1 removes right skewness. Lambda > 1 removes left skewness. Smaller values of lambda are more aggressive in removing skewness.
qplot(x, y, geom="line")
Creates a line plot of x and y.
MAR
Missing At Random Missingness depends on observed variables only.
Ways to Deal with Missing Data
1.) Remove data with missing values. 2.) Replace missing values with a substitute (e.g. mean). 3.) Estimate a probability model and replace with values from that model.
Binning (or discretization)
Taking a numeric variable (real number), dividing its range into several bins, and replacing it with a number representing the corresponding bin.
Power Transformation
A data transformation for dealing with skewed data.
dataframe concatenation
taking two dataframes with identical columns and combining them together
plot(faithful$waiting, faithful$eruption, xlab="waiting time (min)", ylab="waiting time (min)")
Creates a scatter plot with the Graphics library which puts waiting time on the x axis and eruption time on the y axis.
Section 5
(13 cards)