3.1 Intro to Outliers
Outliers
What is an outlier?
An outlier is an observation that is extreme or distant from the overall pattern in a sample. However, “distant” is intentionally a subjective term. Many times what is considered distant is left to the discretion of the analyst or data scientist, who must determine if the observation should be included or not. In order to make wise decisions about which data points to include and exclude, the observer must take into account the main objective at hand, make an extra effort to understand the data, and know its limitations.
Why look for outliers?
In machine learning, algorithms are designed to learn from the data. Outliers, which are not entirely representative of the population, can bias the results and have a disproportionate influence on models. This can lead analysts to make incorrect judgments or decisions. For this reason, it is important to check for outliers before creating models based on the data.
The influence of outliers
To illustrate how outliers can influence descriptive statistics from a sample, let’s begin by observing the effect an outlier can have on simple statistics. Take, for example, five numbers 22, 24, 21, 26, and 22. These numbers have a mean of 23, a median of 22, and a standard deviation of 2. Now if we add an outlier with a number of 299, some of our statistics suddenly become highly influenced. With 299 included, our new statistics show the mean is 69, the median is 23, and the standard deviation is 112.69. From our observations, we can see that the mean and the standard deviation are highly influenced by the outlier. The median, however, manages to be more resistant to the influence of outliers. Drastic distances between the mean and the median could be a good indicator that an outlier is influencing your data.
Outliers can distort fully implemented models as well. In the BeesAndPollen dataset, we can see the affect that an outlier has on a simple linear regression model and its line of fit, which will be explained in more detail in Chapter 6. The BeesAndPollen dataset contains how much pollen was removed from queen and worker bees based upon the duration of their visit. We can still see in Figure 3.2 that the red line of fit for pollen collected by queen bees does not seem entirely representative of the sample data. This is due to the two outliers that appear distant to the right of the cluster of data.
However, after removing these data points we can see in Figure 3.3 the green line of fit has adjusted dramatically and now appears to be more representative of the majority of the sample population. Notably, two outliers are all it took to influence and change the results of our model.
Anomaly Detection
Sometimes the entire purpose of data modeling is to find and discover outliers. This is also known as anomaly detection. Anomaly detection involves identifying items, events, or observations that do not conform to a specific pattern in the data. This also involves finding unique spikes and changes in the data over time.
Detecting unauthorized expenditures or fraud within a company is an example of anomaly detection. Amongst the many transactions and noise, the analyst’s objective is to discover who may be committing fraud. Different models that we will discuss later can be used to identify potential outliers in anomaly detection.
The following are some common applications of anomaly detection:
Cybersecurity and network intrusion
Identifying spam
Airport security screening
Credit card fraud
Falsified currency
Stock market
Common Causes of Outliers
Outliers can come about in many ways. Below is a list of common causes for outliers in data:
Data entry errors (human errors)
Measurement errors (instrument errors)
Experimental errors (data extraction or experiment planning/execution errors)
Intentional (dummy outliers made to test detecting methods)
Data processing errors (data manipulation or dataset unintended mutilations)
Sampling errors (extracting nonrepresentative data or mixing data from wrong or various sources)
Naturally occurring outliers (is not an error)
Outlier Detection Methods in This Chapter
This chapter will discuss some of the most common methods for identifying outliers:

Univariate Methods
Logical Detection
ZScore/Standard Deviation Approach
Tukey’s Boxplots

Multivariate Methods
Scatter Plots
Density Plots
Mahalanobis Distances
Jackknife Distances
KNearest Neighbor