Linear summary (straight line)
Standard regression with all examples below will mislead you, because you assume,when using regression, that it is a good idea to use a (single) line as summary for allpoints at the same time.
See Also
Least squares regression: Definition, Calculation and example - VoxcoAdvantages and Disadvantages of Linear RegressionFive Obstacles faced in Linear RegressionLinear Regression is inaccurate and misleading! - Super Heuristics 
Two perfect, but non-linear relationships
- There are three distinct groups!
- A single line summary is stupid, but two lines would be perfect... but you have to see it!
- A line is misleading, because it does not tell you that the spread around is very small on the left, but big at right (heteroscedastis distribution)
- Three vertical "clouds"...could X be a categorical variable?
A perfect linear relationship, without the outlier. Note how the (least squares) line is attracted by the single outlier (influence point). Note Least squares regression is not resistant to the presence of outliers.
A (single) leverage point completely determines the line, although there is no variation at all in X, with one exception. Note that a single point (right) picture can change the direction!
