EDA of Air-Quality

Dataset statistics

Number of variables	5
Number of observations	153
Missing cells	44
Missing cells (%)	5.8%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	6.1 KiB
Average record size in memory	40.8 B

Variable types

DateTime	1
Numeric	4

Alerts

`Ozone` is highly correlated with `Wind` and 1 other fields	High correlation
`Wind` is highly correlated with `Ozone`	High correlation
`Temp` is highly correlated with `Ozone`	High correlation
`Ozone` is highly correlated with `Wind` and 1 other fields	High correlation
`Wind` is highly correlated with `Ozone`	High correlation
`Temp` is highly correlated with `Ozone`	High correlation
`Ozone` is highly correlated with `Temp`	High correlation
`Temp` is highly correlated with `Ozone`	High correlation
`Ozone` is highly correlated with `Wind` and 1 other fields	High correlation
`Solar` is highly correlated with `Temp`	High correlation
`Wind` is highly correlated with `Ozone` and 1 other fields	High correlation
`Temp` is highly correlated with `Ozone` and 2 other fields	High correlation
`Ozone` has 37 (24.2%) missing values	Missing
`Solar` has 7 (4.6%) missing values	Missing
`Date` has unique values	Unique

Reproduction

Analysis started	2022-07-08 18:02:06.452324
Analysis finished	2022-07-08 18:02:08.801309
Duration	2.35 seconds
Software version	pandas-profiling v3.1.1
Download configuration	config.json

Date
Date

UNIQUE

Distinct	153
Distinct (%)	100.0%
Missing	0
Missing (%)	0.0%
Memory size	1.3 KiB

Minimum	1976-05-01 00:00:00
Maximum	1976-09-30 00:00:00

Histogram

Histogram with fixed size bins (bins=50)

Ozone
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
MISSING

Distinct	67
Distinct (%)	57.8%
Missing	37
Missing (%)	24.2%
Infinite	0
Infinite (%)	0.0%
Mean	42.12931034

Minimum	1
Maximum	168
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	1.3 KiB

Quantile statistics

Minimum	1
5-th percentile	7.75
Q1	18
median	31.5
Q3	63.25
95-th percentile	108.5
Maximum	168
Range	167
Interquartile range (IQR)	45.25

Descriptive statistics

Standard deviation	32.98788451
Coefficient of variation (CV)	0.7830150611
Kurtosis	1.290302678
Mean	42.12931034
Median Absolute Deviation (MAD)	17.5
Skewness	1.241796404
Sum	4887
Variance	1088.200525
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
23	6	3.9%
21	4	2.6%
16	4	2.6%
20	4	2.6%
18	4	2.6%
13	4	2.6%
14	4	2.6%
32	3	2.0%
44	3	2.0%
11	3	2.0%
Other values (57)	77	50.3%
(Missing)	37	24.2%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
1	1	0.7%
4	1	0.7%
6	1	0.7%
7	3	2.0%
8	1	0.7%
9	3	2.0%
10	1	0.7%
11	3	2.0%
12	2	1.3%
13	4	2.6%

Value	Count	Frequency (%)
168	1	0.7%
135	1	0.7%
122	1	0.7%
118	1	0.7%
115	1	0.7%
110	1	0.7%
108	1	0.7%
97	2	1.3%
96	1	0.7%
91	1	0.7%

Solar
Real number (ℝ_≥0)

HIGH CORRELATION
MISSING

Distinct	117
Distinct (%)	80.1%
Missing	7
Missing (%)	4.6%
Infinite	0
Infinite (%)	0.0%
Mean	185.9315068

Minimum	7
Maximum	334
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	1.3 KiB

Quantile statistics

Minimum	7
5-th percentile	24.25
Q1	115.75
median	205
Q3	258.75
95-th percentile	311.5
Maximum	334
Range	327
Interquartile range (IQR)	143

Descriptive statistics

Standard deviation	90.05842223
Coefficient of variation (CV)	0.4843634291
Kurtosis	-0.9684667515
Mean	185.9315068
Median Absolute Deviation (MAD)	66.5
Skewness	-0.4280445256
Sum	27146
Variance	8110.519414
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
238	4	2.6%
259	4	2.6%
175	3	2.0%
223	3	2.0%
220	3	2.0%
250	2	1.3%
264	2	1.3%
127	2	1.3%
273	2	1.3%
291	2	1.3%
Other values (107)	119	77.8%
(Missing)	7	4.6%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
7	1	0.7%
8	1	0.7%
13	1	0.7%
14	1	0.7%
19	1	0.7%
20	1	0.7%
24	2	1.3%
25	1	0.7%
27	1	0.7%
31	1	0.7%

Value	Count	Frequency (%)
334	1	0.7%
332	1	0.7%
323	1	0.7%
322	2	1.3%
320	1	0.7%
314	1	0.7%
313	1	0.7%
307	1	0.7%
299	1	0.7%
295	1	0.7%

Wind
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	31
Distinct (%)	20.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	9.95751634

Minimum	1.7
Maximum	20.7
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	1.3 KiB

Quantile statistics

Minimum	1.7
5-th percentile	4.6
Q1	7.4
median	9.7
Q3	11.5
95-th percentile	15.5
Maximum	20.7
Range	19
Interquartile range (IQR)	4.1

Descriptive statistics

Standard deviation	3.523001352
Coefficient of variation (CV)	0.3538032208
Kurtosis	0.111418252
Mean	9.95751634
Median Absolute Deviation (MAD)	2.3
Skewness	0.3478177747
Sum	1523.5
Variance	12.41153853
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=31)

Value	Count	Frequency (%)
11.5	15	9.8%
10.3	11	7.2%
8	11	7.2%
9.7	11	7.2%
7.4	10	6.5%
6.9	9	5.9%
6.3	8	5.2%
9.2	8	5.2%
10.9	8	5.2%
8.6	8	5.2%
Other values (21)	54	35.3%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
1.7	1	0.7%
2.3	1	0.7%
2.8	1	0.7%
3.4	1	0.7%
4	1	0.7%
4.1	1	0.7%
4.6	4	2.6%
5.1	3	2.0%
5.7	3	2.0%
6.3	8	5.2%

Value	Count	Frequency (%)
20.7	1	0.7%
20.1	1	0.7%
18.4	1	0.7%
16.6	3	2.0%
16.1	1	0.7%
15.5	3	2.0%
14.9	8	5.2%
14.3	6	3.9%
13.8	5	3.3%
13.2	2	1.3%

Temp
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	40
Distinct (%)	26.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	77.88235294

Minimum	56
Maximum	97
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	1.3 KiB

Quantile statistics

Minimum	56
5-th percentile	60.2
Q1	72
median	79
Q3	85
95-th percentile	92
Maximum	97
Range	41
Interquartile range (IQR)	13

Descriptive statistics

Standard deviation	9.465269741
Coefficient of variation (CV)	0.1215329196
Kurtosis	-0.4035053804
Mean	77.88235294
Median Absolute Deviation (MAD)	6
Skewness	-0.3778844643
Sum	11916
Variance	89.59133127
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=40)

Value	Count	Frequency (%)
81	11	7.2%
82	9	5.9%
76	9	5.9%
86	7	4.6%
77	7	4.6%
78	6	3.9%
79	6	3.9%
85	5	3.3%
80	5	3.3%
92	5	3.3%
Other values (30)	83	54.2%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
56	1	0.7%
57	3	2.0%
58	2	1.3%
59	2	1.3%
61	3	2.0%
62	2	1.3%
63	1	0.7%
64	2	1.3%
65	2	1.3%
66	3	2.0%

Value	Count	Frequency (%)
97	1	0.7%
96	1	0.7%
94	2	1.3%
93	3	2.0%
92	5	3.3%
91	2	1.3%
90	3	2.0%
89	2	1.3%
88	3	2.0%
87	5	3.3%

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

The correlation heatmap measures nullity correlation: how strongly the presence or absence of one variable affects the presence of another.

The dendrogram allows you to more fully correlate variable completion, revealing trends deeper than the pairwise ones visible in the correlation heatmap.

First rows

	Date	Ozone	Solar	Wind	Temp
0	1976-05-01	41.0	190.0	7.4	67
1	1976-05-02	36.0	118.0	8.0	72
2	1976-05-03	12.0	149.0	12.6	74
3	1976-05-04	18.0	313.0	11.5	62
4	1976-05-05	NaN	NaN	14.3	56
5	1976-05-06	28.0	NaN	14.9	66
6	1976-05-07	23.0	299.0	8.6	65
7	1976-05-08	19.0	99.0	13.8	59
8	1976-05-09	8.0	19.0	20.1	61
9	1976-05-10	NaN	194.0	8.6	69

Last rows

	Date	Ozone	Solar	Wind	Temp
143	1976-09-21	13.0	238.0	12.6	64
144	1976-09-22	23.0	14.0	9.2	71
145	1976-09-23	36.0	139.0	10.3	81
146	1976-09-24	7.0	49.0	10.3	69
147	1976-09-25	14.0	20.0	16.6	63
148	1976-09-26	30.0	193.0	6.9	70
149	1976-09-27	NaN	145.0	13.2	77
150	1976-09-28	14.0	191.0	14.3	75
151	1976-09-29	18.0	131.0	8.0	76
152	1976-09-30	20.0	223.0	11.5	68

Overview

Variables

Interactions

Correlations

Spearman's ρ

Pearson's r

Kendall's τ

Phik (φk)

Missing values

Sample

First rows

Last rows