data trove noaa significant storms

This column contains geographic latitude values, spanning from -14.3236 to 70.1269 degrees, consistent with worldwide location data. The distribution is tightly clustered between Q1=33.63 and Q3=41.13 (IQR ~7.5), suggesting the bulk of records concentrate around mid-latitude Northern Hemisphere locations (roughly US/Europe range), with the mean (37.28) and median (37.12) nearly identical indicating only mild skew (-0.18). The leptokurtic shape (kurtosis 3.34) and 159 outliers (~1.1%) reflect a small tail of equatorial or high-latitude records that an analyst should verify are not geocoding errors. Treatment: Use as-is or pair with longitude for spatial modelling; consider binning into regions or projecting to avoid Euclidean distance distortion. high · anthropic:default

n

14,770

nulls

0 (0.0%)

unique

7,810

min

-14.32

max

70.13

mean

37.28

median

37.12

std

5.247

q1

33.63

q3

41.13

iqr

7.499

skew

-0.1787

kurtosis

3.341

n_outliers

159

outlier_rate

0.01077

zero_rate

0

This column represents geographic longitude, with values spanning from -170.7316 to 171.4689 degrees. The bulk of observations cluster around the Americas (mean -90.94, IQR roughly -96.4 to -84.23, consistent with the central/eastern US or Caribbean), but the extreme kurtosis of 55.6 and 623 outliers (4.2%) indicate a heavy-tailed distribution with a notable minority of records far outside this core region — including values near +171, suggesting Pacific or Asian locations. The positive skew (1.29) and tight IQR relative to the full range confirm most records concentrate in a narrow band while a long right tail pulls toward positive (eastern hemisphere) longitudes. Treatment: Retain as-is for geospatial modelling; investigate the 623 outliers for data-entry errors or legitimate international records before clustering or bounding-box filtering. high · anthropic:default

n

14,770

nulls

0 (0.0%)

unique

8,828

min

-170.7

max

171.5

mean

-90.94

median

-90.22

std

11.7

q1

-96.4

q3

-84.23

iqr

12.17

skew

1.286

kurtosis

55.61

n_outliers

623

outlier_rate

0.04218

zero_rate

0

This column contains structured event description labels of the form '[Weather Event Type] in [STATE, COUNTY]', effectively serving as a composite label combining event type and geographic location. The duplicate rate is strikingly high at 54.9%, with 8,110 duplicates across 14,770 rows and only 6,660 unique values, indicating that the same event type/location combinations recur frequently — consistent with repeated weather incidents in the same areas. The multilingual alert is almost certainly a false positive from language detection mis-classifying US place names and weather terminology as non-English; dominant language is English (4,796 of sampled values) and top values are entirely English-structured strings. Vocabulary size of 1,980 across ~14k rows and a mean of ~4.6 words per entry confirm the formulaic, low-variety nature of the text. Treatment: Parse into two structured features (event_type, state_county) via regex split on ' in ' before modelling; do not embed as raw text. high · anthropic:default

n

14,770

nulls

0 (0.0%)

unique

6,660

len_min

17

len_max

134

len_mean

30.22

len_median

29

len_p95

41

word_mean

4.588

word_median

4

n_empty

0

n_duplicates

8,110

duplicate_rate

0.5491

vocab_size

1,980

readability_flesch_mean

31.16

emoji_rate

0

url_rate

0

one_word_rate

0

allcaps_rate

0

boilerplate_rate

0

This column contains structured event descriptions summarising disaster or incident outcomes — specifically property damage amounts, injury counts, fatalities, and seismic magnitudes (e.g., 'Magnitude 0; $2.5M property damage'). The duplicate rate is strikingly high at 60.76%, with 8,974 duplicates across 14,770 rows and only 5,796 unique values, indicating these are templated strings generated from a small set of outcome combinations rather than free-form text. The Flesch readability mean of 29.86 reflects the dense, numeric, shorthand nature of the content. A small multilingual signal exists (10 Norwegian, 5 French, 1 Japanese entries) which may indicate data sourced from multiple regional systems and warrants review. Treatment: Parse structured fields (damage amount, injuries, fatalities, magnitude) via regex into separate numeric columns rather than embedding as text. high · anthropic:default

n

14,770

nulls

0 (0.0%)

unique

5,796

len_min

3

len_max

259

len_mean

50.09

len_median

36

len_p95

166

word_mean

7.393

word_median

5

n_empty

0

n_duplicates

8,974

duplicate_rate

0.6076

vocab_size

4,289

readability_flesch_mean

29.86

emoji_rate

0

url_rate

0

one_word_rate

0.0002708

allcaps_rate

0.0002708

boilerplate_rate

0

This column is a dataset category tag, holding a single constant value 'significant_us_storms' across all 14,770 rows with no nulls. It carries zero information entropy (entropy = 0.0) and a top_rate of 1.0, meaning it is entirely invariant. This is a metadata label describing the dataset itself, not a feature with predictive or analytical value. Treatment: Drop before modelling; constant column adds no signal and will cause issues with variance-based methods. high · anthropic:default

n

14,770

nulls

0 (0.0%)

unique

1

top_value

significant_us_storms

top_rate

1

cardinality

1

entropy

0

entropy_ratio

0

This column contains ISO-8601 date strings (YYYY-MM-DD format), stored as text rather than a native date type — all 14,770 values are exactly 10 characters with zero nulls. The duplicate rate of 65.75% (9,712 duplicates across only 5,058 unique dates) is notable and suggests this is a grouping/event date used as a foreign-key-style attribute rather than a unique record timestamp. The top date, 1974-04-03, appears 126 times, and several 2011 dates cluster heavily, which may reflect significant event concentrations worth investigating. Treatment: Parse to native date type, then use as a grouping/join key or engineer calendar features (year, month, day-of-week) for modelling. high · anthropic:default

n

14,770

nulls

0 (0.0%)

unique

5,058

len_min

10

len_max

10

len_mean

10

len_median

10

len_p95

10

word_mean

1

word_median

1

n_empty

0

n_duplicates

9,712

duplicate_rate

0.6575

vocab_size

5,058

readability_flesch_mean

121.2

emoji_rate

0

url_rate

0

one_word_rate

1

allcaps_rate

1

boilerplate_rate

0

This column represents the country of origin or scope for all records in the dataset, and every single one of the 14,770 rows contains the value 'USA' — making it a zero-entropy constant. The column carries no discriminative information whatsoever and will contribute nothing to any model or analysis. Its uniformity may also indicate the dataset is intentionally scoped to a single country, which is worth confirming before joining with broader datasets. Treatment: Drop before modelling; constant column with zero variance and entropy of 0.0. high · anthropic:default

n

14,770

nulls

0 (0.0%)

unique

1

top_value

USA

top_rate

1

cardinality

1

entropy

0

entropy_ratio

0

This column contains categorical labels for natural weather/disaster event types across 14,770 records, with 17 distinct categories and no nulls. The dominant class is 'Tornado' at 42.9% (6,334 occurrences), creating notable class imbalance — the top 5 categories ('Tornado', 'Flash Flood', 'Thunderstorm Wind', 'Flood', 'Hail') account for the vast majority of records, while tail categories like 'Marine Thunderstorm Wind' (25) and 'Debris Flow' (43) are sparsely represented. The entropy ratio of 0.572 confirms moderate but uneven spread across classes, which will challenge classifiers without resampling or class-weight adjustment. Treatment: Encode as nominal category; apply class weights or oversample minority classes (e.g., 'Marine Thunderstorm Wind' n=25) before classification modelling. high · anthropic:default

n

14,770

nulls

0 (0.0%)

unique

17

top_value

Tornado

top_rate

0.4288

cardinality

17

entropy

2.336

entropy_ratio

0.5715

This column represents the US state associated with each record, stored as full uppercase state names. With 65 unique values against the expected 50 US states, there are likely extra entries such as territories (e.g., Puerto Rico, Guam), non-standard labels, or minor data quality issues worth auditing. Texas dominates at 9.8% of records (1,450), and the top-10 states are heavily weighted toward the South and Midwest. The high entropy ratio of 0.86 indicates a relatively even spread across categories, though Texas is a clear outlier compared to the rest. Treatment: Standardize to a canonical list (resolve the 65→50+ mapping), then one-hot encode or use target encoding for modelling. high · anthropic:default

n

14,770

nulls

0 (0.0%)

unique

65

top_value

TEXAS

top_rate

0.09817

cardinality

65

entropy

5.182

entropy_ratio

0.8605

This column appears to represent a magnitude measure (likely seismic, stellar, or similar scientific scale) stored as a categorical type despite containing numeric-looking values spanning a wide range (e.g., 1.75, 2.75, 50.00, 70.00). Two surprises stand out: first, 51.78% of rows are null, triggering an alert; second, the dominant value '0' accounts for 54.24% of non-null rows (3,863 of ~7,124 non-null records), suggesting zero may encode 'none', 'unknown', or a sentinel rather than a true zero magnitude. The presence of both small decimal values (1.75, 2.00, 2.50) and large round integers (50.00, 61.00, 65.00, 70.00) hints at a possible mixed-scale or mixed-source column. Treatment: Investigate zero sentinel vs. true zero, impute or drop nulls based on missingness mechanism, cast to float, then assess whether log-transform or binning is appropriate before modelling. medium · anthropic:default

n

14,770

nulls

7,648 (51.8%)

unique

170

top_value

0

top_rate

0.5424

cardinality

170

entropy

3.586

entropy_ratio

0.484

This column represents a count of injuries per record, stored as a categorical type despite being fundamentally numeric. The dominant value is '0' appearing in 68.1% of rows (10,064 of 14,770), indicating most records involve no injuries. With 178 unique values and top counts following a steep drop-off consistent with a zero-inflated count distribution, the categorical encoding is likely a data-type artifact — the values are clearly ordinal integers and should be treated as numeric. Treatment: Cast to integer, then model with zero-inflated Poisson or apply log1p transform before regression given heavy zero inflation. high · anthropic:default

n

14,770

nulls

0 (0.0%)

unique

178

top_value

0

top_rate

0.6814

cardinality

178

entropy

2.468

entropy_ratio

0.3301

This column records fatality counts per incident, stored as strings but representing non-negative integers ranging from 0 to at least 10 across 49 distinct values. The dominant value is '0' at 69.1% of rows (10,209 of 14,770), indicating most incidents involve no fatalities. The distribution is heavily right-skewed, with counts dropping sharply: 1 fatality appears 3,208 times, 2 appears 649 times, and values thin out rapidly beyond that — yet 49 unique values suggests some high-count outliers exist beyond the top 10 shown. Low entropy (1.42, ratio 0.25) confirms the extreme concentration on zero. Treatment: Cast to integer, treat as count variable; consider zero-inflated modelling or log1p-transform given severe right skew and 69.1% zero mass. high · anthropic:default

n

14,770

nulls

0 (0.0%)

unique

49

top_value

0

top_rate

0.6912

cardinality

49

entropy

1.423

entropy_ratio

0.2535

This column represents property damage amounts stored as formatted currency strings (e.g., '2.5M', '250K', '0.00K'), typical of NOAA storm event or similar disaster/insurance datasets. With only 1,014 unique values across 14,770 rows, the duplicate rate is extremely high at 93.1%, reflecting heavy rounding/bucketing of damage estimates rather than precise measurements. All values are single tokens (one_word_rate: 1.0) and 87.2% are uppercase, consistent with a coded categorical-style encoding of numeric magnitudes. There are 368 empty strings (null_rate reported as 0.0 but n_empty=368), which should be treated as missing values. Treatment: Parse suffix notation (K=thousands, M=millions) to convert to numeric float, treat empty strings as null, then log-transform before modelling. high · anthropic:default

n

14,770

nulls

0 (0.0%)

unique

1,014

len_min

0

len_max

8

len_mean

4.381

len_median

5

len_p95

7

word_mean

1

word_median

1

n_empty

368

n_duplicates

13,756

duplicate_rate

0.9313

vocab_size

1,013

readability_flesch_mean

117

emoji_rate

0

url_rate

0

one_word_rate

1

allcaps_rate

0.8724

boilerplate_rate

0

This column identifies the data source, and every single one of the 14,770 rows carries the identical value 'NOAA Storm Events Database' — cardinality of 1 with top_rate of 1.0 and entropy of 0.0. It is a constant metadata field, almost certainly a provenance tag added during ingestion. It carries zero predictive or analytical signal. Treatment: Drop before modelling; constant column adds no variance. high · anthropic:default

n

14,770

nulls

0 (0.0%)

unique

1

top_value

NOAA Storm Events Database

top_rate

1

cardinality

1

entropy

0

entropy_ratio

0