data trove us disasters mashup

This column is a disaster/incident type label with exactly 4 categories: aviation_accident, storm, earthquake, and shipwreck. The distribution is notably skewed — aviation_accident dominates at 59.4% of all 54,575 rows (32,410 records), while earthquake and shipwreck are each underrepresented at roughly 6.7% apiece. The entropy ratio of 0.74 confirms meaningful but unbalanced spread across classes, which could bias classifiers trained on this target without resampling. Treatment: Use as classification target; apply class-weighting or oversampling for minority classes (earthquake, shipwreck) before modelling. high · anthropic:default

n

54,575

nulls

0 (0.0%)

unique

4

top_value

aviation_accident

top_rate

0.5939

cardinality

4

entropy

1.483

entropy_ratio

0.7415

This column contains geographic latitude values ranging from -77.42 to 82.17, consistent with global coordinate data. The median of 38.38 and IQR of 9.12 suggest the bulk of records cluster around mid-latitude Northern Hemisphere locations (roughly US/Europe), but the negative minimum (-77.42) indicates some Southern Hemisphere entries. Highly surprising is the negative skew of -2.51 combined with extreme kurtosis of 15.97 and 4,302 outliers (7.88% of rows), pointing to a heavy tail of anomalous low-latitude or Southern Hemisphere observations that likely warrant geographic subsetting or anomaly review. Treatment: Retain as-is for geo-spatial modelling; investigate the 4,302 outliers for data quality issues before binning or clustering by region. high · anthropic:default

n

54,575

nulls

0 (0.0%)

unique

32,209

min

-77.42

max

82.17

mean

38.16

median

38.38

std

11.96

q1

33.65

q3

42.77

iqr

9.12

skew

-2.51

kurtosis

15.97

n_outliers

4,302

outlier_rate

0.07883

zero_rate

0

This column contains geographic longitude values, spanning from -179.28° to +178.83°, consistent with worldwide coordinates. The mean (-92.97°) and median (-92.81°) are tightly clustered in the central United States, suggesting the bulk of records are North American, yet 4,320 outliers (7.9% of rows) and an extreme kurtosis of 15.13 indicate a heavy-tailed distribution with a substantial minority of globally dispersed points. The positive skew of 2.84 confirms an asymmetric pull toward higher (less-negative or positive) longitude values, i.e., non-US locations. Treatment: Retain as-is for geospatial modelling; consider pairing with latitude and clustering by region to handle the bimodal/heavy-tailed distribution before feeding into non-spatial models. high · anthropic:default

n

54,575

nulls

0 (0.0%)

unique

34,804

min

-179.3

max

178.8

mean

-92.97

median

-92.81

std

39.5

q1

-112

q3

-82.18

iqr

29.86

skew

2.843

kurtosis

15.13

n_outliers

4,320

outlier_rate

0.07916

zero_rate

0

This column contains descriptive incident or event names, predominantly aviation accidents and natural disaster events (floods, tornadoes). The duplicate rate is strikingly high at 62.3% — with 33,988 duplicates across only 20,587 unique values out of 54,575 rows — largely driven by generic labels like 'Unnamed Wreck' (2,184 occurrences) and repeated aircraft model patterns (e.g., 'Aviation Accident - CESSNA 172' variants). While 86.6% of detected-language tokens are English, 14 other languages appear (French: 60, German: 58, Spanish: 46, Japanese: 32), indicating a multilingual dataset that may require language-aware processing. Treatment: Normalize case variants (e.g., 'CESSNA 172' vs 'Cessna 172') before grouping or embedding; treat as a categorical label with high cardinality rather than free text. high · anthropic:default

n

54,575

nulls

0 (0.0%)

unique

20,587

len_min

2

len_max

153

len_mean

32.38

len_median

31

len_p95

48

word_mean

4.782

word_median

5

n_empty

0

n_duplicates

33,988

duplicate_rate

0.6228

vocab_size

8,062

readability_flesch_mean

8.744

emoji_rate

0

url_rate

0

one_word_rate

0.007971

allcaps_rate

0.001264

boilerplate_rate

0

This column contains date strings in ISO-8601 format (YYYY-MM-DD), stored as text rather than a native date type. Nearly all top values fall on January 1st of their respective years (2002–2012), suggesting dates are truncated or snapped to year-start, which is analytically significant and likely not raw event timestamps. The duplicate rate is extremely high at 81.99%, consistent with annual granularity across 54,575 rows, and 9,264 unique values hint that some finer dates do exist beyond the dominant Jan-1 entries. Null rate is low at 5.74%. Treatment: Parse to date type, investigate year-start snapping before using as a time feature, and consider extracting year as an ordinal variable. high · anthropic:default

n

54,575

nulls

3,134 (5.7%)

unique

9,264

len_min

2

len_max

10

len_mean

9.98

len_median

10

len_p95

10

word_mean

1.002

word_median

1

n_empty

0

n_duplicates

42,177

duplicate_rate

0.8199

vocab_size

4,710

readability_flesch_mean

121.2

emoji_rate

0

url_rate

0

one_word_rate

0.9985

allcaps_rate

0.9983

boilerplate_rate

0

This column is a categorical event subcategory, most likely classifying incident or hazard reports across domains such as aviation, geophysical (seismic), meteorological (Tornado, Flash Flood, Thunderstorm Wind), and maritime events. 'aviation' dominates heavily at 59.4% of all 54,575 rows, creating pronounced class imbalance. A subtle data quality issue is present: some values use title case ('Tornado', 'Flash Flood', 'Thunderstorm Wind', 'Hail') while others are fully lowercase ('aviation', 'seismic', 'maritime'), suggesting records were ingested from at least two inconsistently formatted sources. Entropy ratio of 0.49 confirms the distribution is far from uniform. Treatment: Normalize casing before use, then one-hot encode or target-encode accounting for the heavy 'aviation' majority (59.4%). high · anthropic:default

n

54,575

nulls

0 (0.0%)

unique

20

top_value

aviation

top_rate

0.5939

cardinality

20

entropy

2.115

entropy_ratio

0.4894

This column represents earthquake or seismic event magnitude, stored as a categorical/string type despite being a numeric measurement with 291 distinct decimal values (e.g., 4.5, 4.6, 4.7). Two signals demand attention: the null rate is extremely high at 80.09%, meaning only ~10,866 of 54,575 rows carry a value. The dominant value '0' accounts for 35.56% of non-null records (3,863 occurrences), which is likely a sentinel or placeholder rather than a true zero magnitude, since genuine zero-magnitude events would be vanishingly rare and the next most frequent values cluster around 4.5–5.1. Treatment: Cast to float after replacing '0' sentinel values with NaN; investigate whether 80.09% nulls are structurally missing or data quality issues before imputing or dropping rows. high · anthropic:default

n

54,575

nulls

43,711 (80.1%)

unique

291

top_value

0

top_rate

0.3556

cardinality

291

entropy

4.732

entropy_ratio

0.5781

This column represents a fatality count per incident, stored as a categorical/string type despite being numeric in nature. The null rate is severe at 72.94%, meaning nearly three-quarters of records have no value recorded — this is the primary alert. Among non-null values, the distribution is heavily right-skewed: '0' dominates at 69.1% of non-null rows, with counts dropping sharply through 49 distinct values, indicating rare but high-fatality events exist in the tail. Treatment: Cast to integer, treat nulls as unknown (not zero), then apply log1p-transform or use as-is for count-based modelling given heavy zero-inflation. high · anthropic:default

n

54,575

nulls

39,805 (72.9%)

unique

49

top_value

0

top_rate

0.6912

cardinality

49

entropy

1.423

entropy_ratio

0.2535

This column represents an injury count per incident, stored as a categorical type despite containing integer values (0, 1, 2, …). The dominant concern is an extreme null rate of 72.94%, meaning nearly three-quarters of rows carry no injury data at all. Among non-null rows, the value '0' accounts for 68.14% of responses, indicating most recorded incidents involved no injuries, with a long tail reaching at least 178 distinct values — suggesting occasional high-casualty outliers. Treatment: Cast to integer, impute or flag nulls explicitly, then consider log-transform or treat as a count target given heavy zero-inflation. high · anthropic:default

n

54,575

nulls

39,805 (72.9%)

unique

178

top_value

0

top_rate

0.6814

cardinality

178

entropy

2.468

entropy_ratio

0.3301

This column contains abbreviated monetary damage estimates (e.g., '2.5M', '250K', '0.00K') stored as free-form text, most likely representing financial loss or property damage figures from incident or insurance records. The null rate is extremely high at 72.94%, meaning nearly three-quarters of rows carry no damage value. The all-caps rate of 87.2% and one-word rate of 100% confirm a consistent but non-numeric encoding; the 1,014 unique values across 54,575 rows with a duplicate rate of 93.1% indicate a relatively coarse discrete scale. Analysts should note that string suffixes (K vs M) encode magnitude and must be parsed before any quantitative use. Treatment: Parse magnitude suffixes (K=thousands, M=millions) and convert to a numeric column; impute or flag the 72.94% nulls before modelling. high · anthropic:default

n

54,575

nulls

39,805 (72.9%)

unique

1,014

len_min

0

len_max

8

len_mean

4.381

len_median

5

len_p95

7

word_mean

1

word_median

1

n_empty

368

n_duplicates

13,756

duplicate_rate

0.9313

vocab_size

1,013

readability_flesch_mean

117

emoji_rate

0

url_rate

0

one_word_rate

1

allcaps_rate

0.8724

boilerplate_rate

0

This column contains US state names (full uppercase spellings), acting as a geographic feature for records in the dataset. The critical issue is a 72.94% null rate, meaning nearly three-quarters of all 54,575 rows carry no state value — this is a severe missingness alert. Among non-null values, cardinality is 65 (slightly above 50 US states, suggesting territories or data anomalies), and distribution is moderately spread (entropy ratio 0.86) with Texas as the dominant value at 9.82% of non-null records. Treatment: Investigate missingness mechanism before use; consider imputation or missingness indicator flag, and audit the 65 unique values to identify non-standard entries beyond the 50 states. high · anthropic:default

n

54,575

nulls

39,805 (72.9%)

unique

65

top_value

TEXAS

top_rate

0.09817

cardinality

65

entropy

5.182

entropy_ratio

0.8605

This column contains aircraft make-and-model designations (e.g., 'Cessna 172', 'Piper PA-28-140') from what appears to be an aviation incident or registration dataset. Two major surprises: first, 40.6% of rows are null, indicating substantial missing coverage; second, case inconsistency is severe — 'CESSNA 172' (360 occurrences) and 'Cessna 172' (189 occurrences) are counted as distinct values despite being the same aircraft, with ~49.5% of values in all-caps, inflating n_unique (9,478) and the duplicate rate (70.8%) artificially. The top words confirm a GA-heavy dataset dominated by Cessna, Piper, and Beech. Treatment: Normalize case (lowercase or title-case), then deduplicate/consolidate variant spellings before using as a categorical feature or grouping key. high · anthropic:default

n

54,575

nulls

22,165 (40.6%)

unique

9,478

len_min

7

len_max

50

len_mean

15.85

len_median

13

len_p95

31

word_mean

2

word_median

2

n_empty

0

n_duplicates

22,932

duplicate_rate

0.7076

vocab_size

7,261

readability_flesch_mean

45.12

emoji_rate

0

url_rate

0

one_word_rate

0

allcaps_rate

0.4955

boilerplate_rate

0

This column is an aviation or safety incident event identifier — the 14-character format (e.g., '20010519X00967') encodes a date prefix followed by an alphanumeric case code, consistent with NTSB accident/incident IDs. Two signals are surprising: a null rate of 40.61% means nearly half of rows lack an event ID entirely, and the duplicate rate of 18.46% (5,983 duplicates across 26,427 unique values) indicates multiple rows share the same event ID, implying a one-to-many relationship where each event spawns several records. All values are exactly 14 characters and fully uppercase, confirming a tightly controlled format with no malformed entries. Treatment: Left-join on this ID to an events dimension table; investigate and handle the 40.61% null rate before joining. high · anthropic:default

n

54,575

nulls

22,165 (40.6%)

unique

26,427

len_min

14

len_max

14

len_mean

14

len_median

14

len_p95

14

word_mean

1

word_median

1

n_empty

0

n_duplicates

5,983

duplicate_rate

0.1846

vocab_size

17,535

readability_flesch_mean

121.2

emoji_rate

0

url_rate

0

one_word_rate

1

allcaps_rate

1

boilerplate_rate

0

This column categorizes the type of vessel involved in an incident or record, with 23 distinct values including 'ship', 'submarine', 'aircraft', and oddly 'car'. Two major data quality issues stand out: the null rate is extreme at 93.31%, meaning only ~3,700 of 54,575 rows carry any value, and the top recorded value is an empty string (3,311 occurrences), which inflates the apparent top_rate to 90.6% — suggesting the true fill rate is even lower than the null_rate implies. The long-tail alert is consistent with rare values like 'schooner' (2), 'sailboat' (2), and 'steamer' (1), while 'car' appearing as a vessel type signals potential data entry errors or schema misuse. Treatment: Treat empty strings as nulls, impute or exclude before modelling given 93.31% missingness, and audit 'car' and 'aircraft' entries for schema validity. high · anthropic:default

n

54,575

nulls

50,922 (93.3%)

unique

23

top_value

top_rate

0.9064

cardinality

23

entropy

0.5764

entropy_ratio

0.1274

This column records the type of cargo carried by vessels or vehicles, with 17 distinct categories including 'human', 'timber', 'coal', 'fertilizer', and 'fish'. It is overwhelmingly sparse: 93.31% of rows are null, and among the non-null rows the top value is an empty string (3,632 occurrences), meaning genuinely populated values number only in the single digits each. The entropy ratio of 0.018 confirms near-total concentration, and the presence of a German-language entry ('Fischkutter (Stahl)') signals a language mix in the rare populated records. Treatment: Exclude from modelling unless the non-null subset is the analytic focus; treat empty strings as nulls, consolidate language variants, and flag the 93.31% missingness as likely structural (field not applicable to most records). high · anthropic:default

n

54,575

nulls

50,922 (93.3%)

unique

17

top_value

top_rate

0.9943

cardinality

17

entropy

0.07302

entropy_ratio

0.01786

This column represents earthquake or geological event depth in kilometres, a continuous numeric feature. The profiler skipped analysis entirely, so no distribution statistics, uniqueness counts, or range information are available. With 54,575 rows and a null rate of 0.0, the data is fully populated, but nothing can be said about skew, outliers, or value range from this evidence alone. An analyst should inspect the column directly before modelling. Treatment: Manually profile for range and skew; apply log-transform if depth distribution is right-skewed before regression. low · anthropic:default

n

54,575

nulls

0 (0.0%)

unique

—