data trove hot pepper varieties pepperscale

This column contains the names of individual pepper varieties — it is a human-readable label for each record in what appears to be a pepper/vegetable catalog. With 175 rows and 175 unique values, every entry is distinct (cardinality = 175, top_rate = 0.0057, i.e. 1/175), making this a perfect natural key. Entropy ratio of ~1.0 confirms near-maximum unpredictability, consistent with a unique-per-row label rather than a grouping variable. Treatment: Treat as a unique row label or display name; drop before modelling or use as an index key — do not encode as a categorical feature. high · anthropic:default

n

175

nulls

0 (0.0%)

unique

175

top_value

Bell Pepper

top_rate

0.005714

cardinality

175

entropy

7.451

entropy_ratio

1

This column is an ordinal heat/spiciness level rating with 5 discrete categories, likely describing food or beverage intensity. 'Medium' dominates at 40% of 175 rows (top_rate 0.4), while the two hottest tiers ('Hot' and 'Extra Hot') together account for only 30 rows — a pronounced skew toward milder levels. Entropy ratio of 0.893 indicates reasonably spread distribution despite the modal imbalance. No nulls and no unexpected values are present. Treatment: Encode as ordered ordinal (Mild < Medium < Hot < Super Hot < Extra Hot) before modelling. high · anthropic:default

n

175

nulls

0 (0.0%)

unique

5

top_value

Medium

top_rate

0.4

cardinality

5

entropy

2.074

entropy_ratio

0.8933

This column represents the minimum Scoville Heat Unit (SHU) value for chili peppers or hot sauces, capturing the lower bound of each item's heat range. The distribution is dramatically right-skewed (skew = 10.31, kurtosis = 120.13): the median is just 15,000 SHU while the mean is 289,208 SHU, driven by extreme outliers reaching 15,000,000 SHU (pure capsaicin territory). With 29 outliers (16.6% of rows) and only 44 unique values across 175 records, the data clusters heavily at low heat levels but has a long tail of superhot entries. The 9.7% zero rate corresponds to genuinely non-hot items (e.g., bell peppers at 0 SHU), which is domain-valid. Treatment: Log-transform (log1p) before modelling to compress the extreme right tail; consider pairing with scoville_max for a range-based heat feature. high · anthropic:default

n

175

nulls

0 (0.0%)

unique

44

min

0

max

1.5e+07

mean

2.892e+05

median

15,000

std

1.218e+06

q1

1,000

q3

75,000

iqr

74,000

skew

10.31

kurtosis

120.1

n_outliers

29

outlier_rate

0.1657

zero_rate

0.09714

This column represents the maximum Scoville Heat Unit (SHU) rating for chili peppers or hot sauce products — a standard measure of capsaicin-driven heat intensity. The distribution is extraordinarily right-skewed (skew=9.45, kurtosis=106.1): the median is only 30,000 SHU while the mean is 384,835 and the maximum reaches 16,000,000 — consistent with extreme outliers like pure capsaicin extracts sitting alongside mild peppers. With 43 outliers (24.6% of rows) and a standard deviation of 1,333,100 dwarfing the median, the bulk of records cluster at low heat levels while a long tail of superhot entries dominates the mean. The 5.7% zero rate likely reflects peppers with no measurable heat (e.g., bell peppers). Treatment: Log-transform (log1p) before modelling to compress the extreme right tail; flag zeros as a separate category if heat presence is a meaningful signal. high · anthropic:default

n

175

nulls

0 (0.0%)

unique

59

min

0

max

1.6e+07

mean

3.848e+05

median

30,000

std

1.333e+06

q1

2,750

q3

100,000

iqr

97,250

skew

9.45

kurtosis

106.1

n_outliers

43

outlier_rate

0.2457

zero_rate

0.05714

This column represents the median Scoville heat unit (SHU) rating for chili peppers or hot sauces — a standard measure of capsaicin-driven spiciness. The distribution is extraordinarily right-skewed (skew=9.79, kurtosis=111.47): the median sits at 22,500 SHU while the mean is pulled to 339,804 SHU by extreme outliers, with a maximum of 15,500,000 SHU (consistent with ultra-hot peppers like Carolina Reaper). 41 of 175 rows (23.4%) are flagged as outliers, and 5.7% of values are exactly zero, suggesting entries for non-spicy varieties (e.g., bell peppers). The IQR spans only 2,000–90,000 SHU, confirming that the bulk of records are mild-to-medium heat while a long tail of superhot entries distorts the scale severely. Treatment: Log-transform (log1p) before modelling to compress the extreme right tail; consider flagging zero-SHU entries as a separate binary indicator. high · anthropic:default

n

175

nulls

0 (0.0%)

unique

80

min

0

max

1.55e+07

mean

3.398e+05

median

22,500

std

1.279e+06

q1

2,000

q3

90,000

iqr

88,000

skew

9.794

kurtosis

111.5

n_outliers

41

outlier_rate

0.2343

zero_rate

0.05714

jalRP is a non-negative numeric column with 175 rows and no nulls, likely representing a monetary amount, duration, or count that is naturally right-bounded near zero. The distribution is extremely right-skewed (skew = 9.79, kurtosis = 111.48): the median is just 4.29 while the mean is 64.72, and a maximum of 2952.38 pulls the standard deviation to 243.61 — 41 rows (23.4%) are flagged as outliers. The interquartile range spans only 0.38 to 17.14, meaning 75% of values are below 17.14, yet the top end reaches nearly 3000, signalling a heavy-tailed, potentially power-law distribution. Treatment: Log-transform (e.g., log1p) before modelling to reduce skew; investigate the 41 outliers for data-entry errors or legitimate extreme events. high · anthropic:default

n

175

nulls

0 (0.0%)

unique

81

min

0

max

2952

mean

64.72

median

4.29

std

243.6

q1

0.38

q3

17.14

iqr

16.76

skew

9.795

kurtosis

111.5

n_outliers

41

outlier_rate

0.2343

zero_rate

0.05714

This column captures Capsicum species type (botanical variety), with 8 apparent unique values across 175 records and no nulls. The dominant value 'annuum' accounts for 59.4% of rows, followed by 'chinense' (46 rows) and 'baccatum' (12 rows). A critical data quality issue exists: 'annuum' and 'Annuum' are treated as distinct values (104 vs. 4 occurrences), as are 'chinense' and 'Chinense' (46 vs. 2), indicating inconsistent capitalisation that inflates apparent cardinality. Additionally, one record holds the sentinel string 'N/A' rather than a true null. Treatment: Normalise to lowercase, remap 'N/A' to null, then use as a categorical grouping variable or stratification key. high · anthropic:default

n

175

nulls

0 (0.0%)

unique

8

top_value

annuum

top_rate

0.5943

cardinality

8

entropy

1.657

entropy_ratio

0.5524

This column records the geographic origin of individuals or items, expressed as country or regional names across 34 distinct values in 175 rows. 'United States' dominates at 26.3% (46 rows), followed by 'Mexico' (26) and several South American/Caribbean entries. Notably, the taxonomy mixes country-level specificity ('Peru', 'Italy') with broad regional labels ('South America', 'Caribbean'), and 7 rows carry the value 'Unknown', indicating inconsistent granularity in data collection. Treatment: Standardize regional vs. country-level labels, encode 'Unknown' as missing, then one-hot or target-encode for modelling. high · anthropic:default

n

175

nulls

0 (0.0%)

unique

34

top_value

United States

top_rate

0.2629

cardinality

34

entropy

3.98

entropy_ratio

0.7823

This column captures the primary use-case category for each record, likely plants or herbs, with four distinct values and no nulls across 175 rows. 'Culinary' dominates at 80.6% (141 of 175), making the distribution heavily skewed. One entry is an empty string rather than a true null, which effectively acts as a fifth implicit category. The dual-label value 'Culinary, Ornamental' (2 occurrences) signals inconsistent multi-value encoding that may need normalisation. Treatment: Normalise the empty-string entry, then either one-hot encode or split multi-value strings (e.g. 'Culinary, Ornamental') into binary indicator columns before modelling. high · anthropic:default

n

175

nulls

0 (0.0%)

unique

4

top_value

Culinary

top_rate

0.8057

cardinality

4

entropy

0.8097

entropy_ratio

0.4049

This column captures flavor descriptors for a food or beverage dataset, stored as free-form comma-separated tag strings (e.g., 'Sweet, Fruity, Smoky'). With 73 unique values across only 175 rows and an entropy ratio of 0.845, the vocabulary is highly fragmented — notably, 'Sweet' (25 occurrences) and 'Sweet, Fruity' (21) appear to be near-duplicates of ordering variants like 'Fruity, Sweet' (6), suggesting inconsistent entry order inflating cardinality artificially. The long-tail alert confirms that most combinations appear very rarely, with the top value covering only 14.3% of rows. Treatment: Split on comma, normalize order, and one-hot or multi-label encode individual flavor tokens before modelling. high · anthropic:default

n

175

nulls

0 (0.0%)

unique

73

top_value

Sweet

top_rate

0.1429

cardinality

73

entropy

5.232

entropy_ratio

0.8453

This column contains fully-qualified URLs pointing to individual pepper variety pages on pepperscale.com, effectively serving as a unique page identifier for each row. Cardinality is exactly 175 with 175 unique values and a null rate of 0.0, meaning every row has a distinct URL — the top_rate of 0.005714 (1/175) confirms no duplicates whatsoever. Entropy ratio of ~1.0 indicates maximum dispersion, consistent with a perfect natural key. The long_tail alert is technically triggered but is structurally expected given perfect uniqueness. Treatment: Retain as a unique row key or use as a foreign key to join additional scraped metadata; do not encode or embed for modelling. high · anthropic:default

n

175

nulls

0 (0.0%)

unique

175

top_value

https://www.pepperscale.com/bell-pepper/

top_rate

0.005714

cardinality

175

entropy

7.451

entropy_ratio

1