data trove hot pepper varieties pepperscale

Overview

Source: /home/coolhand/html/datavis/data_trove/data/quirky/peppers.json

Saturn profiled 175 rows across 11 columns. The stats below are deterministic and machine-readable; the prose is a language-model interpretation of those stats (opt-in, added after the fact, never sees raw rows).

name categorical label

This column contains the names of individual pepper varieties — it is a human-readable label for each record in what appears to be a pepper/vegetable catalog. With 175 rows and 175 unique values, every entry is distinct (cardinality = 175, top_rate = 0.0057, i.e. 1/175), making this a perfect natural key. Entropy ratio of ~1.0 confirms near-maximum unpredictability, consistent with a unique-per-row label rather than a grouping variable.

Treatment: Treat as a unique row label or display name; drop before modelling or use as an index key — do not encode as a categorical feature.

heat categorical label

This column is an ordinal heat/spiciness level rating with 5 discrete categories, likely describing food or beverage intensity. 'Medium' dominates at 40% of 175 rows (top_rate 0.4), while the two hottest tiers ('Hot' and 'Extra Hot') together account for only 30 rows — a pronounced skew toward milder levels. Entropy ratio of 0.893 indicates reasonably spread distribution despite the modal imbalance. No nulls and no unexpected values are present.

Treatment: Encode as ordered ordinal (Mild < Medium < Hot < Super Hot < Extra Hot) before modelling.

scoville_min numeric feature

This column represents the minimum Scoville Heat Unit (SHU) value for chili peppers or hot sauces, capturing the lower bound of each item's heat range. The distribution is dramatically right-skewed (skew = 10.31, kurtosis = 120.13): the median is just 15,000 SHU while the mean is 289,208 SHU, driven by extreme outliers reaching 15,000,000 SHU (pure capsaicin territory). With 29 outliers (16.6% of rows) and only 44 unique values across 175 records, the data clusters heavily at low heat levels but has a long tail of superhot entries. The 9.7% zero rate corresponds to genuinely non-hot items (e.g., bell peppers at 0 SHU), which is domain-valid.

Treatment: Log-transform (log1p) before modelling to compress the extreme right tail; consider pairing with scoville_max for a range-based heat feature.

scoville_max numeric feature

This column represents the maximum Scoville Heat Unit (SHU) rating for chili peppers or hot sauce products — a standard measure of capsaicin-driven heat intensity. The distribution is extraordinarily right-skewed (skew=9.45, kurtosis=106.1): the median is only 30,000 SHU while the mean is 384,835 and the maximum reaches 16,000,000 — consistent with extreme outliers like pure capsaicin extracts sitting alongside mild peppers. With 43 outliers (24.6% of rows) and a standard deviation of 1,333,100 dwarfing the median, the bulk of records cluster at low heat levels while a long tail of superhot entries dominates the mean. The 5.7% zero rate likely reflects peppers with no measurable heat (e.g., bell peppers).

Treatment: Log-transform (log1p) before modelling to compress the extreme right tail; flag zeros as a separate category if heat presence is a meaningful signal.

scoville_median numeric feature

This column represents the median Scoville heat unit (SHU) rating for chili peppers or hot sauces — a standard measure of capsaicin-driven spiciness. The distribution is extraordinarily right-skewed (skew=9.79, kurtosis=111.47): the median sits at 22,500 SHU while the mean is pulled to 339,804 SHU by extreme outliers, with a maximum of 15,500,000 SHU (consistent with ultra-hot peppers like Carolina Reaper). 41 of 175 rows (23.4%) are flagged as outliers, and 5.7% of values are exactly zero, suggesting entries for non-spicy varieties (e.g., bell peppers). The IQR spans only 2,000–90,000 SHU, confirming that the bulk of records are mild-to-medium heat while a long tail of superhot entries distorts the scale severely.

Treatment: Log-transform (log1p) before modelling to compress the extreme right tail; consider flagging zero-SHU entries as a separate binary indicator.

jalRP numeric feature

jalRP is a non-negative numeric column with 175 rows and no nulls, likely representing a monetary amount, duration, or count that is naturally right-bounded near zero. The distribution is extremely right-skewed (skew = 9.79, kurtosis = 111.48): the median is just 4.29 while the mean is 64.72, and a maximum of 2952.38 pulls the standard deviation to 243.61 — 41 rows (23.4%) are flagged as outliers. The interquartile range spans only 0.38 to 17.14, meaning 75% of values are below 17.14, yet the top end reaches nearly 3000, signalling a heavy-tailed, potentially power-law distribution.

Treatment: Log-transform (e.g., log1p) before modelling to reduce skew; investigate the 41 outliers for data-entry errors or legitimate extreme events.

type categorical label

This column captures Capsicum species type (botanical variety), with 8 apparent unique values across 175 records and no nulls. The dominant value 'annuum' accounts for 59.4% of rows, followed by 'chinense' (46 rows) and 'baccatum' (12 rows). A critical data quality issue exists: 'annuum' and 'Annuum' are treated as distinct values (104 vs. 4 occurrences), as are 'chinense' and 'Chinense' (46 vs. 2), indicating inconsistent capitalisation that inflates apparent cardinality. Additionally, one record holds the sentinel string 'N/A' rather than a true null.

Treatment: Normalise to lowercase, remap 'N/A' to null, then use as a categorical grouping variable or stratification key.

origin categorical feature

This column records the geographic origin of individuals or items, expressed as country or regional names across 34 distinct values in 175 rows. 'United States' dominates at 26.3% (46 rows), followed by 'Mexico' (26) and several South American/Caribbean entries. Notably, the taxonomy mixes country-level specificity ('Peru', 'Italy') with broad regional labels ('South America', 'Caribbean'), and 7 rows carry the value 'Unknown', indicating inconsistent granularity in data collection.

Treatment: Standardize regional vs. country-level labels, encode 'Unknown' as missing, then one-hot or target-encode for modelling.

use categorical label

This column captures the primary use-case category for each record, likely plants or herbs, with four distinct values and no nulls across 175 rows. 'Culinary' dominates at 80.6% (141 of 175), making the distribution heavily skewed. One entry is an empty string rather than a true null, which effectively acts as a fifth implicit category. The dual-label value 'Culinary, Ornamental' (2 occurrences) signals inconsistent multi-value encoding that may need normalisation.

Treatment: Normalise the empty-string entry, then either one-hot encode or split multi-value strings (e.g. 'Culinary, Ornamental') into binary indicator columns before modelling.

flavor categorical feature

This column captures flavor descriptors for a food or beverage dataset, stored as free-form comma-separated tag strings (e.g., 'Sweet, Fruity, Smoky'). With 73 unique values across only 175 rows and an entropy ratio of 0.845, the vocabulary is highly fragmented — notably, 'Sweet' (25 occurrences) and 'Sweet, Fruity' (21) appear to be near-duplicates of ordering variants like 'Fruity, Sweet' (6), suggesting inconsistent entry order inflating cardinality artificially. The long-tail alert confirms that most combinations appear very rarely, with the top value covering only 14.3% of rows.

Treatment: Split on comma, normalize order, and one-hot or multi-label encode individual flavor tokens before modelling.

url categorical identifier

This column contains fully-qualified URLs pointing to individual pepper variety pages on pepperscale.com, effectively serving as a unique page identifier for each row. Cardinality is exactly 175 with 175 unique values and a null rate of 0.0, meaning every row has a distinct URL — the top_rate of 0.005714 (1/175) confirms no duplicates whatsoever. Entropy ratio of ~1.0 indicates maximum dispersion, consistent with a perfect natural key. The long_tail alert is technically triggered but is structurally expected given perfect uniqueness.

Treatment: Retain as a unique row key or use as a foreign key to join additional scraped metadata; do not encode or embed for modelling.