data trove megaliths

Overview

Source: /home/coolhand/html/datavis/data_trove/data/quirky/megaliths.json

Saturn profiled 15,464 rows across 14 columns. The stats below are deterministic and machine-readable; the prose is a language-model interpretation of those stats (opt-in, added after the fact, never sees raw rows).

id numeric identifier

This column is a numeric row identifier — every one of the 15,464 rows carries a distinct value with zero nulls, confirming it functions as a unique primary key. The values are large integers spanning roughly 24 million to 13.5 billion, which is consistent with a distributed-system or database auto-increment ID rather than a sequential integer index. Mild positive skew (0.89) and a wide IQR (~4.5 billion) suggest IDs were assigned non-uniformly over time or across sources, but no outliers are flagged.

Treatment: Retain as a join/lookup key; exclude from any feature matrix or model input.

osm_type categorical feature

This column encodes the OpenStreetMap geometry type, distinguishing between point features ('node') and linear/polygon features ('way'). With only 2 distinct values across 15,464 rows and zero nulls, it is a clean binary flag. The distribution is heavily skewed: 'node' dominates at 86.1% (13,311 records) versus 'way' at just 13.9% (2,153 records). The low entropy of 0.582 confirms the imbalance, which may matter if 'way' features behave differently in downstream models.

Treatment: Binary-encode (node=1, way=0) and monitor class imbalance if used as a feature or stratification variable.

name text label

This column contains the local or common name of prehistoric megalithic monuments (dolmens, menhirs, stone circles, nuraghes, etc.), drawing from a multilingual dataset spanning at least English, French, Russian (Cyrillic), and German. Two signals stand out: 30.5% of rows (4,720 of 15,464) are empty strings rather than true nulls, and the duplicate rate is 36.2% (5,595 duplicates), largely driven by generic type-names like 'Долмен' (191), 'Dolmen' (51), 'Menhir' (50), and 'Standing Stone' (48) being reused across many distinct monuments. The one-word rate of 37.8% and word mean of ~2.5 are consistent with short monument names, but the 4,720 empty strings should be treated as missing values.

Treatment: Replace empty strings with NaN, then use as a descriptive label or weak text feature; do not treat as a unique identifier due to 36% duplicate rate and multilingual generic names.

lat numeric feature

This column contains geographic latitude values, ranging from -51.81° to 65.17°, almost certainly representing the latitude coordinate of geolocated records. The vast majority of values cluster in the 43°–51° band (IQR of ~7.6°), suggesting heavy concentration in mid-latitude Europe or North America. The negative skew of -3.09 and extreme kurtosis of 26.33 indicate a sharp central peak with a long left tail — a surprising number of records pull toward lower or even southern-hemisphere latitudes, captured in 134 flagged outliers (~0.87%). Near-uniqueness (15,320 unique out of 15,464 rows) is expected for precise coordinate data.

Treatment: Use as-is or pair with longitude for spatial analysis; investigate the 134 outliers for data-entry errors or genuine remote locations before modelling.

lon numeric feature

This column represents geographic longitude values, with readings spanning from -151.36 to 144.74 degrees — a plausible global range. What is surprising is the severe positive skew (3.65) and extreme kurtosis (34.34), indicating the distribution is heavily concentrated in a narrow band (IQR of only 11.53, centred around Western Europe/Africa longitudes near 0°) with 676 outliers (4.37%) pulled far to the east and west. The mean (2.62) and median (-1.62) diverge noticeably, confirming the asymmetric clustering, likely reflecting a dataset dominated by European locations with a long tail of global outliers.

Treatment: Retain as-is for geo-spatial modelling; investigate and potentially stratify or cap the 676 outlier records before distance-based or regression analyses.

type categorical label

This column classifies archaeological monument types, with 19 distinct categories across 15,464 records and no nulls. It is severely imbalanced: 'megalith' dominates at 97.73% of all records (15,113), leaving the remaining 18 types — menhir, dolmen, standing_stone, stone_circle, nuraghe, etc. — sharing just 351 records. The entropy ratio of 0.049 confirms near-total concentration in one class, which will severely impair any multi-class model trained on this label.

Treatment: Treat as a severely imbalanced categorical label; apply oversampling (SMOTE) or class-weighted losses if used as a target, or collapse rare types into an 'other' bucket for feature use.

megalith_type categorical label

This column classifies prehistoric stone monuments into structural types, with 73 distinct categories across 15,464 records and no nulls. The dominant class is 'menhir' (5,231 records, ~33.8%), followed closely by 'dolmen' (4,501), meaning these two types together account for over 60% of all rows — a moderate concentration reflected in an entropy ratio of 0.44. Notably, the third most frequent value is an empty string ('') with 1,714 occurrences (~11.1%), which masquerades as a non-null entry and represents a meaningful data quality issue that null_rate alone does not capture.

Treatment: Recode empty-string entries as explicit nulls or an 'unknown' category, then one-hot or target-encode for modelling given 73 categories and moderate class imbalance.

description categorical free_text

This column is a text description field for archaeological or heritage site records, containing short labels or names of megalithic structures (e.g., 'Jættestue', 'Großsteingrab', 'Dolmen', 'Stone circle') in multiple languages including Danish, German, Portuguese, and English. The most striking signal is that 95.8% of the 15,464 rows (14,814) carry an empty string, effectively making the field near-empty at scale. The remaining 586 distinct non-empty values are heavily long-tailed, with the most frequent non-empty value ('Jættestue') appearing only 11 times. The entropy ratio of 0.069 confirms extreme imbalance driven by the dominant empty-string value.

Treatment: Treat empty strings as missing; for non-null values, consider as a sparse categorical label or tokenize and embed for similarity/search use cases.

wikipedia text metadata

This column stores Wikipedia article references in a 'language-code:article-title' format (e.g., 'de:Großsteingräber im Haldensleber Forst'), linking dataset records to corresponding Wikipedia pages across multiple languages including German, French, Catalan, Portuguese, and English. The dominant surprise is that 13,060 of 15,464 rows (84.5%) are empty strings, meaning most records have no Wikipedia link at all. Among populated values, 13,406 duplicates exist because the same Wikipedia article is referenced by multiple records — consistent with grouped/list articles covering many individual megalithic sites. The multi-language mix (de, fr, pt, ca, en prefixes visible) is expected for a multilingual cultural-heritage dataset.

Treatment: Parse language prefix and article slug into separate fields; treat empty strings as nulls; use as an optional enrichment join key rather than a model feature.

wikidata text foreign_key

This column stores Wikidata entity identifiers (Q-codes) linking dataset rows to Wikidata knowledge-base entries. Two signals demand immediate attention: 10,819 of 15,464 rows (70%) are empty strings rather than true nulls, and the duplicate_rate is 0.723, meaning many rows share the same Q-code — the top value 'Q106546933' appears 17 times, suggesting a many-to-one entity mapping. The allcaps_rate of 0.300 reflects the uppercase 'Q' prefix on valid codes.

Treatment: Replace empty strings with null, then left-join on this Q-code to enrich with Wikidata properties; expect a many-to-one join given high duplicate rate.

heritage categorical label

This column represents a heritage classification or designation status for records, likely a regulatory or cultural heritage grading field. The dominant 'value' is an empty string, which accounts for 87.96% of all 15,464 rows, indicating that most records carry no heritage designation. The remaining values are a heterogeneous mix of numeric grades (1–4, 7), boolean-style strings ('yes', 'no'), a Portuguese classification phrase ('Em Vias de Classificação'), and a single 'Scheduled Monument' entry — suggesting the column was populated from multiple source systems or locales with no enforced vocabulary.

Treatment: Treat empty strings as a distinct 'undesignated' category; harmonise numeric grades, boolean strings, and foreign-language values into a unified controlled vocabulary before encoding.

heritage_operator categorical label

This column identifies the heritage operator or authority responsible for a record, with 31 distinct coded values across 15,464 rows. The dominant 'value' is an empty string, accounting for 89.5% of all rows (13,848), meaning the vast majority of records have no operator assigned — this blank dominance severely suppresses the entropy ratio to 0.14. Among the 30 non-empty values, 'mhs' (960), 'IE:smr' (229), and 'dgpc' (185) are the most common, suggesting a mix of abbreviated authority codes and occasional full names (e.g., 'Historic Environment Scotland'), indicating inconsistent formatting across sources.

Treatment: Treat empty string as missing; normalise authority codes to a consistent controlled vocabulary before using as a categorical feature or grouping key.

start_date categorical metadata

This column is intended to capture a start date for records, but it is overwhelmingly empty: 15,430 of 15,464 rows (99.78%) contain a blank string, making it nearly useless as a feature. The 34 non-empty values are highly heterogeneous — mixing ISO dates ('2004-07-01'), calendar years ('1999'), approximate historical dates ('~2000 BC'), ranges ('between 3500 and 2800 BCE'), negative year offsets ('-3000 BC'), and even a code-like value ('C-30') — indicating no enforced format or schema. The extreme imbalance (top_rate 0.998) and near-zero entropy (0.032) confirm the column carries almost no information signal.

Treatment: Drop from modelling due to 99.78% blank rate; if historical context is needed, parse and normalise the 34 non-blank values manually before any use.

material categorical feature

This column captures the construction or surface material of a physical feature (likely a wall, path, or structure in a geospatial dataset), with 13 distinct values across 15,464 rows. The dominant 'value' is an empty string, accounting for 98.44% of all records — meaning the field is effectively unpopulated for the vast majority of entries, despite a null_rate of 0.0. The remaining 241 non-empty records span stone-type materials (stone, granite, sandstone, limestone, etc.), with a minor language inconsistency ('Quarzit' appearing in German). Entropy is extremely low (0.133) confirming near-total dominance of the blank value.

Treatment: Treat empty string as missing; recode to NaN, then consider a binary 'has_material' flag or impute/drop depending on task, given 98.44% missingness.