data trove deep sea specimens

Overview

Source: /home/coolhand/html/datavis/data_trove/data/quirky/deep_sea.json

Saturn profiled 200,000 rows across 12 columns. The stats below are deterministic and machine-readable; the prose is a language-model interpretation of those stats (opt-in, added after the fact, never sees raw rows).

scientificName categorical label

This column contains biological taxonomic names (scientific names) spanning multiple ranks — from broad groups like 'Bacteria' and 'Alphaproteobacteria' down to species-level binomials like 'Amperima rosea' and 'Xiphias gladius'. With 1,478 unique values across 200,000 rows and zero nulls, it is a well-populated label field, though the top value 'Alphaproteobacteria' accounts for 8.82% of all rows, indicating moderate concentration at a handful of higher-rank taxa. The high entropy ratio of 0.796 confirms substantial spread across many taxa, and the mix of ranks (class, order, family, genus, species) within a single column is a structural issue that may require rank-disambiguation before analysis.

Treatment: Normalise taxonomic rank before grouping or modelling; consider splitting into rank-specific columns or encoding hierarchy via a taxonomy backbone.

species categorical label

This column contains biological species names (binomial Latin nomenclature), likely from a marine/oceanographic observation dataset given taxa such as swordfish (Xiphias gladius), Atlantic mackerel (Scomber scombrus), and deep-sea holothurians (Amperima rosea). The dominant 'value' is an empty string, accounting for 73.2% of all 200,000 rows (146,400 records), which is a critical data quality issue — species was not recorded for nearly three-quarters of observations. Among the 678 distinct non-empty values, entropy ratio is only 0.33, indicating heavy concentration in a handful of species.

Treatment: Treat empty string as missing/unknown; impute or filter before modelling, then encode remaining 677 species (e.g. target-encode or embed taxonomic hierarchy) given high cardinality.

genus categorical label

This column contains biological genus names, likely from a marine species observation or biodiversity dataset given genera such as Xiphias (swordfish), Thunnus (tuna), Prionace (blue shark), and deep-sea taxa like Amperima and Chrysogorgia. The most striking issue is that 54.9% of rows (109,800 of 200,000) carry an empty string rather than a null, meaning missingness is systematically masked and will not be caught by standard null checks. The remaining 840 distinct genus values show moderate entropy (4.90, entropy_ratio 0.50), indicating a reasonably spread but skewed distribution.

Treatment: Replace empty-string values with NaN to expose true missingness (~55%), then use as a categorical label or grouping key; consider hierarchical encoding with higher taxonomic ranks if available.

family categorical label

This column contains biological family-level taxonomic names (e.g., Nitrosopumilaceae, Elpidiidae, Scombridae), consistent with a marine species or biodiversity dataset. The most striking issue is that the top value is an empty string, accounting for 40.18% of all 200,000 rows (80,360 records) — a substantial proportion of missing taxonomy that is masked by a null_rate of 0.0, meaning blanks were stored as empty strings rather than true nulls. With 606 unique values and moderate entropy (5.50), the remaining distribution is fairly spread but dominated by a handful of families.

Treatment: Replace empty strings with NaN, then use as a categorical grouping variable or encode (e.g., target/ordinal encoding) for modelling; investigate whether blank family records can be imputed from genus/species columns.

order categorical label

This column represents the biological taxonomic rank 'Order' for marine organisms, containing 310 distinct taxonomic order names drawn from bacteria, archaea, protists, invertebrates, and fish. The most striking signal is that the top value is an empty string, accounting for 28.16% of all 200,000 rows (56,320 records) — suggesting a substantial proportion of specimens could not be classified at this rank, which is common in metagenomic or environmental sampling datasets. The entropy ratio of 0.659 indicates moderate diversity across the 310 categories, with a long tail of rarer orders beneath the dominant few.

Treatment: Treat empty string as a distinct 'unclassified' category or null before encoding; encode remaining values as nominal categories (one-hot or target encoding depending on model).

class categorical label

This column contains biological taxonomic class-level classifications (e.g., Alphaproteobacteria, Teleostei, Octocorallia), consistent with a marine or environmental biodiversity dataset. With 138 unique values across 200,000 rows, cardinality is moderate and entropy is reasonably high (4.89, ratio 0.69), indicating a fairly broad but not flat distribution. The top value 'Alphaproteobacteria' accounts for 11.42% of records, suggesting mild concentration at the top. A notable concern is the second-most-frequent entry being an empty string ('') with 21,920 occurrences (≈11% of rows), which likely represents missing or unclassified taxa rather than a true category and should be treated as null.

Treatment: Replace empty-string entries with null, then encode as a nominal categorical feature (e.g., target-encode or embed) for modelling.

phylum categorical label

This column contains biological phylum classifications, covering 65 distinct phyla across 200,000 rows with no nulls. The dominant value is Proteobacteria (17.74%), followed by Cnidaria and Chordata, suggesting a marine or mixed ecological dataset spanning bacteria, invertebrates, and vertebrates. Notably, empty strings appear as the 5th most frequent 'value' with 13,200 occurrences (6.6%), which are functionally missing values masked as non-null entries. The entropy ratio of 0.68 indicates moderate concentration — a few phyla dominate while the long tail of 65 categories is spread unevenly.

Treatment: Replace empty-string entries (13,200 rows) with explicit nulls, then encode as a categorical feature (e.g., ordinal or one-hot for low-cardinality models).

latitude numeric feature

This column contains geographic latitude values, ranging from -75.0 to 89.06 degrees, consistent with a near-global spatial dataset. With only 2,617 unique values across 200,000 rows, each distinct latitude is reused on average ~76 times, suggesting coordinates are discretized or snapped to a coarse grid rather than recorded at full precision. The distribution is nearly symmetric (skew 0.12) with a platykurtic shape (kurtosis -1.22) and an IQR of ~72 degrees, indicating broad global coverage with no strong concentration in any hemisphere. No nulls, no outliers, and zero_rate of 0.0 are all clean signals.

Treatment: Pair with longitude for spatial joins or clustering; investigate grid resolution given only 2,617 unique values across 200,000 rows before treating as continuous.

longitude numeric feature

This column represents geographic longitude, with values spanning the full valid range from approximately -180 to +180 degrees and a mean of -51.58, suggesting a dataset skewed toward the Western Hemisphere (median -94.29, well west of the prime meridian). The IQR of 233.75 and near-flat kurtosis (-1.12) indicate values are broadly spread across the globe with no sharp central peak — consistent with global or multi-continental coverage. Surprisingly, only 2,654 unique values exist across 200,000 rows, implying heavy coordinate quantization or snapping to a fixed grid rather than continuous GPS precision. The zero_rate of 0.0004 is negligible but worth monitoring as a potential null-proxy sentinel.

Treatment: Use as-is for spatial joins or map projections; investigate coordinate quantization (2654 unique values over 200000 rows) before using as a continuous feature in regression.

depth numeric feature

This column almost certainly represents depth measurements (e.g., ocean depth, well depth, or seismic depth) in meters, ranging from 1,000 m to 11,000 m with a mean of ~2,406 m and median of ~1,962 m. The distribution is right-skewed (skew ≈ 1.09) with a wide IQR of 2,174 m, indicating most observations cluster at shallower depths while a tail extends toward very deep values — the maximum of 11,000 m is consistent with ocean trench depths. Only 1,938 unique values across 200,000 rows suggests the depth values are rounded or binned rather than continuous measurements, which is worth noting for precision-sensitive analyses. Outlier count is modest (560, 0.28%) and no nulls or zeros are present.

Treatment: Apply log-transform or quantile transformation before regression/modelling to reduce right skew; verify whether discretisation into 1,938 unique values is intentional.

year numeric feature

This column represents a calendar year, spanning from 1875 to 2024 with 98 distinct values across 200,000 rows. The bulk of records cluster tightly between 2004 and 2016 (IQR of 12 years, median 2016), but the distribution is heavily left-skewed (skew = -3.57, kurtosis = 19.65), driven by a long tail of historically old entries stretching back to 1875. Roughly 6.3% of rows (12,080) are flagged as outliers, almost certainly the pre-20th/early-20th century records that sit far below the modern core; analysts should verify whether these antique years are legitimate data or encoding errors.

Treatment: Inspect and potentially cap or bin pre-1950 outlier years; use as an ordinal/numeric feature or derive 'age relative to reference year' before modelling.

country categorical feature

This column represents the country of origin or registration for records in the dataset, with 57 distinct values across 200,000 rows. The dominant signal is that 51.92% of rows (103,840) have an empty string — effectively a missing value masked as a non-null entry, which would go undetected by null checks. Among populated values, 'Australia' accounts for 79,320 rows (~39.7%), making this a heavily Australia-centric dataset; a further data quality concern is the presence of both 'United States' (8,160) and 'USA' (680) as separate values, indicating inconsistent country name standardisation.

Treatment: Replace empty strings with null, then standardise country name variants (e.g. 'USA' → 'United States') before encoding or aggregating.