data trove bioluminescence

Overview

Source: /home/coolhand/html/datavis/data_trove/data/quirky/bioluminescence.json

Saturn profiled 43,060 rows across 14 columns. The stats below are deterministic and machine-readable; the prose is a language-model interpretation of those stats (opt-in, added after the fact, never sees raw rows).

scientificName categorical label

This column contains scientific (Latin) names of marine organisms, covering 245 distinct taxa across 43,060 records with no nulls. The values span both genus-only entries (e.g., 'Lingulodinium', 'Photobacterium', 'Vibrio') and full binomial species names, suggesting inconsistent taxonomic resolution across records. The top value 'Mnemiopsis leidyi' appears 2,000 times (~4.6% of rows), and the top 10 values together account for a substantial share of records, indicating the dataset is dominated by a relatively small set of species. Entropy ratio of 0.747 confirms moderate-to-high concentration for a 245-cardinality field.

Treatment: Standardise taxonomic resolution (genus vs. species) before grouping; one-hot or target-encode for modelling.

genus categorical label

This column contains biological genus names for marine organisms — including bioluminescent dinoflagellates (Noctiluca, Pyrocystis, Lingulodinium, Alexandrium), jellyfish (Pelagia, Atolla, Periphylla), and ctenophores (Mnemiopsis, Beroe) — suggesting a marine biology or bioluminescence dataset. With 27 unique genera across 43,060 rows and no nulls, the distribution is remarkably flat: every visible top value appears exactly 2,000 times, implying a deliberately balanced or stratified dataset. The entropy ratio of 0.9586 is very high for only 27 categories, confirming near-uniform representation across genera. No skew or imbalance alerts were triggered.

Treatment: Use as a classification target or stratification key; one-hot encode or ordinal encode for modelling given 27 balanced classes.

family categorical label

This column contains biological family-level taxonomic classifications, covering 22 distinct families across 43,060 records with no nulls. The distribution is notably near-uniform for the top entries: four families (Pyrocystaceae, Euphausiidae, Cypridinidae, Vibrionaceae) each have exactly 4,000 records, strongly suggesting deliberate stratified sampling or synthetic balancing rather than natural occurrence frequencies. The high entropy ratio of 0.932 (close to maximum for 22 categories) confirms an unusually flat distribution. Families represented include bioluminescent marine organisms (dinoflagellates, krill, ostracods, bacteria), hinting this is a marine bioluminescence or plankton dataset.

Treatment: Use as a categorical grouping variable or classification target; encode with integer or one-hot encoding, noting the artificially balanced class distribution may not reflect real-world priors.

phylum categorical label

This column encodes biological phylum classifications across 43,060 records with exactly 7 distinct values and no nulls, making it a clean taxonomic label field. The distribution spans both animal (Arthropoda, Cnidaria, Ctenophora, Mollusca, Annelida) and non-animal (Myzozoa, Proteobacteria) kingdoms, suggesting the dataset covers a broad range of marine or environmental organisms. Arthropoda dominates at 28.6% (12,297 records), while the entropy ratio of 0.923 indicates a fairly well-spread distribution across categories. The presence of Proteobacteria (bacteria) and Myzozoa (protists) alongside metazoans may surprise analysts expecting a purely animal-focused dataset.

Treatment: One-hot encode or use as a stratification/grouping variable; verify whether cross-kingdom mixing is intentional before modelling.

class categorical label

This column contains biological taxonomic class labels for marine/aquatic organisms, with 13 distinct classes across 43,060 records and zero nulls. The entropy ratio of 0.927 indicates a near-uniform distribution across classes, though mild imbalance exists: 'Dinophyceae' is the most frequent at 18.6% (8,000 records), while several classes like 'Scyphozoa' and 'Malacostraca' each hold exactly 6,000 records, suggesting some classes may have been deliberately sampled to round numbers. The mix spans protists (Dinophyceae, Gammaproteobacteria), crustaceans (Malacostraca, Ostracoda, Copepoda), molluscs (Cephalopoda), and cnidarians/ctenophores (Scyphozoa, Hydrozoa, Nuda, Tentaculata), pointing to a plankton or marine biodiversity dataset.

Treatment: Use as classification target or stratification key; check class imbalance before modelling and consider stratified splits given the ~2× spread between largest and smallest classes.

order categorical label

This column contains biological taxonomic order classifications for marine organisms, with 17 distinct orders spanning bacteria (Vibrionales), dinoflagellates (Gonyaulacales, Noctilucales), jellyfish (Coronatae, Leptothecata), crustaceans (Euphausiacea, Calanoida), and cephalopods (Oegopsida) among others. The distribution is moderately uneven — Gonyaulacales dominates at 13.9% (6,000 rows) while several orders share exactly 4,000 rows, suggesting possible stratified or quota-based sampling rather than natural observation frequencies. Entropy ratio of 0.949 indicates near-uniform spread across the 17 classes, which is unusually high for a taxonomic label and reinforces the structured-sampling hypothesis. No nulls are present.

Treatment: Use as a categorical target or stratification variable; one-hot encode or ordinal-encode for modelling, and verify whether the equal-count groupings (4,000 each) reflect intentional sampling design.

latitude numeric feature

This column contains geographic latitude values, spanning from -76.619° (deep Southern Hemisphere) to 88.29° (near the North Pole), with 14,146 unique values across 43,060 rows suggesting many repeated locations. The mean (19.1°) is notably lower than the median (36.7°), and the IQR of 69.6° spans nearly the full usable latitude range, indicating records are spread across both hemispheres rather than clustered in any one region. The slight negative skew (-0.66) and near-zero outlier rate confirm a broadly spread but reasonably uniform distribution, which is unusual — most real-world geo datasets cluster in populated mid-latitude bands. The 0.046% zero-rate (≈20 rows) warrants inspection as 0.0° latitude may represent missing/default values.

Treatment: Verify ~20 zero-latitude rows for data quality; use as-is or pair with longitude for spatial joins/clustering; consider binning into latitude bands if used as a categorical feature.

longitude numeric feature

This column contains geographic longitude values, spanning the full valid range from -179.9987 to 179.99 degrees, confirming global coverage. The distribution is remarkably flat and near-symmetric (skew 0.138, kurtosis -0.646) with an IQR of 124.12 degrees, indicating records are spread broadly across both hemispheres rather than clustered in any region. The mean (9.64) is notably higher than the median (3.06), hinting at a slight eastward bias in the dataset. A zero_rate of 0.11% warrants a check for null-substituted zeros masquerading as the Prime Meridian.

Treatment: Use as-is for geospatial joins or pair with latitude for coordinate-based features; verify ~48 zero-value rows are genuine Prime Meridian locations and not null substitutions.

depth numeric feature

This column represents a physical depth measurement (e.g., depth below surface for geological, seismic, or oceanographic observations), ranging from -53.0 to 10,000.0 with a median of only 52.5 — meaning most records are shallow, but a long tail of deep measurements drives extreme skew (4.72) and very high kurtosis (35.89). The null rate of 24.75% and outlier rate of 10.63% (3,444 rows) are both flagged as alerts, and 11.92% of values are exactly zero, suggesting possible default-fill or surface-level records that may need special handling. The IQR of 313.5 against a std of 570.18 confirms the heavy-tailed distribution driven by a minority of extreme deep values up to 10,000.0.

Treatment: Impute nulls carefully (median preferred over mean given skew), investigate zero values for validity, then log-transform or apply a signed-log transform before modelling to reduce the heavy right tail.

date text timestamp

This column contains publication or recording date strings stored as text, using multiple formats including year ranges ('1962/1964'), year-month ranges ('2010-05/2010-06'), year-month ('2013-08'), and full ISO dates ('2017-05-30'). The heterogeneous format mix and wide length range (min 4, max 51 chars) will require normalization before any temporal analysis. A 67.4% duplicate rate across 43,060 rows with only 12,338 unique values indicates many records share the same date, consistent with batch or periodical publication data. The 12.0% null rate is also notable and should be investigated for systematic missingness.

Treatment: Parse and normalize the mixed date formats (range, year-month, ISO) into structured start/end date fields before use in temporal analysis or modelling.

year categorical feature

This column represents a calendar year associated with each record, stored as a categorical type despite being numeric in nature. The top value is '2000' with 1,287 occurrences (~5.2% of total), and the 137 unique values span a range that includes at least 1979 through 2016. Two signals stand out: the null rate is 42.18%, which is severe enough to warrant an alert, and the year 2000 is notably over-represented relative to adjacent years (e.g., 2001 has only 703), suggesting either a data collection artifact or a large batch of undated records defaulted to that year.

Treatment: Investigate the 2000 spike for default-value contamination, impute or flag the 42.18% nulls, then cast to integer for temporal modelling.

country categorical feature

This column captures the country associated with each record, with 130 distinct values across 43,060 rows. The dominant 'value' is an empty string, accounting for 63.7% of all records (27,422 rows) — a critical data quality issue that functionally resembles a very high null rate. The remaining values show inconsistent normalisation: mixed casing ('PERU', 'SOVIET UNION' vs. 'Australia', 'Canada'), abbreviations ('GB' instead of 'United Kingdom'), and anachronistic entities ('SOVIET UNION'), suggesting data was collected from heterogeneous or historical sources without standardisation.

Treatment: Treat empty strings as nulls, then standardise to ISO 3166-1 country codes before modelling or aggregation.

dataset categorical metadata

This column identifies the source dataset or survey program for each observation, with 214 distinct named sources across 43,060 rows. The dominant value is an empty string, accounting for 61.1% of all records (26,317 rows), meaning the majority of observations carry no dataset attribution — a significant data quality concern. The remaining records span named marine/environmental monitoring programs (trawl surveys, coastal monitoring, jellyfish sightings, etc.), with the largest named source ('Environmental Monitoring database (MOD) DNV') covering only ~4% of rows.

Treatment: Treat empty string as missing/unknown; encode named values as a categorical feature or use as a grouping/stratification variable, but investigate whether the 61.1% blank rate reflects a systematic data gap before modelling.

bioluminescence_group categorical label

This column classifies observations into one of 26 named bioluminescent organism groups (e.g., 'Dinoflagellate', 'Crystal jelly (source of GFP)', 'Krill (many species bioluminescent)'), covering marine taxa from dinoflagellates to jellyfish and crustaceans. The distribution is remarkably uniform: the top category 'Dinoflagellate' holds only 9.3% of rows, and the entropy ratio of 0.95 (near-maximum for 26 categories) indicates near-flat class balance. With no nulls across 43,060 rows and exactly 2,000 records for all visible non-top categories, the dataset appears deliberately balanced or synthetically constructed.

Treatment: Use as classification target or stratification variable; near-perfect class balance means no resampling required.