deepsky ngc

Overview

Source: /home/coolhand/data/celestial/deepsky/NGC.csv

Saturn profiled 13,969 rows across 32 columns. The stats below are deterministic and machine-readable; the prose is a language-model interpretation of those stats (opt-in, added after the fact, never sees raw rows).

Name text identifier

This appears to be an astronomical object designation field — short, all-caps, single-token codes like 'NED01', 'NED02', and 'IC' prefixed catalog numbers. Every one of the 13,969 rows is unique with zero nulls, lengths tightly bounded between 6 and 13 characters, and 97.4% are single-word tokens. The 'NED01'/'NED02' values recurring 175 and 168 times sit oddly against the n_unique=13969 claim, suggesting these are prefixes/substrings counted at the word level rather than full duplicates.

Treatment: Treat as a primary key; drop from modelling features and use only for joins or lookups.

Type categorical label

A categorical type code with 20 distinct values across 13,969 rows and no nulls. The distribution is highly imbalanced: 'G' alone accounts for 75.0% of records (10,481), with the next categories ('OCl', 'Dup', '*', 'Other') each below 5%, yielding an entropy ratio of just 0.38. The codes ('G', 'OCl', 'GPair', 'GCl', 'PN', 'Neb') suggest astronomical object classifications (galaxies, open/globular clusters, planetary nebulae, nebulae).

Treatment: Group rare categories or stratify by 'G' dominance before any classification task.

RA unknown other

The column is named RA, which in astronomical datasets typically denotes Right Ascension, but saturn skipped profiling so kind and uniqueness are unresolved. The only confirmed signals are 13969 rows with a 0.0 null rate; no distributional statistics are available.

Treatment: Re-profile with an explicit type cast before deciding on use.

Dec text feature

This column holds astronomical Declination coordinates formatted as signed sexagesimal strings (e.g. '-19:28:17.6'), with every value 9-11 characters long and exactly one token. It is near-unique (13,282 distinct of 13,969) yet shows 680 duplicates (4.87%), suggesting repeated observations of the same sky positions. The 'allcaps' and Flesch=121 signals are artefacts of the numeric format, not real prose.

Treatment: Parse the sexagesimal string into signed decimal degrees before any numeric use.

Const categorical feature

This column holds three-letter constellation abbreviations (Vir, Com, Leo, Cet, UMa…), with 89 distinct values across 13,969 rows — consistent with the 88 IAU constellations plus possibly one stray code. Distribution is fairly even (entropy ratio 0.85), though Virgo leads at 8.85% and the top three constellations account for roughly a fifth of records. Nulls are negligible (0.05%).

Treatment: One-hot or target-encode as a categorical feature.

MajAx numeric feature

MajAx is a numeric measurement (likely a major-axis length, e.g. of a galaxy or ellipse) with a tight central distribution—median 1.2, IQR 0.82–1.87—but an extreme right tail reaching 299.92. Skew of 21.89 and kurtosis of 641.85 are both severe, and 10.36% of values flag as outliers. Null rate is also non-trivial at 14.06%.

Treatment: log-transform and impute missing before modelling; consider winsorising the upper tail.

MinAx numeric feature

MinAx is a numeric measurement (likely a minor-axis dimension) with a tight central mass — median 0.69, IQR 0.45–1.07 — but an extreme right tail stretching to 179.89. Skew of 26.82 and kurtosis near 981 are extraordinary, and 760 outliers (6.88%) sit alongside a 20.91% null rate. Only 465 distinct values across 13,969 rows suggests rounding or a discretised scale.

Treatment: Impute or flag the 20.91% nulls and apply a log transform before modelling to tame the heavy right tail.

PosAng numeric feature

PosAng is a numeric column bounded between 0 and 180 with mean 87.27 and median 87, consistent with a position angle in degrees (a common astronomical measurement). The distribution is nearly symmetric (skew 0.047) and platykurtic (kurtosis -1.21), spread broadly across the range with IQR 40-133 and no outliers. Notable: 23.17% of rows are null, and despite n=13969 there are only 181 unique values, suggesting integer-degree quantisation.

Treatment: Treat as a circular/angular feature (e.g. encode as sin/cos) and impute or flag the 23% missingness before modelling.

B-Mag numeric feature

B-Mag is a numeric photometric measurement, almost certainly the B-band apparent magnitude of astronomical sources, with values concentrated between 13.42 and 15.2 (median 14.42). The distribution is strongly left-skewed (skew -1.69, kurtosis 5.58) with a long faint-end tail down to 1.51 and 572 outliers (5.05%). Roughly 18.86% of rows are null, so a sizeable share of objects lack a B measurement.

Treatment: Impute or flag the 18.86% missing values and consider winsorising the bright-end tail before modelling.

V-Mag numeric feature

V-Mag is a numeric column almost certainly recording visual (apparent) magnitude of astronomical objects, with values spanning 1.69 to 20.41 and a median of 12.38 consistent with that scale. The distribution is left-skewed (skew -1.22) with 323 outliers (7.66%) on the bright end, and 69.83% of rows are null, so usable coverage is limited to roughly 30% of the catalogue. The interquartile band is tight (11.31-13.28, IQR 1.97) around faint magnitudes.

Treatment: Impute or flag the 69.83% missing values before modelling; consider keeping raw scale since magnitudes are already logarithmic.

J-Mag numeric feature

This is the J-band magnitude (near-infrared photometry, ~1.25 μm) for catalogued sources, with values centered near 11.42 and spanning 1.11 to 17.02. The distribution is mildly left-skewed (-0.55) with modest excess kurtosis (2.51) and 333 outliers (3.45%), consistent with a mix of bright and faint sources. A notable concern is the 30.9% null rate, meaning nearly a third of rows lack a J-Mag measurement.

Treatment: Impute or flag missing values before modelling; consider robust scaling given the left skew and outliers.

H-Mag numeric feature

H-Mag is a numeric astronomical magnitude (likely absolute H-magnitude, common in asteroid/minor-planet catalogs), ranging from 0.83 to 16.67 with a mean of 10.70 and median 10.74. The distribution is mildly left-skewed (-0.45) with light tails (kurtosis 2.18) and a tight IQR of 1.55, indicating most objects cluster around magnitude 10-11.5. The notable concern is a 30.6% null rate, plus 341 low-end outliers (3.5%) representing unusually bright objects.

Treatment: Impute or flag the 30.6% missing values before modelling; consider keeping raw scale since skew is mild.

K-Mag numeric feature

Numeric K-band magnitude readings (likely 2MASS K-mag photometry), centered around a median of 10.46 and ranging from 0.72 to 15.76. The distribution is mildly left-skewed (-0.45) with 315 outliers (3.3%) and a notable 30.9% null rate, suggesting many sources lack K-band coverage. Spread is tight (std 1.36, IQR 1.57) across 823 unique values.

Treatment: Impute or flag the 31% missing values before modelling; no transform needed given near-symmetric spread.

SurfBr numeric feature

SurfBr is a numeric measurement tightly clustered around 23.31 (median 23.33) with a narrow IQR of 0.71 and standard deviation 0.61, consistent with a surface brightness magnitude. The distribution is mildly left-skewed (-0.30) with elevated kurtosis (2.65) and 288 outliers (2.8%), and roughly 27% of rows are null which is a notable coverage gap.

Treatment: Impute or flag the 27% missing values before modelling; no transform needed given the tight, near-symmetric spread.

Hubble categorical label

Hubble appears to be the Hubble morphological classification of galaxies, with familiar types like E (elliptical), S0 (lenticular), and the spiral sequence Sa/Sb/Sc dominating. Across 13,969 rows it has 30 distinct codes and high entropy ratio (0.83), so the type distribution is fairly spread rather than concentrated — the top value E only accounts for 15.2%. Notably, 27.26% of rows are null, meaning over a quarter of galaxies lack a morphological label.

Treatment: Treat as categorical label; impute or filter the 27% nulls before any class-conditional analysis.

Pax numeric feature

Pax is a sparse numeric measurement, populated for only ~5% of rows (null_rate 0.9485) with values ranging from 0.003 to 22.8 and a median of 0.4829. The distribution is severely right-skewed (skew 7.21, kurtosis 68.87) with 65 outliers among the non-null values, and the mean (0.919) sits well above the median, indicating a long upper tail. With 676 unique values across 13,969 rows and no zeros, this looks like a continuous rate or ratio observed only on a small subpopulation.

Treatment: Log-transform and add a missingness indicator before modelling, given the 94.85% null rate and heavy right skew.

Pm-RA numeric feature

Pm-RA is almost certainly a proper-motion-in-right-ascension measurement (mas/yr) for catalog objects, with values centered slightly negative (median -0.9015, mean -1.374) and a tight IQR of 3.54. Two things stand out: 92.53% of rows are null, suggesting this is only populated for a subset of sources, and the distribution has heavy tails (skew 2.66, kurtosis 50.19) with extremes from -43.57 to 76.0 and a 6.8% outlier rate.

Treatment: Impute or flag the 92.53% missingness explicitly and winsorize/robust-scale before modelling.

Pm-Dec numeric feature

Pm-Dec is almost certainly proper motion in declination (mas/yr), a sparse astrometric feature with 92.53% nulls and only 1044 populated rows out of 13969. Values center near zero (median -1.088, mean -1.402) but span -58.961 to 64.0 with std 5.9, kurtosis 43.8, and 74 outliers (7.09% of non-null), indicating heavy tails typical of high-proper-motion sources.

Treatment: Impute or mask the 92.53% missing before modelling, and consider a robust scaler given the heavy tails.

RadVel numeric feature

RadVel reads as a radial velocity measurement (likely km/s or m/s) for ~14k objects, with values ranging from -483 to 52,025 and a median of 4,885. The distribution is right-skewed (skew 1.53, kurtosis 5.16) with 341 outliers (3.2%) and a notable 24.2% null rate flagged as an alert. Only 6,691 unique values across 13,969 rows suggests some repeated/quantized measurements.

Treatment: Impute or flag the 24% missing values and consider a robust scaler given the right skew and outliers.

Redshift numeric feature

Redshift values cluster tightly between q1 0.008056 and q3 0.02579 with a median of 0.01643, consistent with cosmological redshift measurements for relatively nearby objects. The distribution is right-skewed (skew 1.65, kurtosis 6.29) with a max of 0.191616 and a small negative min of -0.00161, and 24.24% of rows are null.

Treatment: Impute or filter the 24% nulls and consider a log1p transform before modelling to tame the right skew.

Cstar U-Mag numeric feature

Cstar U-Mag appears to be the U-band magnitude of a central/companion star, populated for only ~0.11% of rows (null_rate 0.9989) — just 16 unique non-null values across 13969 records. Where present, values span 9.3 to 14.75 with mean 11.93 and median 12.09, roughly symmetric (skew 0.09) and platykurtic (kurtosis -1.12), with no flagged outliers. The extreme sparsity is the dominant signal and limits any aggregate use.

Treatment: Drop or treat as a presence indicator; too sparse (99.89% null) for direct modelling.

Cstar B-Mag numeric feature

Apparent B-band magnitude of a companion star (Cstar B-Mag), populated for only 0.79% of rows (null_rate 0.9921). Among the 111 non-null values there are just 97 unique magnitudes spanning 9.93 to 21.1, with mean 15.23 and median 15.5, roughly symmetric (skew -0.17) and no flagged outliers.

Treatment: Treat as sparse optional feature; impute or model missingness explicitly rather than relying on the value.

Cstar V-Mag numeric feature

Visual magnitude of a companion star ('Cstar V-Mag'), populated for only ~0.7% of rows (null_rate 0.9931). The 96 non-null values span 9.42 to 19.6 with median 15.145 and a roughly symmetric distribution (skew -0.15, kurtosis -0.36), consistent with stellar photometry on the magnitude scale. Severe sparsity is the dominant signal—this field is only meaningful for systems where a companion star was characterized.

Treatment: Treat as optional astrophysical feature; impute with a missing-indicator or drop given >99% nulls.

M categorical identifier

Column M is a sparsely populated categorical code, present in only ~0.77% of the 13969 rows (null_rate 0.9923). Among the 107 non-null values, every one appears exactly once (top_rate ≈ 0.0093, entropy_ratio ≈ 1.0), so each observation is unique — values are zero-padded 3-digit strings like '024','025','110'. The combination of 99.23% nulls and perfect uniqueness on the remainder suggests an incidental tag or sub-identifier rather than a usable feature.

Treatment: Drop; near-unique values with >99% nulls offer no modelling signal.

NGC categorical identifier

This is an NGC (New General Catalogue) astronomical object identifier, populated for only 6.5% of rows (null_rate 0.935). Among the 908 non-null entries it spans 891 unique codes with near-maximal entropy_ratio 0.999 and a top value '3497' appearing just 3 times (top_rate 0.0033), so it behaves almost like a sparse identifier. Some codes carry letter suffixes (e.g. '5619B'), confirming it's a catalog string rather than a clean integer.

Treatment: Treat as a sparse cross-reference key; drop or left-join to an NGC catalog rather than using as a model feature.

IC categorical foreign_key

IC appears to be a sparse categorical code, likely an industry/identifier classification, populated for only ~3.3% of rows (null_rate 0.9671). Among the 460 non-null entries it spreads across 452 distinct values with near-maximal entropy (entropy_ratio 0.9984) and a top frequency of just 3 (top_rate 0.0065), so essentially every present value is unique. This combination of overwhelming nulls and near-unique codes makes it unusable as a grouping feature without enrichment.

Treatment: Drop or treat as a sparse lookup key; do not use as a categorical feature given 96.7% nulls and near-unique values.

Cstar Names categorical identifier

Companion-star catalogue identifiers (HD/BD designations), occasionally listing multiple names comma-separated. Effectively empty: 99.38% null with only 87 distinct values across 13,969 rows, each appearing once (top_rate 0.0115, entropy_ratio ~1.0).

Treatment: Drop or retain only as a cross-reference key; too sparse and unique to model.

Identifiers text identifier

This column holds astronomical object identifiers, with 12,179 unique values across 13,969 rows and almost everything in uppercase (allcaps_rate 0.998). The token distribution reveals catalog prefixes like 2MASX (9,805), MWSC, ESO, MCG, SDSS, and PGC, suggesting each cell concatenates cross-catalog designations (word_mean 5.26). Note the 12.81% null rate and that the column is near-unique, so it carries little aggregate signal on its own.

Treatment: Treat as a join key to external catalogs; do not use as a model feature.

Common names categorical metadata

Vernacular labels for astronomical objects (e.g., 'Antennae Galaxies', 'Flame Nebula'), occasionally comma-concatenating multiple aliases in one cell. The column is 99.06% null across 13,969 rows, leaving only ~127 unique populated values with near-uniform distribution (entropy_ratio 0.998, top_rate 1.5%). Multi-name cells like 'Butterfly Galaxies,Siamese Twins' indicate the field is a delimited list rather than a clean category.

Treatment: Treat as sparse free-form labels: split on comma into a name list and use only for display or lookup, not modelling.

NED notes text metadata

Short astronomical annotations from the NGC/IC NED catalog, averaging 6.7 words and capped at 80 characters, describing object positions and identification caveats (e.g. 'In the Large Magellanic Cloud.', 'Confused HIPASS source'). 83.6% of rows are null and only 1,198 distinct strings appear across 13,969 rows, with a 47.6% duplicate rate driven by a small set of canned remarks. Language detection flags the field as multilingual but 780 of 783 detected samples are English; the bs/it/pt hits are almost certainly false positives on terse astronomy jargon.

Treatment: Treat as sparse categorical-ish notes: bucket the top recurring phrases as flags and ignore the long tail rather than embedding free text.

OpenNGC notes categorical free_text

Free-text curator notes on OpenNGC catalog entries, present for only 1.52% of rows (null_rate 0.9848). Among the 213 populated cells there are 159 distinct strings with high entropy (6.72, ratio 0.92), and the most common note — 'Identification taken from Corwin's catalog.' — covers just 13.6% of non-nulls. Content is heterogeneous provenance commentary referencing LEDA, NED, SIMBAD, and Corwin sources rather than a controlled vocabulary.

Treatment: Treat as sparse free-text metadata; drop for modelling or keep as a provenance flag (note present vs absent).

Sources categorical metadata

This column encodes a per-row provenance manifest: a pipe-delimited list of astronomical measurement fields (Type, RA, Dec, Const, magnitudes, redshift, etc.) each tagged with a numeric source/catalog code. Despite 344 distinct combinations across 13,969 rows, the distribution is highly concentrated — the top template covers 41.2% of rows and entropy ratio is just 0.41, confirming the long_tail alert. There are no nulls, but the structured-string format means it is not directly usable as a category without parsing.

Treatment: Parse into per-field source codes (one column per measurement) rather than treating the raw string as a category.