data trove nyc housing analysis

Overview

Source: /home/coolhand/html/datavis/data_trove/economic/housing/nyc/nyc_housing_metrics_merged.csv

Saturn profiled 2,327 rows across 23 columns. The stats below are deterministic and machine-readable; the prose is a language-model interpretation of those stats (opt-in, added after the fact, never sees raw rows).

total_renter_households numeric feature

This column counts the total number of renter-occupied households per geographic unit (likely census tract or ZIP code), with values ranging from 0 to 8,209. The distribution is notably right-skewed (skew 1.59) with excess kurtosis of 4.63, indicating a long tail of high-density rental areas pulling the mean (946) well above the median (726). About 4.4% of records are zero, suggesting some areas have no renters at all, and 69 outliers (~3%) represent unusually dense rental markets that may warrant separate treatment.

Treatment: Log-transform (or log1p to handle zeros) before regression or distance-based modelling to reduce skew impact.

rent_30_to_34_9_pct numeric feature

This column represents a count (or weighted count) of households paying 30–34.9% of income toward rent — a standard housing cost-burden threshold bracket. The distribution is severely right-skewed (skew=2.76, kurtosis=13.86), with a median of 51 but a mean of 83 and a max of 1205, indicating a long tail driven by high-density geographies. About 16.2% of rows are zero (areas with no such households or suppressed data), and 124 observations (~5.3%) are flagged as outliers, suggesting a mix of very small and very large geographic units in the dataset.

Treatment: Log-transform (log1p) before modelling to address severe right skew; consider normalising by total renter households to convert counts to rates.

rent_35_to_39_9_pct numeric feature

This column represents the count (or estimated count) of renter households paying 35–39.9% of their income on rent — a standard housing cost-burden bracket used in Census/ACS tabulations. The distribution is severely right-skewed (skew 2.40, kurtosis 9.27), with a median of 35 but a mean of 58 and a maximum of 633, indicating that most geographic units have low counts while a small number of populous areas drive extreme values. Nearly 20% of rows are zero, suggesting many small geographies have no households in this rent-burden band, and 110 observations (4.7%) qualify as outliers. The IQR of 73 against a median of 35 confirms substantial spread relative to the typical value.

Treatment: Log-transform (log1p) before regression or clustering to reduce skew and compress outlier influence.

rent_40_to_49_9_pct numeric feature

This column represents a count (or estimate) of renter households spending 40–49.9% of their income on rent within some geographic unit, likely a census tract or block group. The distribution is heavily right-skewed (skew=2.14, kurtosis=7.14) with a median of 49 but a mean of 74.68 and a maximum of 740, indicating a small number of high-density areas pull the mean far above the typical value. About 15.6% of rows are zero—geographic units with no households in this rent-burden band—and 111 observations (4.8%) are flagged as outliers, consistent with urban concentration effects.

Treatment: Log-transform (log1p) to reduce skew before regression or clustering; retain zeros as valid observations representing absence of this rent-burden group.

rent_50_pct_or_more numeric feature

This column likely represents a count of renter households spending 50% or more of their income on rent (severe rent burden), probably aggregated at a geographic unit such as a census tract or ZIP code. The distribution is right-skewed (skew 1.60) with a mean of 253 and a median of 184, indicating many lower-burden areas alongside a long tail of high-burden zones reaching up to 1,918. Kurtosis of 3.44 and 87 outliers (3.7% of rows) confirm a heavy upper tail, and ~6.3% of records are zero, representing areas with no severely rent-burdened households.

Treatment: Log-transform or apply square-root before regression to reduce right skew; consider normalizing by total renter households to derive a rate.

NAME text label

This column contains fully-qualified U.S. Census tract names for New York City, following the pattern 'Census Tract [number], [County] County; New York' — every one of the 2,327 rows carries this format. All 2,327 values are unique with zero nulls or duplicates, and the extremely tight string-length range (min 38, max 46, mean 41.6) confirms a highly templated naming convention. The top words show 'census', 'tract', and 'county;' each appear exactly 2,327 times, while borough-level counties (Kings 805, Queens 725, Bronx 361, Richmond 126) account for NYC's five boroughs. The vocabulary size of 1,539 relative to 2,327 rows reflects the shared template words plus varying tract numbers and county names.

Treatment: Use as a human-readable label or parse to extract county and tract number as separate structured features; do not use as a model input directly.

state numeric feature

This column appears to encode a categorical 'state' as a numeric code, but every single one of its 2,327 non-null rows holds the identical value of 36.0 — making it a zero-variance constant. This is a strong signal that the dataset was filtered or exported for a single state (code 36, which corresponds to New York in U.S. FIPS encoding, though that interpretation is not confirmed by the evidence). With n_unique = 1 and std = 0.0, this column carries no predictive or discriminative information whatsoever.

Treatment: Drop — constant value across all 2,327 rows provides zero variance and no modelling signal.

county numeric foreign_key

Despite being named 'county', this column is stored as a numeric type with only 5 distinct values (5, 47, 81, and two others within the min–max range of 5–85), strongly suggesting it is a numeric county FIPS code or a coded categorical identifier rather than a free-text name. The concentration of values around Q1=47 and Q3=81 with zero outliers and zero nulls indicates a clean, bounded categorical encoding. The mild negative skew (−0.72) and flat kurtosis (−0.45) suggest the five codes are distributed somewhat unevenly, with lower-numbered counties less represented. Analysts should treat the 5 numeric values as nominal category labels, not ordinal or continuous quantities.

Treatment: Cast to categorical; map numeric codes to county names via a reference table before modelling or reporting.

tract numeric foreign_key

This column contains U.S. Census tract codes stored as integers, serving as a geographic identifier rather than a true numeric measure. The values range from 100 to 990100 with 1,530 unique codes across 2,327 rows, meaning roughly 34% of tract codes appear more than once — consistent with multiple records per tract. The extreme skew (10.14) and kurtosis (189.82) reflect the non-uniform geographic distribution of census tracts and the integer encoding scheme (e.g., tract 101 vs. tract 9901), not a meaningful numeric distribution; 63 outliers at the high end likely represent tracts in high-numbered FIPS areas.

Treatment: Treat as a categorical geographic key; do not use raw numeric value in models — left-join to a census tract reference table for geographic features.

county_name categorical label

This column contains the five boroughs of New York City, functioning as a geographic region label for each record. With only 5 unique values across 2,327 rows and zero nulls, it is clean and complete. Brooklyn (Kings) is the most frequent borough at 34.6% (805 records), while Staten Island (Richmond) is the least represented at 126 records — a roughly 6:1 imbalance worth noting for any borough-level modelling. Entropy ratio of 0.898 confirms the distribution is moderately spread but not uniform.

Treatment: One-hot encode or use as a grouping/stratification variable; be aware of class imbalance between Brooklyn and Staten Island.

moderate_burden numeric feature

This column appears to represent a count or dollar-amount measure of 'moderate burden' — likely a financial, time, or resource burden metric per entity (e.g., household, case, or geographic unit). The distribution is heavily right-skewed (skew=1.93) with high excess kurtosis (6.05), meaning a long upper tail pulls the mean (216.1) well above the median (159.0), and 86 outliers reach as high as 1732. The IQR of 247 spans 64 to 311, while ~6.4% of rows are zero, suggesting a meaningful minority of entities report no burden at all.

Treatment: Log-transform (or apply sqrt) before regression/modelling to address skew; consider zero-inflated treatment given the 6.4% zero mass.

severe_burden numeric numeric_target

This column appears to be a quantitative burden score or count (e.g., disease burden, cost burden, or case load) measured per some unit, given its name 'severe_burden' and integer-like range of 0–1918. The distribution is notably right-skewed (skew = 1.60) with excess kurtosis (3.44), meaning a long upper tail pulls the mean (253.2) well above the median (184.0); 87 outliers (3.7% of rows) extend toward the maximum of 1918. About 6.3% of values are exactly zero, which may represent genuine absence of burden or a data-quality artifact worth investigating.

Treatment: Log-transform (or apply Yeo-Johnson) before regression to reduce skew; investigate zero values for structural zeros vs. missingness.

pct_moderate_burden numeric feature

This column represents the percentage of households (or units) experiencing moderate housing cost burden, likely defined as spending 30–50% of income on housing. The distribution is right-skewed (skew = 1.51) with a heavy tail and high kurtosis (6.70), meaning most observations cluster around a median of 21.8% but a notable minority push toward the maximum of 100.0—59 outliers exist at the upper extreme. The null rate of 4.38% and zero rate of 2.11% are modest but worth flagging as potential data gaps or areas with genuinely zero burden. With only 461 unique values across 2,327 rows, values appear to be rounded or binned percentages rather than continuous measurements.

Treatment: Investigate and possibly winsorize or cap the 59 upper outliers (including any 100.0 values) before regression; impute or flag the 4.38% nulls.

pct_severe_burden numeric feature

This column represents the percentage of some population or housing units experiencing severe cost burden (likely housing costs exceeding 50% of income, a standard HUD metric). Values range from 0 to 100 with a mean of 27.1% and median of 26.2%, indicating a fairly symmetric distribution with mild right skew (0.57) and modest kurtosis (1.22). Notably, 518 unique values across 2,327 rows suggests aggregated geographic or demographic groupings rather than raw microdata. The 4.38% null rate and 30 outliers (reaching 100%) warrant attention but are not extreme.

Treatment: Use as-is or apply mild log-transform if residuals show heteroscedasticity; impute or flag the 4.38% nulls before modelling.

rent_burdened numeric feature

This column likely represents a count of rent-burdened households (those spending >30% of income on rent) per geographic unit such as a census tract or ZIP code. The distribution is right-skewed (skew 1.49, kurtosis 3.00) with a mean of 469 well above the median of 358, indicating a long tail driven by 82 outliers reaching up to 3,153 — consistent with dense urban areas pulling the upper end. The IQR spans 164.5 to 670, suggesting high variability across units, and 4.7% of rows are zero, plausibly reflecting areas with negligible renter populations.

Treatment: Log-transform or apply robust scaling before regression to address right skew and outlier influence.

pct_rent_burdened numeric feature

This column represents the percentage of renters who are rent-burdened (typically defined as spending ≥30% of income on rent) across 2,327 geographic or housing units. Surprisingly, the distribution is almost perfectly symmetric around the median of 50.0 and mean of 49.87, with near-zero skew (-0.038) and moderate IQ range of 17.9 — this is unusually bell-shaped for a percentage metric, which normally skews in one direction. The full 0–100 range is present, 62 outliers exist at the tails, and the near-zero rate is negligible at 0.36%, suggesting very few units with no rent burden at all.

Treatment: Use as-is or apply mild clipping at tails (62 outliers); no transformation needed given near-normal distribution.

median_gross_rent numeric feature

This column represents median gross rent (likely in USD) across 2,327 geographic units, with a healthy central range of $1,441–$2,049 (IQR) and a median of $1,735. The mean of –$41.5M and minimum of –$666,666,666 are severe red flags: a sentinel/placeholder value (e.g., –666666666) has been used for missing or N/A records, which is driving the extreme negative skew (–3.62), kurtosis of 11.1, and the 289 outliers (12.4% of rows). These corrupted values must be treated as nulls before any analysis.

Treatment: Replace –666666666 (and any other negative values) with NaN, then validate the remaining distribution before using as a regression feature or for aggregation.

median_household_income numeric feature

This column represents median household income, likely drawn from census or demographic data, with a plausible central distribution (median $76,833, IQR $49,117–$102,360) that aligns with typical US household income ranges. However, the column is severely corrupted by sentinel/error values: a minimum of -666,666,666 drags the mean to -$36,017,397 and produces extreme negative skew (-3.94) and kurtosis of 13.5. With 208 outliers (8.9% of rows) and a max capped at $250,001 (suggesting a top-coded value), a meaningful subset of records contain invalid or placeholder negative values that must be treated before any analysis.

Treatment: Filter or null-out records where value < 0 (sentinel values like -666666666), then apply log-transform for regression after cleaning.

total_households numeric feature

This column represents a count of total households per geographic or administrative unit, likely at a census-tract or zip-code level. The distribution is right-skewed (skew = 1.479) with leptokurtic tails (kurtosis = 4.38), meaning a minority of units contain disproportionately large household counts — confirmed by 70 outliers reaching up to 8,209 while the median sits at 1,252. A notable 4.1% of rows carry a zero value, which may indicate uninhabited areas, data gaps, or boundary artifacts worth investigating before modelling.

Treatment: Investigate zero values for validity, then log-transform or apply a power transform before regression to address right skew.

owner_occupied numeric feature

This column represents a count of owner-occupied housing units per geographic area (e.g., census tract or block group), with values ranging from 0 to 3,052 and a mean of ~465. The distribution is notably right-skewed (skew = 1.76) with high kurtosis (4.25), and 143 outliers (~6.1% of rows) pull the tail well above the median of 371. A 7.2% zero rate is worth investigating — these may be areas with no owner-occupied units or data gaps.

Treatment: Log-transform or apply a robust scaler before modelling; investigate zero values for data-quality issues.

renter_occupied numeric feature

This column represents the count of renter-occupied housing units, likely aggregated at a census tract or similar geographic level. The distribution is right-skewed (skew 1.59) with a mean of 946 well above the median of 726, indicating many mid-range areas pulled up by a long upper tail reaching 8,209. Kurtosis of 4.63 and 69 outliers (3.0%) confirm a heavy tail of high-density rental areas. A 4.4% zero rate is plausible for owner-dominated or low-density geographies.

Treatment: Log-transform or apply square-root before regression to reduce skew and compress outlier influence.

pct_owner_occupied numeric feature

This column represents the percentage of owner-occupied housing units for geographic areas (e.g., census tracts or zip codes), ranging from 0% to 100% with a mean of 37.5% and median of 34.4%. The distribution is surprisingly flat and broad — the IQR alone spans 39.7 percentage points (Q1=16.4, Q3=56.1) — indicating high variability across areas rather than a clustered norm. Negative kurtosis (−0.85) confirms a platykurtic, spread-out distribution with no heavy tails or outliers. A 3.2% zero rate may reflect areas that are entirely renter-occupied or institutionalized populations, which is worth investigating before modelling.

Treatment: Use as-is or apply mild clipping at boundaries; consider spatial context and impute the 4.1% nulls using geographic neighbours before modelling.

pct_renter_occupied numeric feature

This column represents the percentage of housing units that are renter-occupied, likely derived from census or housing survey data at some geographic unit (e.g., tract, ZIP, or neighborhood). The distribution is notably broad — IQR spans 39.7 points (43.9 to 83.6) with a std of 25.65 — indicating high variability in renter rates across observations. The mean (62.5) and median (65.6) both skew toward higher renter shares, suggesting this dataset may oversample urban or higher-density areas where renting is more common. The near-platykurtic shape (kurtosis −0.85) and mild negative skew confirm a relatively flat, left-leaning distribution with no outliers flagged.

Treatment: Use as-is or normalize to [0,1]; null_rate of 4.13% warrants median or model-based imputation before modelling.