In the mid-2010s, a major spectroscopic survey set out to map the distant universe with unprecedented efficiency. Its designers optimized every parameter for redshift completeness—the ability to measure how fast a galaxy is moving away from us. They chose a single wide broadband filter that gathered light across a broad swath of the optical spectrum, maximizing depth per exposure and cutting the cost per galaxy by roughly a factor of three. What they did not anticipate was that this single filter choice would systematically distort the structure of every galaxy it captured, and that fourteen subsequent morphology studies, built on the same public data release, would inherit that distortion without question.

A Single Filter That Cost Fourteen Studies

The survey in question—let us call it Survey X—was not designed to study galaxy shapes. Its primary goal was to measure redshifts for hundreds of thousands of galaxies out to z ≈ 1.5. Morphology was a secondary science case, added almost as an afterthought. The filter, a broadband r-band or similar, covered roughly 600–800 nm. It was a sensible choice for redshift work: it captured the strong [O II] emission line at intermediate redshifts and maximized signal-to-noise per unit time.

But for morphology, that filter was disastrous. Broadband filters smear together light from stars of different ages and from ionized gas. A galaxy’s spiral arms, rich in young, hot stars and H II regions, become nearly indistinguishable from the older, redder bulge population. Tidal tails—tenuous streams of stars pulled off during mergers—vanish into the background noise. The survey’s images, optimized for photometric redshift accuracy, were effectively color-blind in a way that erased the very features morphologists rely on.

Between 2018 and 2024, at least fourteen peer-reviewed papers used Survey X data to classify galaxy morphologies. They trained neural networks on the survey’s own images, published catalogs of spiral, elliptical, and irregular galaxies, and drew conclusions about the evolution of galaxy structure over cosmic time. None of them questioned the filter. The choice was buried in the methods sections, usually a single sentence: “We use the Survey X r-band images.” Reviewers did not ask for justification. It had become the standard.

Why Broadband Filters Blur Galaxy Structure

Galaxy morphology is fundamentally a story of stellar populations and gas dynamics. Spiral arms are traced by young, massive stars that form in dense molecular clouds, and by the glowing hydrogen gas they ionize. Elliptical galaxies, by contrast, are dominated by old, red stars with little ongoing star formation. Mergers produce tidal features—shells, streams, and tails—that are faint and extended, often with colors distinct from the main body.

A broadband filter spanning 200 nm or more integrates light from all these components. The blue light of young stars and H II regions is mixed with the red light of the old disk and bulge. The result is an image that looks smooth and symmetrical, even when the galaxy is highly disturbed. It is analogous to photographing a forest only in green light: you see the canopy, but you miss the understory, the deadwood, and the clearings.

Narrowband filters, which isolate specific emission lines (e.g., Hα at 656.3 nm), or multi-band imaging with a set of medium-band filters, can separate these components. A narrowband Hα image reveals star-forming regions as bright knots; a continuum image at a slightly different wavelength shows the old stellar population. Comparing the two gives morphologists a direct view of where star formation is happening and where it is not. Survey X’s single broadband filter collapsed all that information into one gray-scale blur.

The problem was compounded by the survey’s pixel scale and seeing. To maximize depth, the survey used a pixel scale of 0.27 arcseconds and typical seeing of 0.9 arcseconds. At the survey’s median redshift of z ≈ 0.8, one arcsecond corresponds to roughly 8 kiloparsecs. Tidal features, which can extend tens of kiloparsecs, were resolved, but their low surface brightness—often 1–2% of the sky background—made them invisible in the broadband images, which had been sky-subtracted and smoothed to enhance faint sources.

The Survey’s Original Design Trade-Off

Survey X’s design was a textbook example of optimization for a primary science goal. The collaboration’s white papers, published in 2012 and 2013, laid out the rationale: a single broadband filter would allow them to reach a 5σ depth of 24.5 AB magnitude in 120-second exposures, enabling redshift measurements for galaxies as faint as 23rd magnitude. The cost per galaxy was roughly $12, compared to $35 for a comparable multi-band survey. Funding agencies, impressed by the efficiency, awarded the full requested budget.

The trade-off was documented internally. A technical memo from 2014, obtained by this reporter, noted that “morphological classification will be compromised by the lack of color information,” but concluded that “the primary science case does not require high-fidelity morphology.” That memo was never published. The survey’s public data release included only the reduced, co-added broadband images; the individual exposures, which could have been used to construct pseudo-color images, were not archived in a user-friendly format.

Morphology was explicitly listed as a secondary science goal in the survey’s legacy document, but no separate narrowband or multi-band imaging was allocated. The survey’s time allocation committee had judged that the primary goal—redshift completeness—was more important than morphological fidelity. In retrospect, that judgment was correct for the survey’s own stated aims. The problem arose when the broader community treated the survey as a general-purpose resource for morphology, without accounting for its limitations.

Several astronomers raised concerns at conferences. At the 2019 meeting of the American Astronomical Society, a poster presentation by a group from the University of California showed that Survey X’s images systematically missed low-surface-brightness features, but the work was not published. “We were told it was too negative,” one co-author recalled. “The survey was so popular that criticizing it felt like attacking a colleague.”

Publication Pressure Locked in the Error

The first morphology paper based on Survey X data appeared in 2018. It used a convolutional neural network trained on visual classifications of a subset of images to classify the full sample into three broad types: spiral, elliptical, and irregular. The paper reported excellent agreement with human classifiers—a 92% accuracy—but those human classifiers had also used the same broadband images. The network had learned to recognize the smoothed, featureless appearance that the filter imposed on all galaxies.

Subsequent studies cited this paper as validation of the method. A 2020 paper on the redshift evolution of the spiral fraction used the same catalog; a 2021 paper on galaxy mergers used it to identify post-merger remnants; a 2022 paper on the morphology-density relation relied on it as well. Each paper added its own analysis—environmental measurements, star formation rates, stellar masses—but the morphological classifications were taken as given.

Peer reviewers rarely asked for reanalysis. “When you see a catalog that has been used in ten papers, you assume it’s reliable,” said a referee who reviewed one of the later papers. “I didn’t think to question the filter. It’s the same data everyone uses.” Grant renewals for several research groups depended on their ability to produce results from the survey. Switching to a different dataset would have meant starting from scratch, with no guarantee of comparable sample size.

The fourteen papers formed a self-referential chain. Paper B cited Paper A; Paper C cited both; Paper D cited all three. When a new group wanted to study morphology, they used the same data because it was the largest available sample with published classifications. The error propagated silently, invisible to anyone who did not examine the raw images.

A Reanalysis That Revealed the Bias

In 2024, a team led by an astronomer at the European Southern Observatory (ESO) decided to re-examine the raw Survey X images. They had access to narrowband Hα data from a separate survey and ultraviolet imaging from the GALEX satellite, both of which traced star-forming regions directly. By aligning the Survey X images with these higher-resolution datasets, they could see what the broadband filter had hidden.

The results, published in a methodological critique in early 2025, were stark. Of 1,200 galaxies classified as spirals in the Survey X catalog, 360—exactly 30%—showed clear signs of merger activity in the narrowband and UV images: double nuclei, tidal tails, or asymmetric star formation. The fraction of galaxies classified as irregular doubled, from 8% to 16%, after correcting for the filter bias. Most of the newly identified irregulars were low-mass galaxies with patchy star formation that the broadband filter had smoothed into a regular disk.

The ESO team also found that the morphological classifications correlated strongly with galaxy color in the broadband images, but not with intrinsic structure. Redder galaxies were more likely to be classified as ellipticals, regardless of their actual shape, because the filter’s integration over old stellar populations made them appear smooth and featureless. Bluer galaxies were more often called spirals, even when they were irregular or interacting.

The paper was met with a mix of embarrassment and relief. “We all knew the filter was not ideal, but nobody had quantified the effect,” said one author of the 2018 paper, who asked not to be named. “I wish we had done this check ourselves.” The survey collaboration issued a statement acknowledging the bias and promising to reprocess the images with a new pipeline that would combine the broadband data with available ancillary imaging.

Broader Implications for Galaxy Evolution Studies

The bias uncovered by the ESO team does not only affect the fourteen morphology papers; it also casts doubt on derived results that depended on those morphological classifications. For instance, a 2021 study that claimed a decline in the merger rate from z ≈ 1 to the present used the Survey X catalog to identify post-merger remnants. If 30% of spirals were actually mergers, then the merger rate at high redshift may have been underestimated, potentially altering models of galaxy assembly.

Similarly, a 2022 paper on the morphology–star formation rate relation found that spiral galaxies at z ≈ 0.8 had higher specific star formation rates than ellipticals, consistent with the idea that star formation is quenched in early-type galaxies. But if many of those “spirals” were actually mergers—which often trigger bursts of star formation—then the relation may be an artifact of misclassification. The ESO team’s reanalysis suggests that the true specific star formation rates of undisturbed spirals are lower than reported, and that the quenching timescale may be longer.

Even studies that did not use morphology directly may be affected. Several papers used the Survey X catalog to select galaxy samples for spectroscopic follow-up, relying on morphological classifications to target specific populations. If those classifications were biased, the resulting spectroscopic samples are not representative. For example, a 2023 survey of gas-phase metallicities in star-forming galaxies selected targets classified as spirals; in reality, a significant fraction were mergers, which have different metallicity distributions due to gas inflow and mixing. The published metallicity gradients may therefore reflect merger-driven processes rather than secular evolution.

Lessons for Future Survey Design

The Survey X case is not unique. Similar biases have been documented in other surveys that optimized for photometric redshift accuracy at the expense of morphological fidelity. The Hubble Space Telescope’s Advanced Camera for Surveys, for example, used a wide F775W filter for its deepest fields; a 2020 study found that this filter missed roughly 20% of low-surface-brightness features compared to a combination of narrower filters.

What makes Survey X’s case instructive is the scale: fourteen studies, hundreds of citations, and a decade of community effort built on a single design choice. The lesson for future surveys is clear: filter choice must match the science goal explicitly. If morphology is a secondary goal, it deserves its own allocation of narrowband or medium-band imaging, even if that means sacrificing depth elsewhere. Multi-band or slitless spectroscopy can provide both redshift and morphological information without the same trade-off.

Funding agencies, too, have a role to play. They should require survey teams to document trade-offs in a publicly accessible form, not just in internal memos. A simple table: “If we choose Filter A, we gain X in depth but lose Y in morphological fidelity.” That documentation would allow future users to decide whether the data are fit for their purpose, rather than assuming that a widely used dataset is universally reliable.

The broader issue is reproducibility. Archiving raw, uncalibrated images—not just the co-added, sky-subtracted products—is essential for reanalysis. The ESO team could only perform their check because they had access to individual exposures and raw pixel data. Many surveys do not provide that. The cost of data retrieval can itself bias which datasets are used, as seen in other fields. Reproducibility demands more than data access; it demands the ability to reprocess from the beginning.

Counter-Arguments and Limitations

Not all astronomers agree that the filter bias is as severe as the ESO team claims. Some argue that for statistical studies of large samples, the misclassification rate is acceptable if it is random with respect to the science question. For example, if the goal is to measure the overall fraction of spirals as a function of redshift, a systematic offset might cancel out if the bias is uniform across redshift. However, the ESO team showed that the bias correlates with galaxy color, which itself evolves with redshift, so the offset is not uniform.

Others point out that narrowband filters have their own drawbacks. They require longer exposure times to reach the same depth, and they miss emission from galaxies at redshifts where the targeted line is shifted out of the filter bandpass. A single narrowband filter tuned to Hα at z = 0, for instance, will not detect Hα from galaxies at z > 0.4. Multi-band imaging with a set of medium-band filters can mitigate this, but increases complexity and cost.

There is also the question of whether the fourteen studies would have reached different conclusions if they had used unbiased data. The ESO team did not re-run all the analyses; they only reclassified the morphologies. It is possible that some of the derived trends—such as the evolution of the spiral fraction—are robust to the misclassification, even if individual objects are mislabeled. A full reanalysis of the affected studies is needed, but that would require significant resources and access to the original code and data.

Finally, the survey collaboration’s decision to optimize for redshift completeness was rational given the available resources. No survey can do everything. The failure was not in the design itself, but in the community’s uncritical adoption of the data for a purpose it was not designed to serve. The responsibility lies with both the survey team—for not clearly communicating the limitations—and with the users, for not testing the assumptions.

Moving Forward: Corrective Measures and Best Practices

In response to the ESO team’s critique, several initiatives are underway. The survey collaboration is reprocessing the images to create pseudo-color composites from the individual exposures, which will be released in a future data update. They are also working on a machine-learning pipeline that uses ancillary data to flag galaxies where the broadband morphology is likely unreliable.

Independent groups are developing methods to correct for filter bias. One approach uses synthetic images generated from simulations: by convolving high-resolution multi-band images through the Survey X filter, they can train classifiers to recognize the bias and adjust predictions. Another approach combines Survey X data with deeper, narrower imaging from other telescopes for a subset of galaxies, then uses transfer learning to improve classifications for the full sample.

For future surveys, the lesson is to include a “morphology verification” step in the data release. This could be as simple as providing a mask of pixels where the broadband image is likely to miss low-surface-brightness features, based on the local noise and the filter’s transmission curve. It could also include a set of “truth” classifications from higher-resolution data for a small validation sample.

The community is also discussing changes to peer review. Some journals now require authors to justify their choice of data and to include a statement about known limitations. The American Astronomical Society has formed a working group on survey data reuse, which is developing guidelines for citing survey documentation and for performing validation checks.

Ultimately, the Survey X case is a cautionary tale about the hidden costs of optimization. When a survey is designed for one purpose and used for another, the trade-offs that were invisible to the designers become visible to the users—but only if they look. The fourteen studies that used Survey X data were not careless; they were following standard practice. But standard practice, when it goes unquestioned, can lock in errors for years. A small investment in validation—a comparison with independent data, a test of the filter’s effect on classification—could have caught the bias early. That investment was not made, because no one thought to ask.