Storm Composite Analysis- Methods, Validation, and Physical Interpretation

Date: 2026-06-16

Purpose

This document summarizes the development of the storm-composite analysis framework used for the Spotter tropical cyclone dataset. It is intended as a technical record of the analysis path, validation steps, major decisions, and physical conclusions that emerged during development.

The primary goals of this work were:

1. Reproduce and extend the original wind-speed and quadrant composite framework.
2. Validate the new Doppler-corrected spectra against previous processed datasets.
3. Investigate uncertainties associated with storm-relative coordinates and quadrant assignment.
4. Explore whether spectra occupying similar bulk sea-state conditions exhibit different high-frequency structure.
5. Determine whether the ISOMAP manifold contains physically meaningful organization beyond traditional bulk metrics such as peak frequency and mean square slope (MSS).

---

1. Composite Construction and Frequency Standardization

Motivation

The original Spotter composite figures were produced from spectra occupying different native frequency grids. Direct comparison of individual observations and construction of composite statistics therefore required a common spectral coordinate system.

To address this, all spectra were interpolated onto a shared logarithmic frequency grid:

$$f_i=\mathrm{geomspace}(0.01,0.5,96)$$

using the Doppler-corrected intrinsic spectra.

Primary datasets:

common_frequency_storm_composite_samples.nc
common_frequency_spotter_storm_composite_samples.nc
frequency_valid_storm_composite_samples.nc

The primary analysis dataset became:

common_frequency_spotter_storm_composite_samples.nc

after later evaluation of Spotter/MicroSWIFT consistency.

---

Frequency-Support Validation

During early testing, composite spectra displayed patchiness and discontinuous behavior at some frequencies.

Investigation showed that the issue originated from invalid Doppler-remapped coordinates rather than plotting procedures.

Samples were retained only when:

$$f_{\mathrm{mapped}}>0$$

and

$$E_{\mathrm{mapped}}>0$$

in at least ten source bins.

Audit output:

spotter_frequency_support_drop_audit.csv

Interpretation

This result demonstrated that not all Doppler-corrected spectra provide usable support across the entire analysis band. In some cases the intrinsic-frequency mapping generates nonfinite or otherwise unusable coordinates over portions of the spectrum.

The resulting patchiness was therefore a coordinate-transformation problem rather than a physical spectral feature.

The frequency-support filter substantially improved composite consistency and became a required preprocessing step for all subsequent analyses.

---

2. Spotter and MicroSWIFT Evaluation

Early composite experiments combined Spotter and MicroSWIFT observations into a single population.

While physically attractive from a sample-size perspective, the resulting composites became more difficult to interpret and exhibited greater variability than expected.

After comparison of spectral behavior and composite structure, the decision was made to use Spotter observations as the primary dataset for the Ian–Idalia–Milton analysis.

Interpretation

This decision was made primarily to maximize interpretability rather than because of an identified problem with the MicroSWIFT observations.

Future work may revisit direct Spotter–MicroSWIFT comparisons, but combining the datasets at this stage introduced additional variability that obscured the physical relationships of interest.

---

3. Storm-Relative Coordinates and Quadrant Assignment

Motivation

A major concern throughout the analysis was whether quadrant assignment accurately represented storm-relative geometry.

The original stored quadrant labels were therefore compared against direct algebraic reconstruction using storm-motion coordinates.

The along-track and cross-track coordinates were defined as

$$s=\mathbf{r}\cdot \hat{\mathbf{t}}$$ $$c=\mathbf{r}\cdot {\hat{\mathbf{n}}}_R$$

where

• $\mathbf{r}$ is the center-to-buoy vector
• $\hat{\mathbf{t}}$ is the storm-motion direction
• ${\hat{\mathbf{n}}}_R$ is the rightward normal

Quadrants were then assigned directly from the signs of (s) and (c).

Relevant outputs:

idalia_processed_spotter_quadrant_algebra_summary.csv
idalia_processed_spotter_quadrant_algebra_confusion_counts.csv
idalia_processed_spotter_algebraic_quadrants.nc

Result

The stored labels were not always algebraically consistent with the storm-relative geometry.

For subsequent analyses, regenerated algebraic quadrants were therefore considered the more robust definition.

---

4. Quadrant Sensitivity and Uncertainty

Motivation

Even if quadrant assignment is algebraically correct, observations near quadrant boundaries may not be physically distinguishable.

Similarly, changing the smoothing window used to estimate storm heading can alter the assigned quadrant.

Heading windows of

0.5 hr
1 hr
2 hr
3 hr

were tested.

Key figure:

all_storms_quadrant_sensitivity_summary.png

Result

A substantial fraction of observations were found to be either:

• near quadrant boundaries,
• sensitive to heading-window choice,
• or both.

Combined statistics:

All storms:
boundary-sensitive: 30.25%
heading-sensitive: 26.58%
both: 13.96%
stable: 57.14%

Interpretation

This result fundamentally changes how quadrant composites should be interpreted.

Quadrants should not be viewed as perfectly discrete categories. Instead, there is a significant population of observations whose assignment depends on relatively small changes in heading estimation or position.

This motivates future confidence weighting and ambiguity flags rather than strict categorical treatment.

---

5. Validation of Doppler-Corrected Spectra

Motivation

A central question was whether the new Doppler correction altered spectra in a physically meaningful way relative to Jake’s processed Idalia dataset.

Variance conservation was evaluated using

$$m_0=\int E(f),df$$

and

$$H_s=4\sqrt{m_0}$$

Key figure:

idalia_one_matched_observation_spectral_variance_curves.png

The comparison included:

• raw Spotter spectra
• Jake processed spectra
• corrected native-grid spectra
• corrected common-grid spectra

A broader comparison was summarized in:

idalia_peak_frequency_shift_summary.png

Result

The common-frequency interpolation introduced negligible systematic peak-frequency shifts:

common-grid / native-grid median peak ratio = 1.000

The dominant differences relative to Jake’s spectra were already present before interpolation:

native-grid / Jake median peak ratio ≈ 1.023

Interpretation

This was an important validation step because it demonstrated that the common-frequency interpolation itself is not responsible for the observed differences.

The source of the shift lies upstream in the Doppler-remapping procedure.

---

6. Fixed Sea-State Analysis

Motivation

The primary physical question became:

Can spectra with similar bulk sea-state properties exhibit systematically different high-frequency structure?

To investigate this, Idalia observations were selected using the actual common-grid spectra rather than inherited feature-table labels.

Selection criteria:

$$0.14<f_p<0.17\mathrm{Hz}$$ $$0.008<MSS<0.010$$

with

$$n=32$$

This ensured that all selected observations occupied a narrow region of traditional sea-state space.

---

High-Frequency Metrics

Three metrics were examined:

Directional spread

$DS$

High-frequency directional spread

$D{S}_{hf}$

High-frequency energy fraction

$R_{hf}=\frac{{\int }_{0.20}^{0.50}E(f),df}{{\int }_{0.05}^{0.50}E(f),df}$

Key figures:

idalia_corrected_rebinned_peak_mss_directional_spread_quartile_medians.png
idalia_corrected_rebinned_peak_mss_hf_tail_fraction_quartile_medians.png

Result

Even within a tightly constrained peak-frequency/MSS category, spectra separated into distinct median shapes when grouped by high-frequency metrics.

The strongest separation occurred for (R_{hf}).

Interpretation

This is one of the most important results of the analysis.

Peak frequency and MSS describe the overall scale and steepness of the sea state, but they do not uniquely determine the structure of the high-frequency tail.

Spectra with similar bulk properties can exhibit substantially different distributions of energy between the dominant waves and the short-wave population.

This observation motivated the subsequent manifold analysis.

---

7. ISOMAP Sheet Structure

Motivation

Dimensionality-reduction analysis suggested that observations occupy two preferred regions of the manifold.

The question became whether these sheets represent physically meaningful wave states or merely mathematical clustering behavior.

Key figure:

all_storms_isomap_lower_upper_sheet_median_spectra_alignment.png

Result

Lower-sheet and upper-sheet populations exhibit systematic differences in:

• high-frequency energy content
• directional spreading
• alignment behavior

despite occupying overlapping regions of peak-frequency/MSS space.

Interpretation

This result provides evidence that the manifold is capturing physically meaningful variability.

The sheets appear to represent different modes of spectral organization rather than simple differences in bulk forcing.

The evidence remains suggestive rather than definitive, but it is currently the strongest indication that multiple high-frequency states can exist within similar bulk sea-state conditions.

---

8. High-Frequency Energy Fraction Relationships

The high-frequency energy fraction was defined as

$R_{hf}=\frac{{\int }_{0.20}^{0.50}E(f),df}{{\int }_{0.05}^{0.50}E(f),df}$

Key figure:

ian_idalia_milton_r_hf_vs_peak_mss_panel_colorbars.png

Result

(R_{hf}) increases strongly with peak frequency and exhibits additional variability associated with MSS and directional organization.

Interpretation

The peak-frequency dependence is expected because increasing peak frequency shifts spectral energy toward shorter waves.

More interesting is the residual variability at similar peak frequencies.

This variability suggests that high-frequency organization contains information not fully captured by either peak frequency or MSS, reinforcing the conclusions from the fixed-sea-state and ISOMAP analyses.

---

Primary Conclusions

1. Frequency-support filtering is required for robust composite construction.
2. Algebraic storm-relative coordinates provide a more reliable quadrant definition than inherited labels.
3. Quadrant assignment contains substantial uncertainty and should be treated probabilistically.
4. Common-grid interpolation preserves spectral structure and is not responsible for observed peak-frequency shifts.
5. Spectra occupying similar peak-frequency/MSS space can exhibit substantially different high-frequency organization.
6. High-frequency energy fraction and directional structure provide information not contained in traditional bulk sea-state metrics.
7. ISOMAP sheet structure appears to capture physically meaningful variability in spectral organization.
8. Collectively, the results support the hypothesis that high-frequency wave organization may vary independently of bulk sea-state properties and may therefore influence air-sea momentum exchange in ways not captured by peak frequency or MSS alone.