.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/03-combat-based/04-plot_combatgam_age_preservation.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_03-combat-based_04-plot_combatgam_age_preservation.py: Analyzing ComBatGAM behavior with imbalance across sites ======================================================== .. GENERATED FROM PYTHON SOURCE LINES 7-9 Imports ------- .. GENERATED FROM PYTHON SOURCE LINES 9-22 .. code-block:: Python import matplotlib.pyplot as plt import pandas as pd import seaborn as sns from uniharmony.datasets import make_multisite_classification from uniharmony.datasets import Covariate, CovariateSiteDistribution from uniharmony import verbosity verbosity("warning") from uniharmony.combat import ComBatGAM sns.set_theme(style="whitegrid") .. GENERATED FROM PYTHON SOURCE LINES 23-25 Data generation --------------- .. GENERATED FROM PYTHON SOURCE LINES 25-39 .. code-block:: Python X, y, sites, covars = make_multisite_classification(n_features=2, signal_type="blobs", covariates=["age", "sex"], site_effect_strength=10) df = pd.DataFrame({"Class": y, "Site": sites, "Age": covars["age"], "Feature1":X[:,0], "sex": covars["sex"]}) plt.figure(figsize=[10, 6]) plt.title("Features vs age/sex distribution") sns.scatterplot(df, y="Age", x="Feature1", hue="Site") plt.grid(axis="y", color="black", alpha=0.5, linestyle="--") .. image-sg:: /auto_examples/03-combat-based/images/sphx_glr_04-plot_combatgam_age_preservation_001.png :alt: Features vs age/sex distribution :srcset: /auto_examples/03-combat-based/images/sphx_glr_04-plot_combatgam_age_preservation_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 40-44 .. caution:: Note that we are harmonizing the whole dataset, which must be avoided in ML scenarios. This is just to illustrate the effect of harmonization. .. GENERATED FROM PYTHON SOURCE LINES 46-48 Harmonization ------------- .. GENERATED FROM PYTHON SOURCE LINES 48-52 .. code-block:: Python combat = ComBatGAM() combat.fit(X, sites, smooth_covariates=covars["age"] ) X_harmonized = combat.transform(X, sites, smooth_covariates=covars["age"]) .. GENERATED FROM PYTHON SOURCE LINES 53-55 Plotting -------- .. GENERATED FROM PYTHON SOURCE LINES 55-77 .. code-block:: Python df_orig = pd.DataFrame(X, columns=["Feature1", "Feature2"]) df_orig["Site"] = sites df_orig["Age"] = covars["age"] df_orig["Phase"] = "Original" df_harm = pd.DataFrame(X_harmonized, columns=["Feature1", "Feature2"]) df_harm["Site"] = sites df_harm["Age"] = covars["age"] df_harm["Phase"] = "Harmonized" fig, axes = plt.subplots(1, 2, figsize=(12, 5), sharex=True, sharey=True) sns.scatterplot(data=df_orig, x="Feature1", y="Age", hue="Site", alpha=0.6, ax=axes[0]) axes[0].set_title("Original data by site") axes[0].grid(alpha=0.3, color="black", linestyle="--") sns.scatterplot(data=df_harm, x="Feature1", y="Age", hue="Site", alpha=0.6, ax=axes[1]) axes[1].set_title("Harmonized data by site") axes[1].grid(alpha=0.3, color="black", linestyle="--") plt.tight_layout() .. image-sg:: /auto_examples/03-combat-based/images/sphx_glr_04-plot_combatgam_age_preservation_002.png :alt: Original data by site, Harmonized data by site :srcset: /auto_examples/03-combat-based/images/sphx_glr_04-plot_combatgam_age_preservation_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 78-80 Preserving the target as covariate ---------------------------------- .. GENERATED FROM PYTHON SOURCE LINES 82-86 .. caution:: This is also wrong in ML context, where you don't have access to the full dataset but may be a good option for statistical analysis. .. GENERATED FROM PYTHON SOURCE LINES 88-108 .. code-block:: Python combat = ComBatGAM() # This is the key line: we need to include the target variable as a covariate # to preserve its relationship with the features during harmonization. combat.fit(X, sites, smooth_covariates=covars["age"]) X_harmonized = combat.transform(X, sites, smooth_covariates=covars["age"]) df_orig = pd.DataFrame(X, columns=["Feature1", "Feature2"]) df_orig["Site"] = sites df_orig["Target"] = y df_orig["Phase"] = "Original" df_harm = pd.DataFrame(X_harmonized, columns=["Feature1", "Feature2"]) df_harm["Site"] = sites df_harm["Target"] = y df_harm["Phase"] = "Harmonized" .. GENERATED FROM PYTHON SOURCE LINES 109-111 Plotting -------- .. GENERATED FROM PYTHON SOURCE LINES 111-119 .. code-block:: Python # Plot data distribution by site before and after harmonization fig, axes = plt.subplots(1, 2, figsize=(12, 5), sharex=True, sharey=True) sns.scatterplot(data=df_orig, x="Feature1", y="Feature2", hue="Site", style="Target", alpha=0.6, ax=axes[0]) axes[0].set_title("Original data by site") sns.scatterplot(data=df_harm, x="Feature1", y="Feature2", hue="Site", style="Target",alpha=0.6, ax=axes[1]) axes[1].set_title("Harmonized data by site") plt.tight_layout() .. image-sg:: /auto_examples/03-combat-based/images/sphx_glr_04-plot_combatgam_age_preservation_003.png :alt: Original data by site, Harmonized data by site :srcset: /auto_examples/03-combat-based/images/sphx_glr_04-plot_combatgam_age_preservation_003.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 5.448 seconds) .. _sphx_glr_download_auto_examples_03-combat-based_04-plot_combatgam_age_preservation.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: 04-plot_combatgam_age_preservation.ipynb <04-plot_combatgam_age_preservation.ipynb>` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: 04-plot_combatgam_age_preservation.py <04-plot_combatgam_age_preservation.py>` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: 04-plot_combatgam_age_preservation.zip <04-plot_combatgam_age_preservation.zip>` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_