An Interesting Question How generally applicable is Backwards approach to PCA? An Attractive Answer: James Damon, UNC Mathematics Key Idea: Express Backwards PCA as

Nested Series of Constraints General View of Backwards PCA Define Nested Spaces via Constraints E.g. SVD Now Define:

Constraint Gives Nested Reduction of Dimn Vectors of Angles Vectors of Angles as Data Objects Slice space with hyperplanes???? (ala Principal Nested Spheres) Vectors of Angles

E.g. , Data w/ Single Mode of Varn Best Fitting Planar Slice gives Bimodal Distn Special Thanks to Eduardo Garca-Portugus Torus Space Try To Fit A

Geodesic Challenge: Can Get Arbitrarily Close Torus Space Fit Nested Sub-Manifold

PNS Main Idea Data Objects: Where is a dimensional manifold Consider a nested series of sub-manifolds: where for and Goal: Fit all of simultaneously to

General Background Call each a stratum, so is a manifold stratification To be fit to New Approach: Simultaneously fit Nested Submanifold (NS)

Projection Notation For let onto denote the telescoping projection

I.e. for Note: This projection is fundamental to Backwards PCA methods PNS Components For a given , represent a point by its Nested Submanifold components: where for

In the sense that means the shortest geodesic arc between & Nested Submanifold Fits Simultaneous Fit Criteria? Based on Stratum-Wise Sums of Squares For

define Uses lengths of NS Components: NS Components in NS Candidate 2 (Shifted

to Sample Mean) Note: Both & Decrease NS Components in

NS based On PC1 Note: Yet is Constant (Pythagorean Thm)

NS Components in NS based On PC2 Note: is Constant (Pythagorean Thm)

NS Components in NS Candidate 1 NS Components in NS Candidate 2 NS Components in

NS based On PC1 NS Components in NS based On PC2

Nested Submanifold Fits Simultaneously fit Simultaneous Fit Criterion? Above Suggests Want:

Works for Euclidean PCA (?) Nested Submanifold Fits Simultaneous Fit Criterion? Above Suggests Want:

Important Predecessor Pennec (2016) AUC Criterion: Pennecs Area Under the Curve Based on Scree Plot 100 %

1 2 3 4 1 2 3 4 Component Index Pennecs Area Under the Curve

Based on Scree Plot Cumulative 100 % 1 2 3 4 1

2 3 4 Component Index Pennecs Area Under the Curve Based on Scree Plot

Cumulative Area = 100 % 1 2 3 4 1 2

3 4 Component Index Torus Space Fit Nested

Sub-Manifold Choice of & in: ??? Torus Space Tiled

embedding is complicated (maybe OK for low rank approx.) Instead Consider Nested Sub-Torii Work in Progress with Garcia, Wood, Le Key Factor: Important Modes of Variation OODA Big Picture New Topic:

Curve Registration Main Reference: Srivastava et al (2011) Collaborators

Anuj Srivastava (Florida State U.) Wei Wu (Florida State U.) Derek Tucker (Florida State U.) Xiaosun Lu (U. N. C.)

Inge Koch (U. Adelaide) Peter Hoffmann (U. Adelaide) J. O. Ramsay (McGill U.) Laura Sangalli (Milano Polytech.) Context Functional Data Analysis Curves as Data Objects Toy Example:

Context Functional Data Analysis Curves as Data Objects Toy Example: How Can We Understand Variation?

Context Functional Data Analysis Curves as Data Objects Toy Example: How Can We Understand Variation?

Context Functional Data Analysis Curves as Data Objects Toy Example: How Can We Understand Variation?

Functional Data Analysis Insightful Decomposition Functional Data Analysis Insightful Decomposition

Horizl Varn Functional Data Analysis Insightful Decomposition

Vertical Variation Horizl Varn Challenge

Fairly Large Literature Many (Diverse) Past Attempts Limited Success (in General) Surprisingly Slippery (even mathematical formulation) Challenge (Illustrated) Thanks to Wei Wu

Challenge (Illustrated) Thanks to Wei Wu Functional Data Analysis Appropriate

Mathematical Framework? Vertical Variation Horizl Varn

Landmark Based Shape Analysis Approach: Identify objects that are: Translations Rotations Scalings of each other Mathematics: Results in:

Equivalence Relation Equivalence Classes Which become the Data Objects Landmark Based Shape Analysis Equivalence Classes become Data Objects

a.k.a. Orbits Mathematics: , Called Quotient Space ,

, , , , Curve Registration

What are the Data Objects? Vertical Variation Horizl Varn

Curve Registration What are the Data Objects? Consider Time Warpings (smooth) More Precisely: Diffeomorphisms

Curve Registration Diffeomorphisms is 1 to 1 is onto (thus is invertible) Differentiable is Differentiable

Time Warping Intuition Elastically Stretch & Compress Axis Time Warping Intuition Elastically Stretch & Compress Axis (identity)

Time Warping Intuition Elastically Stretch & Compress Axis Time Warping Intuition Elastically Stretch & Compress Axis Time Warping Intuition Elastically Stretch & Compress Axis

Curve Registration Say curves and are equivalent, When so that

Curve Registration Toy Example: Starting Curve, Curve Registration Toy Example: Equivalent Curves,

Curve Registration Toy Example: Warping Functions Curve Registration Toy Example: Cannot

Warp Into Each Other Non-Equivalent Curves Data Objects I Equivalence Classes of Curves

(parallel to Kendall shape analysis) Data Objects I Equivalence Classes of Curves (Set of All Warps of

Given Curve) Notation: for a representor Data Objects I Equivalence Classes of Curves (Set of All Warps of

Given Curve) Next Task: Find Metric on Space of Curves Metrics in Curve Space Find Metric on Equivalence Classes Start with Warp Invariant Metric on Curves & Extend

Metrics in Curve Space Traditional Approach to Curve Registration: Align curves, say and By finding optimal time warp, , so:

Vertical varn: PCA after alignment Horizontal varn: PCA on s Metrics in Curve Space Problem: Dont have proper metric Since:

Because: Metrics in Curve Space Thanks to Xiaosun Lu Metrics in Curve Space

Note: Very Different L2 norms Thanks to Xiaosun Lu

Metrics in Curve Space Solution: Look for Warp Invariant Metric Where: Metrics in Curve Space I.e. Have Parallel

Representatives Of Equivalence Classes Metrics in Curve Space Warp Invariant Metric Developed in context of: Likelihood Geometry

Fisher Rao Metric: Metrics in Curve Space Fisher Rao Metric: Computation Based on Square Root Velocity Function (SRVF) Where

Signed Version Of Square Root Derivative Metrics in Curve Space Square Root Velocity Function (SRVF)

Metrics in Curve Space Fisher Rao Metric: Computation Based on SRVF: So work with SRVF, Since much easier to compute. Metrics in Curve Space

Why square roots? Thanks to Xiaosun Lu Metrics in Curve Space Why

square roots? Metrics in Curve Space Why square roots? Metrics in Curve Space

Why square roots? Metrics in Curve Space Why square roots?

Metrics in Curve Space Why square roots? Metrics in Curve Space Why square roots?

Metrics in Curve Space Why square roots? Metrics in Curve Space Why square

roots? Metrics in Curve Space Why square roots? Dislikes Pinching

Focusses Well On Peaks of Unequal Height Metrics in Curve Space Note on SRVF representation: Can show:

Warp Invariance Follows from Jacobean calculation Metrics in Curve Quotient Space Above was Invariance for Individual Curves Now extend to: Equivalence Classes of Curves I.e. Orbits as Data Objects

I.e. Quotient Space Metrics in Curve Quotient Space Define Metric on Equivalence Classes: For &,

i.e. & Independent of Choice of By Warp Invariance &

Mean in Curve Quotient Space Benefit of a Metric: Allows Definition of a Mean Frchet Mean Geodesic Mean Barycenter Karcher Mean Mean in Curve Quotient Space

Given Equivalence Class Data Objects: The Karcher Mean is: Mean in Curve Quotient Space The Karcher Mean is:

Intuition: Recall, for Euclidean Data Minimizer = Conventional Mean in Curve Quotient Space Next Define Most Representative Choice of As Representer of

Mean in Curve Quotient Space Most Representative in Given a candidate Consider warps to each Choose

to make Karcher mean of warps = Identity (under Fisher Rao metric) Mean in Curve Quotient Space Most Representative

Thanks to Anuj Srivastava in Toy Example (Details Later) Estimated Warps (Note:

Represented With Karcher Mean At Identity) Mean in Curve Quotient Space Most Representative

Terminology: in The Template Mean More Data Objects Final

Curve Warps: Data Objects I Warp Each Data Curve, To Template Mean, Denote Warp Functions Gives (Roughly Speaking): Vertical Components

(Aligned Curves) Horizontal Components More Data Objects Data Objects II Final

Curve Warps: Warp Each Data Curve, To Template Mean, Denote Warp Functions Gives (Roughly Speaking): Vertical Components (Aligned Curves) Horizontal Components

~ Kendalls Shapes More Data Objects Final Curve Warps: Data Objects III

Warp Each Data Curve, To Template Mean, Denote Warp Functions Gives (Roughly Speaking): Vertical Components (Aligned Curves) Horizontal Components ~ Changs Transfos

Computation Several Variations of Dynamic Programming Done by Eric Klassen, Wei Wu Toy Example Raw

Data Toy Example Raw Data Both Horizontal And Vertical

Variation Toy Example Conventional PCA Projections Toy Example Conventional

PCA Projections Power Spread Across Spectrum Toy Example Conventional

PCA Projections Power Spread Across Spectrum Toy Example Conventional

PCA Scores Toy Example Conventional PCA Scores Views of 1-d Curve

Bending Through 4 Dimns Toy Example Conventional PCA Scores

Patterns Are Harmonics In Scores Toy Example Scores Plot Shows Data Are 1

Dimensional So Need Improved PCA Decomp. Visualization Vertical Variation: PCA on Aligned Curves,

Projected Curves Toy Example Aligned Curves (Clear 1-d Vertical Varn)

Toy Example Aligned Curve PCA Projections All Varn In 1st Component

Visualization Horizontal Variation: PCA on Warps, Projected Curves Toy Example Estimated

Warps Toy Example Warps, PC Projections Toy Example Warps,

PC Projections Mostly 1st PC Toy Example Warps, PC Projections

Mostly 1st PC, But 2nd Helps Some Toy Example Warps, PC Projections

Rest is Not Important Toy Example Horizontal Varn Visualization Challenge: (Complicated) Warps Hard to Interpret

Approach: Apply Warps to Template Mean (PCA components) Toy Example Warp Componts (+ Mean) Applied to

Template Mean Participant Presentations Xi Yang Multi-View Weighted Network Hang Yu Introduction to multiple kernel learning Zhipeng Ding

Fast Predictive Simple Geodesic Regression