---
abstract: |
    Rankin–Cohen bidifferential operators are the projectors onto irreducible summands in the decomposition of the tensor product of two particular representations of $SL(2,{\mathbb R})$. We consider the general problem to find explicit formulæ for such projectors in the setting of multiplicity-free branching laws for reductive symmetric pairs.

    For this purpose we develop a new method (F-method) based on an *algebraic Fourier transform for generalized Verma modules*, which enables us to characterize those projectors by means of certain systems of partial differential equations of second order.

    We discover explicit formulæ for new equivariant holomorphic differential operators in the six different complex geometries arising from real symmetric pairs of split rank one, and reveal an intrinsic reason why the coefficients of Jacobi polynomials appear in these operators including the classical Rankin–Cohen brackets as a special case.

    Key words and phrases: *branching laws, Rankin–Cohen brackets, F-method, symmetric pair, invariant theory, generalized Verma modules, Hermitian symmetric spaces, Jacobi polynomial.*
author:
- 'Toshiyuki Kobayashi, Michael Pevzner'
title: 'Rankin–Cohen Operators for Symmetric Pairs '
---

Introduction
============

What kind of differential operators do preserve modularity? This question was studied by R. A. Rankin [@Ra] and later investigated by H. Cohen [@C75] in the framework of bidifferential operators. These authors introduced a family of operators transforming a given pair of modular forms into another modular form of a higher weight. Let $f_1$ and $f_2$ be holomorphic modular forms for a given arithmetic subgroup of $SL(2,{\mathbb R})$ of weight $k_1$ and $k_2$, respectively. The bidifferential operator, referred to as the *Rankin–Cohen bracket* of degree $a$ and defined by $$\label{rcb}
\mathrm{RC}_{k_1,k_2}^a(f_1,f_2)(z):=\sum_{\ell=0}^a(-1)^{\ell}\left(
                                         \begin{array}{c}
                                           k_1+a-1 \\
                                           \ell \\
                                         \end{array}
                                       \right)\left(
                                                \begin{array}{c}
                                                  k_2+a-1 \\
                                                a-\ell \\
                                                \end{array} \right)
\frac{\partial^{a-\ell}f_1}{\partial z^{a-\ell}}(z)\frac{\partial ^\ell f_2}{\partial z^{\ell}}(z),$$ yields a new holomorphic modular form of weight $k_1+k_2+2a$.

These operators have attracted considerable attention in recent years particularly because of their various applications in

-   theory of modular and quasimodular forms (special values of $L$-functions, the Ramanujan and Chazy differential equations, van der Pol and Niebur equalities) [@CL11; @MR; @Z1],

-   covariant quantization [@BTY; @CMZ; @CM; @OS; @vDP; @Pev; @UU96],

-   ring structures on representations spaces [@vDP; @Z1].

**Existing methods for the $SL(2,{\mathbb R})$-case.** A prototype of Rankin–Cohen brackets was already found by P. Gordan and S. Guldenfinger [@Go; @Gu] in the 19th century by using recursion relations for invariant binary forms and the Cayley processes. For explicit constructions of such equivariant bidifferential operators several different methods have been developed:

-   Recurrence relations [@C75; @ElG; @HTan; @PevBess; @Z1].

-   Taylor expansions of Jacobi forms [@EZ; @IKO; @Kuz].

-   Reproducing kernels for Hilbert spaces [@PZ; @UU96; @Zh].

-   Dual pair correspondence [@Ban; @Eholzer].

We propose a new method based on branching laws for infinite dimensional representations and the algebraic Fourier transform of generalized Verma modules. We discover new families of covariant differential operators by applying this method to six different complex geometries beyond the $SL(2,{\mathbb R})$ case (see Table \[table:intro\]).

**Branching laws for symmetric pairs.** By *branching law* we mean the decomposition of an irreducible representation $\pi$ of a group $G$ when restricted to a given subgroup $G'$. An important and fruitful source of examples is provided by pairs of groups $(G,G')$ such that $G'$ is the fixed point group of an involutive automorphism $\sigma$ of $G$, called *symmetric pairs*.

The decomposition of tensor product is a special case of branching laws with respect to symmetric pairs $(G,G')$. Indeed, if $G=G_1\times G_1$ and $\sigma$ is an involutive automorphism of $G$ given by $\sigma(x,y)=(y,x)$, then $G'\simeq G_1$ and the restriction of the outer tensor product $\pi_1\boxtimes\pi_2$ to the subgroup $G'$ is nothing but the tensor product $\pi_1\otimes\pi_2$ of two representations $\pi_1$ and $\pi_2$ of $G_1$. The Littlewood–Richardson rule for finite dimensional representations is another classical example of branching laws with respect to the symmetric pair $(GL(p+q,{\mathbb C}), GL(p,{\mathbb C})\times GL(q,{\mathbb C}))$. Our approach relies on recent progress in the theory of branching laws of infinite dimensional representations for symmetric pairs.

**Rankin–Cohen operators as intertwining operators.** From the view point of representation theory the Rankin–Cohen operators are the projectors to irreducible components in the tensor product of two holomorphic discrete series representations of $SL(2,{\mathbb R})$. More precisely, the operator is the projector to the summand $\pi_{k_1+k_2+2a}$ in the following abstract branching law [@Mol; @Repka]: $$\label{eqn:absbra}
\pi_{k_1}\otimes \pi_{k_2}
\simeq
{\sum_{a\in{\mathbb N}}}^{\oplus}\pi_{k_1+k_2+2a}.$$

The subject of this paper is to develop a method to find explicit projectors onto irreducible components of branching laws in a broader setting of symmetric pairs. A reasonable general assumption to work with would be the multiplicity-free decompositions where the projectors are unique up to scalars. This is the case, for instance, if the representation $\pi$ is any highest weight module of scalar type (or equivalently $\pi$ is realized in the space of holomorphic sections of a homogeneous holomorphic line bundle over a bounded symmetric domain) and $(G,G')$ is any symmetric pair (see [@K08; @K12] for the multiplicity-free theorems).

**Geometric setting.** We shall work in the following geometric setting. Let $\mathcal V_X\to X$ be a homogeneous vector bundle of a Lie group $G$ and $\mathcal W_Y\to Y$ a homogeneous vector bundle of $G'$. Then we have a natural representation $\pi$ of $G$ on the space $\Gamma(X,\mathcal V_X)$ of sections on $X$, and similarly that of $G'$ on $\Gamma (Y,\mathcal W_Y)$. Assume $G'$ is a subgroup of $G$. Our main concern is with the following question:

\[question:1\] Find explicit $G'$-intertwining operators from $\Gamma(X,\mathcal V_X)$ to $\Gamma(Y,\mathcal W_Y)$.

For $G'=G$ and $Y=X$ being a flag variety, Question \[question:1\] amounts to the study of intertwining operators between principal series representations, see [@Knapp-Stein; @Ks74]. But not much has been known until know in the case where $G'\subsetneqq G$ and $Y\subsetneqq X$.

In the case where $X$ is a Hermitian symmetric domain, $Y$ a subsymmetric domain, $G'\subset G$ are the groups of biholomorphic transformations of $f: Y\hookrightarrow X$, respectively, and $\mathcal V_X$ is any line bundle $\mathcal L_\lambda$ with sufficiently positive parameter $\lambda$, we prove:

\[thm:intro\] Any continuous $G'$-homomorphism from $\mathcal O(X,\mathcal L_\lambda)$ to $\mathcal O(Y,\mathcal W)$ is given by a unique, up to a scalar, differential operator.

See Theorem \[thm:C\] for precise statement. Theorem \[thm:intro\] includes the tensor product case, namely, $G \simeq G' \times G'$ and $X \simeq Y \times Y$ as a special case. In contrast to Theorem A, we notice that in general, the intertwining operators between two unitary representations of real reductive Lie groups $G'\subset G$ are given by integro-differential operators in geometric models. Among them, equivariant differential operators are very special (e.g. [@Knapp-Stein] for $G'=G$ and [@KS] for $G'\subsetneqq G$).

We observe that the pullback $f^*$ of sections is obviously $G'$-intertwining if $\mathcal W_Y\simeq f^*\mathcal V_X$. Finding all bundles $\mathcal W_Y$ for which such nontrivial intertwining operators exist is a part of the above question, which reduces to abstract branching problems (see Theorem \[thm:surject\]). However, giving explicit formulæ of intertwining operators is more involved even when abstract branching laws are known, as we may note by comparing with in the $SL(2,{\mathbb R})$ case. In fact, to answer Question 1 one has to know a finer structure of branching laws, namely, the precise places where the irreducible summands are located in the whole representation space.

**F-method.** To answer Question \[question:1\], we propose a method, that we call *F-method*, which consists of two stages:

-   Transfer the initial question to an algebraic problem of understanding a fine structure of branching laws for induced modules of enveloping algebras.

-   Characterize projectors by means of certain systems of partial differential equations.

Concerning the stage (a) we first prepare a rigorous notion of ‘differential operators’ between two manifolds $X$ and $Y$ with morphism $f: Y \to X$ and then we prove in Theorem \[thm:surject\] a one–to–one correspondence between $G'$-equivariant differential operators and $\mathfrak g'$-homomorphisms of induced modules (algebraic branching laws). This generalizes a well-known result in the case where $G=G'$ and $X=Y$ are the same flag variety ([@HJ]). The multiplicity-freeness of algebraic branching laws for generalized Verma modules guarantees the uniqueness of the projectors in Question 1 when $X$ and $Y$ are flag varieties.

We proceed to Stage (b) under the assumption that $X$ is defined by a parabolic subalgebra with an abelian nilradical. We then characterize $\mathfrak g'$-homomorphisms of generalized $\mathfrak g$-Verma modules by means of certain systems of partial differential equations. It should be noted that the system is of second order although the resulting covariant differential operators may be of any higher order. The characterization is performed by applying an algebraic Fourier transform (see Definition \[def:hat\]).

The detailed recipe of the F-method is described in Section \[sec:44\] relying on Theorem \[thm:313\] and Proposition \[prop:Fc\].

In Sections \[sec:5\] and \[section:6\] we give an answer to Question \[question:1\] for all symmetric pairs $(G,G')$ of split rank one inducing a holomorphic embedding $Y\hookrightarrow X$ (see Table \[table:1\]). We remark that the split rank one condition does not force the rank of $G/G'$ to be equal to one (see Table \[table:intro\] (1), (5) below).

**Normal derivatives and Jacobi–type differential operators.** In representation theory, taking normal derivatives with respect to an equivariant embedding $Y\hookrightarrow X$ is a standard tool to find abstract branching laws for representations that are realized on $X$ (see S. Martens [@Mart] and H. P. Jakobsen and M. Vergne [@JV]).

However, we should like to emphasize that the common belief “Normal derivatives with respect to $Y\hookrightarrow X$ are intertwining operators in the branching laws" is not true. Actually, it already fails for the tensor product of two holomorphic discrete series of $SL(2,{\mathbb R})$ where the only equivariant operators are Rankin–Cohen brackets.

We discuss when normal derivatives become intertwiners in the following six complex geometries arising from real symmetric pairs of split rank one:

  ----- ------------------------------------------------------------------------ ------------------- --------------------------------------------------------- -- ----- ------------------------------------------ ------------------- ----------------------------------------
                                                                                                                                                                                                                                       
   (1)                                                  $\mathbb P^n{\mathbb C}$  $\hookrightarrow$  $  \mathbb P^n{\mathbb C}\times \mathbb P^n{\mathbb C}$       (4)     $\mathrm {Gr}_{p-1}({\mathbb C}^{p+q})$  $\hookrightarrow$  $ \mathrm {Gr}_{p}({\mathbb C}^{p+q})$
   (2)    $\mathrm{LGr}({\mathbb C}^{2n-2})\times \mathrm{LGr}({\mathbb C}^{2})$  $\hookrightarrow$  $\mathrm{LGr}({\mathbb C}^{2n})$                              (5)                    $\mathbb P^n{\mathbb C}$  $\hookrightarrow$  $\mathrm{Q}^{2n}{\mathbb C}$
   (3)                                                $\mathrm Q^{n}{\mathbb C}$  $\hookrightarrow$  $  \mathrm Q^{n+1}{\mathbb C}$                                (6)    $\mathrm{IGr}_{n-1}({\mathbb C}^{2n-2})$  $\hookrightarrow$  $  \mathrm{IGr}_{n}({\mathbb C}^{2n})$
                                                                                                                                                                                                                                       
  ----- ------------------------------------------------------------------------ ------------------- --------------------------------------------------------- -- ----- ------------------------------------------ ------------------- ----------------------------------------

  : Equivariant embeddings of flag varieties[]{data-label="table:intro"}

Here $\mathrm {Gr}_{p}({\mathbb C}^{n})$ is the Grassmanian of $p$-planes in ${\mathbb C}^n$, $\mathrm Q^{m}{\mathbb C}:=\{z\in\mathbb P^{m+1}{\mathbb C}\,:\, z_0^2+\cdots+z_{m+1}^2=0\}$ is the complex quadric, and $\mathrm{IGr}_{n}({\mathbb C}^{2n}):=\{V\subset{\mathbb C}^{2n}\,:\,\mathrm{dim}\,V=n,
 \,Q|_V\equiv0\}$ is the Grassmanian of isotropic subspaces of ${\mathbb C}^{2n}$ equipped with a quadratic form $Q$, and $\mathrm{LGr}_{n}({\mathbb C}^{2n}):=\{V\subset{\mathbb C}^{2n}\,:\,\mathrm{dim}\,V=n,
 \,\omega|_{V\times V}\equiv0\}$ is the Grassmanian of Lagrangian subspaces of ${\mathbb C}^{2n}$ equipped with a symplectic form $\omega$.

For $Y\hookrightarrow X$ as in Table \[table:intro\] and any equivariant line bundle $\mathcal L_\lambda\to X$ with sufficiently positive $\lambda$ we give a necessary and sufficient condition for normal derivatives to become intertwiners:

${}$

Any $G'$-intertwining operator from $\mathcal O(X,\mathcal L_\lambda)$ to $\mathcal O(Y,\mathcal W)$ is given by normal derivatives with respect to the equivariant embedding $Y\hookrightarrow X$ of type (4), (5) or (6).

None of normal derivatives of positive order is a $G'$-intertwining operator for $Y\hookrightarrow X$ of type (1), (2) and (3).

See Theorem \[thm:nor\_der\] for the precise formulation of the statement (1) . By using the F-method we prove that all the differential intertwining operators appearing in the setting of statement (2) are built on Jacobi polynomials in one variable. Namely, let $P^{\alpha,\beta}_\ell(x)$ be the Jacobi polynomial, and $C^\alpha_\ell(x)$ the Gegenbauer polynomial. We inflate them to polynomials of two variables by $$P_\ell^{\alpha,\beta}(x,y):=y^\ell P_\ell^{\alpha,\beta}\left(2\frac xy+1\right)\quad\mathrm{and}\,\quad
C_\ell^\alpha(x,y):=x^{\frac\ell2}C_\ell^\alpha\left(\frac y{\sqrt x}\right).$$

Then we obtain explicit formulæ for equivariant differential operators correspondingly to (1), (2), and (3) in Table \[table:intro\]:

In that follows, ${\mathcal{L}}_{\lambda}$ stands for a homogeneous holomorphic line bundle, and ${\mathcal{W}}_{\lambda}^a$ a homogeneous vector bundle with typical fiber $S^a({\mathbb{C}}^m)$ ($m=n$ in (1); $=n-1$ in (2)) with parameter $\lambda$ (see Lemma \[lem:64\] for details).

For the symmetric pair $(U(n,1)\times U(n,1), \, U(n,1))$ the differential operator $$D_{X\to Y,a}:=
  P_a^{-\lambda'+{n},\, \lambda'+\lambda'' -{2n}-2a+1}\left(
\sum_{i=1}^{n}v_i \frac{\partial}{\partial z_i},
\sum_{j=1}^{n}v_j \frac{\partial}{\partial z_j}
\right)$$ is an intertwining operator from $\mathcal O(Y,\mathcal L_{(\lambda_1',\lambda_2')})\otimes \mathcal O(Y,\mathcal L_{(\lambda_1'',\lambda_2'')})$ to $\mathcal O(Y,\mathcal W_{(\lambda_1'+\lambda_1'',\lambda_2'+\lambda_2'')}^a)$, where $\lambda_1',
 \lambda_1'',\lambda_2',\lambda_2''\in{\mathbb Z}$, $\lambda'=\lambda_1'-\lambda_2'$, $\lambda''=\lambda_1''-\lambda_2''$, and $a\in{\mathbb N}$.

For the symmetric pair $(Sp(n,{\mathbb R}), Sp(n-1,{\mathbb R})\times Sp(1,{\mathbb R}))$ the differential operator $$D_{X\to Y,a}:=C_a^{\lambda-1}\left(
\sum_{1\leq i, j\leq n-1}2 v_iv_j
\frac{\partial^2}{\partial z_{ij}\partial z_{nn}},
\sum_{1\leq j\leq n-1}v_j\frac{\partial}{\partial z_{jn}}\right)$$ is an intertwining operator from $\mathcal O(X,\mathcal L_\lambda)$ to $\mathcal O(Y,\mathcal W_\lambda^a)$, where $\lambda\in{\mathbb Z}$, $a\in{\mathbb N}$.

For the symmetric pair $(SO(n,2), SO(n-1,2))$ the differential operator $$\begin{aligned}
 D_{X\to Y,a}:=C_a^{\lambda-\frac{n-1}2}\left(-\Delta_{{\mathbb C}^{n-1}},\frac{\partial}{\partial z_n}\right)\end{aligned}$$ is an intertwining operator from $\mathcal O(X,\mathcal L_\lambda)$ to $\mathcal O(Y,\mathcal L_{\lambda+a})$, where $\lambda\in{\mathbb Z}$ and $a\in{\mathbb N}$.

See Theorems \[thm:U(n,1)\], \[thm:Sp\], and \[thm:SO(2,n)\] for precise statements, respectively. The above operators exhaust all intertwining operators for generic parameter $\lambda$ (or $(\lambda', \lambda'')$ in (1)), see Theorem \[thm:C\].

The first statement corresponds to the decomposition of the tensor product, and gives rise to the classical Rankin–Cohen brackets in the case where $n=1$. An analogous formula for the third family was recently found in a completely different way by A. Juhl [@Juhl] in the setting of conformally equivariant differential operators with respect to the embedding of Riemannian manifolds $S^{n-1}\hookrightarrow S^{n}$.

Last but not least, we expect that the F-method could be a useful tool in other settings, for example, in constructing covariant operators in parabolic geometries (cf. [@KOSS]), and also in developing a theory of equivariant differential operators built on orthogonal multivariable polynomials associated to root systems.

Differential intertwining operators
===================================

In this section we discuss equivariant differential operators between homogeneous vector bundles in a more general setting than usual, namely, over different base spaces admitting a morphism. In this generality, we establish a natural bijection between such differential operators and certain Lie algebra homomorphisms, see Theorem \[thm:surject\].

 Pullback of differential operators
-----------------------------------

We understand the notion of differential operators between two vector bundles in the usual sense when the bundles are defined over the same base space. We extend this terminology in a more general setting, where there exists a morphism between base spaces.

\[def:21\] Let $\mathcal V\to X$ and $\mathcal W\to Y$ be two vector bundles and $p:Y\to X$ a smooth map. We say that a linear map $T:C^\infty(X,\mathcal V)\to C^\infty(Y,\mathcal W)$ is a *differential operator* if $T$ is a local operator in the sense that $$\label{eqn:suppT}
\operatorname{Supp}(T f)
 \subset
 {p^{-1}(\operatorname{Supp}f)},\qquad \mathrm{for\, any}\,
 f\in C^\infty (X,\mathcal V).$$ We write $\operatorname{Diff}(\mathcal V_X,\mathcal W_Y)$ for the vector space of such differential operators from $C^\infty(X,\mathcal V)$ to $C^\infty(Y,\mathcal W)$.

Since any smooth map is factorized into the composition of a submersion and an embedding, the following example describes the general situation.

1.  Let $p:Y\twoheadrightarrow X$ be a submersion. Choose an atlas of local coordinates $\{(x_i,z_j)\}$ on $Y$ in such a way that $\{x_i\}$ form an atlas on $X$. Then, every $T\in \operatorname{Diff}(\mathcal V_X,\mathcal W_Y)$ is locally of the form $$\sum_{\alpha\in{\mathbb N}^{\dim X}}h_\alpha(x,z)\frac{\partial^{|\alpha|}}{\partial x^\alpha},$$ where $h_\alpha(x,z)$ are $\operatorname{Hom}(V,W)$-valued smooth functions on $Y$.

2.  Let $i: Y\hookrightarrow X$ be an embedding. Choose an atlas of local coordinates $\{(y_i,z_j)\}$ on $X$ in such a way that $\{y_i\}$ form an atlas on $Y$. Then, every $T\in \operatorname{Diff}(\mathcal V_X,\mathcal W_Y)$ is locally of the form $$\sum_{(\alpha,\beta)\in{\mathbb N}^{\dim X}}g_{\alpha\beta}(y)\frac{\partial^{|\alpha|+|\beta|}}{\partial y^\alpha\partial z^\beta},$$ where $g_{\alpha,\beta}(y)$ are $\operatorname{Hom}(V,W)$-valued smooth functions on $Y$.

Let $\Omega_X:=\vert\bigwedge^{\mathrm {top}}T^*(X)\vert$ be the bundle of densities, and denote by $\mathcal V^*$ the dualizing bundle $\mathcal V^\vee\otimes\Omega_X$. By the L. Schwartz kernel theorem, any continuous linear map $T:C^\infty(X,\mathcal V)\to C^\infty(Y,\mathcal W)$ is given as $$f\mapsto \langle K_T(x,y),f(x)\rangle,$$ for some distribution $K_T\in \mathcal D'(X\times Y,\mathcal V^*\boxtimes \mathcal W)$ such that the second projection $\mathrm{pr}_2:X\times Y\to Y$ is proper on the support of $K_T$.

\[lem:1\] Let $p:Y\to X$ be a smooth map. A continuous operator $T:C^\infty(X,\mathcal V)\to C^\infty(Y,\mathcal W)$ is a differential operator in the sense of Definition \[def:21\] if and only if $\mathrm{Supp}\,K_T\subset\Delta(Y)$, where $\Delta(Y):=
\{(p(y),y):y\in Y\}\subset X\times Y$.

Suppose $\mathrm{Supp}\,K_T\subset\Delta(Y)$. By the structural theory of distributions supported on a submanifold [@SCH Chapter III, Théorème XXXVII] to $\Delta Y\subset X\times Y$, we have $\langle K_T,f\rangle\in\mathcal D'(Y,\mathcal W)$ is locally given as $$\label{eqn:halpha}
\sum_\alpha h_\alpha(y)\frac{\partial^{\vert\alpha\vert}f}{\partial x^\alpha}(p(y)),$$ for every $f\in C^\infty(X,\mathcal V)$ and some $h_\alpha(y)\in \mathcal D'(Y,\mathcal V^*\otimes\mathcal W)$. Thus $T$ is a differential operator in the sense of Definition \[def:21\].

Conversely, by the definition of the kernel distribution $K_T$, for any $(x_o,y_o)\in \mathrm{Supp}\,K_T$ and for any neighborhood $S$ of $x_o$ in $X$ there exists $f\in C^\infty(X,\mathcal V)$ such that $\mathrm{Supp}f\subset S$ and $(x_o,y_o)\in
\mathrm{Supp}\,f\times \mathrm{Supp}\,Tf$. If $T$ is a differential operator then by (\[eqn:suppT\]) $$(x_o,y_o)\in
\mathrm{Supp}\,f\times p^{-1}(\mathrm{Supp}\,f).$$ Since $S$ is an arbitrary neighborhood of $x_o$, then necessarily $(x_o,y_o)\in\Delta(Y)$ and therefore $\mathrm{Supp}\,K_T\subset\Delta(Y)$.

Next, suppose the two vector bundles $\mathcal V\to X$ and $\mathcal W\to Y$ to be equivariant with respect to a given Lie group $G$. Then we have natural actions on the spaces $C^\infty(X,\mathcal{V})$ and $C^\infty(Y,\mathcal{W})$ by translations. We set $$\begin{aligned}
\operatorname{Diff}_G(\mathcal V_X,\mathcal W_Y):=
\operatorname{Diff}(\mathcal V_X,\mathcal W_Y)\cap
\operatorname{Hom}_G(C^\infty(X,\mathcal V),C^\infty(Y,\mathcal W)).\end{aligned}$$

If $X$ and $Y$ are both Euclidean vector spaces and if $G$ contains the subgroup of translations of $Y$ then $\operatorname{Diff}_G(\mathcal V_X,\mathcal W_Y)$ is a subspace of the space of differential operators with constant coefficients.

Induced modules
---------------

Let $\mathfrak g$ be a Lie algebra over ${\mathbb C}$, and $U(\mathfrak g)$ its universal enveloping algebra. Let $\mathfrak h$ be a Lie subalgebra of $\mathfrak g$.

For an $\mathfrak h$-module $V$ we define the induced $U(\mathfrak g)$-module as $${\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V):=U(\mathfrak g)\otimes_{U(\mathfrak h)}V.$$ If $\mathfrak h$ is a Borel subalgebra and $\dim V = 1$, then ${\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V)$ is the standard Verma module.

For further purposes we formulate the following statement in terms of the contragredient representation $V^\vee$. Let $\mathfrak h'$ be another Lie subalgebra of $\mathfrak g$.

\[prop:35\] For finite dimensional $\mathfrak h'$-module $W$ we have

1.  $
    \operatorname{Hom}_{\mathfrak g}(\operatorname{ind}_{\mathfrak h'}^{\mathfrak g}(W^\vee),{\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee))\simeq
    \operatorname{Hom}_{\mathfrak h'}(W^\vee,{\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee)).
    $

2.  If $\mathfrak h'\not\subset\mathfrak h$, then $\operatorname{Hom}_{\mathfrak h'}(W^\vee,{\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee))=\{0\}$.

The first statement is due to the functoriality of the tensor product.

For the second statement it suffices to treat the case where $\mathfrak h'$ is one-dimensional. The assumption $\mathfrak h'\not\subset\mathfrak h$ implies that there is a direct sum decomposition of vector spaces: $$\mathfrak g=\mathfrak h' +\mathfrak q+\mathfrak h,$$ for some subspace $\mathfrak q$ in $\mathfrak g$. Then, by the Poincaré–Birkhoff–Witt theorem we have an isomorphism of $\mathfrak h'$-modules: $${\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee)\simeq U(\mathfrak h')\otimes_{\mathbb C}U'(\mathfrak q)\otimes_{\mathbb C}V^\vee,$$ where $U'(\mathfrak q):={\mathbb C}$-span$\{X_1\cdots X_\ell:\, X_1,\ldots, X_\ell\in \mathfrak q,\,\ell\in{\mathbb N}\}$. In particular, ${\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee)$ is a free $U(\mathfrak h')$-module. Hence no finite dimensional $\mathfrak h'$-submodule exists in ${\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee)$.

Differential operators between homogeneous vector bundles on different base spaces {#sec:33}
----------------------------------------------------------------------------------

Let $G$ be a real Lie group, $\mathfrak g({\mathbb R}):={\mathrm{Lie}(G)}$ and $\mathfrak g:=
\mathfrak g({\mathbb R})\otimes{\mathbb C}$. Analogous notations will be applied to other Lie groups.

Consider two actions $dR$ and $dL$ of the universal enveloping algebra $U(\mathfrak g)$ on the space $C^\infty (G)$ of smooth complex-valued functions on $G$ induced by the regular representation $L\times R$ of $G\times G$ on $C^\infty(G)$: $$\label{eqn:dR}
 (dR(X)f)(x):=\left.\frac {d}{dt}\right\vert_{t=0}f(xe^{tX}),\quad {\mathrm and}\quad
 (dL(X)f)(x):=\left.\frac {d}{dt}\right\vert_{t=0}f(e^{-tX}x).$$

Let $H$ be a closed subgroup of $G$. Given a finite dimensional representation $V$ of $H$ we define the homogeneous vector bundle $\mathcal V_X\equiv\mathcal V:= G\times_HV$ over $X:=G/H$. The space of smooth sections $C^\infty(X,\mathcal V)$ can be seen as a subspace of $C^\infty(G)\otimes V$: $$C^\infty(X,\mathcal V)\simeq C^\infty(G,V)^H\subset C^\infty(G)\otimes V.$$

The right differentiation given by (\[eqn:dR\]) $$C^\infty(G)\times U(\mathfrak g)\to C^\infty(G),
\quad
(f,u)\mapsto dR(u)f$$ together with the canonical coupling $V \times V^\vee \to \mathbb{C}$ induces a well-defined diagram of maps: $$\begin{matrix}
C^\infty(G)\otimes V\times U(\mathfrak g)\otimes_{\mathbb C}V^\vee&
\longrightarrow&
C^\infty(G)\\
\lhookuparrow\qquad\qquad\twoheaddownarrow&
&
\parallel\\
C^\infty(X,\mathcal V)\times {\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee)
&
\, \rule[.55ex]{.23em}{.5pt} \, \rule[.55ex]{.23em}{.5pt} \!\dashrightarrow&
C^\infty(G),
\end{matrix}$$ because $C^\infty(X,\mathcal V)\simeq
C^\infty(G,V)^H$. In turn, we get the following $\mathfrak g$-homomorphism: $$\label{eqn:1234}
{\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee){\longrightarrow}\operatorname{Hom}_G(C^\infty(X,\mathcal V),
C^\infty(G)),$$ where $C^\infty(G)$ is regarded as $G \times \mathfrak{g}$-module via $L\times dR$.

Consider another connected closed Lie subgroup $H'$. Given a finite dimensional representation $W$ of $H'$ we form a homogeneous vector bundle $\mathcal W_Z\equiv\mathcal W:=G\times_{H'}W$ over $Z:=G/H'$. Taking the tensor product of (\[eqn:1234\]) with $W$, we get a morphism: $$\operatorname{Hom}_{\mathbb C}(W^\vee,{\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee)){\longrightarrow}\operatorname{Hom}_G(C^\infty(X,\mathcal V),
C^\infty(G,W)).$$

Taking $\mathfrak h'$-invariants, we have then a morphism $$\label{eqn:morf}
\operatorname{Hom}_{\mathfrak h'}(W^\vee,{\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee)){\longrightarrow}\operatorname{Hom}_G(C^\infty(X,\mathcal V),
C^\infty(Z,\mathcal W)),\quad \varphi\mapsto D_\varphi.$$ We note that the map is injective.

Take any subgroup $G'$ containing $H'$ and form a homogeneous vector bundle $\mathcal W_Y:= G'\times_{H'} W$ over $Y=G'/H'$. Then, the vector bundle $\mathcal W_Y$ is isomorphic to the restriction $\mathcal W_Z\vert_Y$ of the vector bundle $\mathcal W_Z$ to the submanifold $Y$ of the base space $Z$. Let $R_{Z\to Y}:C^\infty(Z,\mathcal W_Z)\to C^\infty(Y,\mathcal W_Y)$ be the restriction map of sections. We set $$\label{eqn:DXY}
D_{X\to Y}(\varphi):=R_{X\to Y}\circ D_\varphi.$$

The next theorem describes explicitly the image $D_{X\to Y}$ which, according to Proposition \[prop:35\], is non–trivial only when $H'\subset H$, that is, when the following diagram exists: $$\xymatrix{
Z=G/H'\ar@{->>}[dr]&\\
Y=G'/H'\ar@{^{(}->}[u]\ar[r]&X=G/H
}.$$

\[thm:surject\] Let $G$ be a Lie group, $H$ and $H'$ are closed connected subgroups such that $H'\subset H$. Consider two finite dimensional representations $V$ and $W$ of $H$ and $H'$, respectively. Then, for any $G'$ containing $H'$ the map $D_{X\to Y} $ establishes a bijection: $$\begin{aligned}
\label{eqn:isoo}
D_{X\to Y}:\operatorname{Hom}_{\mathfrak h'}(W^\vee,{\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee))
\stackrel{\sim}{\longrightarrow}
\operatorname{Diff}_{G'}( \mathcal V_X,\mathcal W_Y).\end{aligned}$$

1.  By the functoriality of the tensor product, the bijection may be restated as $$D_{X\to Y}:\operatorname{Hom}_{\mathfrak g'}(\operatorname{ind}_{\mathfrak{h}'}^{\mathfrak{g}'}(W^\vee),{\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee))
    \stackrel{\sim}{\longrightarrow}
    \operatorname{Diff}_{G'}( \mathcal V_X,\mathcal W_Y).
    \leqno{(\ref{eqn:isoo})'}$$

2.  Theorem \[thm:surject\] is well-known when $X=Y$, i.e. $G' = G$ and $H' = H$, in particular in the setting of complex flag varieties, see e.g. [@HJ; @Ks74].

3.  We shall consider the case where $H'=H \cap G'$ in later applications, however, the setting of Theorem \[thm:surject\] covers also the cases where the natural morphism $Y\to X$ is not injective, i.e. where $H'\subsetneqq H \cap G'$.

4.  The left–hand side of does not depend on the choice of $G'$. This fact is reflected by the commutativity of the following diagram. $$\label{eqn:DXYdiag}
    \xymatrix{
    \operatorname{Hom}_{\mathfrak h'}( W^\vee,{\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee))\ar[dr]_{D_{X\to Y}}^\sim
    \ar[r]^{{\quad\sim}}&
    \operatorname{Diff}_{G}(\mathcal V_X,{\mathcal W_Z}\ar[d]^{R_{Z\to Y}})\\
    & \operatorname{Diff}_{G'}(\mathcal V_X,{\mathcal W_Y})
    }$$

Before giving a proof of Theorem \[thm:surject\] we state its variants, namely, for disconnected subgroups (Corollary \[cor:2.9\]) and for the holomorphic case (Proposition \[prop:2.10\]).

In dealing with a representation $V$ of a disconnected subgroup $H$, we notice that the diagonal $H$-action on $U(\mathfrak{g}) \otimes_{\mathbb{C}} V^\vee$ defines a representation of $H$ on $\operatorname{ind}_{\mathfrak{h}}^{\mathfrak{g}}(V^\vee)$ and thus ${\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee)$ is endowed with a $(\mathfrak g, H)$-module structure.

\[cor:2.9\] Let $H' \subset H$ be (possibly disconnected) closed subgroups of $G$ with Lie algebras $\mathfrak{h}' \subset \mathfrak{h}$, respectively. Suppose $V$ and $W$ are finite dimensional representations of $H$ and $H'$, respectively. Let $G'$ be any subgroup containing $H'$, and $\mathcal V_X:=G\times_{H} V$ and $\mathcal W_Y:=G'\times_{H'} W$ be the corresponding homogeneous vector bundles. Then, there exists the following natural bijection: $$D_{X\to Y}:
\operatorname{Hom}_{H'}(W^\vee,{\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee))
\stackrel{\sim}{\longrightarrow}
\operatorname{Diff}_{G'}\left( \mathcal V_X,
\mathcal W_Y\right),$$ or equivalently, $$D_{X\to Y}:
\operatorname{Hom}_{(\mathfrak g',H')}({\operatorname{ind}_{\mathfrak h'}^{\mathfrak g'}}(W^\vee),{\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee))
\stackrel{\sim}{\longrightarrow}
\operatorname{Diff}_{G'}\left( \mathcal V_X,
\mathcal W_Y\right),$$

\[prop:2.10\] Theorem \[thm:surject\] remains valid for complex Lie groups and equivariant holomorphic differential operators.

The proofs of Corollary \[cor:2.9\] and Proposition \[prop:2.10\] are parallel to the one of Theorem \[thm:surject\], which is based on the explicit construction of the bijection $D_{X\to Y}$.

Proof of Theorem \[thm:surject\]
--------------------------------

The proof for the surjectivity of $D_{X\to Y}$ reduces to a realization of the induced $U(\mathfrak g)$-modules ${\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V)$ in the space of distributions. We begin with some notations.

Let ${\mathbb{C}}_{2\rho}$ denote the one-dimensional representation of $H$ defined by $$h \mapsto \vert \det (\operatorname{Ad}_{G/H}(h):
{\mathfrak {g}}/{\mathfrak {h}}
\to
{\mathfrak {g}}/{\mathfrak {h}})
 \vert^{-1}.$$

The bundle of densities $\Omega_{G/H}$ is given as a $G$-equivariant line bundle, $$\Omega_{G/H}
     \simeq G\times_H\vert \det\nolimits^{-1}\operatorname{Ad}_{G/H}
     \vert\simeq G\times_H{\mathbb C}_{2\rho}.$$ Given an $H$-module $V$, we define a ‘twist’ of the contragredient representation by $$V_{2\rho}^\vee := V^\vee\otimes\vert\det{}^{-1}\operatorname{Ad}_{G/H}\vert
\simeq V^\vee \otimes \mathbb{C}_{2\rho}.$$ Then the dualizing bundle is given, as a homogeneous vector bundle, by: $$\label{eqn:V2rho}
\mathcal V^*\equiv\mathcal V^\vee_{2\rho}:=\mathcal V^\vee\otimes\Omega_{G/H}\simeq G\times_HV_{2\rho}^\vee.$$

In what follows $\mathcal{D}'(X,\mathcal{V}^*)$ (respectively, $\mathcal{E}'(X,\mathcal{V}^*)$) denotes the space of ${V}^*$-valued distributions (respectively, those with compact support). We shall regard distributions as generalized functions à la Gelfand rather than continuous linear forms on $C_c^\infty(X,\mathcal{V})$ (respectively, $C^\infty(X,\mathcal{V})$), and write $\int_X\omega$ for the pairing of $\omega\in\mathcal E'(X,\Omega_X)$ and the constant function $\mathbf 1_X$ on $X$. For a compact subset $S$ of $X$, we write $\mathcal{D}'_S(X,\mathcal{V}^*) = \mathcal{E}'_S(X,\mathcal{V}^*)$ for the space of distributions supported on $S$.

For a homogeneous vector bundle $\mathcal V$, we shall use the notation $\mathcal V^\vee_{2\rho}$ rather than $\mathcal V^*$. Let $o=eH\in X$. Define a vector valued Dirac $\delta$-function $$\delta:V^\vee{\longrightarrow}\mathcal E'_{[o]}(X,\mathcal V^\vee_{2\rho}),\qquad v^\vee\mapsto \delta\otimes v^\vee,$$ by $$\langle f,\delta\otimes v^\vee\rangle:=\langle f(e),v^\vee\rangle,
\quad
\text{for $f\in  C^\infty(X,\mathcal V)\simeq C^\infty(G,V)^H$}.$$ The following lemma is standard:

\[lem:EHom\] ${}$

1.  The $G$-invariant functional $\omega\mapsto\int_X\omega$ induces a $G$-invariant bilinear map $$\label{eqn:Gbilinear}
    C^\infty(X,\mathcal V)\times \mathcal E'(X,\mathcal V^\vee_{2\rho}){\longrightarrow}{\mathbb C}.$$

2.  Let $S$ be a closed subset of $X$ and $U$ an open neighborhood of $S$ in $X$. Then, by restriction, the above map gives rise to a $\mathfrak g$-invariant pairing: $$C^\infty(U,\mathcal V)\times \mathcal E'_S(X,\mathcal V^\vee_{2\rho}){\longrightarrow}{\mathbb C}.$$

3.  The map $$\label{eqn:isoMh}
    {\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee){\stackrel{\sim}{{\longrightarrow}}}\mathcal E'_{[o]}(X,\mathcal V^\vee_{2\rho}),
    \qquad u\otimes v^\vee\mapsto dL(u)\left(\delta\otimes v^\vee\right),$$ is a $\mathfrak g$-isomorphism.

Now let us consider the setting of Theorem \[thm:surject\] where we have a $G'$-equivariant (but not necessarily injective) morphism from $Y=G'/H'$ to $X=G/H$.

\[lem:A\] Suppose that $G'$ is a subgroup of $G$. Then the multiplication map $$m:G\times G'\to G,\quad (g,g')\mapsto (g')^{-1}g,$$ induces the isomorphism: $$m^*:\left(\mathcal D'(X,\mathcal V_{2\rho}^\vee)\otimes W\right)^{\Delta(H')}
\stackrel{\sim}{{\longrightarrow}}
\mathcal D'(X\times Y,\mathcal V_{2\rho}^\vee\boxtimes \mathcal W)^{\Delta(G')}.$$

The image of the coproduct $m^*: \mathcal D'(G)\to\mathcal D'(G\times G')$ is $\mathcal D'(G\times G')^{\Delta(G')}$, where $G'$ acts diagonally from the left. Thus, considering the remaining $G\times G'$ action from the right, we take $H\times H'$-invariants with respect to the diagonal action in $$\mathcal D'(G)\otimes V_{2\rho}^\vee\otimes W
\stackrel{\sim}{{\longrightarrow}}\mathcal D'(G\times G')^{\Delta(G')}\otimes V_{2\rho}^\vee\otimes W,$$ and therefore we get Lemma.

\[lem:312\] There is a natural bijection: $$\begin{aligned}
\operatorname{Diff}_{G'}(\mathcal V_X,\mathcal W_Y)
\stackrel{\sim}{{\longrightarrow}}
(\mathcal D'_{[o]}(X,\mathcal V_X)\otimes W)^{\Delta(H')},\quad
T\mapsto (m^*)^{-1}(K_T).\end{aligned}$$

Let $T\in
\operatorname{Diff}_{G'}( \mathcal V_X,\mathcal W_Y)$. By Lemma \[lem:1\] the distribution kernel $K_T$ is supported on the diagonal set $\Delta(Y)\subset X\times Y$. Via the bijection $m^*$ given in Lemma \[lem:A\] we thus have $$\operatorname{Supp}((m^*)^{-1}K_T) \subset\{o\}.$$

Conversely, take any element $F\in (\mathcal D'_{[o]}(X,\mathcal V_X)\otimes W)^{\Delta(H')}$, then $m^*(F)$ is supported on the diagonal set $\Delta(Y)\subset X\times Y$. Since it is invariant under the diagonal action of $G'$, all the distributions $h_\alpha$ in (\[eqn:halpha\]) are smooth. Therefore $m^*(F)$ defines a differential operator from $C^\infty (X,\mathcal V)$ to $C^\infty(Y,\mathcal W)$.

Taking the tensor product of each term in (\[eqn:isoMh\]) with the finite dimensional representation $W$ of $H'$, we get a bijection between the subspaces of $\mathfrak h'$-invariants: $$\operatorname{Hom}_{\mathfrak h'}(W^\vee,{\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee))
\stackrel{\sim}{\longrightarrow}
(\mathcal D'_{[o]}(X,\mathcal V_X)\otimes W)^{\Delta(H')}.$$

Composing this with the bijection in Lemma \[lem:312\], we obtain a bijection from $\operatorname{Hom}_{\mathfrak h'}(W^\vee,{\operatorname{ind}_{\mathfrak h}^{\mathfrak g}}(V^\vee))$ to $\operatorname{Diff}_{G'}(\mathcal V_X,\mathcal W_Y)$, which is by construction nothing but $D_{X\to Y}$ in Theorem \[thm:surject\].

To achieve our goal to describe intertwining operators between discretely decomposable generalized Verma modules we shall now make the correspondence $\varphi\mapsto D_{X\to Y}(\varphi)$ explicit.

F-method
========

In this section we develop the F-method in details. The recipe is given in Section \[sec:44\] relying on the algebraic Fourier transform of generalized Verma modules (Theorem \[thm:313\] and Proposition \[prop:Fc\]). Some useful lemmas for actual computations for vector-bundle valued differential operators are collected in Section \[subsec:36\].

Weyl algebra and algebraic Fourier transform
--------------------------------------------

Let $E$ be a vector space over ${\mathbb C}$. The Weyl algebra $\mathcal D(E)$ is the ring of holomorphic differential operators on $E$ with polynomial coefficients.

\[def:hat\] We define the *algebraic Fourier transform* as an isomorphism of two Weyl algebras on $E$ and its dual space $E^\vee$: $$\mathcal D(E)\to\mathcal D(E^\vee), \qquad T\mapsto \widehat T,$$ induced by $$\widehat{\frac\partial{\partial z_j}}:=-\zeta_j,\quad
\widehat z_j:=\frac\partial{\partial\zeta_j},\quad 1\leq j\leq n=\dim E.$$ where $(z_1,\ldots,z_n)$ are coordinates on $E$ and $(\zeta_1,\ldots,\zeta_n)$ are coordinates on $E^\vee$.

Definition \[def:hat\] does not depend on the choice of coordinates.

The natural action of the general linear group $GL(E)$ on $E$ yields a representation of $GL(E)$ on the ring $\operatorname{Pol}(E)$ of polynomials of $E$. Taking its differential, we get a Lie algebra homomorphism $\operatorname{End}(E)\to\mathcal D(E)$. In the coordinates this homomorphism amounts to $$\label{eqn:endo1}
\operatorname{End}(E)\to\mathcal D(E),\quad
A\mapsto -{{}^t\!}Z\,{{}^t\!}A\,\partial_Z\equiv - \sum_{i,j}A_{ij}z_j\frac{\partial}
{\partial z_i}.$$

Likewise, for the contragredient representation on $E^\vee$ we have $$\label{eqn:endo2}
\operatorname{End}(E)\to\mathcal D(E^\vee),\quad
A\mapsto {{}^t\!}\zeta \,A\,\partial_\zeta\equiv \sum_{i,j}A_{ji}\zeta_j\frac{\partial}
{\partial \zeta_i}.$$

Further, a representation $\sigma$ of $\mathfrak g$ on $E$ gives rise to Lie algebra homomorphisms $\Psi_\sigma:\mathfrak g\to
\mathcal D(E)$ and $\Psi_{\sigma^\vee}:\mathfrak g\to
\mathcal D(E^\vee)$.

\[prop:Phihat\] The algebraic Fourier transform $T\mapsto\widehat T$ relates $\Psi_{\sigma}$ and $\Psi_{\sigma^\vee}$ as follows: $$\widehat {\Psi}_\sigma=\Psi_{\sigma^\vee}+\mathrm{Trace}\circ\sigma.$$

By (\[eqn:endo1\]) and (\[eqn:endo2\]) the computation of the difference of $\widehat {\Psi}_\sigma$ and $\Psi_{\sigma^\vee}$ reduces to the commutation relations $$\frac{\partial}
{\partial \zeta_i}\zeta_i-\zeta_i\frac{\partial}
{\partial \zeta_i}=\delta_{ij},$$ in the Weyl algebra $\mathcal D(E^\vee)$. It yields $\mathrm{Trace}\circ\sigma$.

For actual computations we need in the latter part of this work it is convenient to give another interpretation of the algebraic Fourier transform using specific real forms of $E$.

Fix a real form $E({\mathbb R})$ of $E$. Let $\mathcal E'_{[0]}(E({\mathbb{R}}))$ be the space of distributions on the vector space $E({\mathbb R})$ supported at the origin which is a convolution algebra with unit $\delta$, the Dirac delta function. Define a ‘Fourier transform’ $\mathcal F_{c}$ by the following formula: $$\label{eqn:algF}
\mathcal F_{c}: \mathcal E'_{[0]}(E({\mathbb{R}}))  \xrightarrow{\sim} \operatorname{Pol}(E^\vee),\quad
\mathcal F_{c}f(\xi):=\langle f(x),\Phi( x,\zeta)\rangle,
\qquad \zeta\in E^\vee,$$ where $\Phi$ is given by: $$\Phi: E\times E^\vee\to{\mathbb C}\quad
(x,\zeta)\mapsto \Phi(x,\zeta):=e^{\langle x,\zeta\rangle}.$$ Notice that we do not include $\sqrt{-1}$ in the definition of $\Phi(x,\zeta)$. This map is an algebra isomorphism and its inverse $\mathcal F_{c}^{-1}:  \operatorname{Pol}(E^\vee) \xrightarrow{\sim}
\mathcal E'_{[0]}(E({\mathbb{R}}))$ satisfies: $$\mathcal F_c^{-1}(\mathbf 1)=\delta.$$

\[rem:45\] The algebraic Fourier transform defined in Definition \[def:hat\] satisfies $$\label{eqn:F-trans}
 \widehat T=\mathcal F_{c}\circ T\circ \mathcal F_{c}^{-1}
\quad\text{for $T \in \mathcal{D}(E)$},$$ and the formula characterizes $\widehat{T}$ because the Weyl algebra $\mathcal D(E)$ acts faithfully on $\mathcal E'_{[0]}(E({\mathbb{R}}))$ and so does $\mathcal D(E^\vee)$ on $\operatorname{Pol}(E^\vee)$. The composition $\mathcal F_{c}\circ T\circ \mathcal F_{c}^{-1}$ does not depend on the choice of a real form $E({\mathbb R})$.

Symbol map and reversing signatures
-----------------------------------

In the sequel we let $E$ be a certain nilpotent Lie algebra $\mathfrak{n}_-$. Even in the case when it is abelian, the left and right actions $dL$ and $dR$ cause different signatures. The purpose of this section is to set up carefully and clearly relations involving various signatures in connection with the algebraic Fourier transform.

We define the symbol map $$\begin{aligned}
\operatorname{Symb}:
 \operatorname{Diff}^{\mathrm {const}}(E){\stackrel{\sim}{\longrightarrow}}\operatorname{Pol}(E^\vee),
\quad
D_x \mapsto Q(\zeta)\end{aligned}$$ by the following characterization $$(D_x\Phi) (x,\zeta)=Q(\zeta)\Phi(x,\zeta).$$ The differential operator on $E$ with symbol $Q(\zeta)$ will be denoted by $\partial Q_x$.

For any homogeneous polynomial $P$ on $E^\vee$ of degree $\ell$ and any polynomial $Q$ on $E$ seen as a multiplication operator one has $$\label{eqn:aFP}
\widehat{\partial P_x}=(-1)^\ell P(\zeta),\qquad
\widehat{Q(x)}=\partial Q_\zeta.$$

For further purposes notice that $\widehat {E_z}=- {E_{\zeta}}- n$, where $E_z:=\sum_j z_j\frac{\partial}{\partial z_j}$ stands for the Euler operator on the corresponding vector space.

Denote $\gamma:S(E)\xrightarrow{\sim} \operatorname{Pol}
(E^\vee)$ the canonical isomorphism, and define another algebra isomorphism $$\gamma_{sgn}: S(E)\xrightarrow{\sim} \operatorname{Pol}(E^\vee),$$ by $\gamma\circ a$, where $a:S(E)\to S(E)$ is the automorphism induced by the antipodal map $X\mapsto -X$ for every $X\in E$.

Now we regard $E$ as an abelian Lie algebra over ${\mathbb C}$, and identify its enveloping algebra $U(E)$ with the symmetric algebra $S(E)$. Then, the right and left-infinitesimal actions induce two isomorphisms: $$dR: S(E)\xrightarrow{\sim}\operatorname{Diff}^{\operatorname{const}}(E),
\qquad
dL: S(E)\xrightarrow{\sim}\operatorname{Diff}^{\operatorname{const}}(E).$$

By the definition of the symbol map, we get, $$\operatorname{Symb}\circ dR=\gamma,\qquad\operatorname{Symb}\circ dL=\gamma_{sgn}.$$ On the other hand, it follows from (\[eqn:aFP\]) that $$\widehat{dL(u)}=\gamma(u),\qquad \widehat{dR(u)}=\gamma_{sgn}(u),$$ for every $u\in S(E)\simeq U(E)$, where polynomials are regarded as multiplication operators. Hence we have proved

\[lem:RLFc\] For any $u\in U(E)$, one has: $$\operatorname{Symb}\circ dR(u)=
\widehat{dL(u)},\qquad \operatorname{Symb}\circ dL(u)=\widehat{dR(u)}.$$

Construction of equivariant differential operators by algebraic Fourier transform
---------------------------------------------------------------------------------

In Section \[sec:33\] we explained a general construction of equivariant differential operators between homogeneous vector bundles on different base spaces $X$ and $Y$ equipped with a morphism $Y\to X$ by using the right action $dR$ of $U(\mathfrak g)$ on $C^\infty(G)$. Now we develop another construction based on the algebraic Fourier transform. In Theorem \[thm:313\] we show that both methods give the same operators when the base space $X$ is a flag variety $G/P$, where the parabolic subgroup $P$ has an abelian nilradical.

From now, let $G$ be a real semisimple Lie group, $P$ a parabolic subgroup of $G$ with Levi decomposition $P=LN_+$ and $V$ a finite dimensional representation of $P$. We apply the results of the previous sections to the case when $H=P$.

Let $LN_-$ be the opposite parabolic subgroup and $\mathfrak n_-({\mathbb R})$ the Lie algebra of $N_-$. By the Bruhat decomposition we have the following open dense imbedding $\iota:\mathfrak n_-({\mathbb R})\hookrightarrow G/P$ given by $X\mapsto \exp(X)\cdot o$, where $o=eP\in G/P$. The pullback of the $G$-equivariant vector bundle $\mathcal V^\vee_{2\rho}\to G/P$ via $\iota$ is trivialized into $\mathfrak{n}_-({{\mathbb R}})\times V^\vee\to \mathfrak{n}_-({{\mathbb R}})$ and thus we have a linear isomorphism: $$\label{eqn:iso1}
\mathcal E'_{[o]}(G/P,\mathcal V^\vee_{2\rho})\stackrel{\sim}{\longrightarrow}
 \mathcal E'_{[0]}(\mathfrak n_-({\mathbb R}), V^\vee),$$ through which we induce the action of the Lie algebra $\mathfrak g$ on $\mathcal E'_{[0]}(\mathfrak n_-({\mathbb R}), V^\vee)$ denoted by $d\pi$.

The Killing form of $\mathfrak g$ identifies $\mathfrak n_-^\vee({\mathbb R})\simeq \mathfrak n_+({\mathbb R})$ and thus the algebraic Fourier transform (\[eqn:algF\]) gives rise to a linear isomorphism: $$\mathcal F_c\otimes \operatorname{id}: \mathcal E'_{[0]}(\mathfrak n_-({\mathbb R}), V^\vee)\stackrel{\sim}{\longrightarrow} \operatorname{Pol}(\mathfrak n_+)\otimes V^\vee,$$ through which we induce the action of the Lie algebra $\mathfrak g$ further on the right-hand side, namely, $$\widehat{d\pi}(X):=\mathcal F_c\circ d\pi(X)\circ \mathcal F_c^{-1}.$$ By Remark \[rem:45\] this action $\widehat{d\pi}$ is given by the Fourier transform of operators: $$\widehat{d\pi}(X) = \widehat{d\pi(X)}.$$

In summary we have the following $\mathfrak g$-isomorphisms: $$\label{eqn:indPol}
F_c:{\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(V^\vee)
{\stackrel{(\ref{eqn:isoMh})}{\overset{\sim}{{\longrightarrow}}}}
\mathcal E'_{[o]}(G/P,\mathcal V^\vee_{2\rho}){\stackrel{(\ref{eqn:iso1})}{\overset{\sim}{{\longrightarrow}}}} \mathcal E'_{[0]}(\mathfrak n_-({\mathbb R}), V^\vee)
{\stackrel{\mathcal F_c\otimes \operatorname{id}}{\stackrel{\sim}{\longrightarrow}}}\operatorname{Pol}(\mathfrak n_+)\otimes V^\vee.$$

(1)The map $F_c$ does not depend on the choice of a real form $G$ of $G_{\mathbb C}$ that appears in the two middle terms of (\[eqn:indPol\]). This fact will be used for actual computations (see Section \[section:6\]).

(2)The isomorphism $F_c:{\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(V^\vee)\stackrel{\sim}{\longrightarrow}
 \operatorname{Pol}(\mathfrak n_+)\otimes V^\vee$ only depends on the infinitesimal action of $P$ on $V$, and so does $\widehat{d\pi}$.

Let $E$ be a hyperbolic element of ${\mathfrak {g}}$ defining a parabolic subalgebra ${\mathfrak {p}}(E)={\mathfrak {l}}(E)+{\mathfrak {n}}(E)$, namely, ${\mathfrak {l}}(E)$ and ${\mathfrak {n}}(E)$ are the sum of eigenspaces of $\operatorname{ad}(E)$ with zero and positive eigenvalues, respectively.

Let $G'$ be a reductive subgroup of $G$ and $\mathfrak g'=\operatorname{Lie}(G')\otimes{\mathbb C}$.

\[def:3.2\] A parabolic subalgebra ${\mathfrak {p}}$ is said to be ${\mathfrak {g}}^\prime$-[*[compatible]{}*]{} if there exists a hyperbolic element $E^\prime\in{\mathfrak {g}}^\prime$ such that ${\mathfrak {p}}={\mathfrak {p}}(E^\prime)$.

If ${\mathfrak {p}}={\mathfrak {l}} +{\mathfrak {n}}$ is ${\mathfrak {g}}^\prime$-compatible, then ${\mathfrak {p}}^\prime :={\mathfrak {p}}\cap{\mathfrak {g}}^\prime$ becomes a parabolic subalgebra of ${\mathfrak {g}}^\prime$ with the following Levi decomposition: $${\mathfrak {p}}^\prime ={\mathfrak {l}}^\prime +{\mathfrak {n}}^\prime :=({\mathfrak {l}}\cap{\mathfrak {g}}^\prime)+({\mathfrak {n}}\cap{\mathfrak {g}}^\prime),$$ The key tool for the F-method that we explain in Section \[sec:44\] is the following assertion that the two approaches (the canonical invariant pairing ) and the algebraic Fourier transform ) give rise to the same differential operators:

\[thm:313\] Suppose ${\mathfrak{p}}$ is a parabolic subalgebra ${\mathfrak {g}}$ which is ${\mathfrak {g}}'$-compatible. Assume further the nilradical $\mathfrak n_+$ is abelian. Then the following diagram commutes: $$\begin{matrix}
\operatorname{Hom}_{\mathbb C}(W^\vee,{\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(V^\vee)) \,
{\stackrel{F_c\otimes \mathrm{id}}{\overset{\sim}{{\longrightarrow}}}}
&\operatorname{Pol}(\mathfrak n_+)\otimes \operatorname{Hom}_{\mathbb C}(V,W)
& \stackrel{\operatorname{Symb}\otimes \mathrm{id}}{\stackrel{\sim}{\longleftarrow}}\operatorname{Diff}^{\mathrm {const}}(\mathfrak n_-)\otimes \operatorname{Hom}_{\mathbb C}(V,W)\\
\cup & \circlearrowright& \cup\\
\operatorname{Hom}_{\mathfrak p'}(W^\vee,{\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(V^\vee))
&\underset{D_{X\to Y}}{\overset{\sim}{\longrightarrow}}&
\operatorname{Diff}_{G'}(\mathcal V_X,\mathcal W_Y).
\end{matrix}$$

Take an arbitrary $\varphi\in \operatorname{Hom}_{\mathfrak p'}
 (W^\vee,{\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(V^\vee))$, which may be written as $$\varphi=\sum_j u_j\otimes\psi_j\in U(\mathfrak n_-)\otimes
 \operatorname{Hom}_{\mathbb C}(V,W)$$ by the Poincaré–Birkhoff–Witt theorem $U(\mathfrak{g}) \simeq U(\mathfrak{n}_-) \otimes U(\mathfrak{p})$. Then it follows from (\[eqn:isoMh\]) and (\[eqn:indPol\]) that $$F_c\varphi=\sum_j\mathcal F_c\, dL(u_j) \delta\otimes \psi_j
 \in \operatorname{Pol}(\mathfrak n_+)\otimes\operatorname{Hom}_{\mathbb C}(V,W).$$ Since $\delta=\mathcal F_c^{-1}(\mathbf 1)$, we get $$F_c\varphi=\sum_j\widehat{dL(u_j)}\otimes \psi_j.$$ On the other hand, by the construction (\[eqn:DXYdiag\]), $$D_{X\to Y}(\varphi)=\sum_j dR(u_j)\otimes \psi_j.$$ Now we use the assumption that $\mathfrak n_+$ or equivalently $\mathfrak n_-$ is abelian. Then, in the coordinates $\mathfrak n_-\hookrightarrow G/P$ the operator $dR(u_j)$ for $u_j\in U(\mathfrak n_-)$ defines a constant coefficient differential operator on $\mathfrak n_-$. Thus $ D_{X\to Y}(\varphi)$ can be regarded as an element of $\operatorname{Diff}^{\operatorname{const}}(\mathfrak n_-)\otimes
\operatorname{Hom}_{\mathbb C}(V,W)$.

Applying the symbol map we have $$(\operatorname{Symb}\otimes\operatorname{id})\circ D_{X\to Y}(\varphi)
=\sum_j \operatorname{Symb}\circ dR(u_j)\otimes\psi_j
=\sum_j\widehat{dL(u_j)}\otimes \psi_j,$$ where the last equation follows from Lemma \[lem:RLFc\]. Thus we have proved that $$(F_c\otimes\rm{id})\varphi=(\operatorname{Symb}\otimes\operatorname{id})\circ D_{X\to Y}(\varphi),$$ whence the Theorem.

Fourier transform of principal series representations
-----------------------------------------------------

As a preparation of the $F$-method, we recall some standard facts on principal series representations of complex reductive Lie groups. Let $P_{{\mathbb C}} = L_{{\mathbb C}} N_+$ be a parabolic subgroup of a connected complex reductive Lie group $G_{{\mathbb C}}$. Let $\lambda$ be a holomorphic representation of $L_{{\mathbb C}}$ on $V$, and extend it to $P_{\mathbb C}$ by letting the unipotent radical $N_+$ act trivially. We form a $G_{{\mathbb C}}$-equivariant holomorphic vector bundle $\mathcal{V}$ and $\mathcal{V}^* \equiv \mathcal{V}_{2\rho}^\vee$ over the (generalized) flag variety $G_{{\mathbb C}}/P_{{\mathbb C}}$ associated to $\lambda$ and $\mu:=\lambda^\vee\otimes{\mathbb C}_{2\rho}$, respectively. We consider the representation $\pi_\mu$ of $G_{{\mathbb C}}$ on $C^\infty(G_{{\mathbb C}}/P_{{\mathbb C}},\mathcal{V}_{2\rho}^\vee)$. We use the same notation $\pi_\mu$ for the representation of a given subgroup of $G_{\mathbb C}$ on $C^\infty(U,\mathcal V^\vee_{2\rho}\vert_U)$ if $U$ is an invariant open subset of $G_{\mathbb C}/P_{\mathbb C}$.

According to the Gelfand–Naimark decomposition $\mathfrak g=\mathfrak n_-+\mathfrak l+\mathfrak n_+$ of the complex reductive Lie algebra $\mathfrak{g}$, we have a diffeomorphism $$\mathfrak n_-\times L_{\mathbb C}\times \mathfrak n_+\to G_{\mathbb C},\quad
(X,\ell, Y)\mapsto (\exp X)\ell(\exp Y),$$ into an open dense subset $G^{\mathrm{reg}}_{\mathbb C}$ of $G_{\mathbb C}$. Let $$p_\pm: G^{\mathrm{reg}}_{\mathbb C}{\longrightarrow}\mathfrak n_\pm,\qquad p_o:G^{\mathrm{reg}}_{\mathbb C}\to L_{\mathbb C},$$ be the projections characterized by the identity $$\exp(p_-(g))p_o(g)\exp(p_+(g))=g.$$

For a section $f \in C^\infty(G_{{\mathbb C}}/P_{{\mathbb C}},\mathcal{V}_{2\rho}^\vee)$, we define $F \in C^\infty(\mathfrak{n}_-,V^\vee)$ by $$F(X):=f(\exp X),\qquad\text{for $X\in\mathfrak n_-$}.$$ We use the same letter $\pi_\mu$ to denote the ‘action’ of $G_{{\mathbb C}}$ on $C^\infty(\mathfrak{n}_-,V^\vee)$, more precisely, $$\left(\pi_\mu(g)F\right)(X)=\mu(p_o(g^{-1}\exp X))^{-1} F(p_-(g^{-1}\exp X))$$ for $g\in G_{\mathbb C}$ and $X\in\mathfrak n_-$ such that $g^{-1}\exp X\in G^{\mathrm{reg}}_{\mathbb C}$. In particular, for $g=m\exp Z$, with $m\in L_{\mathbb C}, Z\in\mathfrak n_-$, we have $$\label{eqn:simpler}
\left(\pi_\mu(g)F\right)(X)=\mu(m)F(\mathrm{Ad}(m)^{-1}X-Z).$$

The goal of this subsection is to analyze the infinitesimal action $d\pi_\mu(Y)$ for $X \in \mathfrak{g}$ and its algebraic Fourier transform $\widehat{d\pi_\mu}(Y)$, in particular for $Y \in \mathfrak{n}_+$. For this, we introduce the following two maps: $$\begin{aligned}
\label{eqn:alpha}
\alpha:\mathfrak g\times \mathfrak n_-\to\mathfrak l,&\quad&
 (Y,X)\mapsto \left.\frac {d}{dt}\right\vert_{t=0}p_o\left(e^{tY}e^X\right),\\
\beta:\mathfrak g\times \mathfrak n_-\to\mathfrak n_-,&\quad&
 (Y,X)\mapsto \left.\frac {d}{dt}\right\vert_{t=0}p_-\left(e^{tY}e^X\right).\label{eqn:beta}\end{aligned}$$

We may regard $\beta(Y,\cdot\,)$ as a vector field on $\mathfrak n_-$ through the following identification $\mathfrak n_-\ni X\mapsto\beta(Y,X)\in\mathfrak n_-\simeq T_X\mathfrak n_-.$

\[example:410\] $G_{\mathbb C}=GL(p+q,{\mathbb C})$, $L_{\mathbb C}=GL(p,{\mathbb C})\times GL(q,{\mathbb C})$, and $\mathfrak n_-\simeq M(p,q;{\mathbb C})$.

We note that $\mathfrak n_-$ is realized as upper block matrices. Then for $g^{-1}=
\begin{pmatrix}
a&b\\c&d
\end{pmatrix}
\in G_{\mathbb C}$ , $
Y=
\begin{pmatrix}
A&B\\C&D
\end{pmatrix}
\in M(p+q;{\mathbb C})
$ and $X \in M(p,q;{\mathbb C})$ we have $$\begin{aligned}
p_-(g^{-1})=&bd^{-1},
\\
p_o(g^{-1})=&(a-bd^{-1}c,d)\in GL(p,{\mathbb C})\times GL(q,{\mathbb C}),
\\
\alpha(Y,X)=&(A-XC,CX+D)\in  \mathfrak{gl}_p({\mathbb C})+ \mathfrak{gl}_q({\mathbb C}),
\\
\beta(Y,X)=&AX+B-XCX-XD.\end{aligned}$$

With the notation and , the infinitesimal action $d\pi_\mu$ on $C^\infty(\mathfrak{n}_-,V^\vee)$ is given by: $$\label{eqn:35}
\left(d\pi_\mu(Y)F\right)(X)=\mu(\alpha(Y,X))F(X)-(\beta(Y,\cdot\,)F)(X)
\quad\text{for $Y \in \mathfrak{g}$},$$ where, by a little abuse of notations $\mu$ stands for the infinitesimal action as well.

The right-hand side of (\[eqn:35\]) makes sense whenever $\mu$ is a representation of the Lie algebra $\mathfrak l$ without assuming that it lifts to a representation of $L_{\mathbb C}$, and thus we get a Lie algebra homomorphism $$\label{eqn:dpi1}
d\pi_\mu:\mathfrak g\to\mathcal D(\mathfrak n_-)\otimes\operatorname{End}(V^\vee).$$ If $\mathfrak{n}_+$ is abelian, we get another Lie algebra homomorphism by the algebraic Fourier transform on the Weyl algebra $\mathcal D(\mathfrak n_-)$, see Definition \[def:hat\]: $$\label{eqn:dpi2}
\widehat{d\pi_\mu}:\mathfrak g\to\mathcal D(\mathfrak n_+)\otimes\operatorname{End}(V^\vee).$$

We pin down a concrete formula, which follows immediately from the definition :

\[prop:Fc\] Let $(\lambda,V)$ be an $\mathfrak{l}$-module, and set $\mu:=\lambda^\vee\otimes\mathbb{C}_{2\rho}$. We extend $\lambda^\vee$ to a $\mathfrak{p}$-module by letting $\mathfrak{n}_+$ act trivially, and define $\widehat{d\pi_\mu}$ by . Then the algebraic Fourier transform of generalized Verma modules (see ) $$F_c: \operatorname{ind}_{\mathfrak{p}}^{\mathfrak{g}}(V^\vee) 
\overset{\sim}{\longrightarrow} 
\operatorname{Pol}(\mathfrak{n}_+)\otimes V^\vee$$ intertwines the $\mathfrak{g}$-action on the generalized Verma module $\operatorname{ind}_{\mathfrak{p}}^{\mathfrak{g}}(V^\vee)  =U(\mathfrak{g})\otimes_{U(\mathfrak{p})}(\lambda^\vee,V^\vee)$ with $\widehat{d\pi_\mu}$.

We begin by analyzing $\widehat{d\pi_\mu}$ on the subalgebra $\mathfrak{l}$. Let $L^+:\mathfrak l\to\mathcal D(\mathfrak n_+)$ be the representation of $\mathfrak l$ by vector fields on $\mathfrak n_+$ given by $$Y\mapsto L^+(Y)_x:=\left.\frac d{dt}\right|_{t=0}\operatorname{Ad}(e^{-tY})x.$$

\[lem:410\] The following two representations of $\mathfrak l$ on $\operatorname{Pol}(\mathfrak n_+)\otimes V^\vee $are isomorphic: $$\label{eqn:dpiell}
\widehat{d\pi_\mu}\big\vert_{\mathfrak l}\simeq
L^+\otimes \mathrm{id}+\mathrm{id}\otimes(\mu-2\rho)(Y).$$

For $Y\in\mathfrak l,\,X\in\mathfrak n_-$ we have $\alpha(Y,X)=Y$, and the formula reduces, in $\mathcal D(\mathfrak n_-)\otimes\mathrm{End}(V^\vee)$, to $$d\pi_\mu(Y)=\mathrm{id}\otimes \mu(Y)-\beta(Y,\,\cdot\,)\otimes
\mathrm{id}.$$

We apply Lemma \[prop:Phihat\] to the case where $(\sigma, E)$ is the adjoint representation of $\mathfrak l$ on $\mathfrak n_-$. With the notation therein we remark that $\Psi_{\mathrm{ad}}=-\beta(Y,\,\cdot\,)$. Then, we get $$\begin{aligned}
\widehat{d\pi_\mu}(Y)&=&\mathrm{id}\otimes\mu(Y)+
\Psi_{\mathrm{ad}}(Y)\otimes \mathrm{id}+\mathrm{Trace}\circ\mathrm{ad}(Y)\Big\vert_{\mathfrak n_-}\\&=&
\mathrm{id}\otimes\mu(Y)+L^+(Y)\otimes \mathrm{id}-\mathrm{id}
\otimes 2\rho(Y)\mathrm{id}.\end{aligned}$$ Thus, Lemma follows.

The differential operators $\widehat{d\pi_\mu}(Y)$ with $Y\in\mathfrak n_+$ play a central role in the F-method. We describe their structure by the following

\[lem:315\] For every $Y\in\mathfrak n_+$ the operator $\widehat{d\pi_\mu}(Y)$ is of degree $-1$ and more precisely it is of the form $$\label{eqn:degree}
\sum a_i^{jk}\xi^i\frac{\partial ^2}{\partial\xi^j\partial\xi^k}+ \sum b^j\frac{\partial}{\partial\xi^j},$$ where $a_i^{jk}$ and $b^j$ are constants depending only on $Y$.

Since $\mathfrak n_+$ is abelian, we can take a characteristic element $H$ such that $$\mathrm{Ad}(e^{tH})Y=e^tY,\quad\mathrm{for\, any}\,Y\in \mathfrak n_+.$$

The equality (\[eqn:dpiell\]) implies that $$\left(\widehat\pi_\mu(\ell)h\right)(\xi)=\lambda^\vee(\ell) h(\mathrm{Ad}(\ell^{-1})\xi),\quad\mathrm{for}\,\xi\in\mathfrak n_+,\,{\mathrm{and}}
\,\mu:=\lambda^\vee\otimes{\mathbb C}_{2\rho}.$$ In particular, we have $
\widehat{\pi}_\mu(e^{tH})=a^{\langle \lambda, H\rangle}R_a,
$ where $a:=e^{-t}$ and $(R_ah)(\xi):=h(a\xi)$. Then the identity $$\widehat\pi_\mu(e^{tH})\widehat{d\pi_\mu}(Y)\widehat\pi_\mu(e^{-tH})=e^t\widehat{d\pi_\mu}(Y),$$ implies $$R_a\widehat{d\pi_\mu}(Y)R_a^{-1}=a^{-1}\widehat{d\pi_\mu}(Y).$$ Hence $\widehat{d\pi_\mu}(Y)$ is of degree $-1$ for any $Y\in\mathfrak n_+$. As $d\pi_\mu(X)$ is a vector field there is no derivatives of higher order in the expression (\[eqn:degree\]).

Recipe of the F-method {#sec:44}
----------------------

Our goal is to find an explicit form of a $G'$-intertwining differential operator from $\mathcal V_X$ to $\mathcal W_Y$. Equivalently, what we call F-method yields an explicit element in $\displaystyle\operatorname{Hom}_{\mathfrak g'}({\operatorname{ind}_{\mathfrak p'}^{\mathfrak g'}}(W^\vee),
{\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(V^\vee))  \simeq \operatorname{Hom}_{\mathfrak{p}'}(W^\vee,\operatorname{ind}_{\mathfrak{p}}^{\mathfrak{g}}(V^\vee))$. Our assumption here is that $\mathfrak{p}=\mathfrak{l}+\mathfrak{n}_+$ is a $\mathfrak{g}'$-compatible parabolic subalgebra of $\mathfrak{g}$ with abelian nilradical $\mathfrak{n}_+$. In particular, $\mathfrak{p}' := \mathfrak{p} \cap \mathfrak{g}'$ is a parabolic subalgebra of $\mathfrak{g}'$ with a Levi decomposition $\mathfrak{p}' = \mathfrak{l}' + \mathfrak{n}'_+$ where $\mathfrak{l}' := \mathfrak{l} \cap \mathfrak{g}'$ and $\mathfrak{n}'_+ := \mathfrak{n}_+ \cap \mathfrak{g}'$.

The method we develop is as follows:

1.  Fix a finite dimensional representation $(\lambda,V)$ of $\mathfrak p=\mathfrak l+\mathfrak n_+$. In case $\lambda$ lifts to a group $P_{\mathbb C}$, we form a $G_{\mathbb C}$-equivariant holomorphic vector bundle $\mathcal V_X$ on $X=G_{\mathbb C}/P_{\mathbb C}$.

2.  Consider a representation $\mu:=\lambda^\vee\,\otimes\, {\mathbb C}_{2\rho}$ of the Lie algebra $\mathfrak p$ on $V^\vee_{2\rho}:= V^\vee\otimes {\mathbb C}_{2\rho}(\simeq V^\vee)$, and compute (see (\[eqn:dpi1\]) and (\[eqn:dpi2\])), $$\begin{aligned}
    d\pi_\mu:\mathfrak g&\to&\mathcal D(\mathfrak n_-)\otimes\operatorname{End}(V^\vee),\\
    \widehat{d\pi_\mu}:\mathfrak g&\to&\mathcal D(\mathfrak n_+)\otimes\operatorname{End}(V^\vee).\end{aligned}$$

3.  Find a finite dimensional representation $(\nu,W)$ of the Lie algebra $\mathfrak p'$ such that $$\operatorname{Hom}_{\mathfrak p'}(W^\vee,{\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(V^\vee))\neq\{0\}.$$ in case $W$ lifts to a group $P'_{\mathbb C}$ we form a $G'_{\mathbb C}$-equivariant holomorphic vector bundle $\mathcal W_Y$ on $Y=G'_{\mathbb C}/P'_{\mathbb C}$.

4.  Consider the system of partial differential equations for $\psi\in \operatorname{Pol}(\mathfrak n_+)\otimes\operatorname{Hom}(V,W)$: $$\begin{aligned}
    (\widehat{d\pi_\mu}(A)\otimes\operatorname{id}_W+\operatorname{id}\otimes\nu(A))\psi=0 &&\mathrm{for}\, A\in\mathfrak l',\label{eqn:Fmethod1}\\
    (\widehat{d\pi_\mu}( C)\otimes\operatorname{id}_W+\operatorname{id}\otimes\nu(C)) \psi=0 &&\mathrm{for}\, C\in\mathfrak n_+'.\label{eqn:Fmethod2}\end{aligned}$$ Notice that equations (\[eqn:Fmethod1\]) are of first order, whereas the equations (\[eqn:Fmethod2\]) are of second order.

5.  Use invariant theory and reduce the system of differential equations (\[eqn:Fmethod1\]) and (\[eqn:Fmethod2\]) to another system of differential equations on a lower dimensional space $S$. Solve it.

6.  Let $\psi$ be a polynomial solution to (\[eqn:Fmethod1\]) and (\[eqn:Fmethod2\]) obtained in Step 4. Compute $(\operatorname {Symb}\otimes \operatorname{id})^{-1}(\psi)$. This short-cut gives the desired equivariant differential operator in the coordinates $\mathfrak{n}_-$ of $X$ by Theorem \[thm:313\]. As a byproduct, $(F_c\otimes\operatorname{id})^{-1}(\psi)$ gives an explicit element in $\operatorname{Hom}_{\mathfrak{p}'}(W^\vee,\operatorname{ind}_{\mathfrak{p}}^{\mathfrak{g}}(V^\vee))$ ($\simeq \operatorname{Hom}_{\mathfrak{g}'} (\operatorname{ind}_{\mathfrak{p}'}^{\mathfrak{g}'}(W^\vee),\operatorname{ind}_{\mathfrak{p}}^{\mathfrak{g}}(V^\vee))$), which is sometimes referred to as a *singular vector*.

For actual applications in this work we assume in Step 0 that $\dim V=1$ and that $X$ is a Hermitian symmetric space $G/K$ so that $\mathcal V_X$ is a holomorphic line bundle over $X$, and in Step 2 that $(\nu,W)$ is irreducible so that $\mathfrak{n}'_+$ acts trivially on $W$. In this case the equation is given as $(\widehat{d\pi_{\mu}}(C)\otimes\operatorname{id}_W)\psi=0$, for which we shall simply write as $\widehat{d\pi_\mu}(C)\psi=0$.

In Step 2 we can use explicit branching laws (see Fact \[fact:B\] and Theorem \[thm:C\]) to find all such $W$ when $(\mathfrak g({\mathbb R}),\mathfrak g'({\mathbb R}))$ is a reductive symmetric pair, $\mathfrak n_+$ is abelian, $V$ is irreducible with a sufficiently positive parameter $\lambda$.

In Step 3, branching laws (Step 2) assures the existence of solutions to (\[eqn:Fmethod1\]) and (\[eqn:Fmethod2\]). Conversely, these differential equations are useful in certain cases to get a finer structure of branching laws, e.g., to find the Jordan–Hölder series of the restriction for exceptional parameters $\lambda$ (see [@KOSS]).

In Step 4, we can take $S$ to be one-dimensional in the case where $G/G'$ is a reductive symmetric space of split rank one.

F-method – supplement for vector valued cases. {#subsec:36}
----------------------------------------------

In order to deal with the general case where the target $\mathcal{W}_Y$ is no longer a line bundle but a vector bundle, i.e., where $W$ is an arbitrary finite dimensional, irreducible $\mathfrak l'$-module, we may find the condition somewhat complicated in practice, even though it is a system of differential equations of first order. In this section we give two useful lemmas to simplify Step 3 in the recipe by reducing to a simpler algebraic question on polynomial rings, so that we can focus on the crucial part consisting of a system of differential equations of second order . The idea here is to work first on the *highest weight variety* of the fiber $W$, and will be used in Sections \[sec:sp\] and \[sec:un1\].

We fix a Borel subalgebra $\mathfrak b(\mathfrak l')$ of $\mathfrak l'$. Let $\chi: \mathfrak b(\mathfrak l')\to{\mathbb C}$ be a character. For a completely reducible $\mathfrak{l}'$-module $U$, we set $$U_\chi:=
 \{u\in U\,:\, Zu=\chi(Z)u \quad\text{for any $Z\in\mathfrak b(\mathfrak l')$}\}.$$ Notice that $\dim U_\chi=1$ if and only if $U$ is irreducible and with highest weight $\chi$.

Let $W$ be an irreducible representation of $\mathfrak l'$ as before, and $\chi$ the highest weight of the contragredient representation $W^\vee$. We fix $w^\vee\in (W^\vee)_\chi$ a nonzero highest weight vector. Then we have the following:

\[lem:412\] For an element $\psi\in \operatorname{Pol}(\mathfrak n_+)\otimes\operatorname{Hom}(V,W)$, we set $P:=\langle\psi,w^\vee\rangle\in\operatorname{Pol}(\mathfrak{n}_+)\otimes V^\vee$. Then $\psi$ satisfies and if and only if the polynomial $P$ belongs to $\left(\operatorname{Pol(\mathfrak n_+})\otimes V^\vee\right)_\chi$ and satisfies the equation .

Denote $\lambda$ the action of $\mathfrak l$ on $V$ and $\nu$ the action of $\mathfrak l'$ on $W$. Owing to Lemma \[lem:410\], we have the identity: $$\{ \psi \in \operatorname{Pol}(\mathfrak{n}_+) \otimes \operatorname{Hom}_{\mathbb{C}}(V,W):
 \text{$\psi$ satisfies \eqref{eqn:Fmethod1}} \}
= (\operatorname{Pol}(\mathfrak{n}_+) \otimes \operatorname{Hom}_{\mathbb{C}}(V,W))^{\mathfrak{l}'},$$ where $\mathfrak{l}'$ acts on $\operatorname{Pol}(\mathfrak{n}_+) \otimes \operatorname{Hom}_{\mathbb{C}}(V,W)$ by $L^+\otimes\operatorname{id} + \operatorname{id} \otimes (\lambda^\vee \otimes \operatorname{id} + \operatorname{id} \otimes \nu)$ on the right-hand side.

For an $\mathfrak{l}'$-module $U$, we notice that the following contraction $$U\otimes W \otimes \mathbb{C} w^\vee \to U,
\quad
u \otimes w \otimes w^\vee \mapsto \langle w,w^\vee \rangle u$$ is a $\mathfrak{b}(\mathfrak{l}')$-homomorphism. Therefore, if $U$ is completely reducible, the evaluation map $$U\otimes W \to U, 
\quad
\psi\mapsto \langle\psi,w^\vee\rangle,$$ induces a bijection between two subspaces: $$\label{eqn:111}
(U\otimes W)^{\mathfrak{l}'}
\stackrel{\sim}{{\longrightarrow}}
U_\chi.$$ Applying to $U:=\operatorname{Pol}(\mathfrak{n}_+) \otimes \operatorname{Hom}_{\mathbb{C}}(V,W)$, we get Lemma \[lem:412\].

Since any nonzero vector in $W^\vee$ is cyclic, the next lemma explains how to recover $
 D_{X\to Y}(\varphi)$ from $P$ given in Lemma \[lem:412\].

We assume, for simplicity, that the $\mathfrak{l}$-module $(\lambda,V)$ lifts to $L_{\mathbb C}$, the $\mathfrak{l}'$-module $(\nu,W)$ lifts to $L'_{\mathbb C}$, and use the same letters to denote their liftings.

\[lem:hwv\] For any $\varphi\in\operatorname{Hom}_{\mathfrak p'}(W^\vee,{\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(V^\vee))$, $\ell\in L'_{\mathbb C}$ and $w^\vee\in W^\vee$, $$\label{eqn:Scalea}
\left\langle D_{X\to Y}(\varphi), \nu^\vee(\ell)w^\vee\right\rangle=
(\mathrm{Ad}(\ell)\otimes \lambda^\vee(\ell))
\left\langle D_{X\to Y}(\varphi), w^\vee\right\rangle.$$

We write $\varphi=\sum_j u_j\otimes \psi_j\in U(\mathfrak n_-)\otimes \operatorname
{Hom}_{\mathbb C}(V,W)$. Since $\varphi$ is $\mathfrak p'$-invariant, we have the identity: $$\sum_j u_j\otimes \psi_j=\sum_j \mathrm{Ad}(\ell)u_j\otimes \nu(\ell)\circ\psi_j\circ\lambda(\ell^{-1})
\quad\text{for $\mathfrak{l}\in L'_{\mathbb{C}}$}.$$ In turn, we have $$\begin{aligned}
\left\langle D_{X\to Y}(\varphi),\nu^\vee(\ell)w^\vee\right\rangle &=&
\sum_j dR(\mathrm{Ad}(\ell)u_j)\otimes \langle \psi_j, w^\vee\rangle\circ \lambda(\ell^{-1})\\
&=&\left((\mathrm{Ad}(\ell)\otimes\lambda^\vee(\ell)\right)
\left\langle D_{X\to Y}(\varphi),w^\vee\right\rangle.\end{aligned}$$ Thus, we have proved Lemma.

We notice that the right-hand side of can be computed by using the identity in $\mathrm{Diff}^{\mathrm{const}}(\mathfrak n_-)\otimes V^\vee$: $$\langle  D_{X\to Y}(\varphi),w^\vee\rangle=(\mathrm{Symb}^{-1}\otimes
 \mathrm{id}_{V^\vee})(P ),$$ once we know the polynomial $P = \langle \psi,w^\vee\rangle$ with $\psi = (F_c \otimes \operatorname{id}) (\varphi)$ (see Theorem \[thm:313\]). In Sections \[sec:sp\] and \[sec:un1\], we find explicit formulæ for vector-bundle valued equivariant differential operators by solving equations for the polynomials $P$.

Branching laws and Hermitian symmetric spaces {#sec:4}
=============================================

From now we apply the general theory developed in the previous sections to the case where $(G,G')$ is a reductive symmetric pair and where $D_{X\to Y}$ intertwines holomorphic discrete series representations.

Branching laws
--------------

Our subject is to construct an explicit covariant differential operator from $\mathcal{V}_X$ to $\mathcal{W}_Y$. The existence, respectively the uniqueness (up to scaling) of such operators are subject to the conditions $$\label{eqn:mfbdle}
\dim\operatorname{Diff}_{G'} (\mathcal{V}_X,\mathcal{W}_Y) \ge 1, \ \text{respectively}\ \le 1.$$ We then encounter a question to find the geometric settings (i.e. the pair $Y \subset X$ of generalized flag varieties and two homogeneous vector bundles $\mathcal{V}_X \to X$ and $\mathcal{W}_Y \to Y$) that satisfies . This is the main ingredient of Step 2 in the recipe of the F-method, and thanks to Theorem \[thm:surject\], the existence and uniqueness are equivalent to the following question of (abstract) branching laws: Given a $\mathfrak p$-module $V$, find all finite dimensional $\mathfrak p'$-modules $W$ such that $\dim\operatorname{Hom}_{\mathfrak p'}(W^\vee,{\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(V^\vee))=1$, and equivalently, $$\label{eqn:homverma}
\dim\operatorname{Hom}_{\mathfrak g'}({\operatorname{ind}_{\mathfrak p'}^{\mathfrak g'}}(W^\vee),{\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(V^\vee))
=1.$$ This subsection briefly reviews what is known on this question (see Fact \[fact:A\]).

Let $\mathfrak g$ be a complex semisimple Lie algebra, and $\mathfrak j$ a Cartan subalgebra of $\mathfrak g$. We fix a positive root system $\Delta^+\equiv\Delta^+(\mathfrak g,\mathfrak j)$, write $\rho$ for half the sum of positive roots, $\alpha^\vee$ for the coroot for $\alpha \in \Delta$, and $\mathfrak g_\alpha$ for the root space. Define a Borel subalgebra $\mathfrak b=\mathfrak j+\mathfrak n$ with the nilradical $\mathfrak n:=\bigoplus_{\alpha\in\Delta^+}\mathfrak g_\alpha$.

The BGG category $\mathcal O$ is defined as the full subcategory of $\mathfrak g$-modules whose objects are finitely generated, $\mathfrak j$-semisimple and locally $\mathfrak n$-finite [@BGG].

As in the previous section, fix a standard parabolic subalgebra $\mathfrak p$ with Levi decomposition $\mathfrak p=\mathfrak l +\mathfrak n_+$ such that the Levi factor $\mathfrak l$ contains $\mathfrak j$. We set $\Delta^+(\mathfrak l):=\Delta^+\cap\Delta(\mathfrak l,\mathfrak j)$. The parabolic BGG category $\mathcal O^{\mathfrak p}$ is defined as the full subcategory of $\mathcal O$ whose objects are locally $\mathfrak{l}$-finite.

The set of $\lambda\in\mathfrak j^*$ whose restrictions to $\mathfrak j\cap[\mathfrak l,\mathfrak l]$ are dominant integral is denoted by $$\Lambda^+(\mathfrak l):=\{ \lambda\in\mathfrak j^*:\,\langle\lambda,\alpha^\vee\rangle
\in{\mathbb N}\text{ for any } \alpha\in\Delta^+(\mathfrak l)\}.$$ We write $V_\lambda$ for the finite dimensional simple $\mathfrak l$-module with highest weight $\lambda$, regard it as a $\mathfrak p$-module by letting $\mathfrak n_+$ act trivially, and consider the generalized Verma module $${\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(\lambda)\equiv
{\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(V_\lambda):=
U(\mathfrak g)\otimes_{U(\mathfrak p)}V_\lambda.$$ Then ${\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(\lambda)\in\mathcal O^\mathfrak p$ and any simple object in $\mathcal O^\mathfrak p$ is the quotient of some generalized Verma module. If $$\label{eqn:condimu}
\langle\lambda,\alpha^\vee\rangle=0\qquad \mathrm{for\, all}\qquad\alpha\in\Delta(\mathfrak l),$$ then $V_\lambda$ is one-dimensional, to be denoted also by ${\mathbb C}_\lambda$. In this case we say ${\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(\lambda)$ is of *scalar type*.

Let $\tau\in \operatorname{Aut}(\mathfrak g)$ be an involutive automorphism of the Lie algebra $\mathfrak g$. We write $$\mathfrak{g}^{\pm \tau}:=\{v \in \mathfrak{g}: \tau v = \pm v\}$$ for the $\pm 1$ eigenspaces of $\tau$, respectively. We say that $(\mathfrak g,\mathfrak g')$ is a symmetric pair if $\mathfrak g'=\mathfrak g^\tau$ for some $\tau$.

For a general choice of $\tau$ and ${\mathfrak {p}}$, the space considered in (\[eqn:homverma\]) may be reduced to zero for all $\mathfrak p'$-modules $W$. Suppose $V\equiv V_\lambda$ with $\lambda\in\Lambda^+(\mathfrak l)$ generic. Then a necessary and sufficient condition for the existence of $W$ such that the left-hand side of (\[eqn:homverma\]) is nonzero is given by the geometric requirement on the generalized flag variety $G_{\mathbb{C}}/P_{\mathbb{C}}$, namely, the set $G^\tau_{\mathbb C}P_{\mathbb C}$ is closed in $G_{\mathbb C}$, see [@K12 Proposition 3.8].

Consider now the case where the nilradical $\mathfrak n_+$ of $\mathfrak p$ is abelian. Then, the following result holds :

\[fact:AA\] If the nilradical $\mathfrak n_+$ of $\mathfrak p$ is abelian, then for any symmetric pair $(\mathfrak g,\mathfrak g^\tau)$ the restriction of a generalized Verma module of scalar type $ {\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(-\lambda)\vert_{\iota(\mathfrak g^\tau)}$ is multiplicity-free for any embedding $\iota:\mathfrak g^\tau\to\mathfrak g$ such that $\iota(G^\tau_{\mathbb C})P_{\mathbb C}$ is closed in $G_{\mathbb C}$ and for any sufficiently positive $\lambda$.

A combinatorial description of the branching law is given as follows. Suppose that ${\mathfrak {p}}$ is ${\mathfrak {g}}^{\tau}$-compatible (see Definition \[def:3.2\]). Then the involution $\tau$ stabilizes $\mathfrak l$ and $\mathfrak{n}_+$, respectively, the nilradical $\mathfrak{n}_+$ decomposes into a direct sum of eigenspaces $\mathfrak n_+=\mathfrak n_+^\tau+\mathfrak n_+^{-\tau}$ and $G^\tau_{\mathbb C}P_{\mathbb C}$ is closed in $G_{\mathbb C}$. Fix a Cartan subalgebra $\mathfrak j$ of $\mathfrak l$ such that $\mathfrak j^\tau:=\mathfrak j\cap\mathfrak g^\tau$ is a Cartan subalgebra of $\mathfrak l^\tau$.

We define $\theta \in \operatorname{End}(\mathfrak{g})$ by $\theta|_{\mathfrak{l}}=\operatorname{id}$ and $\theta|_{\mathfrak{n}_++\mathfrak{n}_-}=-\operatorname{id}$. Then $\theta$ is an involution commuting with $\tau$. Moreover it is an automorphism if $\mathfrak{n}_+$ is abelian. The reductive subalgebra $
\mathfrak{g}^{\tau\theta}= {\mathfrak {l}}^{\tau}
                  + {\mathfrak {n}}_-^{-\tau}
                  + {\mathfrak {n}}_+^{-\tau}
$ decomposes into simple or abelian ideals $\bigoplus_i\mathfrak{g}_i^{\tau\theta}$, and we write the decomposition of $\mathfrak{n}_-^{-\tau}$ as ${\mathfrak {n}}_{-}^{-\tau}=\bigoplus_i{\mathfrak {n}}_{-,i}^{-\tau}$ correspondingly. Each ${\mathfrak {n}}_{-,i}^{-\tau}$ is a ${\mathfrak {j}}^{\tau}$-module, and we denote by $\Delta({\mathfrak {n}}_{-,i}^{-\tau}, {\mathfrak {j}}^{\tau})$ the set of weights of ${\mathfrak {n}}_{-,i}^{-\tau}$ with respect to ${\mathfrak {j}}^{\tau}$. The roots $\alpha$ and $\beta$ are said to be [*[strongly orthogonal]{}*]{} if neither $\alpha + \beta$ nor $\alpha - \beta$ is a root. We take a maximal set of strongly orthogonal roots $\{\nu_1^{(i)}, \cdots, \nu_{k_i}^{(i)}\}$ in $\Delta({\mathfrak{n}}_{-,i}^{-\tau}, {\mathfrak{j}}^{\tau})$ inductively as follows:

1.  $\nu_1^{(i)}$ is the highest root of $\Delta({\mathfrak {n}}_{-,i}^{-\tau}, {\mathfrak {j}}^{\tau})$.

2.  $\nu_{j+1}^{(i)}$ is the highest root among the elements in $\Delta({\mathfrak{n}}_{-,i}^{-\tau}, {\mathfrak{j}}^{\tau})$ that are strongly orthogonal to $\nu_1^{(i)}, \cdots, \nu_{j}^{(i)}$ ($1 \le j \le k_i-1$).

We define the following subset of ${\mathbb{N}}^k$ ($k=\sum k_i$) by $$\label{eqn:Aplus}
  A^+:= \prod_i A_i, \quad
  A_i :=
  \{
(a_{j}^{(i)})_{1 \le j \le k_i}\in {\mathbb{N}}^{k_i}
:
 a_1^{(i)} \ge \cdots \ge a_{k_i}^{(i)}\ge 0
\}.$$

Introduce the following positivity condition: $$\label{eqn:lpos}
\langle\lambda-\rho_{\mathfrak{g}},\alpha\rangle>0
\quad\text{for any $\alpha \in \Delta(\mathfrak{n}_+,\mathfrak{j})$}.$$

\[fact:A\] Suppose ${\mathfrak {p}}$ is ${\mathfrak {g}}^{\tau}$-compatible, and $\lambda$ satisfies and . Then the generalized Verma module ${\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(-\lambda)$ decomposes into a multiplicity-free direct sum of irreducible $\mathfrak g^\tau$-modules: $$\label{eqn:branchV}
{\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(-\lambda)\vert_{\mathfrak g^\tau}
\simeq
 \bigoplus
_{(a_{j}^{(i)}) \in A^+}
{\operatorname{ind}_{\mathfrak p^\tau}^{\mathfrak g^\tau}}(-\lambda\vert_{\mathfrak j^\tau}
-\sum_i\sum_{j=1}^{k_i} a_j^{(i)}\nu_j^{(i)}).$$ In particular, for a simple ${\mathfrak {p}}^{\tau}$-module $W$ (namely, a simple ${\mathfrak{l}}^{\tau}$-module with trivial action of ${\mathfrak {n}}^{\tau}$), $$\dim {\operatorname{Hom}}_{{\mathfrak {g}}^{\tau}}
   ({\operatorname{ind}_{\mathfrak p^\tau}^{\mathfrak g^\tau}}(W^{\vee}), {\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}({\mathbb{C}}_{-\lambda}))=1$$ if and only if the highest weight of the ${\mathfrak {l}}^{\tau}$-module $W$ is of the form $\lambda|_{{\mathfrak {j}}^{\tau}}+\sum_i \sum_{j=1}^{k_i}
 a_j^{(i)}\nu_j^{(i)}$ for some $(a_j^{(i)}) \in A^+$.

In the latter part of the paper we shall construct a family of equivariant differential operators for all symmetric pairs $(\mathfrak{g},\mathfrak{g}^\tau)$ with $k=1$ (in particular, $\Delta(\mathfrak n_{-,i}^{-\tau},\mathfrak j^\tau)$ is empty for all but one $i$).

Local versus non-local intertwining operators for branching laws {#subsec:4.2}
----------------------------------------------------------------

Discrete branching laws for the restriction of a unitary representation of $G$ to $G'$ assure that there exist continuous $G'$-intertwining operators from the representation of $G$ to irreducible summands of $G'$. In general such operators are integro-differential operators, e.g., [@KS]. In this section, however, we formulate and prove a quite remarkable phenomenon that any continuous $G'$-intertwining operators between two representation spaces consisting of holomorphic sections is given simply by differential operators in certain settings, see Theorem \[thm:C\].

Let $G$ be a simple, connected, simply connected Lie group and $K$ a maximal compact subgroup of $G$. We write ${\mathfrak {c}}({\mathfrak {k}})$ for the center $\mathfrak c(\mathfrak k)$ of the complexified Lie algebra $\mathfrak k:=\operatorname{Lie}(K)\otimes_{\mathbb{R}} {\mathbb{C}}$. Assume ${\mathfrak {c}}({\mathfrak {k}})$ is nonzero. Then ${\mathfrak {c}}({\mathfrak {k}})$ is actually one-dimensional, and there exists a characteristic element $Z \in 
{\mathfrak {c}}({\mathfrak {k}})$ such that the eigenvalues of ${\operatorname{ad}} (Z) \in {\operatorname{End}}({\mathfrak {g}})$ is 0 or $\pm 1$. We decompose $${\mathfrak {g}}={\mathfrak {l}}+{\mathfrak {n}}_+
 +{\mathfrak {n}}_-$$ into the eigenspaces of ${\operatorname{ad}} (Z)$ with eigenvalues $0$, $1$, and $-1$, respectively. We note that ${\mathfrak {l}}={\mathfrak {k}}$ in this setting. Then ${\mathfrak {p}}:= {\mathfrak {l}}+{\mathfrak {n}}_+$ is a parabolic subalgebra with abelian nilradical ${\mathfrak {n}}_+$, and the homogeneous $G/K$ becomes a Hermitian symmetric space, for which the complex structure is induced from the Borel embedding $$G/K\subset G_{\mathbb C}/K_{\mathbb C}\exp\mathfrak n_+ = G_{\mathbb{C}}/P_{\mathbb{C}}.$$

Furthermore, let $\tau$ be an involutive automorphism of $G$. Without loss of generality we may and do assume that $\tau$ commutes with the Cartan involution $\theta$, and therefore induces a diffeomorphism of the symmetric space $G/K$. Thus, the subsymmetric space $G^\tau/K^\tau \subset G/K$ is fixed pointwisely by $\tau$. We use the same letters $\tau$ and $\theta$ to denote the complex linear extensions of their differentials. Since $\tau \theta=\theta \tau$, the involution $\tau$ leaves the center $\mathfrak c(\mathfrak k)$ of $\mathfrak k$ invariant, and therefore one has either $\tau\vert_{\mathfrak c(\mathfrak k)}=
  \operatorname{id}$ or $\tau\vert_{\mathfrak{c}(\mathfrak{k})} = -\operatorname{id}$ because $\dim \mathfrak c(\mathfrak k)=1$. We say $\tau$ is of *holomorphic type* if $\tau\vert_{\mathfrak c(\mathfrak k)}=\operatorname{id}$. In this case, the roles of $\theta$ and $\tau$ are the same as in the previous section, and we have:

1.  the involution $\tau$ acts holomorphically on the Hermitian symmetric space $G/K$;

2.  the embedding $G^\tau/K^\tau\hookrightarrow G/K$ is holomorphic;

3.  ${\mathfrak {p}}$ is ${\mathfrak {g}}^{\tau}$-compatible.

Here the complex structure on $G^\tau/K^\tau$ is induced from the open embedding in the complex flag variety $G_{\mathbb C}^\tau/P_{\mathbb C}^\tau $: $$\begin{matrix}
 & G^\tau/K^\tau &\hookrightarrow &G/K\\
&\llap{${}_  {\mathrm{open}}$}\,\bigcap\ & &\bigcap\,\rlap{${}_{\mathrm{open}}$}\\
  Y=&G_{\mathbb C}^\tau/P_{\mathbb C}^\tau&\hookrightarrow&G_{\mathbb C}/P_{\mathbb C}&=X.
  \end{matrix}$$

Notice that in this setting the complexified Lie algebra of $K^\tau$ coincides with the Levi part $\mathfrak{l}^\tau$ of the parabolic subalgebra $\mathfrak p^\tau$. Given a finite dimensional representation of $K^\tau$, we endow the vector space $\mathcal O(G^\tau/K^\tau,\mathcal W)$ of global sections of the homogeneous vector bundle $\mathcal W\to G^\tau/K^\tau$ with the Fréchet topology of uniform convergence on compact sets.

\[thm:C\] Suppose the line bundle parameter $\lambda$ satisfies .

1.  Any continuous $G^\tau$-homomorphism $$\label{eqn:branch2}
    \mathcal O(G/K,\mathcal L_\lambda){\longrightarrow}\mathcal O(G^\tau/K^\tau,\mathcal W)$$ is given by a differential operator.

2.  Suppose $W$ is irreducible as a $K^\tau$-module. There exists a non-trivial continuous $G^\tau$-homomorphism if and only if the $K^\tau$-module $W$ is of the form $W_\lambda^{\underline a}$ for some $\underline{a}\in A^+$ see .

    In this case, the homomorphism is given, up to a scalar, by the restriction to open subsets $G^\tau/K^\tau\subset \mathfrak{n}_-^\tau$ and $G/K\subset \mathfrak{n}_-$ of the differential operator $D_{X\to Y}$ constructed in Theorem \[thm:313\].

We denote by $\mathcal H^2(M,\mathcal V)$ the Hilbert space of square integrable holomorphic sections of the Hermitian vector bundle $\mathcal V$ over a Hermitian manifold $M$. If holds, then $\mathcal{H}^2(G/K,\mathcal{L}_\lambda)\ne\{0\}$, and $G$ acts unitarily and irreducibly on it.

In order to prove Theorem \[thm:C\], we recall the corresponding branching laws in the category of unitary representations, which are the dual of the formulæ in Fact \[fact:A\]. In general $G^\tau$ is noncompact, and we need to consider infinite dimensional irreducible representations of $G^\tau$ when we consider the branching law $G\downarrow G^\tau$.

Given $\underline{a} = (a_j^{(i)}) \in A^+$ ($\subset \mathbb{N}^k$), we write $\mathcal W^{\underline{a}}_\lambda$ for the $G^\tau$-equivariant holomorphic vector bundle over $G^\tau/K^\tau$ associated to the irreducible representation $\mathcal W^{\underline{a}}_\lambda$ of $\mathfrak l^\tau$ with highest weight $\lambda\vert_{\mathfrak j^\tau}+\sum_i \sum_{j=1}^{k_i}a_j^{(i)}\nu_j^{(i)}$.

\[fact:B\] If the positivity condition is satisfied, then $\mathcal{H}^2(G^\tau/K^\tau,\mathcal{W}_\lambda^{\underline{a}})$ is non-zero and $G^\tau$ acts on it as an irreducible unitary representation of $G^\tau$ for any $\underline{a} \in A^+$. Moreover, the branching law for the restriction $G\downarrow G^\tau$ is given by $$\label{eqn:branch}
\mathcal H^2(G/K,\mathcal L_\lambda)
\simeq \sideset{}{^\oplus}\sum_{\underline{a}\in A^+} \mathcal{H}^2
(G^\tau/K^\tau,\mathcal{W}_\lambda^{\underline{a}})
\quad\text{(Hilbert direct sum)}.$$

For continuous representations $\sigma,\sigma'$ of $G^\tau$, we write $\operatorname{Hom}_{G^\tau}(\sigma,\sigma')$ for the space of continuous $G^\tau$-homomorphisms.

Suppose $\sigma$ is an arbitrary irreducible unitary representation of $G^\tau$. Then for any irreducible $K^\tau$-module $W$, we have $$\label{eqn:proof1}
\dim\operatorname{Hom}_{G^\tau}(\sigma,\mathcal O(G^\tau/K^\tau,
\mathcal W))\leq1.$$ Moreover, suppose $W_1$ and $W_2$ are two irreducible $K^\tau$-modules. Then $$\label{eqn:proof2}
\dim\operatorname{Hom}_{G^\tau}(\sigma,\mathcal O(G^\tau/K^\tau,
\mathcal W_i))\neq0 \quad{\mathrm{for}}\quad i=1,2
\ \Rightarrow W_1 \simeq W_2
\quad\text{as $K^\tau$-modules}.$$

The assertion is an immediate consequence of the propagation theorem for multiplicity-freeness property under visible actions [@mfbundl].

In order to prove the first statement, it is sufficient to assume that $\mathcal{W}$ is associated to an *irreducible* $K^\tau$-module $W$. Then it follows from , and the multiplicity-free branching law that the space $\operatorname{Hom}_{G^\tau}\left(\mathcal H^2\left(G/K,\mathcal L_\lambda\right),\mathcal O(G^\tau/K^\tau,\mathcal W)\right)$ is at most one-dimensional, and is nonzero if and only if there exists $\underline{a}\in A^+$ such that $W\simeq W_\lambda^{\underline a}$.

Conversely, suppose that $W\simeq W_\lambda^{\underline a}$. Then, by the branching law for the generalized Verma modules (see Fact \[fact:A\]), we have $$\operatorname{Hom}_{\mathfrak{g}^\tau}\left({\operatorname{ind}_{\mathfrak p^\tau}^{\mathfrak g^\tau}}(W^\vee), {\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}({\mathbb C}_\lambda^\vee)\right)\neq \{0\}.$$ Taking a non-zero element $\varphi$, we get a $G^\tau$-equivariant holomorphic differential operator $D_{X\to Y} (\varphi) \in
\operatorname{Diff}^{\operatorname{const}}(\mathfrak{n}_-)\otimes\operatorname{Hom}_{\mathbb{C}}(\mathbb{C}_\lambda,W)$ according to Theorem \[thm:313\]. The restriction of $D_{X\to Y}(\varphi)$ gives an element of $\operatorname{Hom}_{G^\tau}\left(\mathcal{O}\left(G/K,\mathcal L_\lambda\right),\mathcal O(G^\tau/K^\tau,\mathcal W_\lambda^{\underline a})\right)$. Since this space is at most one-dimensional, both statements of the theorem are now proved.

In the sequel, by a little abuse of notation, we shall write $X=G/K$ and $Y=G^\tau/K^\tau$ and denote $D_{X\to Y}$ the intertwining differential operator given in Theorem \[thm:C\].

Split rank one reductive symmetric pairs of holomorphic type 
-------------------------------------------------------------

The homogeneous space $G/G^\tau$ is endowed with a $G$-invariant pseudo-Riemannian structure $g$ induced from the Killing form, and becomes an affine symmetric space with respect to the Levi-Civita connection.

Geometrically, the *split rank* of the semisimple symmetric space $G/G^\tau$ is the dimension of a maximal flat, totally geodesic submanifold $B$ in $G/G^\tau$ such that the restriction of $g$ to $B$ is positive definite. Algebraically, it is the dimension of the maximal abelian subspace of $\mathfrak g({\mathbb R})^{-\tau,-\theta}:=\{Y\in\mathfrak g({\mathbb R})\,:
\tau Y=\theta Y=-Y\}$. We denote it by $\mathrm{rank}_{\mathbb R}\, G/G^\tau$.

Assume from now that the split rank of the semisimple symmetric space $G/G^\tau$ is equal to 1, or equivalently, the real rank of $\mathfrak g({\mathbb R})^{\tau\theta}$ is one. Table \[table:1\] above gives the infinitesimal classification of split rank one irreducible semisimple symmetric pairs $(\mathfrak g({\mathbb R}), \mathfrak g({\mathbb R})^\tau)$ of holomorphic type.

  ---------------------------------------------------------------------------------------------------------------------------------------------------------------------
                $\mathfrak g({\mathbb R})$                            $\mathfrak g({\mathbb R})^\tau$                       $\mathfrak g({\mathbb R})^{\tau\theta}$
  --- ---------------------------------------------- ------------------------------------------------------------------ -----------------------------------------------
   1   $\mathfrak{su}(n,1)\oplus\mathfrak{su}(n,1)$                         $\mathfrak{su}(n,1)$                                     $\mathfrak{su}(n,1)$

   2         $\mathfrak{sp}(n+1,{\mathbb R})$         $\mathfrak{sp}(n,{\mathbb R})\oplus\mathfrak{sp}(1,{\mathbb R})$                $\mathfrak{u}(1,n)$

   3               $\mathfrak{so}(n,2)$                                    $\mathfrak{so}(n-1,2)$                        $\mathfrak{so}(n-1)\oplus \mathfrak{so}(1,2)$

   4               $\mathfrak{su}(p,q)$                   $\mathfrak s(\mathfrak{u}(1)\oplus\mathfrak{u}(p-1,q))$            $\mathfrak s(\mathfrak{u}(1,q)\oplus
                                                                                                                                      \mathfrak u(p-1))$

   5              $\mathfrak{so}(2,2n)$                                     $\mathfrak{u}(1,n)$                                       $\mathfrak{u}(1,n)$

   6              $\mathfrak{so}^*(2n)$                        $\mathfrak{so}(2)\oplus\mathfrak{so}^*(2n-2)$                             $\mathfrak{u}
                                                                                                                                           (1,n-1)$
  ---------------------------------------------------------------------------------------------------------------------------------------------------------------------

  : Split rank one irreducible symmetric pairs of holomorphic type[]{data-label="table:1"}

The remaining part of this work is devoted to the explicit description of $G^\tau$-intertwining operators between $\mathcal H^2(X,\mathcal L_\lambda)$ and its irreducible components given in Theorem \[thm:C\]. The next section treats all the cases where such operators are given by normal derivatives. In Section \[section:6\] we analyze all the split rank one symmetric pairs of holomorphic type for which those operators have more intricate structure.

Normal derivatives versus intertwining operators {#sec:5}
================================================

Suppose we are in the setting of Section \[subsec:4.2\], namely, $G^\tau/K^\tau$ is a subsymmetric space of the Hermitian symmetric space $G/K$. Then we can consider the Taylor expansion of any holomorphic function (section) on $G/K$ along the normal direction by using the Borel embedding. This idea was used earlier by Martens, Jakobsen and Vergne [@Mart; @JV] for filtered modules to find abstract branching laws.

However, it should be noted that normal derivatives do not always give rise to intertwining operators. In this section we clarify the reason and give a classification of all pairs ($\mathfrak g({\mathbb R}), \mathfrak g({\mathbb R})^\tau$) of split rank one for which it happens.

Normal derivatives and the Borel embedding
------------------------------------------

Suppose $E=E_++E_-$ is a direct sum of vector spaces. Then every $u\in E_-$ defines a vector field on $E$ $$f(v)\mapsto \left.\frac{d}{dt}\right\vert_{t=0} f(v+tu),$$ which is a *normal derivative along the normal direction $E_-$*.

More generally, we shall use vector-bundle valued operators $\widetilde{T}: C^\infty(E,V) \to C^\infty(E_+,W)$ given by normal derivatives. For this, we begin with the following setting. Let $S(E_-)$ be the space of symmetric tensors on $E_-$, and suppose that $W^\vee$ is a subspace of $S(E_-)\otimes V^\vee$. Let $T$ denote the inclusion map $T:
W^\vee \hookrightarrow S(E_-)\otimes V^\vee
$. Then it determines an element, denoted by the same letter $T$, in $S(E_-)\otimes \operatorname{Hom}_{\mathbb C}(V,W)$, which can be viewed as an element of $\operatorname{Diff}^{\mathrm{const}}(E)\otimes \operatorname{Hom}_{\mathbb C}(V,W)$ via the composition $$S(E_-) \subset S(E)\xrightarrow{\sim}\operatorname{Diff}^{\mathrm{const}}(E).$$

In the coordinates $x=\sum_{j=1 }^qx_je_j\in E_-$, where $\{e_1,\ldots,e_q\}$ is a basis of $E_-$, it is written as $$T=\sum_{{\alpha\in {\mathbb N}^q}}
e^\alpha\otimes T_\alpha\qquad \mathrm{for\, some}\;T_\alpha\in\operatorname{Hom}_{\mathbb C}(V,W).$$ The subspace $E_+$ is given by the condition $x=0$ in $E=\{(x,y)\,:\,x\in E_-, y\in E_+\}$. Then the differential operator $
\widetilde T:C^\infty(E,V){\longrightarrow}C^\infty(E_+,W),\,f(x,y)\mapsto (\widetilde Tf)(y)
$ is of the form $$\label{eqn:normalT}
\left(\widetilde T f\right)(y)=\sum_{\alpha\in {\mathbb N}^q}
T_\alpha\left(\frac{\partial^{|\alpha|} f(x,y)}{\partial x^\alpha}\bigg|_{x=0}\right),$$ which is a normal derivative along the direction $E_-$.

We apply this construction to the subsymmetric space $G^\tau/K^\tau$ in $G/K$. As in Section \[sec:4\], let $\mathcal V_X$ be a homogeneous vector bundle over $X=G/K$ associated to an irreducible representation $V$ of $K$. Similarly, let $\mathcal W_Y$ be a homogeneous vector bundle over the subsymmetric space $Y=G^\tau/K^\tau$ associated with an irreducible representation $W$ of $K^\tau$.

If there is a homomorphism $T:W^\vee\longrightarrow S(\mathfrak n_-^{-\tau})\otimes V^\vee$ then it defines a holomorphic normal derivative $\widetilde T$ with respect to $\mathfrak n_-=\mathfrak n_-^\tau+\mathfrak n_-^{-\tau}$, which induces a holomorphic differential operator between two vector bundles by $$\begin{aligned}
&\xymatrix{
\mathcal O({\mathfrak {n}_-}, V)
\ar[r]^{ \widetilde T}
\ar[d]_{\mathrm{restriction}}
& \mathcal O({\mathfrak {n}}_-^{\tau}, W)
\ar[d]^{\mathrm{restriction}}
\\
\mathcal O(G/K, \mathcal V_X)\ar@{.>}[r]
& \mathcal O(G^\tau/K^{\tau},\mathcal W_Y).
}
\label{diag:5.1}\end{aligned}$$

We shall denote it by the same letter $\widetilde T$, and call it a normal derivative along the direction $\mathfrak{n}_-^{-\tau}$.

When are normal derivatives intertwining operators?
---------------------------------------------------

Let dim $V=1$, and we write as before $\mathcal L_\lambda$ for the homogeneous line bundle over $X=G/K$ associated to the character ${\mathbb C}_\lambda$ of $K$.

\[thm:nor\_der\] Suppose $(\mathfrak g({\mathbb R}),\mathfrak g({\mathbb R})^\tau)$ is a split rank one irreducible symmetric pair of holomorphic type. Then, the following three conditions on the pair $(\mathfrak g({\mathbb R}),\mathfrak g({\mathbb R})^\tau)$ are equivalent:

1.  For any $\lambda$ satisfying and for any irreducible $K^\tau$-module $W$, all continuous $G^\tau$-homomorphisms $$\mathcal{O}(X,\mathcal L_\lambda)\longrightarrow
    \mathcal O(Y,\mathcal W),$$ are given by normal derivatives along the direction $\mathfrak{n}_-^{-\tau}$.

2.  For some $\lambda$ satisfying and for some irreducible $K^\tau$-module $W$, there exists a non-trivial $G^\tau$-intertwining operator $$\mathcal{O}(X,\mathcal L_\lambda)\longrightarrow
    \mathcal O(Y,\mathcal W)$$ which is given by normal derivatives of positive order.

3.  The symmetric pair $(\mathfrak g({\mathbb R}),\mathfrak g({\mathbb R})^\tau)$ is isomorphic to one of $(\mathfrak{su}(p,q),\mathfrak s(\mathfrak{u}(1)\oplus\mathfrak{u}(p-1,q)))$, $(\mathfrak{so}(2,2n),\mathfrak{u}(1,n))$ or $(\mathfrak{so}^*(2n),\mathfrak{so}(2)\oplus\mathfrak{so}^*(2n-2))$.

Notice that the geometric nature of embeddings $Y\hookrightarrow X$ mentioned in the condition (iii) corresponds to the following inclusions of flag varieties: $$\begin{aligned}
\mathrm{Gr}_{p-1}({\mathbb C}^{p+q})&\hookrightarrow& \mathrm{Gr}_{p}({\mathbb C}^{p+q});\\
\mathbb P^n{\mathbb C}&\hookrightarrow & Q^{2n}{\mathbb C};\\
\mathrm{IGr}_{n-1}({\mathbb C}^{2n-2})&\hookrightarrow&\mathrm{IGr}_{n}({\mathbb C}^{2n}),\end{aligned}$$ where $\mathrm{Gr}_{p}({\mathbb C}^{k}):=\{V\subset{\mathbb C}^{k}\,:\,\mathrm{dim}\,V=p\}$ is the complex Grassmanian, $Q^m{\mathbb C}:=\{z\in\mathbb P^{m+1}{\mathbb C}\,:\, z_0^2+\cdots+z_{m+1}^2=0\}$ is the complex quadric and $\mathrm{IGr}_{n}({\mathbb C}^{2n}):=\{V\subset{\mathbb C}^{k}\,:\,\mathrm{dim}V=n,
 \,Q\big\vert_{V}\equiv0\}$ is the isotropic Grassmanian for ${\mathbb C}^{2n}$ equipped with a quadratic form $Q$.

Outline of the proof of Theorem \[thm:nor\_der\]
------------------------------------------------

The implication (i)$\Rightarrow$(ii) is obvious. On the other hand, for split rank one symmetric spaces there are three other cases (i.e., (1), (2) and (3) in Table \[table:1\]) where the $G^\tau$-intertwining operators are not given by normal derivatives. In Section \[section:6\] we construct them explicitly. This will conclude the implication (ii)$\Rightarrow$(iii). For the rest of this section we shall give a proof for the implication (iii)$\Rightarrow$(i).

Consider a homomorphism: $
T:W^\vee\longrightarrow S(\mathfrak n_-^{-\tau})\otimes V^\vee.
$ We regard $S(\mathfrak n_-^{-\tau})\otimes V^\vee$ as a subspace of $\operatorname{Pol}(\mathfrak n_+)\otimes V^\vee$ on which the Lie algebra $\mathfrak g$ acts by $\widehat{d\pi_\mu}$, see (\[eqn:dpi2\]). If $T$ is a $K^\tau$-homomorphism, the differential operator $\widetilde T:
\mathcal O(G/K,\mathcal V_X)\to\mathcal O(G^\tau/K^{\tau},\mathcal W_Y)$ is $K^\tau$-equivariant. The following statement gives a sufficient condition for $\widetilde T$ to be $G^\tau$-equivariant.

\[prop:415\] The normal derivative $\widetilde T$ induces a $G^\tau$-equivariant differential operator from $\mathcal V_X$ to $\mathcal W_Y$ if $T$ is a $K^\tau$-homomorphism and $T(W^\vee)$ is contained in $\left( \operatorname{Pol}(\mathfrak n_+)\otimes V^\vee\right)^{\widehat{d\pi_\mu}(\mathfrak n_+^\tau)}$.

The proof is a direct consequence of the F-method as follows. First of all, we note that $D\in 
\operatorname{Diff}^{\mathrm {const}}(\mathfrak n_-)\otimes \operatorname{Hom}_{\mathbb C}(V,W)$ is a normal derivative along the normal direction $\mathfrak n_-^{-\tau}$ if and only if $(\operatorname{Symb}\otimes\operatorname{id})(D)
\in \operatorname{Pol}(\mathfrak n_+^{-\tau})\otimes \operatorname{Hom}(V,W)$ in the diagram of Theorem \[thm:313\]. Furthermore, by Proposition \[prop:Fc\], $F_c\otimes\operatorname{id}_W$ induces an isomorphism of two subspaces: $$\label{eqn:622}
\operatorname{Hom}_{\mathfrak p^\tau}(W^\vee,{\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}(V^\vee))\simeq
\left(\operatorname{Pol}(\mathfrak n_+)\otimes \operatorname{Hom}(V,W)\right)^{\widehat{d\pi_\mu}(\mathfrak p^\tau)}.$$ As $\mathfrak l^\tau$ is the complexification of the Lie algebra of $K^\tau$, $T\in\operatorname{Hom}_{K^\tau}(W^\vee,S(\mathfrak n_-^{-\tau})\otimes V^\vee)$ defines an element in $\left(\operatorname{Pol}(\mathfrak n_+)\otimes\operatorname{Hom}(V,W)\right)^{\widehat{d\pi_\mu}(\mathfrak l^\tau)}$, and therefore the invariance with respect to $\mathfrak p^\tau=\mathfrak l^\tau+\mathfrak n_+^\tau$ reduces to that with respect to $\mathfrak n_+^\tau$.

Hence, by Theorem \[thm:313\], the normal derivative $\widetilde T$ coincides with the $G^\tau$-equivariant differential operator $D_{X\to Y}(\varphi_T)$, where $\varphi_T$ is the preimage of $T$ in (\[eqn:622\]).

\[lem:64\] Suppose $(\mathfrak g({\mathbb R}),\mathfrak g({\mathbb R})^\tau)$ is a split rank one irreducible symmetric pair of holomorphic type and $\lambda$ satisfying . For $a\in{\mathbb N}$ we define an $\mathfrak l^\tau$-module: $$\label{eqn:wa}
W^a_\lambda:= S^a(\mathfrak n_+^{-\tau})\otimes{\mathbb C}_\lambda.$$

1.  The module $W^a_\lambda$ is irreducible for any $a\in{\mathbb N}$.

2.  If for an irreducible $K^\tau$-module $W$ there exists a nonzero continuous $G^\tau$-homomorphism $\mathcal{O}(G/K,\mathcal L_\lambda) \to \mathcal O(G^\tau/K^\tau,\mathcal W)$, then the module $W$ is isomorphic to $W^a_\lambda$ for some $a\in{\mathbb N}$.

3.  Assume that $$\label{eqn:homs}
    \operatorname{Hom}_{\mathfrak l^\tau}(S^a(\mathfrak n_-^{-\tau}),
    S^{a_1}(\mathfrak n_-^\tau)\otimes S^{a-a_1}(\mathfrak n_-^{-\tau}))=\{0\}\quad \mathrm{for\,  any}\, 1\leq a_1\leq a.$$ Then, the normal derivative $\widetilde T$ corresponding to the natural inclusion $T:(W_\lambda^a)^\vee\to S(\mathfrak n_-^{-\tau})\otimes({\mathbb C}_\lambda)^\vee$ is a $G^\tau$-equivariant differential operator.

If $\mathrm{rank}_{\mathbb R}\, G/G^\tau=1$, then the noncompact part of $\mathfrak{g}(\mathbb{R})^{\tau\theta}$ is isomorphic to $\mathfrak{su}(1,n)$ for some $n$. Thus the first statement follows from the observation that the action of $\mathfrak l^\tau$ on $\mathfrak n_+^{-\tau}$ corresponds to the natural action of $\mathfrak{gl}_n({\mathbb C})$ on ${\mathbb C}^n$.

The second statement is due to Theorem \[thm:C\] for $k=\mathrm{rank}_{\mathbb R}\, G/G^\tau=1$.

We have the following natural inclusions $A\subset B\supset C$, where $$A:=\operatorname{Pol}^a(\mathfrak n_+^{-\tau})\otimes{\mathbb C}_\lambda^\vee,\,
B:=\operatorname{Pol}^a(\mathfrak n_+)\otimes{\mathbb C}_\lambda^\vee,\, C:=
(\operatorname{Pol}^a(\mathfrak n_+)\otimes{\mathbb C}_\lambda^\vee)^{\widehat{d\pi(\mathfrak n_+^\tau)}}.$$ Therefore $$\operatorname{Hom}_{\mathfrak l^\tau}((W^a_\lambda)^\vee,
A)\hookrightarrow
\operatorname{Hom}_{\mathfrak l^\tau}((W^a_\lambda)^\vee,B)\hookleftarrow\operatorname{Hom}_{\mathfrak l^\tau}((W^a_\lambda)^\vee, C).$$ The left-hand side is an isomorphism because of the assumption (\[eqn:homs\]). Moreover, since $A$ is isomorphic to the irreducible $\mathfrak l^\tau$-module $W_\lambda^a$, the first term is one-dimensional by Schur’s lemma. The last one is also one-dimensional according to the multiplicity-one decomposition given in Fact \[fact:A\]. Therefore, all three terms coincide.

Hence the canonical isomorphism $T:(W_\lambda^a)^\vee\to S(\mathfrak n_-^{-\tau})\otimes({\mathbb C}_\lambda)^\vee$ satisfies the assumption of Proposition \[prop:415\]. Thus Lemma follows.

The highest weight vectors of the generalized Verma module ${\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}({\mathbb C}_\lambda^\vee)$ with respect to $\mathfrak p^\tau$ have a significantly simple form if the condition (\[eqn:homs\]) is satisfied. In fact, by Poincaré–Birkhoff–Witt theorem ${\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}({\mathbb C}_\lambda^\vee)$ is isomorphic, as an $\mathfrak l$-module, to $S(\mathfrak n_-)\otimes{\mathbb C}_\lambda^\vee$, when $\mathfrak n_-$ is abelian. Under the assumption we thus have $$\left({\operatorname{ind}_{\mathfrak p}^{\mathfrak g}}({\mathbb C}_\lambda^\vee)\right)^{\mathfrak n_+^\tau}\simeq
\bigoplus_{a=0}^\infty S^a(\mathfrak n_-^{-\tau})\otimes {\mathbb C}_\lambda^\vee.$$ This formula is an algebraic explanation of the fact that $G^\tau$-equivariant operators are given by normal derivatives in this setting.

In order to conclude the proof of Theorem \[thm:nor\_der\] we have to show that in all cases mentioned in (iii) the condition (\[eqn:homs\]) is fulfilled. It will be done in the next subsection.

An application of the classical branching rules {#section:53}
-----------------------------------------------

In what follows, we shall verify the condition (\[eqn:homs\]) for the last three cases (4), (5) and (6) in Table \[table:1\] by using some classical branching rules of irreducible representations of $\mathfrak{gl}_m({\mathbb C})$.

Denote by $F(\mathfrak{gl}_m({\mathbb C}),\mu)$ the finite dimensional irreducible $\mathfrak{gl}_m({\mathbb C})$-module with highest weight $\mu$. For example, the natural representation of the Lie algebra $\mathfrak{gl}_m({\mathbb C})$ on ${\mathbb C}^m$ corresponds to $F(\mathfrak{gl}_m({\mathbb C}),\,(1,0,\ldots,0))$ and its contragredient representation on $({\mathbb C}^m)^\vee$ to $F(\mathfrak{gl}_m({\mathbb C}),\,(0,0,\ldots,0,-1))$, while the action of $\mathfrak{gl}_m({\mathbb C})$ on the space of symmetric matrices $\operatorname{Sym}(m,{\mathbb C})\simeq S^2({\mathbb C}^m)$ given by $C\mapsto XC\,{{}^t\!}X$ for $X\in\mathfrak{gl}_m({\mathbb C})$ and $C\in Sym(m,{\mathbb C})$ corresponds to $F(\mathfrak{gl}_m({\mathbb C}),\,(2,0,\ldots,0))$. More generally, the action of $\mathfrak{gl}_m({\mathbb C})$ on the space of $i$-th symmetric tensors is no longer irreducible and decomposes as follows: $$\begin{aligned}
\label{eqn:Si-deco}
S^i\left(\operatorname{Sym}(m,{\mathbb C})\right)&\simeq &S^i\left(S^2({\mathbb C}^m)\right)\nonumber
\\
&\simeq&\bigoplus_{\substack{i_1\geq\ldots\geq i_m\geq0\\
i_1+\dots+i_m=i}}F(\mathfrak{gl}_m({\mathbb C}),\,(2i_1,2i_2,\ldots,2i_m)).\end{aligned}$$

In turn, the classical Pieri’s rule implies the following decomposition for the tensor product of such modules: $$\begin{aligned}
\label{eqn:Pieri}
S^i\left(S^2({\mathbb C}^m)\right)\otimes S^k\left({\mathbb C}^m\right)
\simeq\bigoplus_{\substack{i_1\geq\ldots\geq i_m\geq0,\\
i_1+\dots+i_m=i}}
\bigoplus_{\substack{\ell_1\geq 2i_1\geq\ldots\geq\ell_m\geq2i_m,\\ \sum_{r=1}^m(\ell_r-2i_r)=k}}
F(\mathfrak{gl}_m({\mathbb C}),\,(\ell_1,\ldots,\ell_m)).\nonumber\end{aligned}$$

\[rem:pieri\] The summand of the form $F(\mathfrak{gl}_m({\mathbb C}),\,(\ell,0,\ldots,0))$ occurs in the above formula if and only if $i_2=\cdots=i_m=0$, hence $i_1=i$ and $\ell-2i=k$. This remark will be used in Section \[sec:sp\].

Let $G=U(p,q)$, $G^\tau=U(1)\times U(p-1,q)$ and $\mathfrak l^\tau=\mathfrak k^\tau({\mathbb R})\otimes_{\mathbb R}{\mathbb C}\simeq \mathfrak{gl}_1({\mathbb C}) +\mathfrak{gl}_{p-1}({\mathbb C})+\mathfrak{gl}_q({\mathbb C})$. Then, the decomposition $
\mathfrak n_-\equiv \mathfrak n_-^\tau\oplus\mathfrak n_-^{-\tau}
$ as an $\mathfrak l^\tau$-module amounts to $$({\mathbb C}^p)^\vee\boxtimes{\mathbb C}^{q}
\simeq({\mathbb C}\boxtimes({\mathbb C}^{p-1})^\vee\boxtimes{\mathbb C}^q)\oplus({\mathbb C}_{-1}\boxtimes
{\mathbb C}\boxtimes{\mathbb C}^q).$$ Where $\boxtimes$ stands for the outer tensor product representation. Since the $\mathfrak l^\tau$-module $S^a(\mathfrak n_-^{-\tau})$ is isomorphic to the irreducible module ${\mathbb C}_{-a}\boxtimes{\mathbb C}\boxtimes S^a({\mathbb C}^q)$, the space $$\bigoplus_{a_1+a_2=a}\operatorname{Hom}_{\mathfrak l^\tau}
(S^a(\mathfrak n_-^{-\tau}),{\mathbb C}_{-a_2}\boxtimes S^{a_1}({\mathbb C}^{p-1})^\vee\boxtimes
(S^{a_1}({\mathbb C}^{q})\otimes S^{a_2}({\mathbb C}^{q})))$$ is not reduced to zero if and only if $a_1=0$ and $a_2=a$. Therefore, the condition is satisfied.

\[example:SO-U\] Let $G=SO(2,2n)$, $ G^\tau=U(1,n)$ and $\mathfrak l^\tau=\mathfrak{gl}_1({\mathbb C})+\mathfrak{gl}_n({\mathbb C}).$ Then the decomposition $
\mathfrak n_-\equiv \mathfrak n_-^\tau\oplus\mathfrak n_-^{-\tau}
$ as an $\mathfrak l^\tau$-module amounts to $${\mathbb C}_{-1}\boxtimes{\mathbb C}^{2n}\simeq({\mathbb C}_{-1}\boxtimes{\mathbb C}^n)\oplus({\mathbb C}_{-1}\boxtimes
({\mathbb C}^n)^\vee).$$ Thus, $\mathfrak l^\tau$-module $S^a(\mathfrak n_-^{-\tau})$ is isomorphic to the irreducible module ${\mathbb C}_{-a}\boxtimes\left(S^a({\mathbb C}^n)\right)^\vee$, whereas we have an isomorphism of ${\mathfrak {l}}^{\tau}$-modules: $$S^{a_1}(\mathfrak n_-^\tau)\otimes S^{a_2}(\mathfrak n_-^{-\tau})\simeq
{\mathbb C}_{-a_1-a_2}\boxtimes(S^{a_1}({\mathbb C}^n)\boxtimes S^{a_2}({\mathbb C}^n)).$$ Thus, the space $\operatorname{Hom}_{\mathfrak l^\tau}
(S^a(\mathfrak n_-^{-\tau}), S^{a_1}(\mathfrak n_-^\tau)\otimes
S^{a_2}(\mathfrak n_-^{-\tau}))$ is not reduced to zero if and only if $a_1=0$ and $a_2=a$. Therefore, the condition is satisfied.

Let $G=SO^*(2n)$, $G^\tau=SO^*(2n-2)\times SO(2)$ and $\mathfrak l^\tau=\mathfrak{gl}_{n-1}({\mathbb C})+ \mathfrak{gl}_1({\mathbb C})$. In this case, the decomposition $
\mathfrak n_-\equiv \mathfrak n_-^\tau\oplus\mathfrak n_-^{-\tau}
$ as an $\mathfrak l^\tau$-module amounts to $$(\operatorname{Alt}
({\mathbb{C}}^{n-1})^{\vee}
\boxtimes {\bf{1}}
)
\oplus 
(({\mathbb{C}}^{n-1})^{\vee}\boxtimes{\mathbb{C}}_{-1}).$$ Thus the ${\mathfrak {l}}^{\tau}$-modules $S^{a}({\mathfrak {n}}_-^{-\tau})$ and $S^{a_1}({\mathfrak {n}}_-^{\tau})
 \otimes
 S^{a_2}({\mathfrak {n}}_-^{-\tau})$ are given as $$\begin{aligned}
S^{a}({\mathfrak {n}}_-^{-\tau})
\simeq&
 S^{a}(({\mathbb {C}}^{n-1})^{\vee}) \boxtimes {\mathbb{C}}_{-a}, 
\\
S^{a_1}({\mathfrak {n}}_-^{\tau})
\otimes
S^{a_2}({\mathfrak {n}}_-^{-\tau})
\simeq&
(S^{a_1}(\operatorname{Alt}({\mathbb {C}}^{n-1})^{\vee}) 
\otimes
S^{a_2}(({\mathbb {C}}^{n-1})^{\vee}))
\boxtimes
{\mathbb{C}}_{-a_2}.  \end{aligned}$$ Hence the compatibility condition for the action of $\mathfrak{gl}_1({\mathbb C})\subset\mathfrak l^\tau$ implies .

Thus we have verified the assumption for all the three symmetric pairs $(\mathfrak g({\mathbb R}),\mathfrak g({\mathbb R})^\tau)$ and have proved the implication $(iii)\Rightarrow(i)$ in Theorem \[thm:nor\_der\] by Lemma \[lem:64\] (3).

Partial differential operators built on Jacobi polynomials {#section:6}
==========================================================

In this section we apply the F-method to the remaining cases of split rank one symmetric pairs of holomorphic type (see Table \[table:1\], (1), (2) and (3)). In all three cases, the families of equivariant differential operators we discover are built on special values of Jacobi polynomials in one variable.

We shall give a detailed account of each step of the F-method according to the recipe (see Section \[sec:44\]) in the first case. For the remaining two cases, we skip similar computations and highlight new features, e.g. a trick that allows to describe vector-bundle valued differential operators.

Case of $SO(n,2)\downarrow SO(n-1,2)$
-------------------------------------

Let $n\ge2$ and $\widetilde Q$ be a non-degenerate quadratic form on ${\mathbb C}^{n+2}$ defined by $\widetilde Q(w)=w_0^2+\cdots+w_n^2-w_{n+1}^2$. Then the special orthogonal group $G_{\mathbb C}:=SO({\mathbb C}^{n+2},\widetilde Q)$ acts transitively on the isotropic cone $$\Xi_{\mathbb C}:=\{w\in{\mathbb C}^{n+2}\setminus\{0\}:\,\widetilde Q(w)=0\},$$ and also on the complex quadric $$\mathrm Q^n{\mathbb C}:=\Xi_{\mathbb C}/{\mathbb C}^*\subset \mathbb P^{n+1}{\mathbb C},$$ which is a flag variety containing ${\mathbb C}^n$ as an open dense subset (open Bruhat cell): $$\label{eqn:bruhat}
{\mathbb C}^n\to\mathrm Q^ n{\mathbb C},\quad z\mapsto\left[1-\frac{\vert z\vert^2}{4}:z:
1+\frac{\vert z\vert^2}{4}\right].$$

We write $p:\Xi_{\mathbb C}\to Q^n{\mathbb C}$ for the natural projection.

The quadratic form $\widetilde Q$ is of signature $(n,2)$ when restricted to the real vector space $E_{\mathbb R}:= \sqrt{-1}e_0+\sum_{j=1}^{n+1}{\mathbb R}e_j$, where $\{e_j: 0\leq j\leq n+1\}$ is the standard basis in ${\mathbb C}^{n+2}$. Thus we have an isomorphism: $$SO({\mathbb C}^{n+2}, \widetilde Q)\cap GL_{\mathbb R}(E_{\mathbb R})\simeq SO(n,2).$$ Let $G$ be its identity component $SO_o(n,2)$. Then the $G$-orbit through the origin in the open Bruhat cell ${\mathbb C}^n$ ($\subset Q^n{\mathbb C}$) is still contained in ${\mathbb C}^n$, and is identified with the Lie ball $\displaystyle X:=\{z\in {\mathbb C}^n:\vert z{{}^t\!}z\vert^2+1
-2\overline z{{}^t\!}z>0,\, \vert z{{}^t\!}z\vert<1\}$.

Let $\tau$ be the involution of $G$ acting by conjugation by $\mathrm{diag}(1,\ldots,1,-1,1)$. It leaves $G$ invariant, and we denote by $G'$ the identity component of $G^\tau$. The group $G'=SO_o(n-1,2)$ acts on the subsymmetric domain $$Y:= X\cap\{z_n=0\}.$$ Note that $X\simeq SO_o(n,2)/SO(n)\times SO(2)$ is the bounded Hermitian symmetric domain of type IV in $ \mathfrak n_-\simeq {\mathbb C}^n$ and $Y\simeq SO_o(n-1,2)/SO(n-1)\times SO(2)$ a subsymmetric space of complex codimension one.

We construct explicit holomorphic differential operators that give the projectors onto irreducible summands in the branching laws with respect to the symmetric pair $(SO_o(n,2), SO_o(n-1,2))$.

By using the Gegenbauer polynomial $C_\ell^\alpha(x)$ (see Appendix \[sec:A2\]), we set $$\label{eqn:Cxy}
C_\ell^\alpha(x,y):=x^{\frac\ell2}C_\ell^\alpha\left(\frac y{\sqrt x}\right)=
\sum_{k=0}^{\left[\frac \ell2\right]}(-1)^k\frac{\Gamma(\ell-k+\alpha)}
{\Gamma(\alpha)\Gamma(k+1)\Gamma(\ell-2k+1)}(2y)^{\ell-2k}x^{k}.$$ For instance, $C_0^\alpha(x,y)=1$, $C_1^\alpha(x,y)=2\alpha y$, $C_2^\alpha(x,y)=2\alpha(\alpha+1)y^2-\alpha x$, etc. Notice that, $C_\ell^\alpha(x^2,y)$ is a homogeneous polynomial of $x$ and $y$ of degree $\ell$.

\[thm:SO(2,n)\] Suppose $\lambda\in\mathbb{Z}$ satisfies $\lambda>n-1$, and $a\in {\mathbb N}$. Then, any $G^\tau$-intertwining operator from $\mathcal O(G/K,\mathcal L_\lambda)$ to $\mathcal O(G^\tau/K^\tau,\mathcal L_{\lambda+a})$ is given, up to a scalar, by the differential operator of degree $a$: $$\label{eqn:thm66}
D_{X\to Y,a}:=C_a^{\lambda-\frac{n-1}2}\left(-\Delta_{{\mathbb C}^{n-1}},\frac{\partial}{\partial z_n}\right).$$

${}$

1.  The same statement remains true for intertwining operators between the unitary representations $\mathcal H^2(G/K,\mathcal L_\lambda)$ and $\mathcal H^2(G^\tau/K^\tau,\mathcal L_{\lambda+a})$.

2.  For every $\lambda\in{\mathbb C}$ and $a\in{\mathbb N}$, $D_{X\to Y,a}$ given by the same formula, is a $\widetilde G^\tau$-equivariant holomorphic differential operator between two homogeneous bundles $\mathcal L_{\lambda}\to G/K$ and $ \mathcal L_{\lambda+a}\to G^\tau/K^\tau $.

This result is a ‘holomorphic version’ of the conformally covariant operator considered by A. Juhl [@Juhl] in the setting $S^n\hookrightarrow S^{n+1}$, with equivariant actions of the pair of groups $SO(n+1,1) \subset SO(n+2,1)$, respectively. For the case $n=1$ we recover the celebrated Rankin–Cohen brackets.

In order to prove Theorem \[thm:SO(2,n)\] we apply the F-method (see Section \[sec:44\]). We set $H_o:=E_{0,n+1}+E_{n+1,0}$, and $\mathfrak a={\mathbb C}H_o$. Then, the Lie algebra $\mathfrak g=\mathfrak{so}({\mathbb C}^{n+2},\widetilde Q)$ is a sum $$\mathfrak g=\mathfrak n_-+(\mathfrak a+\mathfrak  m)+\mathfrak n_+$$ of $-1,0,$ and $1$ eigenspaces of $\mathrm{ad}(H_o)$, respectively. The stabilizer of the base point $w_o=[1:0:\cdots:0:1] \in Q^n{\mathbb C}$ is a parabolic subgroup $P_{\mathbb C}$ with Langlands decomposition $P_{\mathbb C}=M_{\mathbb C}A_{\mathbb C}N_{\mathbb C}$, where $A_{\mathbb C}= \exp\mathfrak a$ and $N_{\mathbb C}=\exp\mathfrak n_+$.

As Step 1 of the F-method we define a standard basis of $\mathfrak n_+\simeq {\mathbb C}^n$ by $$\begin{aligned}
C_j&:=&E_{j,0}-E_{j,n+1}-E_{0,j}-E_{n+1,j}\quad (1\leq j\leq n), \end{aligned}$$ and similarly a standard basis of $\mathfrak n_-\simeq {\mathbb C}^n$ by $$\begin{aligned}
\overline C_j&:=&E_{j,0}+E_{j,n+1}-E_{0,j}+E_{n+1,j}
\quad (1\leq j\leq n), \end{aligned}$$

Let $Z=\frac12\sum_{i=1}^n z_i\overline C_i\in\mathfrak n_-$ and $Y=\sum_{j=1}^n y_j C_j\in\mathfrak n_+$. Notice that $\exp(Z)\cdot w_o=z\in{\mathbb C}^n\subset\mathrm Q^n{\mathbb C}$ via . Moreover, with the notations of and , one has $$\begin{aligned}
\alpha(Y,Z)&=&-(z,y) H_o \quad\mathrm{mod}\,\mathfrak m;\\
\beta(Y,Z)&=&(z,y)E_z-\frac12 Q_n(z)\sum_{j=1}^n y_j\frac\partial{\partial z_j},\end{aligned}$$ where $(z,y)=z_1y_1+\cdots+ z_ny_n$ and $\displaystyle Q_n(z):=z_1^2+\cdots+z_n^2.$

For $\mu\in{\mathbb C}$, we define a character $\mu:\mathfrak a={\mathbb C}H_o\to{\mathbb C}$ by $H_o\mapsto \mu$, and extend it to a character of $\mathfrak p$, denoted by the same letter $\mu:\mathfrak p\to{\mathbb C}$, or simply by ${\mathbb C}_\mu$, as the composition of the projection $\mathfrak p\to\mathfrak p/(\mathfrak m+\mathfrak n)$ and the above map $\mu:\mathfrak a\to{\mathbb C}$.

Then the infinitesimal action $d\pi_\mu({C_j})$ is given by $$\label{eqn:dpic22}
d\pi_\mu(C_j)=-\mu z_j-
z_jE_z+\frac12Q_n(z)\frac\partial{\partial z_j},$$ where $E_z:=\sum_{j=1}^{n}z_j\frac\partial{\partial z_j}$.

For $C\in{\mathbb C}^n\simeq\mathfrak n_+$ and $Z\in{\mathbb C}^n\simeq\mathfrak n_-$ one has, $$\begin{aligned}
\widehat{d\pi_\mu}(C_j)&=&\lambda \frac\partial{\partial \zeta_j}+E_\zeta
\frac\partial{\partial \zeta_j}-\frac12\zeta_j\Box_\zeta,
\qquad 1\leq j\leq n,\end{aligned}$$ where $\mu=\lambda^\vee\otimes{\mathbb C}_{2\rho}=-\lambda+n$, $E_\zeta:=\sum_{j=1}^{n}\zeta_j\frac\partial{\partial \zeta_j}$ and $\Box_\zeta$ is the d’Alembertian differential operator of symbol $Q_n(z)$.

The proof is straightforward from Definition \[def:hat\] and .

As for Step 2 we write $\zeta=(\zeta',\zeta_n)\in{\mathbb C}^n$ with $\zeta'=(\zeta_1,\ldots,\zeta_{n-1})\in{\mathbb C}^{n-1}$ and, according to Lemma \[lem:64\] (2), we set for $\lambda\in{\mathbb C}$ $$\label{eqn:Wmuso}
W_\lambda^a:=S^a(\mathfrak n_+^{-\tau})\otimes{\mathbb C}_\lambda.$$ The vector space $W_\lambda^a$ is one-dimensional because $\mathfrak n_-^{-\tau}\simeq {\mathbb C}$. We denote by $\nu$ the action of $\mathfrak l^\tau$ on $W_\lambda^a$. In our setting where $\dim V=\dim W_\lambda^a=1$, Step 3 reduces to find polynomials $\psi$ which are, according to (\[eqn:Fmethod1\]) and (\[eqn:Fmethod2\]), $\mathfrak m$-invariant, homogeneous of degree $a$ and satisfying $$\widehat{d\pi_\lambda}(C_j)\psi=
\left(\lambda \frac\partial{\partial \zeta_j}+E_\zeta
\frac\partial{\partial \zeta_j}-\frac12\zeta_j\Box_\zeta\right)\psi=0,
\qquad 1\leq j\leq n-1.$$

${}$

1.  For the irreducible $\mathfrak l^\tau$-module $W_\lambda^a$ given in the highest weight of $(W_\lambda^a)^\vee$ is given by $
    \chi:= \lambda+a.
    $

2.  For the $\mathfrak m$-module $\mathrm{Pol}(\mathfrak n_+)\otimes V^\vee$, the $\chi$-weight space for $\mathfrak b(\mathfrak l^\tau)$ is given by $$(\mathrm{Pol}(\mathfrak n_+)\otimes V^\vee)_\chi\simeq
    \operatorname{Pol}^a[\xi_n^2,Q(\xi')],$$ where we identify $\operatorname{Pol}(\mathfrak n_+)\otimes V^\vee$ with $\operatorname{Pol}(\mathfrak n_+)$ as vector spaces.

First notice that in the present setting $\mathfrak l^\tau= \mathfrak{so}(n-1,{\mathbb C})+\mathfrak{so}(2,{\mathbb C})$. The decomposition $\mathfrak n_-\equiv \mathfrak n_-^\tau\oplus\mathfrak n_-^{-\tau}$ as an $\mathfrak l^\tau$-module amounts to $${\mathbb C}^{n} \boxtimes{\mathbb C}_{-1}\simeq({\mathbb C}^{n-1}\boxtimes{\mathbb C}_{-1})\oplus({\mathbb C}\boxtimes{\mathbb C}_{-1}
).$$

Since we have isomorphisms as ${\mathfrak {l}}^{\tau}$-modules: $$S^a(\mathfrak n_-)
\simeq 
\bigoplus_{a_1+a_2=a} S^{a_1}(\mathfrak n_-^\tau)
\otimes S^{a_2}(\mathfrak n_-^{-\tau})=
\bigoplus_{a_1=0}^a S^{a_1}({\mathbb C}^{n-1}) \boxtimes {\mathbb C}_{-a}$$ and since the $\mathfrak l^\tau$-module $ S^a(\mathfrak n_-^{-\tau})$ is isomorphic to the irreducible module ${\mathbb C}_{-a}\boxtimes{\mathbb C}$, we obtain $$\begin{aligned}
\operatorname{Hom}_{\mathfrak l^\tau}(S^a(\mathfrak n_-^{-\tau}), S^a(\mathfrak n_-))
&\simeq&\bigoplus_{a_1+a_2=a}\operatorname{Hom}_{\mathfrak l^\tau}
({\mathbb C}\boxtimes{\mathbb C}_{-a},  S^{a_1}({\mathbb C}^{n-1}) \boxtimes{\mathbb C}_{-a_1})
\otimes({\mathbb C}\boxtimes {\mathbb C}_{-a_2})\\
&=&\bigoplus_{a_1+a_2=a}\operatorname{Hom}_{\mathfrak{so}(n-1)}
({\mathbb C}, S^{a_1}({\mathbb C}^{n-1}))\\
&=&\bigoplus_{0\leq j\leq\frac a2}{\mathbb C}Q_{n-1}(\zeta')^{j}\cdot\vert \zeta_n\vert^{a-2j}.\end{aligned}$$

Suppose $n\geq3$. There exists a polynomial $$\label{eqn:gs}
g(t)\in{\mathbb C}\operatorname{-Span}\left\langle t^{a-2j}:0\leq j\leq\left[\frac a2\right]
\right\rangle$$ of one variable $t$ such that every $\psi\in\operatorname{Pol}^a[\xi_n^2,Q(\xi')]$ can be written as $$\label{eqn:Dmodmorph2}
\psi(\zeta)=\left( T_ag\right)(\zeta):=Q_{n-1}(\zeta')^{\frac a2}g\left(\frac{\zeta_n}{\sqrt{Q_{n-1}(\zeta')}}\right).$$

We note that for a more general $g\in{\mathbb C}[t]$ the map $T_ag$ is a (multi-valued) meromorphic function of $\zeta_1,\ldots,\zeta_n$.

To implement Step 4 of the F-method we say that a differential operator $R$ on ${\mathbb C}^n$ is *$T$-saturated* if there exists an operator $S$ on ${\mathbb C}$ such that the following diagram commutes: $$\xymatrix{
{\mathbb C}[t]\ar[r]^{T_a\quad}\ar[d]_{S}& {\mathbb C}(\zeta_1,\ldots,\zeta_n)\ar[d]^{R}\\
{\mathbb C}[t]\ar[r]^{T_a\quad}&{\mathbb C}(\zeta_1,\ldots,\zeta_n)
.}$$ Such an operator $S$ is unique (if exists), and we denote it by $T_a^\sharp R$. We allow $R$ to have meromorphic coefficients. We note that $$\label{eqn:tt}
T_a^\sharp(R_1\cdot R_2)=T_a^\sharp(R_1)\cdot T_a^\sharp(R_2),$$ whenever it makes sense.

We give several examples of saturated differential operators that we shall use later.

\[lem:Ta\] For every $0\leq j<n$ one has: $$\begin{aligned}
T_a^\sharp\left(\zeta_j E_{\zeta'}-Q_{n-1}(\zeta')\frac{\partial}{\partial \zeta_j}\right)&=&0,\label{eqn:TaA}\\
T_a^\sharp\left((a-1)\zeta_n-E_\zeta\frac\partial{\partial\zeta_j}\right)&=&0.
\label{eqn:TaB}\end{aligned}$$

The proof of both statements is straightforward from the definition of $T_a$.

\[lem:T\] Let $\vartheta_t:=t\frac d{dt}$ and $\Delta_{{\mathbb C}^{n-1}}:=\frac{\partial^2}{\partial \zeta_1^2}+\cdots+
\frac{\partial^2}{\partial \zeta_n^2}$. One then has:

1.  $T_a^\sharp(E_{\zeta'})=  a-\vartheta_t$.

2.  $T_a^\sharp\left
    (\frac{Q_{n-1}(\zeta')}{\zeta_j}\frac\partial{\partial \zeta_j}\right)
    =(a-\vartheta_t)$.

3.  $T_a^\sharp\left
    (\frac{Q_{n-1}(\zeta')}{\zeta_j}E_ \zeta\frac\partial{\partial \zeta_j}\right)
    =(a-1)(a-\vartheta_t)$.

4.  $T_a^\sharp({\zeta_n^2\Delta_{{\mathbb C}^{n-1}}})=
    t^2(\vartheta_t-a)(\vartheta_t-n-a+3)$.

5.  $T_a^\sharp({Q_{n-1}(\zeta')\Delta_{{\mathbb C}^{n-1}}})=
    (\vartheta_t-a)(\vartheta_t-n-a+3)$.

6.  $T_a^\sharp({Q_{n-1}(\zeta')\frac{\partial^2}{\partial \zeta_n^2}})=
     t^{-2}(\vartheta_t^2-\vartheta_t)$.

7.  $T_a^\sharp({\zeta_n\frac\partial{\partial \zeta_n}})=\vartheta_t$.

8.  $T_a^\sharp({\zeta_n^2\frac{\partial^2}{\partial \zeta_n^2}})=
    \vartheta_t^2-\vartheta_t$.

Notice first that the identity (1) is equivalent to (2) according to (\[eqn:TaA\]) and that the identity (3) may be deduced from (1) or (2) by (\[eqn:TaB\]). Furthermore, identities (4) and (5) on the one hand and (6) and (8) on the other are equivalent according to the very definition of $T$-saturation as $t=\frac{\zeta_n}{\sqrt{Q_{n-1}(\zeta')}}$.

Thus, it would be enough to show only identities (1), (4), (7) and (8). We give a proof for the first statement, the remaining cases can be treated in a similar way. Let $1\leq j\leq n-1$. Then $$\begin{aligned}
&&\left(T_a^\sharp(E_{\zeta'})g\right)(t)=\sum_{j=1}^{n-1} \zeta_j\frac{\partial}{\partial \zeta_j}\left(Q_{n-1}(\zeta')^{\frac a2}
g\left(\frac{\zeta_n}{\sqrt{Q_{n-1}(\zeta')}}\right)\right)\\
&=&
aQ_{n-1}(\zeta')^{\frac a2-1}g\left(\frac{\zeta_n}{\sqrt{Q_{n-1}(\zeta')}}\right)\sum_{j=1}^{n-1}  \zeta_j^2
-Q_{n-1}(\zeta')^{\frac a2}g'\left(\frac{\zeta_n}{\sqrt{Q_{n-1}(\zeta')}}\right)\sum_{j=1}^{n-1} 
\frac{\zeta_j^2 \zeta_n}{\sqrt{Q_{n-1}^3(\zeta')}}\\
&=&aQ_{n-1}(\zeta')^{\frac a2}g\left(\frac{\zeta_n}{\sqrt{Q_{n-1}(\zeta')}}\right)-
\frac{\zeta_n}{\sqrt{Q_{n-1}(\zeta')}}Q_{n-1}(\zeta')^{\frac a2}g'\left(\frac{\zeta_n}{\sqrt{Q_{n-1}(\zeta')}}\right)
\\
&=&\left(a -t\frac{d}{dt}\right)
g(t).\end{aligned}$$

Thus Step 4 of the F-method in this setting is given as follows:

\[prop:55\] Let $T_a$ be as in .

1.  Any polynomial $\psi$ on $\mathfrak n_+$ of degree $a$ satisfying is of the form $\psi=T_a g$ for some polynomial $g(t)$ of one variable, belonging to (\[eqn:gs\]).

2.  The polynomial $\psi(\zeta)=(T_a g)(\zeta)$ of $n$-variables satisfies the system of partial differential equations if and only if $g(t)$ satisfies the following single ordinary differential equation: $$\label{eqn:JacobiDE3}
    \left((1-s^2)\vartheta_s^2-(1+(2\lambda-n+1)s^2)\vartheta_s
    +a(a+2\lambda-n+1)s^2\right)g(-\sqrt{-1}s)=0,$$ or equivalently, $g(-\sqrt{-1}s)$ is, up to a scalar, the Gegenbauer polynomial $C_a^{\lambda-\frac{n-1}2}(s)$.

The first statement of Proposition follows from and Lemma \[lem:410\].

Consider for any fixed $1\leq j<n$ the differential operator $\frac{Q_{n-1}(\zeta')}{\xi_j}\widehat{d\pi_\mu}(C_j)$. According to identities (1-6) in Lemma \[lem:T\] we have following identities: $$\begin{aligned}
T_a^\sharp\left(\frac{ Q_{n-1}(\zeta')}{\zeta_j}\frac{\partial}{\partial \zeta_j}\right)
&=&a-\theta_t,\\
T_a^\sharp\left(\frac{ Q_{n-1}(\zeta')}{\zeta_j}E \zeta\frac{\partial}{\partial \zeta_j}\right)&=&
(a-\vartheta_t)(a-\vartheta_t-1)+((a-\vartheta_t)\vartheta_t)= (a-1)(a-\vartheta_t),\\
T_a^\sharp\left(\frac{ Q_{n-1}(\zeta')}{\zeta_j} \zeta_j\Box_ \zeta\right)
&=& T_a^\sharp\left(Q_{n-1}(\zeta')\left(\Delta_{{\mathbb C}^{n-1}}+
\frac{\partial^2}{\partial \zeta_n^2}\right)\right)\\
&=&(\vartheta_t-a)(\vartheta_t-n+3-a)+ t^{-2}(\vartheta_t^2-\vartheta_t).\end{aligned}$$

Summing up these terms we get $$T_a^\sharp\left(\frac{Q_{n-1}(\zeta')}{\zeta_j}\widehat{d\pi_\mu}(C_j)\right)=
 (1+t^2)\vartheta_t^2-(1-(2\lambda-n+1)t^2)\vartheta_t
-a(a+2\lambda-n+1)t^2.$$ Thus Proposition follows after the change of variable $t=-\sqrt{-1}s$.

Thus we have carried out the crucial part of the F-method. Let us complete the proof of Theorem \[thm:SO(2,n)\].

\[Proof of Theorem \[thm:SO(2,n)\]\]

The uniqueness of the intertwining operator follows from the multiplicity-freeness of the spectral decomposition in the unitary case (Theorem \[thm:C\]).

The explicit form of the intertwining operator $D_{X\to Y,a}$ follows from Step 5 of the F-method, which is given by Proposition \[prop:55\].

Case of $Sp(n,{\mathbb R})\downarrow Sp(n-1,{\mathbb R})\times Sp(1,{\mathbb R})$ {#sec:sp}
---------------------------------------------------------------------------------

Let $G=Sp(n,{\mathbb R})$ be the Lie group of all linear transformations of ${\mathbb R}^{2n}$ preserving the anti-symmetric bilinear form given by $J_n:=\begin{pmatrix}
0 &-I_n\\I_n&0
\end{pmatrix}$. Let $\tau$ be the involution of $G$ acting by conjugation by $I_{2n-2,2}$, where $I_{p,q}:=
\begin{pmatrix}
I_{p} &0\\0&-I_q
\end{pmatrix}$. We consider the symmetric pair $(G,G^\tau)$, where $G^\tau=Sp(n-1,{\mathbb R})\times Sp(1,{\mathbb R})$.

Denote $H_n$ the Siegel upper half-plane $\{Z\in\operatorname{Sym}(n,{\mathbb C})\,:\, {\mathrm{Im}}\,Z\gg0\}$. It is a Hermitian symmetric domain of type CI in $\mathfrak n_-\simeq \operatorname{Sym}(n,{\mathbb C})$. The Lie group $G=Sp(n,{\mathbb R})$ acts biholomorphically on $H_n$ by $$g\cdot Z=(aZ+b)(cZ+d)^{-1}\quad{\mathrm{for}}\quad
g=\begin{pmatrix}a&b\\c&d\end{pmatrix}\in G,\,Z\in H_n,$$ where the isotropy subgroup $K$ of the origin is isomorphic to $U(n)$.

We set $X:=H_n$ and $Y:=H_{n-1}\times H_1$. Thus $Y\hookrightarrow X$ is a complex $G^\tau$-equivariant submanifold of $X$.

Consider the following decomposition of the Lie algebra $\mathfrak g=\mathfrak{sp}(n,{\mathbb C})=\mathfrak n_-+\mathfrak k+\mathfrak n_+$, $
\left(\begin{matrix}
A&B\\C&-{{}^t\!}A
\end{matrix}
\right)\mapsto (B,A,C)$ with $B={{}^t\!}B$ and $C={{}^t\!}C$. Notice that in the present setting $\mathfrak k=\mathfrak l\simeq\mathfrak{gl}_n({\mathbb C})$.

We write $\widetilde G$ for the universal covering of $G$, and $\widetilde K$ for its connected subgroup with Lie algebra $\mathfrak k$.

For $\lambda\in{\mathbb C}$, the map $ \mathfrak k\to {\mathbb C}$, $A\mapsto  -\lambda\,\mathrm{Trace}\, A$ is a character of $\mathfrak k$, which we denote by ${\mathbb C}_\lambda$ according to that we have chosen a realization of $\mathfrak n_+$ in the *lower* triangular matrices. The character ${\mathbb C}_\lambda$ lifts to $\widetilde K$ and defines a $\widetilde G$-equivariant holomorphic line bundle $\mathcal L_\lambda$ over $X= \widetilde G/\widetilde  K\simeq
G/K$. Notice that in the present case ${\mathbb C}_{2\rho}={\mathbb C}_{n+1}$. Then, according to branching law in Fact \[fact:B\], we have to deal with vector bundles rather than line bundles because there exists a nontrivial $G^\tau$-intertwining operator $D_{X\to Y}(\varphi)$ from $\mathcal O(X,\mathcal L_\lambda)$ to $ \mathcal O(Y,\mathcal W_Y)$ only if $\dim W>1$ except for the trivial case $\mathcal W_Y= \mathcal L_\lambda\vert_Y$. In order to give a concrete model of such a representation space $W$, we define a module of $\mathfrak l^\tau\simeq\mathfrak {gl}_{n-1}({\mathbb C})+\mathfrak{gl}_1({\mathbb C})$ by $$\begin{aligned}
\label{eqn:611}
W_\lambda^a&:= &S^a(\mathfrak n_+^{-\tau})\otimes(-\lambda\,\mathrm{Trace}_n)\\&\simeq&\nonumber
\left(S^a\left(({\mathbb C}^{n-1})^\vee\right)\otimes(-\lambda\,\mathrm{Trace}_{n-1})\right)\boxtimes F(\mathfrak{gl}_1,(-\lambda-a)e_n).\end{aligned}$$ As a vector space $W_\lambda^a$ is isomorphic to the space $\operatorname{Pol}^a(\mathfrak n_-^{-\tau})$ of homogeneous polynomials on $\mathfrak n_-^{-\tau}\simeq {\mathbb C}^{n-1}$. Thus, if $W$ is taken to be $W_\lambda^a$, then the intertwining differential operator can be thought of as an element of ${\mathbb C}\left[\frac{\partial}{\partial z_{ij}}\right]\otimes
\operatorname{Pol}^a[v_1,\ldots,v_{n-1}],$ where $(v_1,\ldots,v_{n-1})$ are standard coordinates on $\mathfrak n_-^{-\tau}\simeq{\mathbb C}^{n-1}$ and $z_{ij}$ those on $\mathfrak n_-\simeq\operatorname{Sym}(n,{\mathbb C}).$

\[thm:Sp\] Suppose $\lambda\in\mathbb{Z}$ satisfies $\lambda>n$, and $a\in{\mathbb N}$. Then any $G^\tau$-intertwining operator from $\mathcal O(X,\mathcal L_\lambda)$ to $\mathcal O(Y,\mathcal W_{\lambda}^a)$ is given, up to a scalar, by $$\label{eqn:thmspn}
D_{X\to Y,a}:=C_a^{\lambda-1}\left(
\sum_{1\leq i, j\leq n-1}2 v_iv_j
\frac{\partial^2}{\partial z_{ij}\partial z_{nn}},
\sum_{1\leq j\leq n-1}v_j\frac{\partial}{\partial z_{jn}}\right)
\in {\mathbb{C}}[\frac{\partial}{\partial z_{ij}}]
 \otimes {\operatorname{Pol}}^a[v_1, \cdots, v_{n-1}],$$ where the polynomial $C_a^{\lambda-1}(x,y)$ was defined in by the Gegenbauer polynomial.

${}$

1.  The same statement remains true for intertwining operators between the unitary representations $\mathcal H^2(X,\mathcal L_\lambda)$ and $\mathcal H^2(Y,\mathcal W_{\lambda}^a)$.

2.  For any $\lambda\in{\mathbb C}$ and $a\in{\mathbb N}$, the operator $D_{X\to Y,a}$ defined by is a $\widetilde G^\tau$-equivariant holomorphic differential operator between two homogeneous bundles $\mathcal L_\lambda\to X$ and $\mathcal W_\lambda^a\to Y$.

In order to prove Theorem \[thm:Sp\] we apply the F-method. Its Step 1 is given by

For $C\in Sym(n,{\mathbb C})\simeq \mathfrak n_+$ and $Z\in Sym(n,{\mathbb C})\simeq \mathfrak n_-$ one has $$\begin{aligned}
d\pi_\mu(C )&=&\mu\operatorname{Trace}(CZ)+
\sum_{i\leq j}\sum_{k,\ell}C_{k\ell}z_{ik}z_{j\ell}\frac\partial{\partial z_{ij}},\\
\widehat{d\pi_\mu}(C ) &=&-\lambda\sum_{i\leq j}C_{ij}\frac{\partial}{\partial\zeta_{ij}}
-\frac12\left(\sum_{i\leq k,j\leq\ell}C_{k\ell}\zeta_{ij}\frac{\partial^2}{\partial\zeta_{ik}\partial\zeta_{j\ell}}
+\sum_{i\geq k,j\geq\ell}C_{k\ell}\zeta_{ij}\frac{\partial^2}{\partial\zeta_{ik}\partial\zeta_{j\ell}}\right),\end{aligned}$$ where $\mu =\lambda^\vee\otimes
{\mathbb C}_{2\rho}=-\lambda+n+1$.

We embed the group $Sp(n,{\mathbb R})$ into $U(n,n)$ and apply the results of Example \[example:410\] with $p=q=n$. Thus, the first statement follows from the formula .

We consider a bilinear form $$\mathfrak n_+\times \mathfrak n_-\to {\mathbb C},\qquad (C,Z)\mapsto \operatorname{Trace}
(C{{}^t\!}Z),$$ where $\mathfrak n_+\simeq\operatorname{Sym}(n,{\mathbb C})\simeq\mathfrak n_-$. Recall that $\zeta_{ij}$ with $1\leq i\leq j\leq n$ are the coordinates on $\mathrm{Sym}(n,{\mathbb C})$. However, it is convenient for the computations below to allow writing $\frac{\partial}{\partial \zeta_{ji}}$ to denote $\frac{\partial}{\partial \zeta_{ij}}$. Then $$\widehat{z_{ij}}=\frac12\left(1+\delta_{ij}\right)\frac{\partial}{\partial \zeta_{ij}},\quad
 \widehat{\frac{\partial}{\partial z_{ij}}}=(\delta_{ij}-2)\zeta_{ij}.$$ Thus the algebraic Fourier transform of the first term of $d\pi_\mu( C)$ amounts to $$({\operatorname{Trace}(CZ)})^{\widehat{}}=\frac12\sum_{i,j}C_{ij}(1+\delta_{ij})\frac{\partial}{\partial \zeta_{ij}}=\sum_{i\leq j}C_{ij}\frac{\partial}{\partial \zeta_{ij}},$$ whereas that of the second term of $d\pi_\mu(C )$ amounts to $$\begin{aligned}
&&\left({\sum_{i\leq j}\sum_{k,\ell}C_{k\ell}z_{ik}z_{j\ell}\frac\partial{\partial z_{ij}}}\right)^{\widehat{}}=-(n+1)\sum_{i\leq j}C_{ij}\frac{\partial}{\partial\zeta_{ij}}
-\frac14\sum_{i,j,k,l}C_{kl}(1+\delta_{ik})(1+\delta_{jl})\zeta_{ij}
\frac{\partial^2}{\partial\zeta_{ik}\partial\zeta_{j\ell}}\\
&=&-(n+1)\sum_{i\leq j}C_{ij}\frac{\partial}{\partial\zeta_{ij}}
-\frac12\left(\sum_{i\leq k,j\leq\ell}C_{k\ell}\zeta_{ij}\frac{\partial^2}{\partial\zeta_{ik}\partial\zeta_{j\ell}}
+\sum_{i\geq k,j\geq\ell}C_{k\ell}\zeta_{ij}\frac{\partial^2}{\partial\zeta_{ik}\partial\zeta_{j\ell}}\right).\end{aligned}$$ Hence the formula for $\widehat{d\pi_\mu}( C)$ follows.

The Step 2 of the F-method is given by the branching law in Fact \[fact:A\] and leads us to . For Step 3 we apply Lemma \[lem:412\] and we get:

\[lem:Hom\_sp\] Let $W_\lambda^a$ be the irreducible $\mathfrak l^\tau$-module defined in .

1.  The highest weight of $(W_\lambda^a)^\vee$ is given by $$\chi=(a,0,\ldots,0;a)+(\lambda,\ldots,\lambda;\lambda).$$

2.  For the $\mathfrak l$-module $\operatorname{Pol}(\mathfrak n_+)\otimes {\mathbb C}_\lambda^\vee$, the $\chi$-weight space for $\mathfrak b(\mathfrak l^\tau)$ is given by: $$\label{eqn:xi11}
    (\operatorname{Pol}(\mathfrak n_+)\otimes {\mathbb C}_\lambda^\vee)_\chi\simeq\bigoplus_{2j+k=a}{\mathbb C}\zeta_{11}^j
    \zeta_{1n}^k\zeta_{nn}^j,$$ where we identify $\operatorname{Pol}(\mathfrak n_+)\otimes {\mathbb C}_\lambda^\vee$ with $\operatorname{Pol}(\mathfrak n_+)$ as vector spaces.

The statement (1) is clear from the definition of $W_\lambda^a$ given in .

Notice then that in our convention $\Delta(\mathfrak n_-)=\{e_i+e_j;\,1\leq i\leq j\leq n\}$. Thus $\mathfrak n_-$ decomposes into irreducible representations of $\mathfrak l^\tau$ as $$\begin{aligned}
\mathfrak n_-&\simeq&\left(\mathrm{Sym}(n-1),{\mathbb C})\boxtimes{\mathbb C}\right)\oplus
({\mathbb C}\boxtimes{\mathbb C}_2)\oplus({\mathbb C}^{n-1}\boxtimes{\mathbb C}_1)\nonumber\\
&\simeq&\left( F(\mathfrak {gl}_{n-1},2e_1)\boxtimes F(\mathfrak {gl}_{1},0)\right)
\oplus
\left( F(\mathfrak {gl}_{n-1},0)\boxtimes F(\mathfrak {gl}_{1},2e_n)\right)
\label{eqn:25-1}\\
&&\oplus
\left( F(\mathfrak {gl}_{n-1},e_1)\boxtimes F(\mathfrak {gl}_{1},e_n)\right)\nonumber.\end{aligned}$$ Accordingly we get an isomorphism of $\mathfrak l^\tau$-modules: $$\begin{aligned}
\label{eqn:25-2}
\operatorname{Pol}(\mathfrak n_+)\simeq S(\mathfrak n_-)\simeq
 \bigoplus_{i,j,k} \left( S^i(\mathrm{Sym}(n-1),{\mathbb C}))\otimes S^k({\mathbb C}^{n-1})\right)\boxtimes
{\mathbb C}_{2j+k}.\end{aligned}$$

Since $\zeta_{11}, \zeta_{nn}$ and $\zeta_{1n}$ are highest weight vectors in the $\mathfrak l^\tau$-module $\mathfrak n_-$ with respect to $\Delta^+(\mathfrak l^\tau)$ (see (\[eqn:25-1\])), so is any monomial $\zeta_{11}^i\zeta_{nn}^j\zeta_{1n}^k$ in the $\mathfrak l^\tau$-module $S(\mathfrak n_-)\simeq\operatorname{Pol}(\mathfrak n_+)$ of weight $(2i+k)e_1+(k+2j)e_n$.

According to the irreducible decomposition (\[eqn:25-2\]) and Remark \[rem:pieri\], it follows that the right-hand side of (\[eqn:xi11\]) exhausts all highest weight vectors in $\operatorname{Pol}(\mathfrak n_+)$ of weight $a(e_1+e_n)$. Thus, taking into account the $\mathfrak l^\tau$-action on ${\mathbb C}_\lambda^\vee\simeq
\lambda\,\mathrm{Trace}_n$, we get Lemma.

As Step 4, we reduce the system of differential equations , i.e. $\widehat{d\pi}_\mu( C)\psi=0$, to an ordinary differential equation as follows:

\[prop:gegpro\] ${}$

1.  Any polynomial $\psi(\zeta)\equiv \psi(\zeta_{ij})$ in the right–hand side of (\[eqn:xi11\]) is given by $$\label{eqn:Dmodmorph1}
    \psi(\zeta)=\left(T_ag\right)(\zeta):=(\sqrt{2\zeta_{11}\zeta_{nn}})^ag\left(\frac{\zeta_{1n}}{\sqrt{2\zeta_{11}\zeta_{nn}}}\right),$$ where $g(t)$ is a polynomial in one variable $t$ of degree at most $a$.

2.  The polynomial $\psi(\zeta)$ of $\frac 1 2 n(n+1)$ variables satisfies the system of partial differential equations $\widehat{d\pi}_\mu( C)\psi=0$ for any $X\in
    {\mathfrak n_+^\tau}$ if and only if $g(t)$ satisfies the Gegenbauer differential equation $$\label{eqn:JacobiDE2}
    \left((1-t^2)\vartheta_t^2-(1+2(n-\mu) t^2)\vartheta_t +a(a+2(n-\mu))t^2\right)g(t)=0,$$ where we denote $\vartheta_t=t\frac{d}{dt}$ as before.

The first statement is clear from . The proof of the second assertion is similar to the one of Proposition \[prop:55\] and uses the following identities: $$T_a^\sharp{\vartheta_{\xi_{11}}}=
T_a^\sharp{\vartheta_{\xi_{nn}}}= \frac12(a- \vartheta_t),\quad
T_a^\sharp{\vartheta_{\xi_{1n}}}=  \vartheta_t,$$ where $\vartheta_{\zeta_{ij}}=\zeta_{ij}\frac{\partial}{\partial\xi_{ij}}$.

We are ready to complete the proof of Theorem \[thm:Sp\].

\[Proof of Theorem \[thm:Sp\]\] The existence and uniqueness of the continuous $G^\tau$-intertwining operator are guaranteed by Theorem \[thm:C\]. Let us prove that $D_{X\to Y,a}$ defined in belongs to $\mathrm{Diff}_{G^\tau}(\mathcal L_\lambda,\mathcal W_\lambda^a)$. Using the F-method we have proved that if $D\in \mathrm{Diff}_{G^\tau}(\mathcal L_\lambda,\mathcal W_\lambda^a)$ and $w^\vee$ is a highest weight vector in $(W_\lambda^a)^\vee$, then $\langle D,w^\vee\rangle$ is of the form $(\mathrm{Symb}^{-1}\otimes\mathrm{id})T_ag$, where $g(t)$ is a polynomial satisfying . Hence $g(t)$ is, up to a scalar multiple, the Gegenbauer polynomial $C_a^{\lambda-1}(t)$. In turn, $(T_ag)(\zeta)=C_a^{\lambda-1}
(2\zeta_{11}\zeta_{nn},\zeta_{1n})$ up to a scalar.

Thus, in order to show $D_{X\to Y,a}\in\mathrm{Diff}_{G^\tau}(\mathcal L_\lambda,\mathcal W_\lambda^a)$ it is sufficient to verify for all $\ell\in L_{\mathbb C}^\tau$: $$\label{eqn:666}
(\mathrm{Symb}\otimes\mathrm{id})\langle D_{X\to Y,a},\nu^\vee(\ell^{-1})w^\vee\rangle=(\mathrm{Ad}(\ell^{-1})\otimes\lambda^\vee(\ell^{-1}))(T_ag),$$ by Lemma \[lem:hwv\] and by the observation that every nonzero $w^\vee\in W^\vee$ is cyclic. The left-hand side of amounts to $$\begin{aligned}
&&\left\langle C_a^{\lambda-1}\left(\sum_{1\leq i,j\leq n-1}2v_iv_j\zeta_{ij}\zeta_{nn},
\sum_{1\leq j\leq n-1}v_j\zeta_{jn}\right),\nu^\vee(\ell^{-1})w^\vee\right\rangle\\
&=& (\det\ell)^{-\lambda}
\left\langle C_a^{\lambda-1}\left(\sum_{1\leq i,j\leq n-1}2(\ell v)_i(\ell v)_j\zeta_{ij}\zeta_{nn},
\sum_{1\leq j\leq n-1}(\ell v)_j\zeta_{jn}\right),w^\vee\right\rangle,\end{aligned}$$ where $v={{}^t\!}(v_1,\ldots,v_{n-1})$ stands for the column vector. Since $\langle Q(v),w^\vee\rangle$ gives the coefficients of $v_1^a$ in the polynomial $Q(v)$, it is equal to $$\begin{aligned}
&&(\det\ell)^{-\lambda}
C_a^{\lambda-1}\left(\sum_{1\leq i,j\leq n-1}2\ell_{i1}\ell_{j1}\zeta_{ij}\zeta_{nn},
\sum_{1\leq j\leq n-1}\ell_{j1}\zeta_{jn}\right)\\
&=& (\det\ell)^{-\lambda}
C_a^{\lambda-1}\left(\sum_{1\leq i,j\leq n-1}2({{}^t\!}\ell\zeta\ell)_{11}\zeta_{nn},
\sum_{1\leq j\leq n-1}({{}^t\!}\ell\zeta)_{1n}\right).\end{aligned}$$

On the other hand, the action of $\mathrm{Ad}(\ell^{-1})$ on $\operatorname{Pol}(\mathfrak n_+)$ is generated by $$\begin{aligned}
\zeta_{ij}\mapsto ({{}^t\!}\ell\zeta\ell)_{ij},\quad
\zeta_{in}\mapsto ({{}^t\!}\ell\zeta)_{in}.\end{aligned}$$ Hence, the right–hand side of amounts to $$(\det\ell)^{-\lambda}
C_a^{\lambda-1}\left(\sum_{1\leq i,j\leq n-1}2({{}^t\!}\ell\zeta\ell)_{11}\zeta_{nn},
\sum_{1\leq j\leq n-1}({{}^t\!}\ell\zeta)_{1n}\right),$$ which concludes the proof of Theorem \[thm:Sp\].

Case of $U(n,1)\times U(n,1)\downarrow U(n,1)$ {#sec:un1}
----------------------------------------------

Let $U(n,1)$ be the Lie group of all matrices preserving the standard Hermitian form given by $I_{n,1}$. Consider the Lie group $G:=U(n,1)\times U(n,1)$ and let $\tau$ be the involution of $G$ acting by $\tau:(g,h)\to (h,g)$. We consider the symmetric pair $(G,G^\tau)$, where $G^\tau\simeq \Delta(U(n,1))\subset G$.

Let $D$ be the unit ball $\{Z\in{\mathbb C}^n\,:\, \Vert Z\Vert<1\}$. It is a Hermitian symmetric domain of type AIII in ${\mathbb C}^n$. We adapt the notations of Example \[example:410\] with $p=n$ and $q=1$. Then the Lie group $U(n,1)$ acts biholomorphically on $D$ by $$g\cdot Z=(aZ+b)(cZ+d)^{-1}\quad{\mathrm{for}}\quad
g=\begin{pmatrix}a&b\\c&d\end{pmatrix}\in U(n,1),\,Z\in D,$$ and the isotropy subgroup $K$ at the origin is isomorphic to $U(n)\times U(1)$.

We set $X:=D\times D$ and $Y:=\Delta(Y)$. Thus, we have the following diagram: $$\begin{matrix}
X=D\times D & \subset & {\mathbb C}^n\times{\mathbb C}^n &\simeq&\mathfrak n_-&\subset& \mathbb P^n{\mathbb C}\times\mathbb P^n{\mathbb C}\\
\cup& &\cup& & \cup& &\cup\\
Y=\Delta(D)&\subset &\Delta({\mathbb C}^n)&\simeq&\mathfrak n_-^\tau&\subset&
\Delta(\mathbb P^n{\mathbb C}).
\end{matrix}$$

For $\lambda_1,\lambda_2\in{\mathbb C}$, the map $ \mathfrak{gl}_n({\mathbb C})+\mathfrak{gl}_1({\mathbb C})\to {\mathbb C}$, $(A,d)\mapsto  -\lambda_1\,\mathrm{Trace}\, A-\lambda_2d$ is a character, which we denote by ${\mathbb C}_{(\lambda_1,\lambda_2)}$ according to that we have chosen a realization of $\mathfrak n_+$ in the lower triangular matrices. For integral values of $\lambda_1$ and $\lambda_2$ the character ${\mathbb C}_{(\lambda_1,\lambda_2)}$ lifts to $U(n)\times U(1)$.

According to the branching law in Fact \[fact:B\], there exists a nontrivial $G^\tau$-intertwining operator $D_{X\to Y}(\varphi)$ from $\mathcal O(X,\mathcal L_{(\lambda_1',\lambda_2')}
\otimes \mathcal L_{(\lambda_1'',\lambda_2'')} )$ to $ \mathcal O(Y,\mathcal W_Y)$ only if $\dim W>1$ except for the case $\mathcal W_Y\simeq
(\mathcal L_{(\lambda_1',\lambda_2')}
\otimes \mathcal L_{(\lambda_1'',\lambda_2'')})\vert_Y$. In order to give a concrete model of such a representation space $W$, we define a module of $\mathfrak l^\tau\simeq\mathfrak {gl}_{n}({\mathbb C})+\mathfrak{gl}_1({\mathbb C})$ by $$\begin{aligned}
\label{eqn:6116}
W_{(\lambda_1,\lambda_2)}^a&:= &S^a(\mathfrak n_+^{-\tau})\otimes{\mathbb C}_{(\lambda_1,\lambda_2)}\\&\simeq&\nonumber
\left(S^a\left(({\mathbb C}^{n})^\vee\right)\otimes(-\lambda_1\,\mathrm{Trace}_{n})\right)\boxtimes F(\mathfrak{gl}_1,(-\lambda_2+a)e_{n+1}).\end{aligned}$$ As a vector space $W_\lambda^a$ is isomorphic to the space $\operatorname{Pol}^a(\mathfrak n_-^{-\tau})$ of homogeneous polynomials on $\mathfrak n_-^{-\tau}\simeq {\mathbb C}^{n}$. Thus, if $W$ is taken to be $W^a_{(\lambda_1,\lambda_2)}$, then the intertwining differential operator can be thought of as an element of $$\label{eqn:W12}
{\mathbb C}\left[\frac{\partial}{\partial z_{1}'},\ldots,
\frac{\partial}{\partial z_{n}'},\frac{\partial}{\partial z_{1}''},\ldots,
\frac{\partial}{\partial z_{n}''}
\right]\otimes
\operatorname{Pol}^a[v_1,\ldots,v_{n}],$$ where $(v_1,\ldots,v_{n})$ are the standard coordinates on $\mathfrak n_-^{-\tau}\simeq{\mathbb C}^{n}$ and $z_{i}', z_j''$ are those on $\mathfrak n_-\simeq{\mathbb C}^n+{\mathbb C}^n.$

Let $P_\ell^{\alpha,\beta}(t)$ be the Jacobi polynomial defined by $$\begin{aligned}
 P^{\alpha,\beta}_\ell(t)&=&
 \frac{\Gamma(\alpha+\ell+1)}{\Gamma(\alpha+\beta+\ell+1)}\sum_{m=0}^\ell
 \begin{pmatrix}
 \ell\\m
 \end{pmatrix}
\frac{\Gamma(\alpha+\beta+\ell+m+1)}{\ell!\Gamma(\alpha+m+1)}\left(\frac{t-1}2\right)^m,\end{aligned}$$ see Appendix for more details. We set $$P_\ell^{\alpha,\beta}(x,y):=y^\ell P_\ell^{\alpha,\beta}\left(2\frac xy+1\right).$$ For instance, $P_0^{\alpha,\beta}(x,y)=1$, $P_1^{\alpha,\beta}(x,y)=(2+\alpha+\beta)x+(\alpha+1)y$, etc.

\[thm:U(n,1)\] Suppose that $\lambda_1',\lambda_2', \lambda_1'',\lambda_2''\in{\mathbb Z}$ satisfy $\lambda':=\lambda_1'-\lambda_2'>0$ and $\lambda'':=\lambda_1''-\lambda_2''>0$, and that $a\in{\mathbb N}$. Then any $G^\tau$-intertwining operator from $ {\mathcal{O}}^2(Y, {\mathcal{L}}_{(\lambda_1', \lambda_2')})
\otimes
{\mathcal{O}}^2(Y, {\mathcal{L}}_{(\lambda_1'', \lambda_2'')})$ to ${\mathcal{O}}^2(Y, {\mathcal{W}}_{(\lambda_1'+\lambda_1'', \lambda_2'+\lambda_2'')}^a)$ is given, up to a scalar, by $$\label{eqn:DXYUN1}
 D_{X\to Y,a}:=
  P_a^{-\lambda'+{n},\, \lambda'+\lambda'' -{2n}-2a+1}\left(
\sum_{i=1}^{n}v_i \frac{\partial}{\partial z_i},
\sum_{j=1}^{n}v_j \frac{\partial}{\partial z_j}
\right).$$

${}$

1.  The fiber of the vector bundle ${\mathcal{W}}_{(\lambda_1, \lambda_2)}^{a}$ is isomorphic to the space $S^a({\mathbb{C}}^n)$ of symmetric tensors of degree $a$. It is a line bundle if and only if $a=0$ or $n=1$. In the case $n=1$, the formula is nothing but the classical Rankin–Cohen bidifferential operator (we note that $SU(1,1)$ is isomorphic to $SL(2,{\mathbb{R}})$).

2.  $D_{X\to Y,a}$ belongs to , and is thus regarded as a ${\mathcal{W}}_{(\lambda_1'+\lambda_1'', \lambda_2'+\lambda_2'')}^a$-valued differential operator (see Diagram \[diag:5.1\]).

3.  The same statement remains true for the unitary representations.

4.  For any $\lambda_1',\lambda_2', \lambda_1'',\lambda_2''\in{\mathbb C}$ and $a\in{\mathbb N}$, the operator $  D_{X\to Y,a}$ defined by the same formula, is a $\widetilde{U(n,1)}$-equivariant holomorphic differential operator between two homogeneous bundles $\left(\mathcal L_{(\lambda_1',\lambda_2')})\otimes 
    \mathcal L_{(\lambda_1'',\lambda_2'')})\right)\to X$ and $\mathcal W_{(\lambda_1,\lambda_2)}^a\to Y$.

In order to prove Theorem \[thm:U(n,1)\] we apply again the F-method. Its Step 1 is given by

Let $C:=C'+C''=(c_1',\ldots, c_n')+(c_1'',\ldots, c_n'')\in \mathfrak n_+\simeq{\mathbb C}^n\oplus{\mathbb C}^n$,

$$\begin{aligned}
d\pi_{\mu_1',\mu_2'}( C')\oplus d\pi_{\mu_1'',\mu_2''}( C'' )&=&
\sum_{i=1}^nc_i'z_i'(E_{z'}-(\mu_1'-\mu_2'))+\sum_{j=1}^nc_j''z_j''(E_{z''}-(\mu_1''-\mu_2''))\\
\widehat{d\pi}_{\mu_1',\mu_2'}(C' )\oplus \widehat{d\pi}_{\mu_1'',\mu_2''} (C'' )
&=& -\left((\mu_1'-\mu_2'+n+1) 
\sum_{i=1}^nc_i'\frac\partial{\partial \zeta_i'}
+\sum_{i,j=1}^nc_i'\zeta'_j\frac{\partial^2}{\partial\zeta_i'\partial\zeta'_j} \right)\\
   &&- \left((\mu_1''-\mu_2''+n+1) 
   \sum_{j=1}^nc_j''\frac\partial{\partial \zeta_j''}
   +\sum_{i,j=1}^nc_i''\zeta_j''\frac{\partial^2}{\partial\zeta_i''\partial\zeta_j''} \right).\end{aligned}$$

The Step 2 of the F-method is given by the branching law in Fact \[fact:A\] and leads us to . For Step 3 we apply Lemma \[lem:412\] and we get:

For $\lambda_1,\lambda_2\in{\mathbb C}$ and $a\in N$ we recall from that $W_{(\lambda_1,\lambda_2)}^a$ is an irreducible module of $\mathfrak l^\tau
\simeq\mathfrak{gl}_n({\mathbb C})+\mathfrak{gl}_1({\mathbb C})$.

1.  The highest weight of $\left(W_{(\lambda_1,\lambda_2)}^a\right)^\vee$ is given by $$\chi=(a,0,\cdots,0;-a)+(\lambda_1,\cdots,\lambda_1;\lambda_2).$$

2.  For the $\mathfrak l$-module $\operatorname{Pol}(\mathfrak n_+)
    \otimes({\mathbb C}_\lambda)^\vee$, the $\chi$-weight space with respect to $\mathfrak b(\mathfrak l^\tau)$ is given by $$\label{eqn:polUn1}
    \left(\operatorname{Pol}(\mathfrak n_+)\otimes{\mathbb C}_\lambda^\vee\right)_\chi\simeq\bigoplus_{i+j=a}{\mathbb C}(\zeta'_1)^i(\zeta_1'')^j,$$ where we identify $\operatorname{Pol}(\mathfrak n_+)\otimes{\mathbb C}_\lambda^\vee$ with $\operatorname{Pol}(\mathfrak n_+)$ as vector spaces.

According to the natural decomposition $
\mathfrak n_+\simeq {\mathbb C}^n\oplus{\mathbb C}^n$, one has $$\begin{aligned}
\operatorname{Pol}(\mathfrak n_+)&\simeq&
\bigoplus_{i,j}\operatorname{Pol}^i({\mathbb C}^n)\otimes \operatorname{Pol}^j({\mathbb C}^n)\\
&\simeq& \bigoplus_{i,j}\bigoplus_{\underline s}
F(\mathfrak{gl}_n({\mathbb C}),(s_1,\cdots,s_n))\otimes
F(\mathfrak{gl}_1({\mathbb C}),-(i+j)e_n),\end{aligned}$$ where the sum in the second factor is taken over all $\underline s=(s_1,\cdots, s_n)\in{\mathbb N}^n$ satisfying $s_1\geq\cdots\geq s_n\geq0$, $\max(i,j)\geq s_1\geq\min (i,j)$ and $s_1+\cdots+s_n=i+j$. In particular, the $\chi$-weight space for $\mathfrak b(\mathfrak l^\tau)$ occurs in $\operatorname{Pol}(\mathfrak n_+)\otimes{\mathbb C}_\lambda^\vee$ if and only if $i+j=a$ and $s_2=\cdots=s_n=0$. In this case the weight vectors are the monomials $ (\zeta_1')^i(\zeta_1'')^j$. Lemma follows.

As Step 4, we reduce the system of differential equations , i.e. $\widehat{d\pi}_\mu( C)\psi=0$, to an ordinary differential equation as follows:

\[prop:723\] ${}$

1.  Any polynomial $\psi(\zeta',\zeta'')$ of $2n$-variables in the right-hand side of is given by $$\psi(\zeta',\zeta'')=(T_ag)(\zeta',\zeta''):=(\zeta_1'')^ag\left(\frac{\zeta_1'}{\zeta_1''}\right),$$ where $g(t)$ is a polynomial in one variable $t$ of degree at most $a$.

2.  The polynomial $\psi(\zeta',\zeta'')$ satisfies $\widehat{d\pi}(X)\psi=0$ for any $X\in
    {\mathfrak n_+^\tau}$ if and only if the polynomial $g(t)$ solves the single equation $$\label{eqn:JacobiDE1}
    \left((1-s^2)\frac{d^2}{ds^2}+q(s)\frac d{ds}
    +{a(1-a_2-a)}\right)g\left(\frac{s-1}2\right)=0,$$ where $a_1=\mu_1'-\mu_2'+n+1$, $a_2
    =\mu_1''-\mu_2''+n+1$ and $q(s)=s(a_2-2+2a)-2a_1-a_2-2a+2$.

For the proof of Proposition \[prop:723\] we state following identities for $T_a$-saturated operators whose verification is similar to one for Lemma \[lem:T\].

\[lem:T3\] One has:

1.  $T_a^\sharp\left(\zeta_1''\frac{\partial}{\partial \zeta_1'}\right)
    =\frac d{dt}$.

2.  $T_a^\sharp\left(\zeta_1'\zeta''_1\frac{\partial^2}{\partial (\zeta_1')^2}\right)
    =t\frac {d^2}{dt^2}$.

3.  $T_a^\sharp\left(\zeta_1''\frac{\partial}{\partial \zeta_1''}\right)
    =a-\frac d{dt}$.

4.  $T_a^\sharp\left((\zeta_1'')^2\frac{\partial^2}{\partial (\zeta_1'')^2}\right)
    =a(a-1)-2(a-1)t\frac d{dt}+t^2\frac{d^2}{dt^2}$.

The general conditions (\[eqn:Fmethod1\]) and (\[eqn:Fmethod2\]) of the F-method reduce in the present case to one non-trivial differential equation that holds for $C_1=(1,0,\ldots,0)+(1,0,\ldots,0)\in\mathfrak n_+^\tau$: $$\label{eqn:DE1}
\left((\mu_1'-\mu_2'+n+1)\frac\partial{\partial\zeta_1'}+\zeta_1'\frac{\partial^2}{\partial(\zeta_1')^2}
+
(\mu_1''-\mu_2''+n+1)\frac\partial{\partial\zeta_1''}+\zeta_1''\frac{\partial^2}{\partial(\zeta_1'')^2}\right)\psi(\zeta',\zeta'')=0.$$ According to Lemma \[lem:T3\] the above requirement corresponds to the following condition on $g(t)$: $$\label{eqn:Tsat3}
(t+t^2)\frac{d^2}{dt^2} + (a_1-t(a_2+2a-2))\frac{d}{dt}+a(a_2+a-1) g(t)=0,$$ where $a_1=\mu_1'-\mu_2'+n+1$ and $a_2
=\mu_1''-\mu_2''+n+1$.

To complete the proof, notice that the change of variable $t\mapsto \frac12(s-1)$ transforms the above equation into (\[eqn:JacobiDE1\]).

Now we have all the necessary ingredients to apply Lemma \[lem:hwv\]. The remaining part of the proof is parallel to one for Theorem \[thm:Sp\].

\[rem:sing\] In all the three cases we have reduced a system of partial differential equations to a single ordinary differential equation in Step 4 of the F-method. The latter equation has regular singularities at $t = \pm 1$ and $\infty$. We describe the corresponding singularities via the map $T_a$ as below:

1.  The singularities of the differential equation (\[eqn:JacobiDE3\]) correspond to the varieties given by $\zeta_n=0$ and $Q_{n-1}(\zeta')=0$.

2.  The singularities of the differential equation (\[eqn:JacobiDE2\]) correspond to the varieties given by $\zeta_{1n}=0$ and $\det
    \left\vert\begin{matrix}
    \zeta_{11}&\zeta_{1n}\\\zeta_{1n}&\zeta_{nn}
    \end{matrix}\right\vert=0$.

3.  The singularities of the differential equation (\[eqn:JacobiDE1\]) correspond to the varieties given by $\zeta_1'=0$ and $\zeta_1'=\pm \zeta_1''$.

Jacobi polynomials and Gegenbauer polynomials
=============================================

Jacobi polynomials
------------------

Let $\alpha>-1$ and $\beta>-1$. Jacobi polynomials $ P^{\alpha,\beta}_\ell(t)$ of one variable $t$ and of degree $\ell$ form an orthogonal basis in $L^2([-1,1],
(1-t)^\alpha(1+t)^\beta dt)$ and satisfy the Jacobi differential equation $$\left((1-t^2)\frac{d^2}{dt^2}+(\beta-\alpha-(\alpha+\beta+2)t)\frac d{dt}+\ell(\ell+\alpha+\beta+1)\right)y=0.$$

This equation is a particular case of the Gauss hypergeometric equation. Therefore its unique polynomial solution $y(t)$ satisfying $y(1)=\frac{\Gamma(\alpha+\ell+1)}{\Gamma(\alpha+1)\ell!}$ is given by: $$\begin{aligned}
 P^{\alpha,\beta}_\ell(t)&=&
 \frac{\Gamma(\alpha+\ell+1)}{\Gamma(\alpha+1)\ell!}
 {}_2F_1\left(-\ell,1+\alpha+\beta+\ell;\alpha+1;\frac{1-t}2\right)\\
 &=&\frac{\Gamma(\alpha+\ell+1)}{\Gamma(\alpha+\beta+\ell+1)}\sum_{m=0}^\ell
 \begin{pmatrix}
 \ell\\m
 \end{pmatrix}
\frac{\Gamma(\alpha+\beta+\ell+m+1)}{\Gamma(\alpha+m+1)\ell!}\left(\frac{t-1}2\right)^m.\end{aligned}$$

The Jacobi polynomials are subject to the Rodrigues formula $$(1-t)^\alpha(1+t)^\beta P_\ell^{\alpha,\beta}(t)=\frac{(-1)^\ell}{2^\ell\ell!}\left(\frac d{dt}\right)^\ell\left((1-t)^{\ell+\alpha}(1+t)^{\ell+\beta} \right).$$ When $\alpha=\beta$ these polynomials yield Gegenbauer polynomials (see the next paragraph for more details) and they reduce to Legendre polynomials in the case when $\alpha=\beta=0$.

Here are the first four Jacobi polynomials.

-   $P_0^{\alpha,\beta}(t)= 1.$

-   $P_1^{\alpha,\beta}(t)=\frac{1}{2} (\alpha-\beta+(2+\alpha+\beta) t)$.

-   $P_2^{\alpha,\beta}(t)=
    \frac{1}{2} (1+\alpha) (2+\alpha)+\frac{1}{2} (2+\alpha) (3+\alpha+\beta) (t-1)+\frac{1}{8} (3+\alpha+\beta) (4+\alpha+\beta) (t-1)^2$.

Gegenbauer Polynomials {#sec:A2}
----------------------

The Gegenbauer (or ultraspherical) polynomial $C_\ell^\alpha(t)$ is a polynomial of $t$ of degree $\ell$ satisfying the Gegenbauer differential equation $$\label{eqn:ODEC}
\left((1-t^2)\vartheta_t^2-(1+2\alpha t^2)\vartheta_t +\ell(\ell+2\alpha)t^2\right)y=0.$$

It is a specialization of the Jacobi polynomial $$\label{eqn:Gegen-def}
C_\ell^\alpha(t)=\frac{\Gamma(\alpha+\frac12)\Gamma(\ell+2\alpha)}
{\Gamma(2\alpha)\Gamma(\ell+\alpha+\frac12)}P_\ell^{\alpha-\frac12,\alpha-\frac12}(t).$$

This equation (\[eqn:ODEC\]) is a particular case of the Gauss hypergeometric differential equation. Thus $$\begin{aligned}
\label{eqn:Gegen-def2}
C_\ell^\alpha(t)&=&\frac{\Gamma(\ell+2\alpha)}{\Gamma(2\alpha)\Gamma(\ell+1)}
{}_2F_1\left(-\ell,\ell+2\alpha;\alpha+\frac12;\frac{1-t}2\right)\\
&=&\sum_{k=0}^{\left[\frac \ell2\right]}(-1)^k\frac{\Gamma(\ell-k+\alpha)}
{\Gamma(\alpha)\Gamma(k+1)\Gamma(\ell-2k+1)}(2t)^{\ell-2k}.\end{aligned}$$

Here are the first five Gegenbauer polynomials.

-   $C_0^\alpha(t)=1.$

-   $C_1^\alpha(t)=2\alpha t.$

-   $C_2^\alpha(t)=-\alpha(1-2(\alpha+1)t^2)$.

-   $C_3^\alpha(t)=-2\alpha(\alpha+1)(t-\frac23(\alpha+2)t^3)$.

-   $C_4^\alpha(t)=
    \frac12 \alpha (\alpha+1)(1-4(\alpha+2)t^2 + 
     \frac43 (\alpha+2) (\alpha+3) t^4)$.

Acknowledgements {#acknowledgements .unnumbered}
----------------

T. Kobayashi was partially supported by Institut des Hautes Études Scientifiques, France and Grant-in-Aid for Scientific Research (B) (22340026), Japan Society for the Promotion of Science. Both authors were partially supported by Max Planck Institute for Mathematics (Bonn) where a large part of this work was done.

[99]{} K. Ban, On Rankin–Cohen–Ibukiyama operators for automorphic forms of several variables. *Comment. Math. Univ. St. Pauli*, **55** (2006), pp. 149–171. I. N. Bernstein, I. M. Gelfand, S. I. Gelfand, A certain category of ${\mathfrak g}$-modules. *Funkcional. Anal. i Prilozhen.* **10** (1976), pp. 1–8.

P. Bieliavsky, X. Tang, Y. Yao, Rankin–Cohen brackets and formal quantization. *Adv. Math.* **212** (2007), [pp. 293–314](http://www.sciencedirect.com/science/article/pii/S0001870806003410).

Y. Choie, M. H. Lee, Notes on Rankin–Cohen brackets. *Ramanujan J.* **25** (2011), [pp.141–147](http://dx.doi.org/10.1007/s11139-010-9229-2).

H. Cohen, [Sums involving the values at negative integers of $L$-functions of quadratic characters]{}, *Math.Ann*. **217** (1975), [pp. 271–285](http://dx.doi.org/10.1007/BF01436180).

P. B. Cohen,Y. Manin, D. Zagier, Automorphic pseudodifferential operators. In *Algebraic aspects of integrable systems*, pp. 17–47, Progr. Nonlinear Differential Equations Appl., **26**, Birkhäuser Boston, 1997.

A. Connes, H. Moscovici, Rankin–Cohen brackets and the Hopf algebra of transverse geometry, *Mosc. Math. J.* **4**, (2004), pp. 111–130 .

G. van Dijk, M. Pevzner, Ring structures for holomorphic discrete series and Rankin–Cohen brackets. *J. Lie Theory*, **17**, (2007), [pp. 283–305](http://www.heldermann.de/JLT/JLT17/JLT172/jlt17015.htm).

M. Eichler, D. Zagier, *The theory of Jacobi forms.* Progress in Mathematics, [**55**]{}. Birkhäuser, Boston, 1985.

W. Eholzer, T. Ibukiyama, Rankin–Cohen type differential operators for Siegel modular forms. *Int. J. Math.* **9**, no. 4 (1998), [pp. 443–463](http://www.worldscientific.com/doi/abs/10.1142/S0129167X98000191).

P. Gordan, *Invariantentheorie*, Teubner, Leipzig, 1887. S. Gundelfinger, Zur der binären Formen, *J. Reine Angew. Math.* **100** (1886), pp. 413–424.

A. El Gradechi, The Lie theory of the Rankin–Cohen brackets and allied bi-differential operators. *Adv. Math.* **207** (2006), [pp. 484–531](http://www.sciencedirect.com/science/article/pii/S0001870805004135).

M. Harris, H. P. Jakobsen, Singular holomorphic representations and singular modular forms. Math. Ann. [**[259]{}**]{} (1982), [pp.227–244](http://dx.doi.org/
10.1007/BF01457310).

R. Howe, E. Tan, *Non-Abelian Harmonic Analysis. Applications of* ${SL}(2,{\mathbb R})$, Universitext, Springer-Verlag, New York, 1992.

T. Ibukiyama, T. Kuzumaki, H. Ochiai, Holonomic systems of Gegenbauer type polynomials of matrix arguments related with Siegel modular forms. *J. Math. Soc. Japan* [**64**]{} (2012), [ pp. 273–316](http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.jmsj/1327586984).

H. P. Jakobsen, M. Vergne, Restrictions and extensions of holomorphic representations, *J. Funct. Anal.* [**[34]{}**]{} (1979), [pp. 29–53](http://www.sciencedirect.com/science/article/pii/0022123679900235).

A. Juhl, *Families of conformally covariant differential operators, $Q$-curvature and holography.* Progress in Mathematics, [**[275]{}**](http://link.springer.com/book/10.1007/978-3-7643-9900-9/page/1). Birkhäuser, Basel, 2009.

A. W. Knapp, E. M. Stein, Intertwining operators for semisimple groups. *Ann. of Math*. **93** (1971), [pp. 489–578](http://www.jstor.org/stable/1970887?origin=crossref&).

T. Kobayashi, Discrete decomposability of the restriction of $A_{\mathfrak q}(\lambda)$ with respect to reductive subgroups and its applications, Part I. *Invent. Math.* **117** (1994), [181–205](http://dx.doi.org/10.1007/BF01232239); Part II, *Ann. of Math.* (2) **147** (1998), [709–729](http://dx.doi.org/10.2307/120963); Part III, *Invent. Math.* **131** (1998), [229–256](http://dx.doi.org/10.1007/s002220050203).

T. Kobayashi, Multiplicity-free theorems of the restrictions of unitary highest weight modules with respect to reductive symmetric pairs. Representation theory and automorphic forms, 45–109, Progr. Math., [**255**](http://dx.doi.org/10.1007/978-0-8176-4646-2_3), Birkhäuser, Boston, 2008.

T. Kobayashi, Restrictions of generalized Verma modules to symmetric pairs. *Transformation Groups* [**[17]{}**]{}, (2012), [pp. 523–546](http://dx.doi.org/10.1007/s00031-012-9180-y).

T. Kobayashi, Propagation of multiplicity-freeness property for holomorphic vector bundles, Progr. Math., [**[306]{}**]{}, Birkhäuser, Boston, (in press). [arXiv:0607004](http://uk.arxiv.org/abs/math.DG/0607004).

T. Kobayashi, B. [Ø]{}rsted, P. Somberg, V. Souček, Branching laws for Verma modules and applications in parabolic geometry. Part I. Preprint.

T. Kobayashi, B. Speh, Some intertwining operators for rank one orthogonal groups, in preparation.

B. Kostant, Verma modules and the existence of quasi-invariant differential operators, Lecture Notes in Math. **466**, Springer-Verlag, (1974), 101–129.

N. V. Kuznetsov, A new class of identities for the Fourier coefficients of modular forms. *Acta Arith.* [**27**]{} (1975), pp. 505–519.

S. Martens, The characters of the holomorphic discrete series, *Proc. Nat. Acad. Sci. USA*, [**72**]{} (1975), pp.3275–3276.

F. Martin, E. Royer, Rankin–Cohen brackets on quasimodular forms. *J. Ramanujan Math. Soc.* **24** (2009), pp. 213–233.

V.F. Molchanov, Tensor products of unitary representations of the three-dimensional Lorentz group. *Math. USSR, Izv.* **15** (1980), pp.113–143.

P. J. Olver, J. A. Sanders, Transvectants, modular forms, and the Heisenberg algebra. *Adv. in Appl. Math.* **25** (2000), [pp. 252–283](http://dx.doi.org/10.1006/aama.2000.0700).

L. Peng, G. Zhang, Tensor products of holomorphic representations and bilinear differential operators. *J. Funct. Anal.* [**[210]{}**]{} (2004), [pp. 171–192](http://www.sciencedirect.com/science/article/pii/S0022123603003422).

M. Pevzner, Rankin–Cohen brackets and associativity. *Lett. Math. Physics*, **85**, (2008), pp. [195–202](http://dx.doi.org/10.1007/s11005-008-0266-3).

M. Pevzner, Rankin–Cohen brackets and representations of conformal groups. *Ann. Math. Blaise Pascal* **19** (2012), [pp. 455–484](http://ambp.cedram.org/ambp-bin/item?id=AMBP_2012__19_2_455_0).

R. A. Rankin, The construction of automorphic forms from the derivatives of a given form, *J. Indian Math. Soc.* **20** (1956), pp. 103–116.

J. Repka, Tensor products of holomorphic discrete series representations. *Can. J. Math.* **31**,(1979), [pp. 836–844](http://cms.math.ca/10.4153/CJM-1979-079-9).

L. Schwartz, *Théorie des distributions*. Hermann, Paris, 1966.

A. Unterberger, J. Unterberger, *Algebras of symbols and modular forms*, [ J. Anal. Math.]{} **68** (1996), [pp. 121–143](http://dx.doi.org/10.1007/BF02790207).

D. Zagier, Modular forms and differential operators, *Proc. Indian Acad. Sci. (Math. Sci.)* **104** (1994), [pp. 57–75](http://dx.doi.org/10.1007/BF02830874).

G. Zhang, Rankin–Cohen brackets, transvectants and covariant differential operators. *Math. Z.* [**264**]{} (2010), [pp. 513–519](http://dx.doi.org/10.1007/s00209-009-0477-z).
