ON MICROFUNCTIONS AT THE BOUNDARY ALONG CR MANIFOLDS Andrea D’Agnolo Giuseppe Zampieri

ALONG CR MANIFOLDS

Andrea D’Agnolo Giuseppe Zampieri

Abstract. Let X be a complex analytic manifold, M ⊂ X a C

submanifold, Ω ⊂ M an open set with C

boundary S = ∂Ω. Denote by µ

(O

) (resp. µ

(O

)) the microlocalization along M (resp. Ω) of the sheaf O

of holomorphic functions.

In the literature (cf [A-G], [B-C-T], [K-S 1,2]) one encounters two classical results concerning the vanishing of the cohomology groups H

µ

(O

)

for p ∈ T ˙

X.

The first one gives the vanishing outside a range of indices j whose length is equal to s

(M, p) (where s

(M, p) denotes the number of positive, negative and null eigenvalues for the “microlocal” Levi form L

(p)). The second, sharpest result gives the concentration in a single degree, provided that the difference s

(M, p

)−γ(M, p

) is locally constant for p

∈ T

X near p (where γ(M, p) = dim

(T

X ∩ iT

X)

for z the base point of p).

The first result was restated for the complex µ

(O

) in [D’A-Z 2], in the case codim

S = 1. We extend it here to any codimension and moreover we also restate for µ

(O

) the second vanishing theorem.

We also point out that the principle of our proof, related to a criterion for con- stancy of sheaves due to [K-S 1], is a quite new one.

§1. Notations

Let X be a complex analytic manifold and M ⊂ X a C ² submanifold. One denotes by π : T ^∗ X → X and π : T _M ^∗ X → M the cotangent bundle to X and the conormal bundle to M in X respectively. Let ˙ T ^∗ X be the cotangent bundle with the zero section removed, and let ρ : M × _X T ^∗ X → T ^∗ M be the projection associated to the embedding M ,→ X.

For a subset A ⊂ X one defines the strict normal cone of A in X by N ^X (A) :=

T X \ C(X \ A, A) where C(·, ·) denotes the normal Whitney cone (cf [K-S 1]).

Let z _◦ ∈ M , p ∈ ( ˙ T _M ^∗ X) _z

. We put T _z ^C

M = T _z

M ∩ iT _z

M λ M (p) = T p T _M ^∗ X

λ 0 (p) = T p (π ⁻¹ π(p)),

ν(p) = the complex Euler radial field at p,

and we set γ(M, p) = dim ^C (T _M ^∗ X ∩ iT _M ^∗ X) _z

. If no confusion may arises, we will sometimes drop the indices z _◦ or p in the above notations.

Let φ be a C ² function in X with φ| _M ≡ 0 and p = (z _◦ ; dφ(z _◦ )). In a local system of coordinate (z) at z _◦ in X we define L φ (z _◦ ) as the Hermitian form with

Appeared in: Compositio Math. 105 (1997), no. 2, 141–146.

1

matrix (∂z i ∂z j φ) ij . Its restriction L M (p) to T _z ^C

M does not depend on the choice of φ and is called the Levi form of M at p. Let s ^+,−,0 (M, p) denote the number of positive, negative and null eigenvalues of L M (p).

One denotes by D ^b (X) the derived category of the category of bounded complexes of sheaves of C-vector spaces and by D ^b (X; p) the localization of D ^b (X) at p ∈ T ^∗ X, i.e. the localization of D ^b (X) with respect to the null system {F ∈ D ^b (X); p / ∈ SS(F )} (here SS(F ) denotes the micro-support in the sense of [K-S 2], a closed conic involutive subset of T ^∗ X).

Remark 1.1. We recall that a complex F which verifies SS(F ) ⊂ T _M ^∗ X in a neighborhood of p ∈ T _M ^∗ X is microlocally isomorphic (i.e. isomorphic in D ^b (X; p)) to a constant sheaf on M . This criterion, stated in [K-S 1] for a C ² manifold M , extends easily to C ¹ manifolds (cf [D’A-Z 1]).

Let O _X be the sheaf of germs of holomorphic functions on X and C _A (A ⊂ X locally closed) the sheaf which is zero in X \ A and the constant sheaf with fiber C in A. We will consider the complex µ _A (O _X ) := µhom(C _A , O _X ) of microfunctions along A (where µhom(·, ·) is the bifunctor of [K-S 1]). Of particular interest are the complexes µ _M (O _X ) and µ _Ω (O _X ) for Ω being an open subset of the manifold M (cf [S]).

§2. Statement of the results

Let X be a complex analytic manifold of dimension n, M ⊂ X a C ² submanifold of codimension l, Ω ⊂ M an open set with C ² boundary S = ∂Ω, and set r = codim M S (we assume Ω locally on one side of S for r = 1). Define:

d M (p) = codim X M + s ⁻ (M, p) − γ(M, p), c M (p) = n − s ⁺ (M, p) + γ(M, p).

Let us recall the following classical results concerning the cohomology of µ M (O X ):

Theorem A. ( [A-G], [B-C-T], [K-S 1]) Assume dim ^R (ν(p) ∩ λ _M (p)) = 1. Then H ^j µ M (O X ) p = 0 for j / ∈ [d M (p), c M (p)].

Theorem B. ( [H], [K-S 1]) Assume dim ^R (ν(p) ∩ λ M (p)) = 1 and s ⁻ (M, p ⁰ ) − γ(M, p ⁰ ) is constant for p ⁰ ∈ T _M ^∗ X close to p. Then

H ^j µ _M (O _X ) _p = 0 for j 6= d _M (p).

Dealing with µ _Ω (O _X ), one knows that:

Theorem C. ( [D’A-Z 2]) Assume codim M S = 1 and dim ^R (ν(p) ∩ λ M (p)) = 1.

Then

H ^j µ Ω (O X ) p = 0 for j / ∈ [d M (p), c M (p)].

The aim of the present note is, on the one hand, to extend Theorem C to the case of any codimension for S in M , and, on the other hand, to state the analogue of Theorem B for the complex µ Ω (O X ). We point out that the method of our proof, based on the criterion of [K-S 1, Proposition 6.2.2] (with its C ¹ -variant of [D’A-Z 1]), is a quite new one.

Our results, valid for any r = codim _M S, go as follows.

Theorem 2.1. Assume

(2.1) dim ^R (ν(p) ∩ λ S (p)) = 1.

Then

(2.2) H ^j µ _Ω (O _X ) _p = 0 for j / ∈ [d _M (p), c _M (p) + r − 1].

Theorem 2.2. Assume (2.1) and moreover

(2.3)



 

 

s ⁻ (M, p ⁰ ) − γ(M, p ⁰ ) is constant for p ⁰ ∈ Ω × _M T _M ^∗ X near p,

s ⁻ (S, p ⁰ ) − γ(S, p ⁰ ) is constant for p ⁰ ∈ T _S ^∗ X ∩ ρ ⁻¹ (N ^M (Ω) ^◦a ) near p, s ⁻ (M, p) − γ(M, p) = s ⁻ (S, p) − γ(S, p).

Then

(2.4) H ^j µ _Ω (O _X ) _p = 0 for j / ∈ [d _M (p), d _M (p) + r − 1].

Remark 2.3. We notice that the sets appearing in (2.3) are very natural in this context, one has in fact:

 T _M ^∗ X ∩ SS(C _Ω ) = Ω × _M T _M ^∗ X,

T _S ^∗ X ∩ SS(C Ω ) = T _S ^∗ X ∩ ρ ⁻¹ (N ^M (Ω) ^◦a ).

§3. Proofs of the results

Proof of Theorem 2.1. We set Ω ⁻ = M \ Ω and use the distinguished triangle (3.1) µ S (O X ) → µ M (O X ) → µ Ω (O X ) ⊕ µ _Ω

(O X ) ⁺¹ → .

Recall that (cf [D’A-Z 4]):

c _M (p) ≤ c _S (p) ≤ c _M (p) + r, d _M (p) ≤ d _S (p) ≤ d _M (p) + r.

(3.2)

The vanishing of (2.2) for j > c _M (p) + r − 1 then follows by applying Theorem A to M and S.

The vanishing of (2.2) for j < d _M (p) is immediate for d _S (p) > d _M (p) due to Theorem A and (3.1).

When d _S (p) = d _M (p) it remains to be proven that

(3.3) H ^d

^(p) µ _S (O _X ) _p → H ^d

^(p) µ _M (O _X ) _p is injective.

To this end we perform a contact transformation χ near p which interchanges (setting q = χ(p))

(3.4)

( T _M ^∗ X → T ^∗

M f X codim f M = 1, s ⁻ ( f M , q) = 0 T _S ^∗ X → T ^∗

S e X codim e S = 1

(cf [D’A-Z 3]). Let f M ⁺ and e S ⁺ be the closed half spaces with boundary f M and e S

and inner conormal q. We have

Proposition 3.1. Let d S = d M . Then in the above situation (3.5)

( s ⁻ ( e S, q) = 0, S e ⁺ ⊂ f M ⁺ .

Proof. Quantizing χ by a kernel K ∈ Ob(D ^b (X × X)) we get by [K-S 1, Proposition 11.2.8]:

( φ _K (C _M ) ∼ = C

M f

[d _M (p) − 1] in D ^b (X; q), φ _K (C _S ) ∼ = C

S e

[d _S (p) − s ⁻ ( e S, q) − 1] in D ^b (X; q).

Moreover the natural morphism C M → C S is transformed via φ K to a non null morphism C

M f

[d _M (p) − 1] → C

S e

[d _S (p) − s ⁻ ( e S, q) − 1]. Thus Hom _D

(X;q) (C

M f

[d _M (p) − 1], C

S e

[d _S (p) − s ⁻ ( e S, q) − 1]) =

= H ⁰ (RΓ

M f

(C

S e

) _y [d _S (p) − d _M (p) − s ⁻ ( e S, q)]) 6= 0,

where y = π(q). Since we are assuming d M (p) = d S (p), (3.5) follows. q.e.d.

End of the proof of Theorem 2.1. From the proof of Proposition 3.1 it follows that φ K transforms the morphism (3.3) in

(3.6) H ¹

S e

(O X ) y → H ¹

M f

(O X ) y ,

where y = π(q), which is clearly injective. The proof of Theorem 2.1 is now complete q.e.d.

Proof of Theorem 2.2. From now on we will drop p in our notations, due to the constancy assumptions (2.3).

If r > 1 one has Ω = M and N ^M (Ω) ^◦a = T ^∗ M . Thus, by (2.3), we enter the hypotheses of Theorem B for both M and S. The claim follows in this case from (3.1), (3.3) and from the inequalities (3.2).

We may then assume r = 1. The problem in this case is that (2.3) holds only along SS(C Ω ).

Let χ : T ^∗ X → T ^∗ X be a contact transformation from a neighborhood of p to a neighborhood of q = χ(p), such that:

( T _M ^∗ X → T ^∗

M f X codim f M = 1 T _S ^∗ X → T ^∗

S e X codim e S = 1, s ⁻ ( e S, q ⁰ ) ≡ 0.

Notice that, for y = π(q), T y M = T f y S. Quantizing χ by a kernel K, we thus have e that either φ _K (C _Ω ) or φ _K (C _Ω ) is a simple sheaf along the conormal bundle to a C ¹ submanifold Y ⊂ X. Since d _M = d _S − 1, then s ⁻ ( f M , q) = 0, f M ⁺ ⊂ e S ⁺ and φ K (C Ω ) = C Y [d M − 1]. Denoting by W the open domain with boundary Y and exterior conormal q, we have by [Z] that W is pseudoconvex at y, and one concludes since

χ _∗ µ _Ω (O _X ) _q [−d _M ] ∼ = H ¹ _X\W (O _X ) _y ,

q.e.d.

References

[A-G] A. Andreotti, H. Grauert, Th´ eor` emes de finitude pour la cohomologie des espaces com- plexes, Bull. Soc. Math. France 90 (1962), 193–259.

[B-C-T] M. S. Baouendi, C. H. Chang, F. Treves, Microlocal hypo-analyticity and extension of C-R-functions, J. Diff. Geometry 18 (1983), 331-391.

[D’A-Z 1] A. D’Agnolo, G. Zampieri, A propagation theorem for a class of sheaves of microfunc- tions, Rend. Mat. Acc. Lincei, s. 9 1 (1990), 53–58.

[D’A-Z 2] A. D’Agnolo, G. Zampieri, Vanishing theorem for sheaves of microfunctions at the boundary on CR-manifolds, Commun. in P.D.E. 17 (1992), no. 5& 6, 989–999.

[D’A-Z 3] A. D’Agnolo, G. Zampieri, Generalized Levi forms for microdifferential systems (M. Ka- shiwara, T. Monteiro Fernandes, P. Schapira, eds.), D-modules and microlocal geome- try, Walter de Gruyter & Co., 1992.

[D’A-Z 4] A. D’Agnolo, G. Zampieri, Levi’s forms of higher codimensional submanifolds, Rend.

Mat. Acc. Lincei, s. 9, 2 (1991), 29–33.

[H] L. H¨ ormander, An introduction to complex analysis in several complex variables, Van Norstrand. Princeton (1966).

[K-S 1] M. Kashiwara, P. Schapira, Microlocal study of sheaves, Ast´ erisque 128 (1985).

[K-S 2] M. Kashiwara, P. Schapira, Sheaves on manifolds, Springer-Verlag 292 (1990).

[S] P. Schapira, Front d’onde analytique au bord II, Sem. E.D.P. Ecole Polyt., Exp. XIII (1986).

[S-K-K] M. Sato, T. Kawai, M. Kashiwara, Hyperfunctions and pseudo-differential equations, Lecture Notes in Math., Springer-Verlag 287 (1973), 265–529.

[Z] G. Zampieri, The Andreotti-Grauert vanishing theorem for polyhedrons of C

, preprint.