Coherent feedback in optomechanical systems in the sideband-unresolved regime

Preparing macroscopic mechanical resonators close to their motional quantum groundstate and generating entanglement with light offers great opportunities in studying fundamental physics and in developing a new generation of quantum applications. Here we propose an experimentally interesting scheme, which is particularly well suited for systems in the sideband-unresolved regime, based on coherent feedback with linear, passive optical components to achieve groundstate cooling and photon-phonon entanglement generation with optomechanical devices. We find that, by introducing an additional passive element - either a narrow linewidth cavity or a mirror with a delay line - an optomechanical system in the deeply sideband-unresolved regime will exhibit dynamics similar to one that is sideband-resolved. With this new approach, the experimental realization of groundstate cooling and optomechanical entanglement is well within reach of current integrated state-of-the-art high-Q mechanical resonators.


Introduction
Over the past decade, optomechanical systems have seen great progress towards studying fundamental physics and in realizing new applications [1][2][3][4][5][6][7][8]. In particular, microfabricated optomechanical systems with large mechanical resonators and an integrated optical cavity have attracted significant interest, as they provide a versatile and easy-to-use platform in many areas including sensing [2,9], quantum networks [10][11][12], and for studying quantum effects in massive, macroscopic systems [13]. Demonstrating quantum effects in optomechanics has almost exclusively been the realm of systems in the sideband-resolved regime [3,6,7,[14][15][16][17][18], with several notable exceptions [19][20][21]. This is in great part due to the strong suppression of either the phonon creation or the annihilation process through the optical cavity [14,17] when the cavity linewidth is small compared to the mechanical frequency.
For fundamental tests using optomechanics, large and massive mechanical resonators are required however, which typically puts these systems far into the sideband-unresolved regime, where the mechanical resonance frequency is smaller than the linewidth of the optical cavity. While significant advancements have been made for device designs in the sideband-unresolved regime, both in terms of the mechanical resonator [22][23][24][25] and the integration with optical cavities with large optomechanical coupling strength [24,26,27], the difference in suppression of the optomechanical sidebands is insignificant due to the large optical linewidth. This makes them incompatible with many of the standard approaches for quantum experiments used to date. At the same time, the large bandwidth of the optical cavity allows obtaining the information of the mechanical resonator and interacting with it very efficiently and with little delay. This has lead to several ideas specifically designed for this regime, including short-pulse [28][29][30][31] and measurement-based [32] approaches for quantum state preparation. However, several challenges to experimentally implement these schemes exist, such as using an optical pulse with a duration much shorter than the mechanical oscillation period typically being limited by noise introduced by unwanted mechanical modes [33]. While measurement-based scheme are mostly focused on feedback cooling, creating photon-phonon entanglement through continuous measurement has also been proposed, with potential squeezing of the Einstein-Podolski-Rosen (EPR) quadratures of up to 50% [34].
In this work, by further exploring the large bandwidth of the optical cavity, and by using either a continuous laser or pulses that are much longer than the mechanical period, we propose schemes based on coherent feedback by external linear, passive optical elements. Effectively, the extra optical element creates an asymmetry in the suppression, similar to the sideband-resolved regime. We show that using this new approach, groundstate cooling and quantum entanglement, with squeezing of the EPR quadratures beyond 50%, is possible with realistic experimental requirements, even with systems deep in the sideband-unresolved regime.

Optomechanical system
We consider an optomechanical system consisting of a single mechanical mode and a single-mode optical cavity. The mechanical mode has an angular resonance frequency Ω m and an energy damping rate Γ m . Its field is bosonic, and the position and momentum quadratures are described by two normalized Hermitian operatorsX m andŶ m . The optical field has a resonance frequency of ω c and an energy damping rate κ c , with the amplitude and phase quadraturesX c andŶ c . Throughout this work, we combine the quadraturesû = (X,Ŷ ) T to simplify our expressions. The quadratures satisfy the commutation relation [û α,i ,û β,j ] = iδ αβ ε ij , where α, β are for c (cavity field) or m (mechanical field), i, j ∈ {1, 2} for the X or Y quadrature. ε ij = 1 for i = 1, j = 2, ε ij = −1 for i = 2, j = 1, and 0 otherwise. The annihilation operators of the two bosonic fields areâ α = (X α + iY α )/ √ 2. We use the the frame rotating with the laser (drive) frequency ω l , and we define the detuning ∆ c = ω l − ω c as being the frequency difference between the input laser and the cavity field. The mechanical resonator and the optical cavity couple dispersively, with a (linearized) coupling strength g = √ n c g 0 [17,35], where g 0 is the single photon coupling rate enhanced by the intra-cavity photon number n c . The mechanical resonator and the optical cavity couple to the environment through their respective loss channels, the energy dissipation rate Γ m and κ c . For the mechanical mode, in a typical experiment in the sideband-unresolved regime at temperature T , the thermal phonon excitation is given by n th ≈ k B T Ωm 1, with k B being the Boltzmann constant. We further assume the mechanical quality factor Q m to be large. The bath only couples to the momentum quadrature of the harmonic oscillator and is approximately Markovian,û in [34]. Due to the high frequency of the cavity field any thermal excitations can be neglected and the cavity input field is in the vacuum state. We consider two loss channels for the cavity, where one is due to the coupling to an external mode with an energy dissipation rate of κ (e) c and all other losses are included in κ The associated optical field areû in,e c andû in,i c , respectively. The linearized dynamics of the system are then described by the Quantum Langevin equation [34,35] Only the field coupled back to the external mode can be collected,û out c u c .

Coherent feedback with linear optical elements
With additional resources included, coherent feedback can have a significant impact on the dynamics of a system [36][37][38][39][40][41][42][43][44][45][46][47][48][49][50]. In our approach, we consider the optomechancial cavity being connected c , is connected to an auxiliary component via an optical path. The output light from the optomechanical cavity couples to the auxiliary component, and then travels back to the optomechanical cavity, forming a feedback loop. The optical path introduces a single-way delay of τs, phase φs, with a single-way efficiency of ηs and can be used as a channel to couple driving laser into the feedback system and to perform measurements. We consider (b) an auxiliary cavity or (c) an auxiliary mirror for the auxiliary component in this work. The auxiliary cavity has a coupling rate κ (1) A to the internal feedback optical path and κ (2) A to the outside. The reflectivity of the auxiliary mirror is R A .
to either an external optical cavity or a mirror via an optical path, as shown in Figure 1. Experimentally, the optical path might be realized by free-space optics, an optical fiber, or an on-chip waveguide. The light traveling through the path acquires a constant single way delay τ s and a phase shift φ s , and the path has a single way efficiency η s . The input and output of the optomechanical cavity are related to the input and output of the auxiliary component, (2) u in fw andû in bw are the input vacuum field due to the loss in the optical connection, and the subscript A denotes the field of the auxiliary component. R is a rotational matrix, due to the phase acquired in the optical path. The auxiliary component in general has two sides, with channel 1 coupling to the optomechanical system, and channel 2 coupling to the outside, which can be used for driving and readout. With a constant drive field from channel 2, the input u in,2 A is a vacuum field since only the fluctuations are considered. To simplify our discussion, we will mostly focus on using an optical cavity as the auxiliary component. The intracavity field is then denoted asû A = (X A ,Ŷ A ) anḋ where κ (1) A and κ (2) A are the loss rates, or coupling, to channel 1 and 2 and the total loss rate of the cavity κ A = κ (1) A + κ (2) A . The output on both channels is given by the input-output relation [17] where k ∈ {1, 2} denotes the index of the coupling channel. For the coherent feedback cooling, we will also explicitly consider the scheme where a mirror with a reflectivity R A is used as the coherent feedback component. In this case, there are no addition fields with a time derivative, and the output is directly given by the input Here, we drop an added phase from the reflection as any additional phases can be included into the phase of the optical path φ s . Combining the dynamics in the system, we obtain a Langevin equation of the form,u D and A n (n = 0, 1) define the interaction between fields in different elements in the system, while the C n matrices give the coupling to the external fields. The delayed response are given in the matrices D and A n , C n with n = 0. All the localized fields are included inû, and all the input fields are included inû in . For example, for the coherent feedback with an auxiliary cavity, we The delay τ depends on the scheme, which for the coherent feedback with a mirror is τ = τ s and with an auxiliary cavity τ = 2τ s . Equation (7) is a delay differential equation, which is hard to solve in general. However, for coherent feedback cooling the system will reach a steady state and with the stability test [51,52], this steady state can be solved in the Fourier domain and the final phonon occupancy can be obtained (Appendix B). For entanglement generation and verification, we will consider the special case with τ s = 0, for which the solution can be obtained by solving the time evolution of the covariance matrix.

Similarity of a sideband-resolved system with auxiliary cavity
Let us first consider a simplified but illustrative model, which can be solved analytically. The optomechanical system is in the deep sideband-unresolved regime, κ c Ω m . The cavity field has a dynamics that is much faster than the dynamics of the mechanical resonator. It is therefore possible to eliminate the derivative to the cavity field by approximating it with an instant response to the mechanical resonator [32,34]. If we choose ∆ c = 0 the optical field in equation (1) is then given by For the feedback part, we consider φ s = 0. This can be achieved experimentally by locking the length of the feedback path to a fixed value. We further choose a short feedback path τ s 2π/Ω m , allowing us to approximate τ s ≈ 0. This results in a linearized Langevin equatioṅ where We have introduced two effective bosonic fields,û in,1 A andû add , whose full expressions are given in Appendix C. Equations (9) results in dynamics that are similar to a bare optomechanical cavity where the mechanical resonator is directly coupled to the auxiliary cavity when compared to equation (1), with modified parameters, Now, the effective optical decay rate isκ A . By using a narrow-linewidth auxiliary cavity, it therefore effectively brings the system into the sideband-resolved regime. In particular, with the optical cavity of the optomechanical system being overcoupled, which can be routinely achieved experimentally [24,27,53], r c < 0 and ξ 2 /ξ 2 1 < 1,κ A < κ A . The effective optical decay rate is smaller than the actual decay rate of the auxiliary, enabling even less stringent linewidth requirements for the auxiliary cavity. Experimentally, external Fabry-Pérot cavities can have a much narrower linewidth than the mechanical frequencies in many integrated optomechanical systems. Connecting to an external cavity, however, introduces an additional delay due to the optical path length, which can be significant for mechanical resonators with high frequencies [12,26,54]. In this regime, an on-chip auxiliary optical cavity [55,56] is able to provide both small linewidth and short optical path to ensure τ s 2πΩ m . The drastic reduction of the effective linewidth of the optical cavity, from κ c toκ A , allows to now realize experiments and applications originally proposed for the sideband-resolved regime, with systems with broad integrated cavities or very low mechanical frequencies. This however comes at the expense of a reduction in the (effective) optomechanical coupling rateg. Furthermore, the added noiseX add increases the effective phonon number of the bath, Note that the added noise vanishes at the limit of η s → 1, i.e., no loss in the optical path, while it does not require a fully overcoupled optomechanical cavity (κ (i) c = 0). To better understand the effect of the coherent feedback, we consider another important figure of merit in optomechanics, the quantum cooperativity C qu = 4g 2 n th κΓm [17]. Larger C qu means a more robust optomechanical interaction compared to the photon and phonon decoherence and is thus favorable for optomechanical experiments in the quantum regime [17]. The quantum cooperativity with the coherent feedback is then given bỹ For a lossless optical path, the effective quantum cooperativity is enhanced as long as κ A > κ A /2, i.e., the auxiliary cavity is overcoupled to the optomechanical cavity. This is due to the fact that an overcoupled auxiliary cavity recycles photons. With an inefficient feedback η s < 1, the enhancement is reduced when the original quantum cooperativity is very large due to the added noise from the optical path. For a set of practical parameters (see Appendix A), the effective quantum cooperativity and the ratio between the effective and the original quantum cooperativity are shown in Figure 2(a,b). Without loss in the optical path, the cooperativity is enhanced by up to a factor of 1.5. Despite the added noise, we note that the enhancement is robust against loss in the feedback path. With a moderate single way efficiency of 70%, the enhancement of the quantum cooperativity persists until C qu ≈ 3.25, which allows for experiments with a relatively large quantum cooperativity. This regime is crucial for many applications at high temperature, such as cooling and non-classical state generation [34,57,58]. Improving the efficiency drastically increases the enhancement region, with a single-way efficiency of 0.8 yielding an upper bound of C qu = 6.98, more than doubling the previous value. The results for changing the coupling efficiency κ (e) c /κ c and κ (1) A /κ A , while fixing κ A and κ A , are plotted in Figure 2(c,d). As expected, increasing the coupling to the internal feedback system increases the quantum cooperativity enhancement.

Coherent feedback cooling
Preparing a mechanical resonator close to its quantum groundstate has been a major driving force in optomechanics over the past decades [57,[59][60][61], enabling many quantum experiments [62]. Inspired by the similarity between a system in the sideband-unresolved regime with coherent feed- back and a sideband-resolved system, we show that it is possible to reduce the phonon occupation with our coherent feedback scheme. With large quantum cooperativity even groundstate cooling can be achieved. Furthermore, the additional parameters available to control the optomechanical system, allows it to perform better than a similar optomechanical system without the coherent feedback. Figure 3(a) shows the average phonon occupancy for an optomechanical system that is coupled to an auxiliary cavity without delay. The phonon number is minimized numerically with respect to the phase acquired on the optical path φ s and the detuning of the auxiliary cavity ∆ A . When the quantum cooperativity approaches 1, the phonon number starts to drop below 1. For a singleway efficiency of 70%, 80% and 90%, it is possible to achieve a minimum phonon number of 0.24, 0.16 and 0.10, respectively. When the quantum efficiency becomes too large, the added noise from the feedback and the back-action noise in the optomechanical system dominates and thus the phonon number increases. The sideband cooling results of an equivalent optomechanical system (Equation (9)) is shown as well. They correspond to a feedback cooling where φ s = 0 and ∆ A = −Ω m . We see that the optimized phonon number is slightly lower and that the reduction is larger when the efficiency is higher. A greater benefit can be obtained when we further include κ (1) A as an optimization parameter (cf. Figure 3b). As the auxiliary cavity is not part of the optomechanical system, we can freely choose its parameters as long as they are experimentally feasible. Here, we keep the coupling to the outer channel the same as before, κ A /2π = 100 kHz and for a single-way efficiency of 70%, 80% and 90%, it is then possible to achieve a minimum A , ∆ A and φs. In (b), the reflectivity of the auxiliary mirror is set to 1. Non-unity reflectivity can be included in the optical path efficiency. The phonon number is minimized with respect to τs and φs.
phonon number of 0.15, 0.10 and 0.07, respectively. The optimized parameters are plotted in Figure 3(c-e), showing the optimal parameters being very different from simply setting φ s = 0 and ∆ A = −Ω m . The optimal detuning is not necessarily on the red sideband due to the extra degree of freedom φ s . The optimal linewidth is larger than or comparable to the mechanical frequency, resulting in an auxiliary cavity that is heavily overcoupled to the feedback system. It is also interesting to note that we find an optimal point that is fixed for different efficiencies over the feedback optical path. It occurs at roughly 500 intra-cavity photons (C qu ≈ 2.8), with the optimal set of parameters φ s ≈ π/4, ∆ A /2π ≈ 0 MHz, and κ (1) A /2π ≈ 2 MHz. The reason for this fixed point is not yet understood and it may require solving the model analytically.
In an actual experiment, the optical path will always introduce a delay, which reduces the coherence of the feedback. For low frequency mechanical resonators, this delay can however be kept small compared to the oscillation period. We would like to stress that the delay does not introduce a significant reduction of the cooling performance if the mechanical decoherence rate is small, i.e., the decoherence has a time scale that is much longer than the feedback delay. In Figure 4(a), the cavity-assisted coherent feedback with delay at different η s is studied, where the round trip delay 2τ s is set to 0, 0.05 and 0.1 µs. This corresponds to a single-way free-space optical path of 0, 7.5, and 15 m, respectively. No significant higher phonon number for different delays with different η s can be observed. The cooling performance is worse at very large photon number, which is however already outside the optimal regime.
Besides the coherent feedback with an auxiliary cavity, coherent feedback cooling can also be achieved by using a mirror as the feedback element (cf. Figure 1(c)) in the presence of optical delay. With a round-trip delay of around 1/4 of the mechanical oscillation cycle, the feedback can introduce a significant damping force onto the mechanical resonator when φ s is properly tuned, which we show in Figure 4(b). For simplicity, we assume a mirror reflectivity of 1, while any deviation from unity can be directly included in the optical path efficiency η s . We find a similar cooling performance compared to the coherent feedback with an auxiliary cavity. At large quantum cooperativity, the resulting phonon number is slightly higher due to the larger delay which introduces an additional incoherent signal to the feedback. However, similar to the feedback with the cavity, the impact is small at the optimal quantum cooperativity. A direct comparison is plotted in Figure 5.
Additonally, we compare the coherent feedback to the measurement-based feedback approach (see Figure 5). For the measurement based feedback cooling, we adapt the results from [32] with : Comparison between coherent feedback by mirror (crosses, results from 3(b)), by auxiliary cavities (circles, results from 4(b)), and from measurement-based feedback (dashed lines, results calculated from [32]). The measurement based feedback cooling has a detection efficiency η det that is equal to ηs of the corresponding coherent feedback case.
where ω fb is the feedback bandwidth. We minimize the phonon number with respect to ω fb and the feedback gain numerically. We use a detection efficiency that matches the single-way optical path efficiency, η det = η s . At small quantum cooperativity, the measurement-based feedback cooling is much more efficient, as the weak cooling power for the coherent feedback cooling is due to the lack of gain in the coherent feedback system proposed here. On the other hand, in the measurement-based feedback cooling, the gain can be tuned to reach the noise-squashing regime [19,63]. However, at large intra-cavity photon number, it is possible to achieve a lower phonon occupation with the coherent feedback. For the coherent feedback, the feedback control signalû fb coherently mixes with the vacuum noise (û vac ) with an efficiency η coupling into the optomechanical cavity,û in c = √ ηû fb + √ 1 − ηû vac . For the measurement-based feedback cooling, the control signal is classical,û in c = √ ηu fb +û vac . The input noise is therefore lower in the coherent feedback case, with 0 < 1 − η < 1 always satisfied.

Entanglement generation and verification between photons and phonons
Quantum entanglement between photons and phonons has been proposed using an optomechanical system in the sideband-resolved regime [14,64] and experimentally demonstrated in various systems [16,65]. Depending on the detuning of the input field with respect to the optomechanical cavity, either the Stokes process can be used to generate entanglement or the anti-Stokes process serves as a readout and to verify the entanglement (state-swap operation). With a sidebandresolved system, the other process can always be strongly suppressed, which makes it possible to efficiently generate and verify entanglement by sending two pulses. In contrast, in a sidebandunresolved system, the difference in the suppression is lacking. Entanglement creation and the readout happens at a similar rate, making experimental implementations challenging. Inferring entanglement through continuous measurement inspired by the similar idea has been proposed, with a maximum squeezing of the EPR quadratures reaching up to 50% of the vacuum noise [34]. The coherent feedback proposed can effectively bring the sideband suppression back and it is again possible to create and verify optomechanical entanglement with the help of an auxiliary cavity. Squeezing of the EPR quadratures beyond 50% of the vacuum noise could be achieved by optimizing the mechanical quality factor with a reasonably high efficiency of the optical path.

Entanglement generation
State swap Optical switch Figure 6: Entanglement generation and verification scheme considered in this work. The optomechanical cavity connects to three auxiliary cavities via an optical switch. The three cavities are for cooling, entanglement generation, and state-swap for entanglement verification. By measuring the output of the last two auxiliary cavities, photon-phonon entanglement can be detected. Between the entanglement generation and verification, a gap without the feedback is inserted. Optionally, as considered in this work, light from the laser is coupled into the optomechanical directly to keep a constant cavity photon number.
We consider the scheme shown in Figure 6, without any delay in the feedback to reduce the complexity of the model. It expands the coherent feedback scheme in Figure 1 to a setup with three cavities. A fast optical switch allows to select which of the auxiliary cavities couples to the optomechanical system. Light is sent into the system through port 2 (the port connecting to the outside) of the auxiliary cavities. To start, the switch is set to the first cavity, which is used to pre-cool the mechanical resonator into a low thermal occupation state. For simplicity, we set κ (1) A /2π = 400 kHz, κ A /2π = 100 kHz, ∆ A = −Ω m , and φ s = 0. We also use an interaction time of 0.1 s, which is sufficient to reach a steady state. We then switch to the second cavity, which is used for the entanglement generation. Finally, the third cavity is used to perform a state-swap in order to read the state of the mechanics. We assume an experimentally achievable switching time of 100 ns between the entanglement generation and the state-swap stage, during which there is no feedback (shown as an empty channel with dotted optical lines in Figure 6). Our approach expands on the scheme presented in [14]. By introducing the switching time between the second and the third cavity, which is much larger than 1/κ c , the entanglement generation cavity and the state-swap cavity are effectively isolated. The light carrying information during the entanglement generation cannot be detected by the state-swap measurement. Optomechanical entanglement can thus be verified by measuring the X and Y quadratures of the output of the two cavities.
We calculate the entanglement through the linearized dynamics described in Equation (7), with D = A 1 = C 1 = 0. The calculation routine is similar to that in reference [18], but without the use of the rotating wave approximation (RWA) for the mechanical resonator. The evolution of the system is of the formû For t being in different stages (pre-cooling, entanglement generation, switching, state-swap), t 0 represents the start of each stage. With this setting, A 0 and C 0 are constant. The output of the auxiliary cavities, for the entanglement generation and verification, are then defined aŝ α ∈ {g, s} denotes the components involved in the entanglement generation and the state-swap phase. Further, we define optical temporal modes [14,34] The integration is carried out only within the corresponding stage, starting from t 0 and ending at t f . R is the rotation matrix defined in Equation (3), with θ g (t) = Ω m t and θ s (t) = −Ω m t + φ s . The rotation matrix is necessary since we do not use the RWA for the sideband-unresolved regime. Also, we take the exponential form for the envelope f α [14,34] f withr = (r g,X ,r g,Y ,r s,X ,r s,Y ). In this work, we use the EPR-variance to quantify the entanglement, ∆ EPR = (σ 11 + σ 22 + σ 33 + σ 44 )/2 + (σ 13 − σ 24 ) [34]. We note that switching with a finite dead time might introduce classical noise to the mechanical resonator due to the resulting change in the cavity photon number. It thus reduces the entanglement and is not captured by the linearized model [66]. It is more relevant to the calculation here, as the mechanical oscillation considered is significantly less coherent compared to the typical optomechanical experiments in the sideband-resolved regime [6,16]. A higher interaction strength with a shorter pulse is favorable, as demonstrated below. However, we stress that the switching time is much shorter than the mechanical oscillation period and thus the disturbance would mainly be at very high frequency, while the impact on a low-frequency mechanical resonator is minimal. Assuming an unchanged average photon number inside the optomechanical cavity, allows to eliminate the effect of the switching. Experimentally this can be realized by introducing another coupling channel, such as another waveguide into the optomechanical system. Alternatively, an additional channel on the optical switch that couples to the laser directly can be introduced. We include this in the scheme shown in Figure 6, plotted as the dotted red lines for the optics and the dashed line inside the switch. By controlling the light intensity on this additional channel, it is then possible to achieve a constant cavity photon number. This approach reduces the entanglement since the photons during the switching are not measured, corresponding to a loss of information. Meanwhile, the photons interact with the mechanical resonator, creating mechanical decoherence. Therefore, a switch with a short switching time is required to reduce the switching impact. We note that on-chip optical switches with low loss and short switching time have been demonstrated experimentally [67,68].
In the scheme considered here, the auxiliary cavities for entanglement generation and verification have the same parameters, except for an opposite detuning. The phases of the optical paths φ s for both stages also have opposite signs. We then minimize ∆ EPR with respect to κ A , ∆ A , φ s , Γ tm and τ p , where the values are given in the appendix (Figure 9). Different from the feedback cooling, we include κ (2) A as an optimization parameter since the entanglement is detected from the output light of channel 2. The result is plotted in Figure 7, with a unity detection efficiency for the light getting out of the feedback system. We consider mechanical resonators with Q m = 2 × 10 7 and 10 8 , corresponding to a thermal decoherence rate, in the unit of mechanical resonance cycles, Q m f m /(k B T /h) of 230 and 1100, respectively. When increasing the cavity photon number, we can observe a reduction in ∆ EPR . For a single-way efficiency of 0.7, 0.8 and 0.9, it is possible to achieve a ∆ EPR of 1.37, 1.22 and 1.05 with Q m = 2 × 10 7 , and 1.23, 1.05 and 0.84 with Q m = 10 8 , respectively. ∆ EPR < 2 is the threshold for entanglement between the temporal modes of the two output light pulses and is a direct result of the entanglement between the photons and the motion of the mechanical resonator. Increasing the photon number beyond an optimal point leads to an increase in ∆ EPR . In the present of optical loss, a stronger interaction leads to a stronger effective noise, reducingC qu . As the interaction strength is increased, the time of the pulse becomes shorter. The fixed switching time, during which the entanglement is reduced, becomes therefore more dominant. The calculation shows that it is beneficial to achieve low loss over the feedback path and a low thermal decoherence. Still, it is remarkably robust against any dissipation in the system, making it feasible for real experimental parameters. In the optimization, an optimal κ A /2π is around 500 kHz. We note that a total linewidth of 220 kHz has been reported recently using on-chip disk resonators [55], which is promising for a fully integrated coherent feedback system, which could minimize optical loss over the feedback path. A reasonable increase of the mechanical frequency would also make a fully integrated system more feasible.

Conclusion
In this work, we propose a coherent feedback scheme with linear, passive optical components. We mainly consider optomechanical systems in the deep sideband-unresolved regime, and with experimentally relevant parameters. We show that an additional, external optical cavity can effectively bring the optomechanical system into the sideband-resolved regime for a specific set of parameters (φ s = 0, τ s = 0, ∆ c = 0). We consider non-unity feedback efficiency, which introduces additional noise to the mechanical resonator. Overall, the effective quantum cooperativity can still be enhanced, depending on the feedback path efficiency and the original quantum cooperativity. Our analysis shows that coherent feedback is a highly promising path for broad applications using sideband-unresolved systems.
We use these results to demonstrate how either an optical cavity or a mirror plus an optical delay path as an auxiliary component can be used to perform groundstate cooling of the mechanical resonator under practical experimental conditions. Furthermore, based on an entanglement protocol with long pulses [14], we then propose an experimental scheme that uses three auxiliary cavities for cooling, entanglement generation and verification. By switching between these cavities in a relatively short time the output light can be used to detect photon-phonon entanglement. We quantify the entanglement of the output light by evaluating the EPR-variance of the temporal optical mode. Even though it is not necessarily the optimal entanglement witness [34,69] it shows a significant squeezing well below the inseparability bound. Experimentally realizing a fully integrated on-chip coherent feedback structure is within reach of state-of-the-art on-chip optical resonators [55,56] and fast optical switches [67,68], even for mechanical frequency as low as 1 MHz. Such an integrated structure would drastically reduce the complexity of an experiment and could help realize novel quantum applications [62] with optomechanical systems in the sideband-unresolved regime.

A Default parameters and convention
If not specified otherwise, we consider the following parameters.

B Steady state and photon number
When performing the feedback cooling, the steady state of the system is considered. It can be analyzed by transforming Equation (7) to the Fourier domain, C n e inωτs û in (ω), (19) where I is the identity matrix. Rearranging Equation ( The input noise has a single-side spectrum [32] S uin i (ω) = 1, Here, u ini are for the elements corresponding to the optical input noise only. It is then possible to get the spectrum ofû [32], S u (ω) = |M (ω)| 2 S uin .
The absolute value and the square are performed entry-wise. This allows extracting the energy of the mechanical resonator by integrating the spectrum of the mechanical field. The corresponding phonon occupancy is given by [32], The scheme is valid only when the system is stable. Determining the stability of the system can be done in a classical way [32]. For our system with delay, we follow the method described in [51,52] to perform the stability test.
They are a result of the combination of the optical vacuum field, and they satisfies the Bosonic commutation relation.