Integrated photonics has been pursued on a large variety of materials and platforms over more than half a century. In recent decades, silicon has emerged as the preferred optical and substrate material. However, since many optical functionalities cannot be, inherently or conveniently, implemented on silicon-based devices, heterogeneous integration of other materials has been an inevitable constituent of silicon photonics from its inception. This paper reviews recent progress in the heterogeneous integration of dielectrics and compound semiconductors as the core optical material for ultracompact nonlinear integrated photonics, preferably on silicon substrates. Occasionally, important devices on native substrates of the core nonlinear material are also reviewed. The focus is on ultracompact photonic devices and circuits formed on submicron films of lithium niobate (LiNbO3), and the aluminium-gallium-arsenide (AlGaAs) and aluminium-gallium-nitride (AlGaN) families for second-order nonlinear processes, such as second-harmonic generation, and techniques to address phase-matching in them. Integration of second- and third-order nonlinear devices on the same chip appears to be a future trend, thus the related preliminary results are reviewed. A broader introduction to heterogeneously integrated photonics, shortcomings of conventional silicon photonics for application in nonlinear optics, as well as an overview of modeling second-order nonlinear integrated-waveguide devices are also provided for completeness.