I am a student learning game design, and I have completed a few levels in unreal so far. but I have wondered how developers create large underground levels. I am trying to figure out what is a good way to create entry for caves and tunnels in the Unreal landscape materials and what is the best way for me to block out a prototype of tunnels and natural interiors (ex. caves)?

Traditional and modern games which use heightmaps to define their terrain are all suffering from this problem. A heightmap is a 2D top-down view of a terrains 3rd dimensional shape, so it isn't possible that easy to integrate structures like cliffs or caves, which are beyond the generated terrain, because terrain generators most of the time work plane.

Different developers may have different solutions, some let an external tool generate a mesh and modify it from hand, some use decal assets that contain the cliff for example and some even don't have a seamless transition from the overworld into a cave (The Elder Scrolls for example) and instead simulate that through a loading screen.

I had a talk with someone some time ago, which you can find here https://www.gamedev.net/forums/topic/692945-creating-a-3d-world-for-an-rpg/​

I tried to cover everything which might be interesting, also how I'd implement a terrain generator which can support cave entrances and cliffs. However, the cave interrior needs to be a mesh nayways because usual terrain generators are quite bad in generating closed structures like caves. This is why caves are usually modelled by hand

Don't know about UE4 specifically, however if you build your terrain with voxels using marching cubes, dual contouring or a number of other algorithms, you can easily incorporate caves and the like. That's one big advantage of voxel based terrain.