The first Caveman can be found in the area where Sergeant Major Avery Johnson's Pelican is shot down. Across the water, above and to the right where a Kig-Yar Sniper will spawn on Heroic and Legendary difficulties, the first Caveman can be seen sitting in a tree. Unlike the other Cavemen, there is no way to reach him.
To find the cavemen family, go past the point where Johnson says his Pelican was shot down. Then, enter the jungle and stick to the left wall. You will have to climb over some rocks, jump over a giant rock in the way, and then crouch under a tight ledge to find them. You can shoot them but not kill them, although blood will still spurt out, which shows that the respawn cycle is not set up, so they will not die. They all vary in size, with the largest holding what appears to be a teddy bear, and the smallest being half the size of said teddy bear.
There is a caveman sitting on a cliff outside the level, in the area where you rescue Sergeant Major Johnson. He is accessible by means of a glitch, but he is still visible by using a scoped weapon, such as a sniper or the binoculars. To get where the Caveman is, you must play in at least two players cooperative play. It is much easier to perform with a Gravity Hammer. When you reach him, you can keep running and get out of the level.
Halo 3: ODST
During the epilogue cinematic, after the final level of Halo 3: ODST, Coastal Highway. When Vergil lights Johnson's cigar during the interrogation, push the right thumbstick to the left on the controller and the scene will pan over to the left. You will see Edward Buck touch a Caveman similar to the ones in the first level of Halo 3. He will then groom it, and appear to eat something taken from its hair; however you will only see him raise his hand towards his mouth, as the scene will black out immediately thereafter.