Glitches are either errors or bugs in any program(game, software, etc) that can cause undisired or interesting results. In relation to Halo, glitches can be exploited in many ways to make the game interesting or annoying.
Halo:CE Examples
Not many were present in Halo:CE, however there was the occasional glitch that allowed you to ride a Pelican, skip a certain amount of a level, or get out of a map.
Halo 2 Examples
In Halo 2, many glitches resulted from unforseen actions the developers did not think could occur. Players could do things such as Super Jumping or Super Bouncing, Get past Invisible Walls to get out of maps, etc. Button combos also existed, such as one that would allow you to kill a player instantly.
Halo 3 Examples
Halo 3 has been greatly combed over for glitches, but Bungie has put alot of effort into making sure that their are few if not any in Halo 3. There are a few that do exist though, such as the Mine Glitches, floating objects, driving a AA Wraith, etc.