When you push the reset button, it resets the AVR. Also, the Arduino IDE sends a special signal that causes the Arduino board to reset the AVR.
After the AVR is powered up or reset, it immediately begins running the bootloader. The Arduino bootloader this watches for a few seconds to see if a new sketch is being downloaded from the Arduino IDE -- if so, it erases whatever was in flash and burns in the new sketch. Then the bootloader starts running whatever sketch is in the flash.
"Automatic reset during program download" is a convenient little circuit on most Arduino boards and most Arduino-compatible boards -- Om328p, etc. The Auto Reset Retrofit describes how to add that circuit to (the few) boards that do not already have it, such as many Atmega standalone chips on breadboards. The disabling auto reset on serial connection page describes how to disable that circuit on the (rare?) cases that you don't want it.
Some people store in EEPROM how many times the sketch has started (i.e., how many times the power has come on or the reset button pressed): "Counting resets with wear leveling".
Unfortunately, once the Arduino has started running a sketch, there is apparently no way to write a software function that has exactly the same effect as pushing the reset button.
(See Arduino Waker and "Is it possible to have the arduino reset itself? Because I'm using an Ethernet shield and sometimes you have to reset it a couple of times to get it to connect." for one situation where programmatically pushing the reset button would have been convenient. LCD4BitLibraryDevelopment describes another weird reset situation).
Several people -- gabriellalevine: "Two ways to reset arduino in software" and "There are 3 types of reset on Teensy 2.0 and Teensy++ 2.0" and "Three ways to reset an Arduino Board by code" (via Linkedin ) And "Reset arduino in software" -- mention three work-arounds that act almost like pushing the reset button:
Linkdump in random order: (Consider summarizing the best practices here)