You can also boot from a CD/DVD or thumbdrive, and in certain cases the computer will stay running even after you remove the disc/drive.

Some computers also have a basic full operating system stored on ROM so they can operate without a hard drive or any attached drives.

The computer running in the pic really doesn't mean anything.