When you first boot the computer, power the monitor on first, and then watch the screen for something that says "Press x to enter setup", where the keywords are setup, and which key to press. It's usually one of the F1, F2 or Delete ("Del") key.
Once in setup, read the instructions (usually on the bottom or to the right) on how to move around the screen and how to change items.
You're looking for something that refers to boot order, or sequence. It'll usually say something like this:
1. Removable Device [Legacy Floppy]
2. IDE Hard Drive [Drive make/model here]
3. ATAPI CD-ROM [Drive Make/model here]
4. INT18 Device [Network]
But yours probably says something like this:
1. ATAPI CD-ROM [Drive Make/model here]
2. Removable Device [Legacy Floppy]
3. IDE Hard Drive [Drive make/model here]
4. INT18 Device [Network]
See how the CD-ROM is first? This means that the BIOS tells the computer to look for that device first. Put a CD in, and try to boot, and it'll say something about "boot drive not found" or some such, and not boot any further.
Take the CD out, and the computer still checks the CD-ROM first, but because there is no disk in it, it then can move on to the next boot device.
Just move your CD-ROM down the list to 3rd and see if that fixes your troubles.