When you crash X nine times out of ten it's driver problems.
It's similar to what happens now in Vista with their new user-mode drivers were the screen blanks out and resets the card. The drivers f-up, X crashes, your set to console (which uses a different set of drivers that reset the display more or less), gdm gets restarted by init and then you log in again.
For X, at least, the video drivers are all user-land with a DRM (direct render manager) driver in the kernel to allow hardware access. Unfortunately this doesn't always happen and the drivers are bad enough they put the video card into a state were it can only be recovered by powering off the machine.
(of course it would be nice to not have these problems in the first place)
Now if your using propriatory Nvidia drivers, it's a bit different. More kernel-land stuff going on there, I beleive. Nobody realy knows how they work since examining them is not allowed by Nvidia and would probably make life hard for somebody contributing code to X.org in the future due to legal issues. But with Nvidia essentially your running Nvidia's own X server. They use a lot of X.org code and peices, to be sure, but essentially it's not the same X server your system shipped with. With those things nobody realy can help you to much except for Nvidia.
For xorg.conf.. it's life is coming torwards a end. Linux has gotten very good at hotplugging stuff and detecting hardware.
For example.. if you install Debian or Ubuntu on a machine the vast majority of the hardware will be detected, configured, and drivers loaded automaticly. Each time you reboot the whole proccess of hardware detection and configuration happens again. /dev is configured automaticly nowadays also. (back in 2.4 days you had he choice of using devfs to try to autoconfigure (it would load up drivers automaticly if you tried to access the paticular /dev/ file... even if you didn't have that hardware) or manually making /dev files. (most were made by default, but not aways).
The major excepts to this is hardware that isn't supported by your kernel (which you would have to obtain drivers seperately), volume/disk entries into /etc/fstab, bootloader configuration, or anything to do with X.
So if you install Debian on machine A and then pulled the drive out of it and plugged it into machine B.. it will 'just work'. Everything is detected and autoconfigured. Worst case is that you have to boot up knoppix or the installation cdrom and manually edit /etc/fstab to the correct layout then modify /boot/grub/menu.lst to give the correct root partition.
However with X it doesn't work that way. It can't use any of Linux's hotplug features for things like mice, keyboards, video cards, or monitors.
The closest you get it to work for mice is by configuring X manually to point at /dev/input/mice. The Linux kernel then takes any mice you have, uses emulation to make them all appear as 'ExplorerPS/2' mice, and outputs that to /dev/input/mice. This obviously doesn't work well if you have many button'd mouse, wacom tablets, touch screens, or touch pads, that you want to use special features for (such as pressure sensitivity, or change of resolution).
So keep in mind that Linux kernel is perfectly capable of configuring that on the fly, but X isn't.
Soo....
Right now the latest release of X.org is 7.2. With 7.3 it's goal is to finally get hotplug capabilities and reduce the dependance on xorg.conf (were you don't need it if everything is configured correctly for your user).
So the goal is if you plug a mouse into X... Linux will detect it and set it up, notifying userspace of the changes through udev/dbus. X will be listening on dbus, detect it and probably get it automaticly. At the same time applets on your desktop (or whatever your Desktop Environment chooses to use or you have configured) will probably have a pop up telling you that a mouse has been plugged in. If the hardware is supported well enough then there may be a user-land program to configure it further and such.
So the same thing will happen, hopefully, with monitors also. Plug in a LCD monitor or projector and it will be detected by X's drivers. It will then pull the EDID information from the monitor cable itself and then configure the correct resolutions and such. It will then turn it on telling everybody who wants to listen on dbus that all of a sudden you have lots more desktop space.
If your Windows manager, then, is smart enough it will expand to fill the new space and you can move things around accordingly.
Then when you turn off the monitor and detatch it then the oppisite should happen and if your Window manager is smart enough it will then shove all your windows over to the remaining monitor so you odn't have to fish around in the dark for it.
All of this should happen without xorg.conf being used at all. So this sort of thing is the goal for X.org 7.3. Of course you'll need proper driver support for all this to happen correctly (so don't hold your breath you propriatory ATI driver users, you) which is probably just going to be Intel hardware for the time being.
(it won't work for video card hotplugging, but since nobody realy does that much anyways it's not a big deal at the current time)
Currently if you look at any presentation by Linux users you will always see them f-ing around with the projector forever and often never realy figuring out how to get it to work correctly.. That sort of thing does not inspire confidence in LoTD.
Same thing if you want to use your wacom pad or setup your trackpad with special features your going to be editing text files and restarting X and logging in and out and all sorts of crap. This doesn't look good either.
It's only a little bit better then having to reboot your entire machine each time you want to make a change.