The AI has no self awareness, but it must make choices. I don't think I'm unique but I have had the situation arise where I risked my life to avoid killing a pedestrian. Again I didn't ponder the situation, but me, as an entity, did just that. No one else has? I have a hard time believing that.
Nope, and that's why I'm also suspicious of the hype and rosy, one-sided arguments for replacing human drivers and personal cars.
I had an incident where I narrowly missed hitting a child on a bike years ago.
Suburban area with a family on the walking on sidewalk with a young kid on a bike learning to ride.
I was looking ahead, saw the kid was wobbly on the bike, so I slowed way down (under speed limit) and moved towards center of road and watched like a hawk.
Kid served way to the left, over the grass and into the road. Slammed on brakes and served into other lane. It was really close, but I didnt hit him.
Had I been just following the rules of the road (speed limit, center of lane) I would have likely struck and possibly killed the kid.
Will the AI's scanners be able to see this scenario, anticipate, and diverge from the rules?
Will it see the birthday party with kids running around and go into super safe mode?
Will it understand and see all the deer running in the fields? Will it learn their hiding spots in the trees? Will be be especially careful at dusk and rutting season?
While I see a scenario for automated drivers for cargo trucks and others on predictable and controlled highways devoid of pedestrians, achieving full AI everywhere will be far more difficult and fraught with legal peril.
Driver supplimental abilities also seem useful (emergency braking/collision avoidance) but roadways are incredibly dynamic and I have a hard time believing software is close to ready for it.
Does anyone recommend blindly following Google maps all the time? How about software landing your plane? This is easy in comparison.