Facing Physical Limits: Learning What NOT To Do