The attractiveness of surfing is mainly due to the surf industry which depicts surfing as a great lifestyle one should follow and surfers as people living on the edge of society in sunny places, caring about nothing except good waves, parties and girls.
The reality is obviously completely different, but people are conditioned by the surf industry ads. That's why they often come at you with awe and prejudices.
In this way, surfers get approached by strangers especially in winter, because they're one of the few daredevils going for a swim in cold waters when everyone is walking by the beach wearing scarves and winter coats. People usually ask surfers if the water is not too cold or if they can still feel the extremities of their limbs...
That said, surfing is a physical activity that requires to be in shape and that's another reason why surfers are attractive.
One could also cite others facts like traveling to find new waves, owning a van (yep, Into The Wild cliché...), being laidback...
We may be attractive but, to our dismay, there's still not enough women in the water. At least the surf spots are less crowded...