Taylor has always been a boy name as it derives from the profession - which has traditionally been male-dominated while seamstresses have traditionally been female. That said, yes, most people are aware of the fact that, nowadays, Taylor is rather popular with girls. But, having said that, LOTS of boy names are currently popular with girls, and lots more boy names will be popular with girls in the near future (with sites like Nameberry advertising Roscoe, Oberon and James as "unisex").
So, with that said, I wouldn't give up Taylor, if I were you. Choosing a name that you genuinely love is far more important than worrying about whether it's "too feminine" or not. Especially since femininity is just as strong as masculinity, so even if Taylor was super girly (which it isn't), that shouldn't be a problem or a shame.
All that aside, I do genuinely like it for a boy and love when people choose to use it for the correct sex.