Maybe I'm missing something here, but why worry about what someone else thinks about you? If you like this girl - and she likes you - then all you need to do is follow that path and not worry about a bunch of wannabe, jealous idiots.
Don't go looking for trouble, obviously. Don't be a bad-ass. Don't be a loud-mouth. Don't invite trouble into your house. Always be nice ... until it's time NOT to be nice. Be non-confrontational until you no longer have that luxury. Walk away and hold your head high ...
BUT ... don't be a push-over or a coward. Stand up for yourself; believe in yourself and who you are. Fuck them ... she chose YOU. If they step to you - and you have no other alternative except for physical ******** - then do what needs to be done ... take some of the fuckers with you.
If they get in your face and start trouble ... DEFEND YOURSELF. Otherwise, if you cower down and refuse to fight back (be it with words, fists, whatever) then they'll always see weak prey. So what if you get an ass-kicking a couple of times? Lord knows I've been in that position more than a couple of times in my past, but I always made sure I either had an equalizer (be it a sock full of quarters, a sap, or something else to even up the odds between us) or else I fought like a fucking wild ******. I had more than a few dumb-ass fucks follow me to the hospital who were in just as bad a shape (or worse) as me.
(Yes, folks, I fight dirty. I grew up in a place where "rules" didn't matter ... survival was the only option. I'm not above gouging eyes, squeezing nuts, or shattering kneecaps. Fuck with me ... and you'll pay dearly. I'll play fair when fair is even throughout ... otherwise, fuck the "rules; it's all about survival.)
Don't let no motherfucker treat you like a cowardly bitch. Hold the line, stand your ground, and give as good as you get.
Otherwise ... you'll run scared for the rest of your life; always looking over your shoulder, wondering if some dumb motherfucker is gonna bitch-slap you the moment you say something they don't like.
It's a hard life ... don't let it steam-roll over you.