You can be raised speaking one language without it affecting your accent in another.
Yes ... accent is not clear cut ... when I was younger I was told that my accent was not "very Australian" and was some middle of the road English one. I also lived in US UK and NZ ... when in the US I spoke with a American accent after some time.... mostly to make myself understood... but would revert to a more Australian one when talking to my parents over the phone..... my sister is 8 years younger and she sounds different to me .... guess it was the difference in environment that we lived in