hey, you wouldn't be wasting any doctors time. it's better to be safe than sorry. anyone would recommend seeing a doctor when you notice any sort of change in your breast, no matter how big or small the change may seem. seeing a doctor now could give you bigger chances of being okay, because they'll have you monitored for breast cancer, especially because your mother has been diagnosed - which I'm sorry to hear by the way. 

you know your body better than anyone, and if your breast changes don't feel right to you or even seem a little off, get it checked. it could help immensely; and I hope it does! good luck & I hope you choose to see a doctor :)