Kevin Durant felt like an outsider during time with Warriors