When to Give Up

I'm getting increasingly frustrated with trying to rebuild my guild. I want to go back to being a casual player who raids, but I can't. I guess it's a sense of duty to the guild my friends founded but aren't really active in anymore.