Setting goals when you have none

So someone has repeatedly said to me that I should have some goals with my workouts, but really I don't unless "being able to eat what I want" is a serious goal. Perhaps it is.

I don't have a desire to bench press a certain weight for a certain number of reps. I don't have a desire to lower my 5K time - perhaps I should? I don't have a desire to hit a certain amount of weight on the leg press machine.

Part of my worry is that typical goals - like being able to bench press a certain weight - would lead to my muscles becoming larger, and that's the last thing I want. I like being lithe -or trying to be. Unfortunately, I think my genetics predispose me to getting bigger, so if I train harder, well...

So, if you're reading this, let me know what you think about setting goals.