I think it's a quite okay way to spend free time, not the best but better than watching TV, reading most magazines, drinking, or other passive hobbies.
There are more productive things you can do of course, but perhaps not 24/7, so games can fill some unproductive hours.
Adults could definitely be right if they tell you you play games too much. Don't neglect your education. Also, some variety in hobbies is good. But other adults overreact because they aren't familiar with playing games, so it's really up to you or people who know you to assess the situation.