When you hear someone say "that's not a game," they usually mean "this is not what I was expecting from a game".
Game definitions are like tools. You need to get something done, so you open your toolbox, pick the right tool, and then use it. As a craftsman, you care about your tools, you polish and clean them, you try to make better use of the ones you have, and judiciously add new ones to your toolbox, all in service of the craft.
A game definition based on Huizinga's magic circle allows me to do different things than a game definition based on Crawford's toy/puzzle/game taxonomy or Costikyan's resource-based definition. All of them are useful, they all make sense and are 'true'. But each operates in completely different perspectives.
And this is the value of a varied and well-selected set of tools. They allow you to look at things from different angles. When you have a screwdriver, you look for screws, for things to open.
Similarly, when you're using the magic circle perspective, you might be thinking of how playing relates to non-playing, how people can be drawn into a magic circle, how their behavior can be changed, or how you can draw attention to their behavior by manipulating the magic circle.
When you're using the toy/puzzle/game perspective, you might be thinking of player goals and replayability. Do you give explicit goals to the player, or do you let them select their own goals? Is the game about figuring out a problem and then moving to the next one, or is it about dealing with an opponent or a complex system?
Costikyan's definition of game draws your attention to resources and allows you to design independent of whether you're working with cardboard and tokens or on a computer. It makes economics applicable to games.
A final perspective - I hesitate to call it a definition - is that of 'it's a computer game because it looks like a computer game'. Here we get games like Proteus or Dear Esther. They are sold and discussed alongside other games. They contain 3D worlds we can navigate through. They don't have a 'serious' purpose - they're not email clients. Note how none of the previous definitions of games said anything about navigation in 3D worlds, or any kind of navigation or worlds at all. Yet the representation of and navigation in worlds is an unspoken assumption underneath a lot of people's idea of what a game is.
The toy/puzzle/game taxonomy is the clearest about games like Proteus and Dear Esther. In this system, seen from this perspective, these games are toys. This helps me understand them. It makes them tractable, understandable, 'known', even before I play them. This doesn't mean they're boring or predictable: just that I can recognize them from a certain perspective.
Game definitions are tools, and knowing your tools well, understanding their differences and their strengths and weaknesses makes you a better designer. In the case of mental tools, knowing exactly which tool you're using is perhaps one of the most useful things to learn.
Game definitions can also be used in a rhetorical way, as a form of argument. This is always about power, and thus political. I define games this way, therefore your game is not a game and thus you lose power. I define games this other way, therefore games become more worthy than your medium and thus you lose power. I define games like so, therefore it should not be a part of your department and I get to raise my own funds.
There is nothing inherently wrong with rhetorics, politics and the use of power. But it's important to distinguish between politics and the truth inside your toolbox.