You've seen hurdles. Those are the obstructions that are placed on a running track to force racers to leap over them.
I imagine that, long ago, some people got bored with running around and around a flat track. Someone dragged a log or something into the path of the race and thought, "This will spice things up!" And it worked.
Thing is: if you're running a hurdles competition, the point of the race isn't how brilliantly, artistically or athletically you clear the hurdles. The point is to cross the finish line ahead of the other runners. The hurdles themselves are just a given. Sure, it takes skill to leap that way. But everyone in the competition, with a few exceptions, has that skill. They all clear the hurdles most of the time. The point is to win the race by being faster than the others. It's not THAT you do it. It's HOW you do it and still manage to win.
As an improviser, doing shortform improv is not really about knowing the rules of the particular game you're playing. Of course you know the rules. That's entry-level stuff. To quickly switch emotions in Emotions; to quickly utter a new choice in New Choice; to quickly portray a movie genre in Genres: That's the basic requirement for being allowed on an improv stage in the first place. Chess isn't about knowing how all the pieces move. It's about how brilliantly you choose to move them.
Shortform improv needs to be more than simply a party game played in front of an audience. You need to do great scene work (in scenic pieces, of course), make true connections with your partner(s) and your audience, transcend the commonplace, the predictable and the purely mechanical. You need to do inspired work DESPITE the obstacles, not simply applaud the fact that you managed to leap over them. And that involves understanding the overall strategy of the performance you're involved in.
Watch city kids play jumprope. It's rarely just about not tripping over the rope as it circles over and beneath them. The good ones do more. They dance. They sing. They choreograph. They do gymnastics. They excel. They aren't happy with basics (although they know the basics very, very well).
How do you achieve this? Hard work. Drill the basics relentlessly. Figure out what you want to accomplish that isn't basic (this requires a director or coach with vision, who isn't afraid to push for better and better results). Work as a team. Don't settle for mere competence. Not for performance-level improv, anyway (obviously, improv that is used for team-building, therapy, fun-and-games needn't be that concerned about this level of excellence).
No. Don't forget the fun. Don't ruin improv by making it pure drudgery. But do realize that there are always new heights to reach if you care to reach them.
Realizing that brilliance, intensity and technique can also be fun is just another hurdle to clear.