I was VIG for a few years. Watchdog hit on most of the perks in his response. The special VIG lounge is also where you can get event tickets without waiting in the long lines, rest your weary muscles after hours of running around, and a bag check if you want to use it. There are also special VIG only events.
The swag bag for VIGs when I did it was pretty good. There was always at least $200 worth of stuff in them with some bigger items like rule books or adventures for games and actual games themselves. I remember getting the D&D 4th edition Players Handbook, a signed copy of a novel, deckbuilding games, etc.
VIG members can also bring in two people as VIG Companions which basically means they have most of the VIG perks. When I last did that the companion badges cost $100. I'm guessing its more now.
For me the biggest draw was first pick for hotel reservations, but it seems like years ago that was a bigger deal than it is now. My friend didn't do anything special, bought 4 day passes, and was able to get exactly what he wanted at the Hyatt. Since the JW Marriott opened about four years back Ive found getting the hotel reservations to be much less stressful. In past Gen Cons, like 2010 and earlier buying badges and getting hotel reservations generally made you miserable due to the extreme slowness and website crashes. You would finally get in and buy your badges after 47 tries only to find a certain hotel was completely booked so for me me it was worth being VIG to bypass all of that misery.
I couldn't attend the last few years, but after going back this year with a 4 day pass I would be interested in VIG again. It just really depends on if you think you will use the perks that come with being a VIG. Its not worth the money if you dont.