My experience is that you're best off either with a swap space that matches or exceeds physical memory, or none at all. The linux kernel swap daemon (kswapd) gets confused and behaves badly with small swap spaces, especially the more memory you have. Good news is that hard drives are cheap and 32G isn't a lot of memory so I'd set it up for 32G + a tiny bit. Once a machine gets to larger memory sizes (128G and more) I just turn off swap.
The really messed up bit is that the problems with the kswapd won't show up for weeks, months, or sometimes even longer. The symptoms of a kswapd problem is that swap is mostly full, but there's LOTS of free memory / kernel cache, and the kswapd is busily swapping in / out. Page faults skyrocket and the problem can last for a few minutes or a few hours, then clear up and not occur again for a long time.
tl;dr: Either physical memory + a little or none.