It's a shame but not unexpected. Ferguson has always been the odd man out in late night: he doesn't seem interested in being a Carson clone, but he hasn't established a different model as Jon Stewart has. My guess is that the stripped-down, low-key atmosphere of his show turned off people before they could tune into his sly humor. Once Colbert was announced to be Letterman's replacement, with Letterman's blessing, Ferguson didn't have a future at CBS, and he knew it.
I hope he finds another gig that will allow more people to discover how talented and funny he is.