I have not had a 4 speed Shadow, but when I was researching them that is the one thing that kept coming up, to get a 5 speed if possible.
Now, if the top gear is the same, then running down the highway would not be an issue. It kind of comes down to where and how you ride, if you do a lot of puttering around at 35-45 maybe it would bother you. It must work out not too badly, they sold a bunch of them with 4 speeds, right?
I suspect it is like most things. If you have not had a 5 speed, you won't miss it. IF you HAVE had a 5 speed in the past, you may not like the four speed? just guessing really and something to think about.