Amen to Bill's advice! I ran the same rig for over 30 years with a 115 V-4, including 6 trips to the Bahamas. I believe the boat will generally ride better with the lighter motor. I couldn't cruise as fast as the V-6's in flat water, but it's always choppy on the Bahama Bank, and I had no trouble staying with them when it got choppy. What you have to remember is that the 20' hull is relatively narrow compared to newer boats, which is one reason it rides so well! When it was designed back in the mid-60's, the biggest motors available only weighed about 300 lbs.
As motor weight goes above that the boat becomes progressively less well balanced IMHO, especially the sf/cc which is more stern heavy than the Seafari to begin with. With my old 1975 115 I could plane at 12-14 mph which is nice capability to have when it gets rough; I suspect min planing speed is a fair amount higher with a heavier V-6. Avg mpg on the Bahamas trips was 2.8 mpg with very heavy load. It's about 4.5-5 mpg with the new motor but you can't justify cost of a new motor based on fuel usage alone, even at $4/gal!

Only downside of the smaller motor is that you may not be able to outrun thunderstorms, so you need to watch the weather a little closer, but good it's good seamanship to always do that anyway!