The gasket needs to have a continuous surface all the way around the float bowl. In the picture it looks either dirty or discontinuous. Teflon tape is usually pretty harmless, because if you put in on properly it does not project past the front of the thread. However yours seems to have been very badly applied. You could use liquid thread-sealer, or nothing - it shouldn't really need sealant, but if you want to end up with the tap horizontal or on any non-random angle, you will need to do something to help it slip. I think I'd try liquid thread-sealer, which you should be able to get from an automotive shop.
There is no need to remove all the rust from the inside of the float-bowl, you just have to make sure there is no loose rust. I'd give it a scrape, wipe it clean, and forget it unless it's badly pitted. No need to replace it.
Can you confirm that the engine was running well at the beginning of this process? If it wasn't, you need to clean out the carburetor - remove jets, blow air through the drillings, reinstall jets. I prefer not to disturb such things if they are already clean, but in this case there could be loose bits of teflon tape all over the place. Your mower guy seems to be more of a plumber than a motor mechanic. If you want to strip the carburetor, you have to note the initial setting of the adjustable idle mixture screw so you can put it back in the same position.
