It should be a clean upgrade but give this a read.
http://codex.wordpress.org/Moving_WordPress
Especially this part.
http://codex.wordpress.org/Moving_WordPress#Moving_to_a_New_Server
If you have a good complete backup of your files and database then it's not hard to do the move. But please do make sure you have that backup safely stored off of your web servers.
On the new server don't install 3.5.1 just install 4.0. The code should see the old database files and make the required changes to the database. If something does go wrong then you'll have that full backup.