What kind of synth are you hooking up and what ports are you using?
USB and/or MIDI ports are generally used exclusively for sending MIDI data back and forth to a computer sequencing program. Generally if you have a USB port on your synth you do not need to actually use the MIDI ports. If you have MIDI ports only and no USB (An older synth) then you need a USB to MIDI adapter (Generally between $20 and $40).
If you want to record audio out, it depends on what sort of audio outs your synth has. If all it has is a headphone jack for output, I'd strongly advise running it thru a small mixer and attenuators before feeding it to the line input on your computer soundcard. Even if it has proper unbalanced or balanced 1/4" phone jack outputs a mixer is still a good idea for level adjustment.
Anyway, keep in mind that MIDI data is recorded and sent with a sequencing program and audio MP3's are created by using a capture program to capture the data into the computer as WAV data and then use a converter like LAME to encode it to MP3 or OGG or FLAC or whatever format you desire.
From the sound of it, if your problem is latency, then you are trying to get your synth to work with a sequencing program. What make and model synth and what MIDI sequencer program are you trying to use with which version of Windows?
We try and help out as much as we can around here, but we need the proper data to make a judgment call.