I don't think it's anything to do with the buttons you use to select the voices.
The issue is that the "keyboard parts" are separate from the "MIDI channels." A multi-timbral keyboard usually has a number of "parts" that it lets you play sounds with-- e.g., it may have a right-hand part and a left-hand part for splitting the keyboard and playing with two different sounds, it may have two parts that can be played as two sounds layered together, etc. On some Yamaha keyboards these parts are called "Main" or "Right 1," "Dual" or "Right 2," and "Split" or "Left." It looks like they're called "Voice 1" and "Voice 2" on the P-115. An arranger keyboard will usually have a number of "style parts" as well-- typically 8 of them-- for playing the tracks of a style, and 16 "song parts" for recording or playing back a MIDI song file that uses 16 channels.
Some keyboards-- e.g., the PSR-S and Tyros models-- have functions which let you assign one or more MIDI IN channels to one or more keyboard parts, so incoming MIDI messages will affect the specified part. But other keyboards don't have functions for that-- e.g., my PSR-E/YPT models don't, nor does the P-115. On page 2 of the P-115's MIDI Reference there's a note that says: "Program change and other like channel messages received will not affect the panel settings of the instrument or the notes you play on the keyboard."
Fortunately, there's still a way to achieve what you want to do, but it requires a sort of workaround:
(1) Turn off the keyboard's LOCAL CONTROL setting, so playing on the keyboard produces no sound.
(2) If necessary, set the keyboard's MIDI TRANSMIT CHANNEL to channel 1, or whichever channel you prefer. (Some keyboards-- e.g., the PSR-E/YPT models-- don't have functions for setting the MIDI OUT channels of the various keyboard parts; instead, each keyboard part is transmitted over a specific channel ("hard-coded" to that channel, so to speak) and can't be reassigned to a different channel. But it looks like you can select the MIDI TRANSMIT CHANNEL on the P-115.)
(3) If necessary, set the keyboard's MIDI RECEIVE CHANNEL to "All," so it will receive and respond to MIDI data coming in on all channels. That doesn't mean you need to use all of the channels, but you could. (Some keyboards-- e.g., the PSR-E/YPT models-- automatically receive all 16 channels, and there are no functions for changing that behavior. But the P-115 has a MIDI RECEIVE CHANNEL function that can be set to "All.")
(4) In your DAW, set up one or more MIDI tracks to receive the MIDI channel(s) being sent by your keyboard.
(5) On the MIDI track(s), set the MIDI OUT to send the data back to the keyboard. The MIDI messages that the keyboard sends to the DAW will be sent back to the keyboard; basically, you're using the keyboard as though it were a MIDI controller to play the keyboard's internal sound engine as though it were an external sound module. MIDI messages inserted into the DAW's outgoing MIDI data-- e.g., Bank Select, Program Change, Control Change, etc.-- will be received by the keyboard and will have the desired effect (if that type of message is recognized by the keyboard).
(6) You will have as many MIDI channels to use as your keyboard allows-- i.e., on the P-115 you could use all 16 channels. You could set up 16 different MIDI tracks, each receiving the same incoming MIDI channel from your P-115, and each sending the data back on a different channel, essentially giving you 16 "zones" to work with. By specifying different Bank Select and Program Change values for each track, you can use a different voice with every zone (track). You would turn each zone on or off by arming or disarming its track.
(7) If you wished, you could add "MIDI effects" to each MIDI track, letting you filter incoming MIDI messages based on message type, Note range, Velocity, etc.; the available options will depend on your DAW's MIDI effects feature. You can usually also apply transposition to the received Note values. This lets you define a Note range for each zone, based on which Note values that particular track is set to allow through. Thus, you can use the 16 zones to create splits and layers far beyond what the P-115 allows via its panel controls.
I don't know how MIDI and DAW savvy you are, so if the preceding information isn't detailed enough for you to set things up by yourself, let me know a specific set up that you'd like to try out and I'll post screenshots from my Cubase Elements showing how I'd do it with my PSR-E/YPT keybaords.
