What is the difference between the following sentences?
The only difference is the first sentence has と (to) and the second sentence has や (ya).
“to” and “ya” both means “and” and connects two or more items but when you use “to”, you are listing all the things in the list. On the other hand if you use “ya” that implies that you are listing up some of the things in the list but not all. So the first sentence means “I study Japanese and Mathematics (and nothing else)”. The second sentence means “I study Japanese and Mathematics (and so on.)”
Remember, “to” and “ya” can only be used to list up two or more “items” you cannot use them to connect “adjectives,” “nouns” and/or “ideas” expressed in nouns.