If your background is mathematics only, you can start with the following two books:
1] Ross, S. M. (2012). A First Course in Probability (9th ed). Pearson Education Limited. - Introductory probability theory.
2] Freedman, D., Pisani, R., & Purves, R. (2007). Statistics (4th ed). New York: W. W. Norton & Company. - Introductory statistics.
If you decide to deepen your knowledge, this relatively concise and non-repetitive literature list on statistics, probability and their applications may help (read just the 2nd half of the linked page, called "Textbooks for independent study"; l you do not have to read the rest).
Finding a single, comprehensive book will be very difficult. If you're asking because you want to do some self-study, get a couple of used texts instead of a single new one. You can get classics for $3-10 dollars if you look around online.
Feller's "Introduction to Probability" is great for its completeness and expository style, but I don't like the exercises much. And the exposition would not make it so good for a reference. He tends to have a lot of long examples, which is great for fostering understanding, and not so great for looking things up.
I enjoyed Allan Gut's "An Intermediate Course in Probability". There is some overlap with Feller, but it goes into greater depth on those topics. He covers the various transformations, order statistics (which, if I recall, Feller only does by example).
Ross' Introduction to Probability Models is pretty comprehensive, but it is very example oriented. Again, that is not my favorite style (I'd rather they saved those examples for exercises with hints, and kept them out of the main flow), but if it works for you, I can recommend it.
You might as well consider Cacoullos' "Exercises in Probability" and Mosteller's "50 Challenging Exercises in Probability".