In this paper we discuss the role of language in the process of teaching and learning mathematics. We begin with the definition of mathematics by Cassiodorus (in its essential features repeated in the famous Kolmogorov’s article “Mathematics” from the “Great Soviet Encyclopedia” and explored in the author’s book Humanitarian oriented mathematics teaching in general school – Ryazan, Ryazan Institute of Education Development, 2000, 112 p.). This definition illuminates the exceptional role of language in the building, discovering, understanding, teaching, and learning mathematics. We show that this role is predetermined by the abstract character of mathematics. The abstract object cannot be shown; it can only be described with the help of language. This gives rise to the problem of identifying abstract entities on the grounds of their descriptions. We show how the language of set theory helps in the solution of this problem. We illustrate this through the examples from elementary geometry and elementary algebra. And we especially underline the importance of this solution for work the with events in the probability theory. After that, we show how proof and language help to form and develop imagery thinking and, thereby, illuminate the possibility of the left brain hemisphere to participate in the formation of imagery thinking.