The key study by Jeremias' [Jerem.CentM, 19-27; see also Jerem.NTT, 65-8] takes a long and detailed look at this special term. To touch upon the high points briefly:
Jeremias notes that "Abba" had only come to mean "Father" when used among human beings - and adds that to use the term when referring to God was unthinkable. It was not used by Jews in such a familiar way until the Hasidim movement, which began in the 18th century. The use of "Abba" is "without analogy in the Jewish prayers of the first millenium A.D."
The only exception noted is that of the prayer of Hanin ha-Nehba, who uses the term quite indirectly for God in imitation of children gathered around him that were using it to address him, but not God.
The significance of Jesus' use of "Abba" is that, for the first-century Jew, "it would have been irreverent and therefore unthinkable to call God by this familiar word." "Abba" as used, therefore "reveals the very basis of (Jesus')communion with God," "not a familiarity and intimacy with God available to anyone," but a unique relationship that was bestowed upon Jesus, representing "the centre of Jesus' awareness of his mission." (see also Dunn.EvJ, 48)
We might qualify this a bit, noting that in the NT world, which was an honor and shame society, "familiarity" and "intimacy" had less to do with what we would call a "personal relationship" and more to do with access. But the implication of the use of "abba" in terms of Jesus' special status is the same.
In Mark 14:36, the word "abba" is used by Jesus, along with "pater," which means "father." Jesus therefore used both the familiar term and the term of respect - as did Paul in the Galatians cite. So these two verses read, "Abba, Father." Dunn [ibid., 47] notes that the retention of both the Aramiac and Greek words is a strong indication that the use of "Abba" goes back to Jesus Himself, since there is no reason why the early Christians should not have been satisfied with Greek equivalents for both words.
Although arguments have been made by some such as Vermes that "Abba" was used to refer to God among the lower class in Palestine, we currently still "have no evidence of ordinary Jews addressing God or praying to God" in that manner. [With.JQ, 109] Jeremias' argument, though a bit aged, is still historically sound today.