Video version!
Did Jesus unfairly kill the fig tree?
Matthew 21:19 And when he saw a fig tree in the way, he came to it, and found nothing thereon, but leaves only, and said unto it, Let no fruit grow on thee henceforward for ever. And presently the fig tree withered away.
Mark 11:13-4 And seeing a fig tree afar off having leaves, he came, if haply he might find any thing thereon: and when he came to it, he found nothing but leaves; for the time of figs was not yet. And Jesus answered and said unto it, No man eat fruit of thee hereafter for ever.

Objection: How can Jesus be sinless when he got irritated and angry at the fig tree?

This is a non-objection. There is no hint of irritation or annoyance in Jesus' attitude; how can this be read into the text?

Even if there was, though, I have yet to see the commandment, "Thou shalt not be irritated." And if this was a sin, what do critics do about the weeds in their yards? Do they zap them with weed killer? Does they get irritated and pull them out, and are they therefore sinning by being unkind to them?

There wouldn't be figs on the trees in April. Jesus was asking for the impossible. Even Mark admits it wasn't the season for figs.

A common explanation that the fig tree in question had not produced the "pre-figs" (somewhat edible, very young figs) that it should have borne along with the leaves. Hence, it was barren and useless - and thus became a prophetic symbol and an object lesson: That which does not produce fruit will be cut down - just like a weed.

This is correct, but incomplete. Throughout the OT, and in the NT, the fig tree as a symbol is tied in with expectation -- and withering is tied in with judgment:

Jeremiah 8:13 I will surely consume them, saith the LORD: there shall be no grapes on the vine, nor figs on the fig tree, and the leaf shall fade; and the things that I have given them shall pass away from them.
Hosea 9:10, 16 I found Israel like grapes in the wilderness; I saw your fathers as the firstripe in the fig tree at her first time: but they went to Baalpeor, and separated themselves unto that shame; and their abominations were according as they loved...Ephraim is smitten, their root is dried up, they shall bear no fruit: yea, though they bring forth, yet will I slay even the beloved fruit of their womb.
Nahum 3:12 All thy strong holds shall be like fig trees with the firstripe figs: if they be shaken, they shall even fall into the mouth of the eater.
Luke 13:6-9 He spake also this parable; A certain man had a fig tree planted in his vineyard; and he came and sought fruit thereon, and found none. Then said he unto the dresser of his vineyard, Behold, these three years I come seeking fruit on this fig tree, and find none: cut it down; why cumbereth it the ground? And he answering said unto him, Lord, let it alone this year also, till I shall dig about it, and dung it: And if it bear fruit, well: and if not, then after that thou shalt cut it down.

Note as well that in the messianic age, fruitfulness was a sign of blessing:

Revelation 22:2 In the midst of the street of it, and on either side of the river, was there the tree of life, which bare twelve manner of fruits, and yielded her fruit every month: and the leaves of the tree were for the healing of the nations.

Witherington (commentary on Mark, 312) adds that the fig tree was a special sign of fruitfulness for Israel. It was often the tree used to bring firstfruits to the Temple. The fig tree was an "emblem of peace and prosperity" and in the messianic age was thought to bear fruit (313).

These points serve to answer the question of why Jesus expected figs -- at a time of year (April; the normal season was much later) when there wouldn't be any normally. As he approached Jerusalem, his acceptance as Messiah would have ushered in the Messianic age.

Checking the fig tree for fruit out of season was a signal: Had he found fruit (which normally came in after the leaves), it would have been a sign of the coming Messianic Kingdom.

Since he did not find fruit, the tree became a symbol for fruitless Israel, and of his rejection, and was withered -- in line with the OT judgments prescribed above. The withering of the fig tree is an enacted parable (that recorded in Luke) and a prophetic demonstration.

To ask why Jesus was "irritated" or "peevish" is to miss the significance of this episode....and Mark is actually offering a double meaning when he says it was "not the season for figs." The "season" in question is not the normal fig season, but the "season" or time of the Messiah.

The Greek word Mark uses is also used in a highly symbolic way: Mark 1:15 And saying, The time is fulfilled, and the kingdom of God is at hand: repent ye, and believe the gospel; Mark 12:2 And at the season he sent to the husbandmen a servant, that he might receive from the husbandmen of the fruit of the vineyard.

There's another problem. There is chronological contradiction between recountings of "whether Jesus overthrew the tables of the money-changers (Matt. 21:12) and subsequently cursed the fig tree (Matt. 21:19), or cursed the fig tree (Mark 11:14) and then threw out the money-changers (Mark 11:15).

The correct solution, which recognizes the ancient historiographical practice of arranging material topically for didactic purposes, rather than following a strict chronology, is dismissed by one Skeptic as a resolution that "borders on the pathetic." I submit that such a bare dismissal of ancient literary practices is more deserving of that description.

But Matthew’s gospel has the fig tree withering "immediately" while Mark has the fig tree withering the next day.

Actually, if we want to be particular, that's not what Mark says. Mark says nothing about when the tree withered; he says that the next day Peter in particular noticed the withered tree.

And if we really want to get down to details, the word translated "immediately" (paracrema) is the same word used in Acts 16:33, where Paul’s former guard was "immediately" baptized along with his family. So, the "clear" chronological marker has a degree of latitude.

More significantly, the amazement of the disciples over the withering of the tree takes place "[w]hen the disciples saw this..." which is certainly non-specific in terms of elapsed time, and might as well be the next day as recounted by Mark (in view of our lesson about dischronologized narrative).

Finally, the clincher on this is that Mark's narrative is clearly and unquestionably fit into an intercalation pattern used throughout his Gospel, so that at once any questions of chronology become academic when this pattern comes to the fore.