In one last flashback, we see the man in black sitting on a hill overlooking the valley. Jacob approaches and sees the man passing the white stone from hand to hand. Jacob points it out and the man tells him not to gloat. The stone signifies Richard's choosing Jacob's side.
Jacob then rhetorically asks if he tried to kill him and the man says that he just wants to leave and asks Jacob to let him in a begging tone. Jacob says that as long as he's alive, the man isn't going to leave and adds that even if he manages to kill him, someone else would just take his place. The man says that he would kill the replacements too.
Jacob pulls out the bottle of wine he shared with Richard earlier and gives to the man to help him pass the time. Once Jacob says "I'll see you around" and then leaves. The man says "sooner than you think" and smashes the bottle on the log.
So, if the island is the cork, what does breaking the glass mean? It also seems, at least by Jacob's tone, that this is the first time the man in black has tried to kill Jacob. If people have come and gone (meaning died) many times before, why has the man waited until now to try to get rid of Jacob?