The Problem of Evil
In the creation myth Tolkien wrote for his cosmos, the world comes into existence through song. The Song of the Ainur (angelic beings) shapes what the world will be like, although only Eru can actually give it primary existence. Morgoth (a close parallel to our Lucifer) wants to increase the importance of his part of the Song, so adds elements to it that become discordant. Eru weaves the discord into the music, making it “more complex and beautiful” than it was before. The world is certainly more complex because of the presence of Evil. Does the freely made choice to act out of love made necessary by Evil's presence make the world more beautiful? As Haldir says, '...though in all lands love is now mingled with grief, it grows perhaps the greater.'
The Song of the Ainur was part of Tolkien's early legendarium; it was around long before The Lord of the Rings was written. There's another image of Evil that Tolkien wrote later, though, that's based on the history of the rings of power, a history he didn't discover until he was writing LotR. It gives its name to volume X of "The History of Middle-earth": Morgoth's Ring.
For this to make sense, it's necessary to keep in mind that Sauron is only a servant of Morgoth: a devil to Morgoth's Satan. At the time of the events that take place in LotR, Morgoth has been chained in the Void by the powers of good, so can openly trouble Middle-earth only through his servants, such as Sauron and the balrog. But before his capture, he was able to carry out a diabolical (literally) scheme that makes anything Sauron could do look insignificant. As Sauron put much of his power into a golden ring, Morgoth put much of his power into Middle-earth itself. Our created world is Morgoth's Ring. We know that the only way to destroy the power of Sauron in the Ruling Ring was to destroy the Ring. The only way to completely eliminate Morgoth's evil present in Middle-earth is to destroy Middle-earth.
Of course, Morgoth's Ring takes place in a secondary creation rather than the primary one we live in. But I think it has a message similar to that of another story: Jesus' parable about the tares and the wheat. In that story, an enemy of the landowner sows weeds among the wheat, but because pulling up the weeds would also dislodge the wheat, the landowner decides to let the two grow together until the harvest, when the wheat will be gathered into his barns and the weeds will be burned. Both stories give us images - rather than logical explanations - of the continuing presence of Evil in the world. Both stories tell us that Evil will be with us until the end of all things, because completely removing Evil before then would also destroy much that is good. The full reason for this is something I don't think we'll ever completely understand, but Eru's weaving of Morgoth's discord into the Song so that it becomes “even more complex and beautiful” may speak to it. I also believe it's related to the ability Tolkien gives to mortals in his secondary creation to affect the course of the Song through their free-will decisions (in an interesting twist, Elves don't have the power to do this). In both Tolkien's secondary creation and in our primary creation as he understood it, free will is so important that The One allows us to sin rather than take it away from us. Does some of its importance come from this power to affect the Song - a Song that Evil has distorted?
The image of Morgoth's Ring is powerful enough that it could be taken as Manichaeism or some other form of dualism. But, although Morgoth can mar Middle-earth, its ultimate fate is in the hands of Eru - as the fate of the wheat in the parable is decided not by the enemy but by the landowner. Morgoth remains a creature, not a creator, and is in no way on an even footing with Eru.
The image does make it clear why Tolkien said it was specifically because of his faith that he saw history as a “long series of defeats” - with, he immediately added, triumph at the end. Evil will ultimately be the losing side, but it's always present to be dealt with and will be until the end. Our weapons against it are our freely made choices, both humdrum and heroic. How has our use of our free will affected the Song today?