Poppies are worn because of the poem "In Flanders Fields" that describes the bright red poppies that grew on the battlefields of WWI. Poppies later became a symbol for veterans and their sacrifices in war. 

?In Flanders Fields?

In Flanders fields the poppies blow

Between the crosses, row on row,

That mark our place; and in the sky

The larks, still bravely singing, fly

Scarce heard amid the guns below.

We are the Dead. Short days ago

We lived, felt dawn, saw sunset glow,

Loved and were loved, and now we lie,

In Flanders fields.

Take up our quarrel with the foe:

To you from failing hands we throw

The torch; be yours to hold it high.

If ye break faith with us who die

We shall not sleep, though poppies grow

In Flanders fields.