to expand on my comments to Prayuktibid's code, here is what I would propose in the loop:
//blink the led
digitalWrite(LED, !digitalRead(LED)); //digitalWrite(LED, state);
//reset the flag
flag = false; //state = !state;
the rest remains unchanged.
here is how it works:
it works as you would expect, without resorting to delays.
edit: btw, the approach shown above (of setting a flag in the isr, processing that flag in the main loop, and resetting the flag after the processing) is a fairly standard way of utilizing interrupts - pretty much the #1 choice unless you have some special reasons to deviate from it.
edit2: to just follow up on that state above (about trying to process the flag in the main loop).
for something this simple, you can also do it in the isr, making the code much closer to your original code:
digitalWrite(LED, flag); //digitalWrite(LED, state);
//flip the flag
flag = !flag; //flag = true;
the code above produced the same waveform as the code posted earlier.
the lessons learned: there is more than one way to solve a problem, as long as you do it right, :)