I know I can send a newline using the option in the bottom right:
But why doesnt it work to use the option 'No line ending', and send a '\n' or '\r\n' after the text using the arduino serial monitor?
The backslash escape encoding for special characters is a feature of double quotes in some programming languages, shells, and other parsers. It is not normally a feature of live text input - an "end user" human typing a backslash is assumed to mean it literally.
"Live" input of a carriage return or newline is more traditionally done with the return/enter key, or with the corresponding control codes - CTRL-J for newline and/or CTRL-M for return. I don't know that the Arduino IDE's Serial Monitor supports either of these, at least once you select "no line ending"