I am trying to use a DC power supply. I am thinking whether I can use this power supply to provide power for Sim800L module. After some research, I gathered that the Sim800L module requires a max current of 2A during GPRS transmission at 3.8-4.2 V (4.0 recommended). With the power supply I can easily configure the voltage at 4.0V. My question is: for this setup, do I need a DC-DC Buck Converter as in this one for regulation/step-down (not sure which)?
The lab power supply you linked is serious overkill for your needs. It shouldn't need a separate voltage regulator or buck converter, since I assume that it provdes a voltage and current regulated supply.
A regulated, fixed voltage 4V power supply that can provide 2A is what you want. That will probably cost < 1/10 of the amount of the lab supply you linked. The challenge is to find a 4V supply. (Most regulated power supplies are either 3.3V or 5V, since those are common voltages for solid state electronics.)
That said, the linear voltage regulators are usually adjustable with a decent voltage range using a resistor.
You could always build your own voltage regulator using a linear voltage regulator rated for 2A, plus a resistor and a couple of capacitors. You could either feed it from an unregulated 6V power supply or use a transformer, bridge rectifier, and capacitor to provide an unregulated DC input to a voltage regulator.
With a little googling I found a few 4V 2A power supplies online. This one, for example:
(That link is to a wholesale manufacturer that sells them in volume, but you may be able to find the same supply for single unit sale. A 2A switching 4V supply would meet your needs as well)