I'm going to create an IoT system. I use a Raspberry Pi as a MQTT broker and as a webserver. Every day, the Pi gets a new IP address, so I need to find its new IP address via ESP8266 to use this information in my code. I know that I can use a fixed IP, but I don't want to, so it is better if I can find a solution.
First of all, if your Raspberry Pi acts as webserver and mqtt broker, in my opinion you should assign a fixed ip-address, either via configuration in the access point/router or via static ip.
However, if you insist on your approach you could do some hands-on method of pinging all addresses in your LAN-subnet and checking for the MAC-address of the respondents.
If the MAC-address matches to that of your Pi, you have found it.
I have not tested it myself, but this library seems to have what you need for this purpose.
After finding a responsive ip-address via ping, it tries to resolve the corresponding MAC-address using the etharp_find_addr function.
This struct field is the reference to a 6 byte character array that you can use for comparing with the Raspberry Pi's MAC address:
If you only want to find the IP address of the Raspberry Pi, using mDNS as suggested by @Majenko is the superior choice.
If you are looking for all devices in your LAN, as your title suggests, pinging makes more sense.