Unless you use galvanic isolation (see below) then yes, you must connect the grounds together. That is vital (see here).
The main concern though is the voltage you are switching. The Arduino cannot connect any of its IO pins directly to any voltage over 5V. Doing so will most likely damage your Arduino.
Instead, you need to use the Arduino to switch a device that will switch the button for you. The simplest is a BJT or MOSFET:

simulate this circuit – Schematic created using CircuitLab
But there are a couple of other caveats you should consider. The most important one is to do with how far away the remote device is, and how it is powered. Connecting the grounds of two devices that are very distant can get quite messy, since "ground" at different points may be at completely different potentials, and the voltage drop across the "ground" wire (wires don't have zero resistance) coupled with the existing ground connection through the power supplies creates a ground loop and introduces massive amounts of interference. For this kind of arrangement, it is good to introduce ...
Galvanic Isolation
This electrically separates the two circuits completely. There is absolutely no electrical connection between your Arduino and the device you are switching.
The most common way of achieving this in low powered digital circuits is to use an opto-isolator (also called an opto-coupler). This is basically an LED and photo-transistor together in a single package. Turn the LED on and the phototransistor turns on. The basic arrangement is exactly the same as the circuit above, but the wire going in from the Arduino is connected to an LED:

simulate this circuit
Now you really don't care what ground potential is at the other end, nor how it is powered. As long as the opto-isolator can cope with the voltage it has to switch all will be good. Nothing in the remote circuit can do any damage at all to the Arduino since there is no way for current to flow back from that circuit to the Arduino.
had my arduino connected to my pc and suddenly it worked too
... what worked? – jsotola Feb 8 '18 at 5:55