In my case both devices are on the same network so I did not have that trouble.
Both the solutions you are proposing would obvioulsy work. ssh would be the simplest to setup with a little port forwarding and care. If you are using certificates (which you would need to) and turn off password authentication then IMHO you should be safe enough for your use case. VPN would be more flexible if you want to run other services at the expense of the extra up front effort. However, now you have a process that is not directly started by Borg that needs to be reliable and needs investigating separately if you have any issue. That is the kind of complexity I try to avoid.
Hope that helps