You should never use the desktop or a folder that already has other files for the location of a profile, but always use a separated/dedicated folder to be used for the profile folder. If you would do this then you should never click the button to remove the files because this will clear all files in the folder including files that already were in this folder. Firefox doesn't use the OS recycle bin, so you would have to use an undelete utility to see whether you can recover the lost files.

What am I supposed to do now? I’ve got no option to recover them

An example of an undelete utility:

https://www.ccleaner.com/recuva