Which set of files I need to backup or store in a version control system to completely recover project settings and related debug settings?

The files you need to store in a version control system:

Any generated folders (like PE Generated_Code, PE Documentation or the folder where the object/make files are placed (usually named by the CPU name) can be stripped off to reduce file size.

You can also export/store the workspace settings, like editor/VCS settings/etc, by selecting, File > Export > Preferences from the IDE menu bar.