To remove your custom wallpaper and return to the default XMB background, simply go to [Settings] > [Theme Settings] > [Wallpaper] and select [Original]. This option will remove your custom wallpaper. If you would like to replace your wallpaper, rather than remove, simply find a photo you want as your wallpaper and press 
If you like to restore your original wallpaper, you can simply select [Wallpaper] from the [Theme Settings] menu and your previously stored wallpaper will be used. There is no way to delete the wallpaper from the system, you can only replace it or choose not to use it.