Right-click on the Taskbar icon and further click on ‘Pin to Taskbar’ That’s all you have to do to permanently base your File Explorer icon on the Windows 11 Taskbar.

Want to make Icons smaller or bigger? You can change Icon size on Desktop, File Explorer or Taskbar in Windows 11, by following these methods.

You can create File Explorer shortcuts in Windows 10 that will open specific folders, but there's a better way.