Yes. Adding CapsLock to your Windows Startup folder is one of the most common uses. This ensures the CAPS LOCK key is always in the state you want when you log in. Press Win+R, type shell:startup, press Enter, and create a shortcut to capslock.exe on or capslock.exe off in that folder.