How do I ensure CAPS LOCK is always off when I start my computer?

Add CapsLock to your Windows Startup folder with the "off" argument. Press Win+R, type shell:startup, press Enter, and create a shortcut with the target set to the full path of capslock.exe off. Every time you log in, CapsLock will turn CAPS LOCK off automatically.