When you create a regular Linux live USB, you can install software, download files, make changes to the system, and so on, but all of these changes are lost after a reboot. A persistent live USB allows saving any changes you make to the live system, so the they are still present the next time you boot to it.
This article explains creating a persistent live USB with Ubuntu (and flavors like Kubuntu, Xubuntu, Ubuntu MATE, etc.), Linux Mint, Debian or elementary OS (it should work with any Debian or Ubuntu based Linux distribution), using mkusb. You'll need to be running Ubuntu, Linux Mint or Debian to install mkusb (it seems you may be able to install mkusb on other Linux distributions, but there's no list of dependencies to install manually, and it may or may not work).
If you're a Microsoft Windows user, you'll probably want to read this instead: Rufus: Creating A Persistent Storage Live USB With Ubuntu Or Debian From Windows.
mkusb can create persistent live drives that work in both UEFI and BIOS mode. The persistence storage partition created by mkusb uses casper-rw, so it can have a size of more than 4 GB, unlike some other similar tools.
This tool can not only create persistent storage live USB drives, but also regular bootable live USBs of Linux distributions, wipe a device, and more. The only downside of mkusb is its user interface, which uses Zenity and can be a bit confusing, but other than that the application works great.
A few related articles from Linux Uprising that might interest you:
- bootiso: Easy ISO To Bootable USB Drive From The Command Line (from Linux or Windows ISOs)
- How To Encrypt An USB Drive With VeraCrypt (Compatible With Windows, macOS and Linux)
- How To Customize Ubuntu Or Linux Mint Live ISO With Cubic