Sure, you can carry around a CD or flash drive with Ubuntu or Knoppix in case you need to do boot any PC you run across into Linux quickly. But Damn Small Linux has long been the king of tiny Linux distributions. Weighing in at under 50MB, you can keep this complete operating system on your flash drive or MP3 player and have plenty of room left over for music.
DSL version 4.0 is out this week. The kernel has been upgraded from 2.4.26 to 2.4.31. And there've been a boatload of updates to the GUI, libraries, options, and programs.
Oh, and if you just want to try DSL out, since it's so tiny it runs great in Virtual PC, VirtualBox, or other free virtualization applications.