The reason that hanging clothes grow stiff is partly that many materials contain substances that are soft when moist, but stiffen as they dry. Also, the washing water is not absolutely pure, but contains dissolved salts and other substances that harden as they dry, some of them from the detergents you use. As the water dries, the salts and so on get left behind and form brittle solids, which make the material stiff and rough.

Notice that if you crumple the dried washing, it softens somewhat as the crystals and other dried, rigid bits get microscopically broken apart.

If however, you tumble-dry the material, the crystals don't get a chance to clump properly as they dry, and the parts that are soft when wet keep on getting fluffed up by the tumbling, so they never form a stiff structure when dry.

If you like you could see it this way: when small hard particles stick together they form large, rough objects when they dry, just as moisture makes grains of salt or sugar stick together in hard lumps when the moisture evaporates. By tumbling them as they dry, you break them apart and stop them from sticking together, and no hard lumps can form.

And in the example of damp washing, you get soft, dry fabric, instead of clothing that feels like sandpaper.