The structure of flames is far more complex than you seem to think. Your observation is good and your thinking is reasonable, but some processing is in fact going on pretty well throughout the flame. The inside portions of the flame must in fact the hot enough at least to produce the gases that burn visibly where they come into contact with the air. The thickness of the visible part of the flame , the "flame wall" as you put it, depends on the nature of the mixing process, the stirring if you like, that brings enough of the air into contact with enough of the hot fuel.

Depending on the nature of your fuel, such as whether it is solid, liquid, or gas, there may be multiple layers to your flame. For example, if you look at a candle flame you will see a barely visible pale outer layer that in fact is very hot. Within that is a thicker, visible yellow layer which is not equally thick all the way up the flame. Inside that, there is the "invisible" inner layer.

But yes, roughly speaking, you have a point. Just don't fool yourself into thinking that things are as simple as they look.