Ah... that's a new block on me.
Doesn't seem to be having the desired effect though, the result is quite erratic. I notice that an actor's z-order never goes above the number of actors in the layer, despite the block trying to set it to a much higher value. Could that be the issue?
If you want to get into the technical details, a "Layer" isn't much different than an "Actor". They're both extensions of the Sprite class. Sprites can be nested into each other. Each sprite has a list of children. In theory, this means you could have "subactors" with some tweaking of the Stencyl engine code. Layers use this functionality, though. Placing an actor into a layer places it into the layer's list of sprites.
The position of the actor in the list of sprites is what 'z-order' refers to.
This (below) is the code snippet I use. The function DrawStackWrapper.actorInFrontOfActor( a, b ) is one I wrote to determine which actor is in front of another. Mine is for isometric perspective. You'd want to replace this function with your own, using the y-value to determine which actor is in front of the other.
var atest:Int = DrawStackWrapper.actorInFrontOfActor( a, b );
// You'll need to use the code blocks provided in the Flow > Advanced palette to utilize 'swapChildren'
// a.parent.getChildIndex(a) is the code that the 'z index of actor' block calls
if ( ( a.parent.getChildIndex( a ) > b.parent.getChildIndex( b ) && atest < 0 ) ||
( a.parent.getChildIndex( a ) < b.parent.getChildIndex( b ) && atest > 0 ) )
a.parent.swapChildren( a, b );