Here's some more information on the solution I used. My game is in landscape mode, and can't display in letterbox format. It also has onscreen controls which must always show up in the same position relative to the screen edges, regardless on the physical dimensions of the device's screen.
If your game is in portrait mode, you'll have to make some adjustments to make this work for you. Specifically, you'll probably need to calculate the vertical offset of the displayed area, rather than the horizontal offset.
I made my game's resolution 600x320. This should be a wider aspect ratio than any screen that's currently available. The game is designed so that it can be played even if you can't see the edges of the screen. I set the game's scale mode to "Scale to Fit (Letterbox)". You'll note that this is the exact opposite of what the game REALLY needs, which is "Scale to Fit (Fill)". As it turns out, in landscape mode, in some situations, Stencyl arbitrarily switches the two scale modes. More on that later. What you need to know right now is that the actual scale mode we want is "Scale to Fit (Fill)".
What happens in this scale mode is that the engine scales the game so that its smaller dimension fills the device's screen, edge to edge. We're in landscape mode, so the smaller dimension is the height. Some part of the view to the right and left is clipped, but I designed my game so that nothing too important happens there. (My game happens to be a scroller - if the player needs to see that part of the level, he can always go there. It works for me.)
If your game doesn't have any on-screen controls or HUD that needs to be positioned precisely, you can skip ahead to the caveats.
To position on-screen controls, you need to know the offset in game pixels (not device pixels) from the edge of the game's screen to the beginning of the window that's actually displayed. Stencyl's engine keeps that information in the engine.root variable, but that's expressed in device pixels, so to get something we can use in the game we need to divide that by the scale constants that Stencyl uses. We can calculate the offset in game pixels using the expression:
-engine.root.x/(engine.root.scaleX*Engine.SCALE)
See the attached image for an example of a Stencyl code block that calculates this and stores it in an attribute.
Once you have the offset available, you can use it to calculate the position of on-screen elements. Position them where they need to be, and anchor them to the screen to show up regardless of the camera position.
For certain types of games, you'll have to use this offset to position game elements (certain types of puzzle games, for example, might need to have game elements positioned very carefully.)
Now for the caveat: When I tested this solution in the iOS simulator, it worked in some configurations, and didn't work in others. Specifically, this worked on the 4", 4.7" and 5.5" simulators, but not in the 3.5" or iPad simulators. In configurations where this didn't work, I was still getting letterbox mode. What's going on?
It turns out that for this to work, the engine must use Scale to Fit (Fill) mode. But it arbitrarily switches to use Letterbox mode when the aspect ratio of the device's screen is 1.5 or less, which includes the 3.5" screens on iPhone 4 or older devices, and all iPad screens.
I can't stress enough how stupid this is.
The only solution I found was to patch the Stencyl engine to at least be consistent about how it applies this switch. To do this, edit line 217 of plaf/haxe/lib/stencyl/1,00/Universal.hx (or around that line) so that instead of:
widescreen = aspectRatio > 1.5
it says:
widescreen = aspectRatio > 1
This makes my solution work consistently across all iPads and all iPhone configurations, with no letterbox.
This works by pretending that all devices are widescreen devices (why is Stencyl even making that distinction? Because.) A better solution would be to force all devices to NOT be detected as widescreen devices by removing that line, and the if statement that follows it, and selecting the correct Scale Mode, and having Stencyl actually respect it.
I hope this helps, and that you can finally get your game published. I can imagine how stressful it is to have your game rejected and to not get any help from your tool vendor after you paid them good money for a subscription.
